False-positives for changed files, when sideloading to speed up `Send Only` folders

Today I submitted a bug report regarding a logic error in Syncthing https://github.com/syncthing/syncthing/issues/8167

I see in the forum there is evidence that this has been a known issue since at least 2018:

I was directed by someone on the GitHub issue tracker, to discuss the topic here, rather than there.

The person who wrote that directive to me, suggested that my own remarks about the behavior of using Send & Receive as the folder type on both devices, offers a potential solution.

However, I respectfully fail to see how switching both folders to Send & Receive can be considered an acceptable risk. Would that not cause any changes inserted by other applications on the target device, to be imposed upon the master-copy files on the source device?

It shouldn’t be necessary - if files are fully identical (mod. time and perm included) between a send-only and any other device, they should get to up-to-date state. Please post screenshots and the metadata for an example file from both sides to look into what’s hapening for you.

What type of metadata would you suggest? I’ve been thinking this malfunction is being caused by the ctime or btime or something else that makes Syncthing think that the files are newer, despite having exactly the same contents. Edit: And yes I have ignore permissions set on all my devices. I would think this should be much faster for other people to recreate than for me to try to explain via non-interactive screenshots, but I’ll see what I can do to help. :slight_smile:

We only consider mtime and posix permissions (and size of course).

Great thanks. Please keep in mind there’s a small amount of volunteers working on this project in their free time and a lot of users reporting issues. For me personally I only act on reports once I got convincing evidence there’s something to act upon - if I ask questions, I am not yet convinced. Nothing against anyone or any problem in particular, just necessary time management.
And just for clarification: Of course even once I am convinced, there’s usually still more things to be acted upon than there is time :slight_smile:

Oh I very much understand, thank you. I have wanted to contribute to Syncthing for many years as it is among the most valuable of open-source projects in my view, and I am just now learning Git, so I hope to be of use in the future.

You may have just now answered my question there: Were you implying that Syncthing deletes files with newer mtime even if the contents are identical?

No it doesn’t delete anything based on mtime. But a change in mtime creates a new version, which can lead to out-of-sync state and the override button appearing. To get in sync with such a new version, it just updates the mtime - no changes to the data happening.

What is the meaning of version in this context? Are you implying Syncthing treats a file with a different mtime but identical content, size, and permissions, such that it would become a duplicate file (by contents) in the .stversions folder? Edit: I re-read your reply and your remark that "it just updates mtime" is the behavior I would expect, rather than deleting the file. (Whereas deleting of the file is what Syncthing does, in my experience. :slightly_frowning_face: )

As far as I understand, Syncthing should just update the mtime if the contents are identical.

Edit: The information below is incorrect. For why, read the follow-up posts.

If you use File Versioning though, then yes, the existing files will be moved there (even if they only differ by mtime). Is this what is happening on your side?

If yes, then this is likely the intended behaviour, and for a good reason. For example, If I open a 20-year-old Word document and by mistake click “save”, its mtime will be updated despite making no changes to its contents. In such a case, I will then go and restore the previous “version” from .stversions (because, at least for myself, the mtime conveys a very important message here, i.e. that this is a very old file and not something created just today).

Well, the good news is that I am thus far unable to re-create the logic error with one-byte files I create myself. (Typically it would occur with folders full of photos.)

Oh…wait…now I’m getting an odd behavior I’ve not seen before…keeping the file I created on the target machine, and then adding .syncthing.2.txt.tmp where 2.txt is the name of the file on both devices.

So that is…odd. Not even creating the .stversions folder.

Here is a screenshot if that helps somehow:

At least I can be proud to be able to show btime in the Nemo filesystem explorer. :smirk:

Something seems up, i.e. Syncthing appears to be unable to replace the original file with the temporary one.

Screenshots of the Web GUI from both sides will be helpful here, but also you can try going to “Actions → Advanced → GUI”, enable "Debugging, save the changes, then go to “Actions” again and click on “Support Bundle”. An archive with various logs, etc., including your config file, will be created automatically, which you can then upload here to help diagnose the problem. Of course, do the above only after reproducing the problem, and if possible, in a clean setup.

PS The category should probably be changed to “Support” :wink:.

1 Like

That’s wrong: Permission and mtime only changes do not get versioned.

I should hope not! :smile: That’s good to hear.

Yet something seems to have caused Syncthing to put the files into the .stversions folder with a date-deleted timestamp appended to their names. And today the anomaly of the .tmp file. I shall continue to investiage the causes.

My bad… I’m sorry for the misinformation then :hushed:.

The Word file example got me confused, as it seems MS Word itself changes more on “save” than just mtime, causing file hash to change too. I’ve just checked on a simple text file, and indeed, no versioning happened there (and the hash stayed the same too).

Which example is this? If you are referring to my example, I provuded a plain-text example in my screenshot as well.

I didn’t follow your examples, but I can provide some principles and you can apply them to your situation.

If you add files on a receive-only side, that side considers itself out of sync and offers to revert the changes – reverting an addition means deleting it, or moving it to the versioning directory if you have versioning enabled.

Otherwise, when a change is synced it is indeed not versioned if the contents are the same, as there would be no point to it.

I don’t generally recommend combining send-only with receive-only, because it can be confusing. Inadvertent changes can be overridden on one side and reverted on the other and you may need to do both to get some changes to sync. It’s conceptually much easier to have one side be send-receive and do the filtering, in whatever direction is appropriate, on the other side.

Yes, that is what I do. I add the files on the send-only side, pause the folder, then add the files on the receive-only side, then resume the folder. I would then expect Syncthing to verify they are the same files, and touch the mtimes on the recieve-only side.

Hmm, so you would combine Send Only with Send & Receive?

Edit: Oh, it’s you! It’s a great honor to meet you. Syncthing has been a resource valuable beyond words. :pray:

1 Like

Yes. Or, do what you do and make sure both sides are scanned and quiescent. Then hit override on the send-only side, and wait until both sides are quiescent again, then hit revert on the receive only side. You skip half of that dance by having send-receive on one of the sides.

Thanks! :slight_smile:

1 Like

Getting a bit off-topic… In fact, Microsoft Office is known to even change file contents when just opening files. Without updating the mtime of course, because naturally the file “didn’t change”.

1 Like

My own example given in https://forum.syncthing.net/t/false-positives-for-changed-files-when-sideloading-to-speed-up-send-only-folders/17992/8, which turned out to be flawed.

1 Like

I don’t remember if it’s Word or Photoshop or both, where opening a file to print it will then give you a “Save changes?” prompt when closing it again. :confounded:

2 Likes