Spurious conflicts prevent a simple mirroring setup from working smoothly

(Oleg Gorfinkel) #1

I am not sure whether I am running into a Syncthing limitation here or just not using it properly. I have what seems like a very simple use case: a folder tree on a Windows PC that needs to be automatically mirrored (one way only) to an Android phone (Samsung Note 4, with Marshmallow 6.1). No files are ever modified on the phone – basically, I’m only using it for backup and reference when I am not near the PC.

So far, I have tried to implement this in two ways. In the beginning, I had set up a bi-directional sync between the PC and phone, and it worked MOST of the time, but then, every once in a while, I would get a massive transfer of files from the phone to the PC with incorrect modification dates, which I clearly had not touched at all at either end (those modification dates were from the very first sync that was done when I originally set up the folder for mirroring). It was as though Syncthing would periodically lose track of some of the files in its index and started seeing them as “modified,” based on the original sync date. I would not lose the data in those files per se, as their contents were unchanged, but this spurious syncing would massively screw up the modification dates throughout my PC folder tree. Most of the times, it was just a few files, or a single subfolder, other times it was much more substantial. I could never find any rhyme or reason as to why certain files suffered this fate at certain times and others did not.

After a couple of major debacles with that first setup, where I ended up having to restore gigabytes of documents from another backup, I clearly could no longer tolerate such risks, so I changed the PC-side folder to “Send only”. This protected it from being overwritten by the spurious syncing events, but now, I regularly get “Out of sync” notifications on the PC, which I have to resolve by manually clicking on the red button to override the warning and force the sync to go ahead.

I see two issues at play here: 1) fundamentally, there is a problem with Syncthing sporadically detecting the original sync dates on the Android device as modifications; 2) there is no configuration option to bypass the “Out of sync” warning on send-only devices, so that the sync can procede without a manual override. I realize that the latter would be just a workaround, but it would still be extremely useful to me under the circumstances, as it would free me from having to constantly go in and override the warning. Mirroring is a common need, so I think an automatic override would be beneficial to many people.

Of course, the root issue of the index losing track of some files and misinterpreting the mdates is the one that really needs to be diagnosed and solved, but for now I would be content with just having an automatic “Out of sync” override option. Or am I missing something here that will allow me solve this problem in some other way?

Thanks!

  • Oleg
0 Likes

(Audrius Butkevicius) #2

Android does not support setting mtimes, so we work around that by storing some magical values in the database.

Perhaps you database gets lost, which causes this, I can’t say.

I don’t think there is a better way to do this, you can try setting android side to receive only, but that just shifts the problem to the other end.

0 Likes

(Oleg Gorfinkel) #3

Thanks for replying so quickly. Yes, I understand that database malfunction is the root cause here, but as a workaround, would it not be possible to have an automatic override option for the “Out of sync” condition in the folder configuration for send-only folders? That would be a lifesaver for people who use Syncthing for simple mirroring.

Also, is there anything that can be done to try and diagnose the issues with the database that are causing the problem?

And finally, from what I understand, mdate setting has become possible in later versions of Android (as of Nougat, if I am not mistaken), isn’t that right? If so, is there (or will there be) a version of Syncthing that bypasses the database step entirely and uses Android datestamps directly?

  • Oleg
0 Likes

(Audrius Butkevicius) #4

No, there are plenty of other threads explaining why, but some automatic process destroying files is not something that we want as an option.

You can check the logs to see if there are any hints as to what causes the database problems.

I don’t think mtime was fixed only in recent versions, and if it works it will be set at which point the database is only kept for higher accuracy than supported by the storage.

0 Likes

(Oleg Gorfinkel) #5

No, there are plenty of other threads explaining why, but some automatic process destroying files is not something that we want as an option

Not to rehash an old argument, but sync, by definition, is a risky, automatic process that destroys files in accordance with certain criteria. I don’t see how having one more criterion that is EXPLICITLY controlled by the user can increase that risk (or at least, any such increase would be insignificant compared to the basic risk inherent in Syncthing’s unrestricted sync mode, whereby files win or lose based only on automatic criteria). I mean, as things stand currently, if neither end is designated as send-only or receive-only, Syncthing goes ahead and overwrites files with complete abandon, without prompting the user. So, what extra safety is gained by having such a prompt in a scenario where the user has already made such a designation explicitly? If anything, by doing so, the user has expressed a higher, not lower, level of authorization to procede with the file comparison and overwriting as compared to the unrestricted mode. In any case, what safety is lost by giving the user an option to automatically override that prompt? Particularly in the case where both send-only and receive-only folders are present at either end, isn’t it only logical that files coming from the sender should ALWAYS win and overwrite those on the receiver?

0 Likes

(Audrius Butkevicius) #6

There are other caveats like two send only folders both with auto override set, overriding each other causing a blackhole to open up at your ISP, etc.

Anyways, this has been discussed tons of times before, the answer was and still is no, but you are free to fork and do whatever you want.

0 Likes

(Oleg Gorfinkel) #7

There are other caveats like two send only folders both with auto override set, overriding each other causing a blackhole to open up at your ISP, etc.

I don’t see why these conditions couldn’t be detected and handled… But anyway I understand the answer is “no” and I’ll leave it at that. Thanks for replying.

I’ll keep an eye on the logs to try and catch a hint of what’s going wrong with the database.

Best Regards, Oleg.

0 Likes