Trying to understand/debug conflicts

Support
1

I am trying to understand what I think is a weird situation with conflicts that I think should not be there (this involves several computers and a couple of androids —and I think the later are the culprits). 1. to 4. below are what I think should be the case, if I understand correctly (but 1. to 4. are not what I am observing).

  1. I think that the <filename>.sync-conflict-<date>-<time>.<ext> file should be present in, at most, K - 1 devices if there are K devices. Is this correct? (Having K - 1 sync-conflict files would mean the file has changed in all K devices).

  2. In 1. I am also impliying that <filename>.sync-conflict-<date>-<time>.<ext> are never themselves synced. This is also correct?

  3. Now, if there are more than one of the <filename>.sync-conflict files (with the same <date> and <time>) over all the devices, each one of these should be different.

    1. Different in content?
    2. Could they differ in time, but have identical content? (where identical is, say, that diff finds no differences).

  4. If we are absolutely certain that only one device has made any changes, there should be at most 1 of those files.

  5. Sometimes I see <filename>.sync-conflict-<date>-<time>.sync-conflict-<date>-<time2>.<ext> files. Should this happen? Why? I mean, as soon as we see a <filename>.sync-conflict we know that the <filename>.sync-conflict file has content that is not on the synced <filename> one. And if there have been simultaneous changes in more than one device, then we know there will be more than one <filename>.sync-conflict file. And we know that in so far as we do not solve that conflict, we will always have a conflict in the <filename>.sync-conflict files. So I don’t understand what it means to have a conflict in a sync-conflict. What extra information is it giving me?

2

Number two is an incorrect assumption (the sync-conflict files are treated as normal files after they are created, so they are propagated), hence maybe the rest as well. We do this because the conflict is detected and resolved on one device, creating the sync-conflict file, but it’s just as much of a conflict everywhere else and we don’t know which of the conflicting files is the “best” from the user point of view.

If there’s something that automatically causes a conflict on change you’ll end up with sync-conflict-...sync-conflict-...-sync-conflict files. That said, that shouldn’t happen just because you have a few androids involved. Those are annoying because we don’t handle modification times correctly on them (or they don’t handle it), but we only create sync-conflict files when the actual content differs.

3

Aha, thanks. Most of my assumptions were wrong (including 1, since the sync-conflict propagate). So I was thinking along the wrong lines

Thanks for confirming that. So mtimes issues alone should not lead to this.

Still, I am seeing things I do not understand. They only happen with folders shared with Android devices (folders without Androids involved are painless). And I have configurations such as (where A? is an android and C? is a linux machine):

A1 <-> C1 A1 <-> C2 A2 <-> C1 A2 <-> C2 A1 <-> A2 C1 <-> C2

Sometimes C1 can be off. And during that time, files can be modified in, say, A1. Now, when I turn on C1 it will connect with A1, A2, C2 (in non-deterministic order), and it might receive the same file from them if A2 and C2 have already got the changes from A1, or maybe from only one, etc, depending on what devices connect with what other device and when.

But as per

this shouldn’t matter. If that always holds, then the explanation to what I see is that some other device is, contrary to my assumptions, actually changing content (not just date). The only thing I can think of is that an application (even the OS?) in, say, A1 is keeping a lock on the file, and releasing it just at the wrong time?

(Sorry, I cannot provide an easily reproducible example: this happens to me, but I cannot trigger it at will).

P.S. Since I found them very clarifying, I am adding several of your comments to the FAQ.

4

I can hypothesize this happening anyway…

Consider a scenario where A1 and A2 are connected and in sync. C1 pushes a change to A1 and A2, lets say file:v1 (I’m simplifying the versioning to a simple number here). Due to them being Androids, they fail to update the mtime and essentially create file:v2A and file:v2B each. These files are in conflict, but that’s not a problem since the contents are the same and if they have time to sync the change.

However, if that change is not synced (due to something being disconnected), or just reoccurs all the time because they take turns stomping on each others mtimes, then A1 and A2 will be almost permanently in conflict. Still just metadata so no visible sync-conflict files.

But then when A1 makes a change so that its copy of the file becomes file:v3A that’s a real sync conflict with A2’s file:v2B, because the contents differ and A1’s new file is not descended from the file already on A2. That’s when you get a sync-conflict copy.

Some of this is probably dependent on timing and bad luck and who’s connected to who, but in principle you can easily end up in the situation where two Androids are permanently in conflict. It’s just this doesn’t “show” until an actual data change is introduced into the system and sync conflicts are created…

5

Thanks a lot for the detailed explanation. A couple of follow up questions/comments:

  1. I am using the workaround of keeping files under Android/data/com.nutomic.syncthingandroid ( Option to not sync mtime · Issue #831 · syncthing/syncthing · GitHub) so that the Androids do not eternally play catch with each other (and they don’t). But this still does not prevent the problem. If I understand the scenario above, file:v3A is not a descendant of file:v2B, but file:v3A is a descendant of a file (file:v2A) which is content-identical to file:v2B. Content-wise, the immediate ancestor of file:v3A is identical to file:v2B. But that cannot be taken into account by the version vectors and the conflict inevitably appears because of the niceties of the android OS?

  2. Is there anything that can be done around this (beyond getting rid of all, or all minus 1, android devices :slight_smile: )? I’ve removed the A1 <-> A2 direct connection (though they are connected through several Cs) and have tried both android clients (silk and official one) but I see no improvement (bad luck should be evenly distributed over all solutions?). Should we just wait until “virtual mtime” gets implemented ( Option to not sync mtime · Issue #831 · syncthing/syncthing · GitHub )? Or am I doing something really twisted? If I understand correctly, this could affect many users with more than one Android device?

6

For what is worth, none of these problems have appeared for a week, running all this time ST in the Androids under Debian for Android. I guess, though, that this kind of solution will soon not be necessary since https://github.com/syncthing/syncthing/issues/831 is now closed.