How does conflict resolution work?

Hey Syncthing Team,

I’m a long-time Syncthing user and fan. I’m trying to deepen my understanding of the system so I can rely on it for more of my data, and I’ve been wondering about how sync works in different situations.

Syncthing’s FAQ answers “What if there is a conflict?”:

Syncthing does recognize conflicts. When a file has been modified on two devices simultaneously and the content actually differs, one of the files will be renamed to <filename>.sync-conflict-<date>-<time>-<modifiedBy>.<ext> . The file with the older modification time will be marked as the conflicting file and thus be renamed. If the modification times are equal, the file originating from the device which has the larger value of the first 63 bits for his device ID will be marked as the conflicting file. If the conflict is between a modification and a deletion of the file, the modified file always wins and is resurrected without renaming on the device where it was deleted.

Beware that the <filename>.sync-conflict-<date>-<time>-<modifiedBy>.<ext> files are treated as normal files after they are created, so they are propagated between devices. We do this because the conflict is detected and resolved on one device, creating the sync-conflict file, but it’s just as much of a conflict everywhere else and we don’t know which of the conflicting files is the “best” from the user point of view.

Dropbox recently rewrote its sync engine and improved resolution of conflicts, discussed in their blog post. A Dropbox employee commented in 2017 that their client handles “hundreds of special cases”.

I have a lot of questions about how Syncthing’s sync works. What happens when directory trees are rearranged in conflicting ways? What about file and directory renames, symlinks, hardlinks, reflinks, filesystems that support different sets of features, unicode, case sensitivity, bind mounts, divergent metadata?

Maybe if I thought about it long enough I’d see that the FAQ answer resolves all of these questions, but I can’t yet reason about what will happen. In general, how can I predict what will happen to my data under a certain set of changes? Is there an overarching theory-of-syncthing sync that will reveal all this? Where is the relevant code handling these cases?

The overarching theory is: Look for the smallest common denominator. E.g. everything is a file (except for directories, but they are anyway mostly metadata, and lets not talk about symlinks) and what Go’s filesystem library does, is what we do (except sometimes). Syncthing generally doesn’t try to represent differing characteristics, it pretends everything is the same. And when there’s stuff that doesn’t behave the same everywhere, Syncthing bends over backwards (very elegantly, mostly) to hide that (e.g. unicode normalization working differently on macs). Case sensitivity is one of the places where Syncthing just knowingly hits the wall head-first repeatedly: It just pretends windows is case sensitive as everything else and that doesn’t work out well (not because that’s fun, but because it’s a nastier problem than it might seem at first glance). The FAQ has an entry detailing what exactly is synced, but essentially it’s data, modtimes und posix permissions unless ignored.

When it comes to renames: That’s also not tracked. There’s stunts on both sides (where the change happens originally by the user and where Syncthing does the change) to make renames detectable/detect renames in a stream of individual file changes, but devices never tell each other that something has been moved/renamed.

As to conflict resolution: That’s mostly just enumeration. Any change gets counted on a device specific counter. If a number increases on two counters, two devices changed the file “at the same time” (before getting in sync), so that’s a conflict. Who wins is determined by “deletion” (file change vs deletion means file change always wins), by modtime or dice roll if equal.