Hello. I have such a naive question about synchronization order.
Imagine we have 2 dirs A and B registered on host Foo.
Both dirs are registered and running on the same syncthing instance.
Another host Bar also has these 2 dirs added.
Folder A contains file 1.jpg in synced state on both hosts.
We MOVE 1.jpg from folder A to folder B on Foo host.
Is it could be guaranteed that file 1.jpg on host Bar will be removed from its A before appearance in Bar’s B?
Does something change if inotify plugin is in use on both hosts? Does something change if A and B are placed on phisically separate disks?
I mean such state sequence:
Foo - A:1.jpg,B; Bar - A:1.jpg,B
Foo - A,B:1.jpg; Bar - A:1.jpg,B
and two possible following states:
a)
Foo - A,B:1.jpg; Bar - A,B
Foo - A,B:1.jpg; Bar - A,B:1.jpg
b)
Foo - A,B:1.jpg; Bar - A:1.jpg,B:1.jpg
Foo - A,B:1.jpg; Bar - A,B:1.jpg
Are a.3 and b.3 both possible or only one of these is correct?
I tried to find something about this topic in docs but found only few words about message ordering in BEP protocol section, idk if it is related.
There is no ordering of operations between folders (the thing that has an ID and is shared between devices), they are completely independent. A move of a file from one to the other is simply a delete in one folder and a create in the other, either of which may be seen and processed before the other. inotify will make the create and delete be seen at roughly at the same time; the exact ordering is still undefined.
By the way, I have another (but similar) question about changes propagation. Is there any specific order of them detected in a single folder and applied to remote host? I mean the same I asked previously but with A and B in the same folder (the one with id)? Are changes applied on remote (‘Bar’) use the same order them are detected on ‘Foo’ or there is no predictable order? BEP docs says that there is ‘no required order or synchronization among BEP messages’. Does that mean that there is no way to determine which is performed first on ‘Bar’ – the deletion of file in A or moving it’s transmitted copy from .syncthing.1.jpg.tmp to B? Do we usually have such an intermitted state on remote host while there is no accessible copy of file that is moved locally from one subfolder to another?
Foo: syncthing_folder/A/1.jpg; syncthing_folder/B – empty
Bar: syncthing_folder/A/1.jpg; syncthing_folder/B – empty
Foo: syncthing_folder/A – empty; syncthing_folder/B/1.jpg
Bar: syncthing_folder/A/1.jpg; syncthing_folder/B – empty
scanning Foo’s folder
index exchange and starting transmission
5.a (no accessible copy of 1.jpg on Bar)
Foo: syncthing_folder/A – empty; syncthing_folder/B/1.jpg
Bar: syncthing_folder/A – empty; syncthing_folder/B/.syncthing.1.jpg.tmp
5.b (Bar’s A/1.jpg still in place)
Foo: syncthing_folder/A – empty; syncthing_folder/B/1.jpg
Bar: syncthing_folder/A/1.jpg; syncthing_folder/B/.syncthing1.jpg.tmp
6.a (final state – synced)
Foo: syncthing_folder/A – empty; syncthing_folder/B/1.jpg
Bar: syncthing_folder/A – empty; syncthing_folder/1.jpg
6.b (Bar’s A/1.jpg still in place)
Foo: syncthing_folder/A – empty; syncthing_folder/B/1.jpg
Bar: syncthing_folder/A/1.jpg; syncthing_folder/B/1.jpg
7.b (final state – synced)
Foo: syncthing_folder/A – empty; syncthing_folder/B/1.jpg
Bar: syncthing_folder/A – empty; syncthing_folder/1.jpg
The order across folders is undefined as it’s completely independant and driven by the rescans. Message dispatch completely independently from other folders too, so all of the scenarios are possible.
Within a certain folder, there is an order of how things are handled.
As you scan the folder and discover files, batches of updates are sent. As soon as the first batch is received the remote device starts pulling down changes (which would be of a partial scan, as it might not finished scanning on the remote side yet). Within the pull cycle (cycle being whatever is outstanding at this point in time, so new messages that arrive are deferred until the next cycle) we create directories first, then group up all new files and all deleted files to try to see if we can recognize renames, then pull down normal files, then delete normal files and finally delete directories.
If you have a lot of files coming out of the scan, some renames might not be identified as creates and deletes might span different messages (hence different cycles), so you’d end up with a copy rather than a rename, as scan sends deletes last.