Conflict files semantics

General
1

Hello! I wonder if there is anything guaranteed about conflict files. What assumption can be made in case of multiple nodes modifying a single file simultaneously? It seems reasonable to create a conflict file for all changes and that way avoid data loss. Not sure if the last statement is true though…

To make a long story short, I am trying to apply model-based testing techniques to Syncthing and need to make some assumptions which are inviolable unless something unexpected or undesired happened.

2

Well there are assumptions of what should be guaranteed, but I am sure we fall short in terms of bugs in the code.

1 Like
3

So this “to create a conflict file for all changes…” property should be satisifed actually?

4

If it’s concurrent modifications, yes. If it’s modify vs delete then the creation wins.

5

Thanks for your answers. :slight_smile: I found that with concurrent modification of even three nodes it is quite easy to make Syncthing discarded some changes…

I am curious if the model-based testing can be applied to Syncthing some way at all…

6

So I am already aware of cases how you can do that. Essentially if you modify on A, modify on B, scan on A, but don’t scan on B, B will happily overwrite what it has as it’s not aware of the local changes.

7

OK, and now I am confused. :slight_smile: I thought that in the described scenerio B will create a conflict file with a copy of the “local content”.

BTW, I apologize if you found my questions silly/pointless.

8

I testet this scenario right now:

modified file on A, some time later modified corresponding file on B

rescanned only on A which hosts the now older file

-> A fetches file from B and
   overwrites file on A and 
   generates a conflict file 
-> B doesn't touch the file

second test:

modified the files the same way, but rescanned on B:

-> B doesn't touch the file on B
-> A fetches file from B and 
   overwrites file on A and 
   generates a conflict file

So it seems like B checks local changes before overwriting the file, even if no scan on B is triggered.

1 Like
9

Yes, exactly. My understanding is that for two nodes it works as intended.

Anyway, if you scale this test to three nodes a change from one of them will be lost.

10

It’s all based on timing. I am sure you could repro it on two. There is am outstanding pr trying to address this.

11

This should work with two, three, or fifty seven devices. The case that’s not covered is conflict with newly created (never scanned) files.

12

Well, or not scanned if we start downloading.

13

Each device handles conflicts individually so the number of devices and simultaneous changes should not matter. For files that are known to exist when we begin syncing (i.e., have been scanned at least once) we verify that it’s locally unchanged just before replacing it. I don’t think timing should enter into it.

There is https://github.com/syncthing/syncthing/issues/4305 which talks about exactly this but which I can’t reproduce.

14

I was talking for the whole time about concurrent modification of existing files, not creating new ones. The problem seems to has a lot of in common with the issue mentioned by calmh.

To give a better overview of my test - I have a single text file with just a single character in it on three nodes. Each node is in a local network and in sync. I concurrently replace character by some other and then force rescan on one node. Eventually, a change from one node is not present anywhere, so not in the file, nor in any conflict files.

Maybe it is relevant that files are small and the connection is fast… (?)

15

So I am pretty sure https://github.com/syncthing/syncthing/pull/4265 is suppose to address that.

16

Probably yes, but a case with files creation seems to be a slightly different.

17

It doesn’t matter, the underlying issue is as follows:

  1. We receive an update to download a file.
  2. We check if the local file has not changed
  3. We start downloading
  4. The local file changes (or local file appears…)
  5. We finish downloading
  6. We overwrite the local file without checking if it has changed or not.
1 Like
18

I’ve proposed this:

Which is essentially the same as the other user has proposed.

19

I’ve run tests once again. This time the expected conflict files were always created. Your fix helps, at least in this particular case. Thanks!