Sync conflicts on identical files in new folder sync

Support
1

Hello,

I have copied a set of folders that syncthing has been syncing on one (source) linux computer to a new hard drive on a new linux computer (using rsync).

Then I set up syncthing on the new computer. I connected the new syncthing to the syncthing source and added the existing folders, being careful to provide the path of the folder that has already been populated with identical files.

After scanning, I find a significant number of files (all of them large >256MB) have been marked as conflicts. Upon researching them, I see that they have identical sizes, checksums, mtimes, and permissions, e.g:

$ ls -l --full-time DNGConverter_10_1.*
-rw-r--r-- 1 jtbr jtbr 343647376 2017-12-24 08:32:48.000000000 +0100 DNGConverter_10_1.exe
-rw-r--r-- 1 jtbr jtbr 343647376 2017-12-24 08:32:48.000000000 +0100 DNGConverter_10_1.sync-conflict-20230829-125346-CHTMNYB.exe

$ sha256sum DNGConverter_10_1.*
8495601252cd40521d908babe4803dee91c52b6c04e46778b6d5933247d349aa  DNGConverter_10_1.exe
8495601252cd40521d908babe4803dee91c52b6c04e46778b6d5933247d349aa  DNGConverter_10_1.sync-conflict-20230829-125346-CHTMNYB.exe

Both computers are running Linux Mint 21 and syncthing 1.23.7. Both hard drives are EXT4 format. Neither side of any folder have send or sync ownership or extended attributes set. I have never set “Large Blocks” (nor do I find such an option).

I’m baffled. Any ideas? I could just let the conflicts propagate before deleting them everywhere, but there are a lot and they are all very big.

Many Thanks

2

Large blocks became default a few years ago. If the files got scanned before that, you’ll have this issue.

1 Like
4

I see, thanks. So large blocks may be set on the new computer but not on the old one. How best to remedy this? Delete the database on the old computer and rescan?

5

That’s effectively what you did now by adding a new device with existing files. I’d suggest just removing the conflicts, or temporarily setting maxConflicts=0 for the folder on that device and letting it do its thing.

6

One detail I forgot to mention is that I set the original folders to send-only rather than send-receive for the time being, and I have not yet connected the new computer to the other two syncthing instances on additional machines (which may or may not be using large blocks already – probably not, I’ve been using syncthing since the early days).

I can set maxConflicts to 0 for the new computer, and complete the sync. But would the database on the original machine have been updated with large blocks change in this case? Also, when I turn on send/receive again will it have the same problem all over again with the other two computers?

Many thanks for your help.

7

Yeah, there’s no great solution for you here. The best is to not “pre-seed” your devices like you did and instead let Syncthing download the data, then there’s no confusion, no block size discrepancy, and no conflicts.

Or, now, you can keep the files as they are in small-blocks mode by forcing the old state on the new device. Since your original device is send-only, you should get a red “override” button it once the new device has done its thing. Press that and the new device will fall in line.

Or, you can upgrade most files to large blocks, easiest is to touch the files (update modification time) on one device so that they are rescanned and sent out as new versions. Lots of re-downloading will happen by all devices. This isn’t precisely the same as starting from scratch and you should still, ideally, not use pre-seeding if you want to avoid conflicts in the future.

Or, you can really start from scratch on your original device, and then use send-only to “override” all the other devices, let them redownload all the large files, and then clean up whatever conflicts resulted.

8

Thanks, I did the first option you recommended and it worked. Let it sync, with max_conficts = 0 on the new machine. Then hit the override button from the source. Both folders were quickly synced then. Then I joined the other machines and turned off send-only (and reverted max_conflicts). Everything is nicely synced now. But I guess the downside is they all remain with the old large blocks setting (off).