I’m looking for a sync solution between several PCs and my NAS, and selected Syncthing for its features and openness, so this is my first try, but I cannot seem to make sync work…
What I did:
Installed v1.5 community package on Synology 1812+, and SyncTrayzor 1.1.24 (Syncthing v1.5) on W10 64b.
I’ve added each device on the other one, and set up a single sync folder called ‘Audio’ on both. That folder had been previously sync’ed on demand using a manual tool, so contents was mostly identical.
On W10, the folder appears “Up to date” but with the Synology device “Syncing” at 0% for the whole day.
On Synology, the folder appears “Out of Sync”, with the W10 device “Syncing” at 99%.
How can I debug the situation ?
As far as I know, I have no particular filtering set up on either device, and the folder on Synology is R/W by user, group, and others.
I guess this sounds like a newbie question, but I’m a Syncthing newbie
That was it indeed! I followed the steps on the page you linked (maybe a link to it in the FAQ would make sense). I changed the settings and within seconds, I had 18175 synced files :-). Perfect.
… or I should say almost perfect, because on the PC, the Global State is 18250 files (75 more thus). Any idea how I can find where the difference comes from ?
… and on both devices, the “Out of Sync Items” line still lists 25 files, just like in the second screenshot above. But the list is the same on both devices. Should I worry about them if the folder shows a green “Up to date” on both sides ?
I have added a new folder to sync and the number has risen to 259 “out of sync” items
Do you know if that known issue only affects the first sync (in which case I can try to “fix” the out of sync items by hand, e.g. by moving them out and back in the folders), or do they come back randomly? The number of synced files now is about 50000 on 2 machines, but if I settle on Syncthing for all my sync needs, that will be around 10 times more, spread on 4 machines, and I’m afraid I’ll soon overlook those warnings and my family and I will end up having an wrong feeling of safety.
Can I help narrow down the issue ?
Edit: Even more weird: the “Device” section mentions 259 “out of sync” files, but on the new “CDRom” folder alone, it says 656 items are “out of sync” (?):
The folder and remote device out of sync displays are independent - it’s totally fine for them to show different numbers. The folder one shows how many files need updating for the local device to get up to date, the remote one shows how many files the remote device needs.
In your case the folder is “send-only”, so this means the remote changed those files and it refuses to accept those changes (because it’s send-only). They should go away when you click override changes, but beware that means the remote will get their changes overwritten by what the send-only device has.
Hmmm, no, the files all originated from Windows, so two files with different case in the same folder is impossible. Could I find the reason for unsync’d files somewhere in the logs ?
Anyway, I performed two tests on two set of files to try and resync.
For the Audio folder, I moved the offending files out of the source (Win) dir, then forced a resync. Syncthing proposed to override (which I did), and remote files were deleted. I then brought back the files into the original Windows folder and a second sync uploaded the files fine (they disappeared from the “out of sync” list on both sides).
For the CDRom folder, I acted differently : I deleted a set of files listed as “out of sync” from the NAS. I then forced a resync and the files did reappear on the NAS, but they are still listed in the “out of sync” list on the NAS UI.
However, I just noticed that the numbers for the “Audio” folder (which shows no “out of sync” files on either side anymore) are different on Win (18 250) vs NAS (18 275). Mmmh 25 files more on the NAS ? Weird.
Thanks for the very helpful links.
Here’s the JSON result for one such “out of sync” files:
I had to struggle a bit to find the .db location but I think I found it (please check below and tell me if it’s not the right path). I had shut down syncthing before to make sure not to mess with a live DB.
Here are the results on the NAS:
sudo /volume1/misc/stindex -mode idxck /var/packages/syncthing/target/var/index-v0.14.0.db
Sequence entry missing for FileInfo "Compilation/VLC-build/_05.06.J-coreutils/libiconv-1.13.1-2-msys-1.0.13-dll-2.tar.lzma", folder "dhftj-klqfe", seq 4191
Sequence entry missing for FileInfo "Compilation/VLC-build/_05.06.J-coreutils/coreutils-5.97-3-msys-1.0.13-ext.tar.lzma", folder "dhftj-klqfe", seq 4193
Sequence entry missing for FileInfo "Raspberry Pi/magPi-FR/The-MagPi-issue-7-fr.pdf", folder "dhftj-klqfe", seq 4181
Sequence entry missing for FileInfo "Compilation/VLC-build/_05.06.J-coreutils/libintl-0.17-2-msys-dll-8.tar.lzma", folder "dhftj-klqfe", seq 4192
Sequence entry missing for FileInfo "Compilation/VLC-build/_03.B-02-msys/MSYS-1.0.11.exe", folder "dhftj-klqfe", seq 4198
Sequence entry missing for FileInfo "Raspberry Pi/magPi-FR/The-MagPi-issue-5-fr.pdf", folder "dhftj-klqfe", seq 4180
Sequence entry missing for FileInfo "Compilation/VLC-build/_08.H-09.2-msys-pkg-config/glib_2.18.2-1_win32.zip", folder "dhftj-klqfe", seq 4196
Sequence entry missing for FileInfo "Compilation/VLC-build/_02.F-04-mingw-utils/mingw-utils-0.3.tar.gz", folder "dhftj-klqfe", seq 4197
Sequence entry missing for FileInfo "Compilation/VLC-build/_08.H-09.2-msys-pkg-config/pkg-config_0.23-3_win32.zip", folder "dhftj-klqfe", seq 4195
Sequence entry missing for FileInfo "Raspberry Pi/magPi-FR/The-MagPi-issue-8-fr.pdf", folder "dhftj-klqfe", seq 4179
Sequence entry missing for FileInfo "Compilation/VLC-build/_08.H-09.2-msys-pkg-config/pkg-config-dev_0.23-3_win32.zip", folder "dhftj-klqfe", seq 4194
Folder 2 missing 3 sequence entries: #4179 - #4181
Folder 2 missing 8 sequence entries: #4191 - #4198
Missing need entry for needed file ".sync", folder "dhftj-klqfe"
Missing need entry for needed file ".sync/ID", folder "dhftj-klqfe"
Missing need entry for needed file ".sync/IgnoreList", folder "dhftj-klqfe"
Missing need entry for needed file ".sync/Archive", folder "dhftj-klqfe"
Missing need entry for needed file ".sync/StreamsList", folder "dhftj-klqfe"
Note that it’s the whole file. The “out of sync” file I checked via REST above (The-MagPi-issue-8-fr.pdf) is included with a “Sequence entry missing” but there are only 10 other files listed while the UI says there are 115 “out of sync”…
Also note that the missing “.sync” (5 last lines) relate to a previous evaluation of Resilio Sync (which of course does not synchronize those folders anymore). The list of excluded patterns does include “.sync” on both sides though…
For reference, running stindex on the Win PC lists 223 “Sequence entry missing” although the UI says there are 234 “out of sync” files, so we’re much closer.
Do you still use Resilio in parallel? If so, some ignore patterns are needed, otherwise you will have problems when folder overlaps.
Regarding ignore patterns, Syncthing has no standard ignore patterns that are defined in principle or globally. In any case, I don’t know anything about it. The problem may be that Windows files are synchronized that Syncthing may not be able to delete from the NAS later if you delete them on Windows. Then you have to intervene manually, otherwise differences can occur.
I have also Synology for myself, DS412+, 415+, 1815+ and Synology is known to be something special with the ACLs. For example, the file Thumbs.db, which is a typical Windows file, had proven to be a “classic problem” for me. Without ignore patterns, this is of course synced and is also on the NAS. Therefore I use an entry in the Ignore list (?D)(?I)Thumbs.db, which means that no distinction is made between upper and lower case and the file is not transferred, because is ignored, but can be deleted if the process needs. There are other files that I have defined for all peers, since then it runs well, for example I define
I just did another test to check if those alerts are real: compare Win and NAS folders using FreeFileSync.
The result shows that the file reported as “out of sync” and analyzed above (“Raspberry Pi/magPi-FR/The-MagPi-issue-8-fr.pdf”) is, in fact, in sync.
However, some others are not, and they are not reported anymore!
Remember there were first 259 “out of sync” files on the NAS, and trying to force a resync, I deleted a part of them (one folder) from the NAS and forced a resync on both sides. They disappeared from the “out of sync” list, bringing the number down to 115 files… but they were not actually copied :
The sequence entry missing errors are the/a problem. While we don’t know the underlying cause of that problem, there is a fix in v1.6.0 that prevents one way how it can occur. I hope it’s the only way, but given I don’t know the root cause, I can’t be sure.
Now for the files that aren’t synced: Are any of those mentioned in the stindex output? If not, can you get the output of the /rest/db/file endpoint for one of those files?
To fix your current sequence related issues without upgrading, you could run Syncthing once with the STRECHECKDBEVERY=1s environment variable set. However lets first investigate the second, more serious issue, and then go on to this step.