"Out Of Sync" - First use of Syncthing beween Synology and W10

vdeconinck · May 25, 2020, 4:51pm

Hi Andy,

Thanks for the suggestion. I had followed instructions on the kastelo.net page linked above so had already checked “Ignore permissions” on folders on the NAS, but I had left it by default (unchecked) on the Windows PC. I now just checked the option on Windows too…

Let’s see what it brings.

Edit: No change…

Vincent

PS: And, agreed, Synology’s take on ACL is really nasty.

vdeconinck · May 25, 2020, 5:20pm

Hi imsodin,

The missing files are not listed in the stindex output on the NAS, but they are listed in the stindex output from the Win PC, as Sequence entry missing.

So basically, that’s the issue to focus on, and I guess the first thing to do would be to upgrade to the test 1.6 version ? (Or can I still help understand what caused those “Sequence entry missing” ?)

If I go for the upgrade, a few questions then:

On Windows, I’m currently using Synctrayzor 1.1.24. What staging version would you suggest to install ?
Do you also suggest upgrading on the NAS ? And if so, is there a “staging” synology community package ?

Kind regards,

Vincent

imsodin · May 25, 2020, 5:45pm

Synctrayzor uses auto-upgrading, i.e. you can select release candidates in the settings menu of the web UI. Try the same for synology, maybe the use that too - I have no clue about that package.

If you don’t want to use release candidates, you can use this suggestion to fix the current problem:

If you can make the problem reappear, logs with db,walkfs enabled might help find the root cause. But beware those logs will be huge, you shouldn’t do that without redirecting your logs and monitoring their. Aka really only do this if you want to invest time.

vdeconinck · May 25, 2020, 7:19pm

Hi imsodin,

First let me thank you again for your help. It’s much appreciated!

Makes sense.

I just tried that route first indeed. I guessed the environment variable should be on the Windows side only, right ?

Here is the log:

[monitor] 20:25:25 INFO: Log output saved to file "C:\Users\Vincent\AppData\Local\Syncthing\syncthing.log"
[start] 20:25:25 INFO: syncthing v1.5.0 "Fermium Flea" (go1.13.10 windows-amd64) teamcity@build.syncthing.net 2020-04-21 20:45:03 UTC
[start] 20:25:25 INFO: Using large-database tuning
[HQMIU] 20:25:25 INFO: My ID: HQMIU6O
[HQMIU] 20:25:26 INFO: Single thread SHA256 performance is 204 MB/s using minio/sha256-simd (156 MB/s using crypto/sha256).
[HQMIU] 20:25:27 INFO: Hashing performance is 173.87 MB/s
[HQMIU] 20:25:27 INFO: Overall send rate is unlimited, receive rate is unlimited
[HQMIU] 20:25:27 INFO: Using discovery server https://discovery.syncthing.net/v2/?noannounce&id=LYXKCHX
[HQMIU] 20:25:27 INFO: Using discovery server https://discovery-v4.syncthing.net/v2/?nolookup&id=LYXKCHX
[HQMIU] 20:25:27 INFO: Using discovery server https://discovery-v6.syncthing.net/v2/?nolookup&id=LYXKCHX
[HQMIU] 20:25:27 INFO: Stored folder metadata for "dhftj-klqfe" is 105h14m45.5423799s old; recalculating
[HQMIU] 20:25:27 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) starting
[HQMIU] 20:25:27 INFO: QUIC listener ([::]:22000) starting
[HQMIU] 20:25:27 INFO: TCP listener ([::]:22000) starting
[HQMIU] 20:25:27 INFO: GUI and API listening on 127.0.0.1:8384
[HQMIU] 20:25:27 INFO: Access the GUI via the following URL: http://localhost:8384/
[HQMIU] 20:25:27 INFO: My name is "PC Vincent"
[HQMIU] 20:25:27 INFO: Device 7PUJGCV is "NAS1812+" at [dynamic]
[HQMIU] 20:25:27 INFO: Established secure connection to 7PUJGCV at [2a02:1811:507d:2d00:88fb:8444:3750:88c]:58699-[2a02:1811:507d:2d00:211:32ff:fe13:c31b]:22000/tcp-client/TLS1.3-TLS_CHACHA20_POLY1305_SHA256
[HQMIU] 20:25:27 INFO: Device 7PUJGCV client is "syncthing v1.5.0" named "NAS1812+" at [2a02:1811:507d:2d00:88fb:8444:3750:88c]:58699-[2a02:1811:507d:2d00:211:32ff:fe13:c31b]:22000/tcp-client/TLS1.3-TLS_CHACHA20_POLY1305_SHA256
[HQMIU] 20:25:29 INFO: Repaired 223 sequence entries in database
[HQMIU] 20:25:29 INFO: Connection to 7PUJGCV at [2a02:1811:507d:2d00:88fb:8444:3750:88c]:58699-[2a02:1811:507d:2d00:211:32ff:fe13:c31b]:22000/tcp-client/TLS1.3-TLS_CHACHA20_POLY1305_SHA256 closed: started folder "CDRom" (dhftj-klqfe)
[HQMIU] 20:25:29 INFO: Ready to synchronize "CDRom" (dhftj-klqfe) (sendonly)
[HQMIU] 20:25:29 INFO: Stored folder metadata for "nq7qk-mczvt" is 140h28m36.5190667s old; recalculating
[HQMIU] 20:25:30 INFO: Ready to synchronize "Audio" (nq7qk-mczvt) (sendonly)
[HQMIU] 20:25:31 INFO: Established secure connection to 7PUJGCV at [2a02:1811:507d:2d00:88fb:8444:3750:88c]:58759-[2a02:1811:507d:2d00:211:32ff:fe13:c31b]:22000/tcp-client/TLS1.3-TLS_CHACHA20_POLY1305_SHA256
[HQMIU] 20:25:31 INFO: Device 7PUJGCV client is "syncthing v1.5.0" named "NAS1812+" at [2a02:1811:507d:2d00:88fb:8444:3750:88c]:58759-[2a02:1811:507d:2d00:211:32ff:fe13:c31b]:22000/tcp-client/TLS1.3-TLS_CHACHA20_POLY1305_SHA256
[HQMIU] 20:25:31 INFO: Connection to 7PUJGCV at [2a02:1811:507d:2d00:88fb:8444:3750:88c]:58759-[2a02:1811:507d:2d00:211:32ff:fe13:c31b]:22000/tcp-client/TLS1.3-TLS_CHACHA20_POLY1305_SHA256 closed: reading length: read tcp [2a02:1811:507d:2d00:88fb:8444:3750:88c]:58759->[2a02:1811:507d:2d00:211:32ff:fe13:c31b]:22000: wsarecv: Une connexion existante a dû être fermée par l’hôte distant.
[HQMIU] 20:25:37 INFO: Completed initial scan of sendonly folder "Audio" (nq7qk-mczvt)
[HQMIU] 20:25:38 INFO: Detected 0 NAT services
[HQMIU] 20:25:46 INFO: quic://0.0.0.0:22000 detected NAT type: Port restricted NAT
[HQMIU] 20:25:46 INFO: quic://0.0.0.0:22000 resolved external address quic://195.16.0.126:22000 (via stun.syncthing.net:3478)
[HQMIU] 20:25:48 INFO: Completed initial scan of sendonly folder "CDRom" (dhftj-klqfe)
[HQMIU] 20:26:05 INFO: Joined relay relay://163.172.60.25:22067
[HQMIU] 20:26:25 INFO: Established secure connection to 7PUJGCV at [fe80::a8be:8658:f1b2:ed41%Ethernet]:22000-[fe80::211:32ff:fe13:c31b%Ethernet]:46649/tcp-server/TLS1.3-TLS_CHACHA20_POLY1305_SHA256
[HQMIU] 20:26:25 INFO: Device 7PUJGCV client is "syncthing v1.5.0" named "NAS1812+" at [fe80::a8be:8658:f1b2:ed41%Ethernet]:22000-[fe80::211:32ff:fe13:c31b%Ethernet]:46649/tcp-server/TLS1.3-TLS_CHACHA20_POLY1305_SHA256
[HQMIU] 20:26:35 INFO: Failed to exchange Hello messages with 7PUJGCV at [2a02:1811:507d:2d00:88fb:8444:3750:88c]:22000-[2a02:1811:507d:2d00:211:32ff:fe13:c31b]:48531/tcp-server/TLS1.3-TLS_CHACHA20_POLY1305_SHA256: EOF
[HQMIU] 20:26:35 INFO: Failed to exchange Hello messages with 7PUJGCV at [2a02:1811:507d:2d00:4086:4afe:153f:6768]:22000-[2a02:1811:507d:2d00:211:32ff:fe13:c31b]:56316/tcp-server/TLS1.3-TLS_CHACHA20_POLY1305_SHA256: write tcp [2a02:1811:507d:2d00:4086:4afe:153f:6768]:22000->[2a02:1811:507d:2d00:211:32ff:fe13:c31b]:56316: wsasend: Une connexion existante a dû être fermée par l’hôte distant.
[HQMIU] 20:26:38 WARNING: Non-increasing sequence detected: Checking and repairing the db...
[HQMIU] 20:26:39 INFO: Repaired %v sequence entries in database 174

So it seems the “Non-increasing sequence detected: Checking and repairing the db…” did the trick.

By the way, you said to run Syncthing once with the env. variable, but why not leave it so that DB is checked at every startup ? Or is it “risky” or does it use system resources during the whole time the program is running ?

Anyway, now the UI reports only 11 “out of sync” items on both sides (the same set, and by the way, the “Raspberry Pi\magPi-FR\The-MagPi-issue-8-fr.pdf” is still part of it ) although they are, in fact, in sync (just checked with a bit-by-bit comparison)

What would be your suggestion to fix these false “out of sync” ?

Kind regards,

Vincent

imsodin · May 25, 2020, 8:36pm

With an interval of 1s it doesn’t only do it on startup, but every time a folder is “opened in the db”, which is e.g. also when you pause and unpause a folder. And it can be costly depending on folder size. Otherwise it’s not risky, just shouldn’t be necessary.

vdeconinck · May 28, 2020, 8:04pm

Hi,

I tried moving the 11 “out of sync” items out of the Windows folder, let a sync complete, then bring those files back in, but the result is the same.

Could you advise the recommended way to get rid of those “false out of sync” files ?

Kind regards,

Vicent

vdeconinck · June 3, 2020, 7:07pm

A few days ago, I added a 3rd folder to synchronize and the “out of sync” number rose to 150. But the good news is: Syncthing upgraded to v1.6.1 and after clicking on “override”, all “out of sync” items disappeared, except 4 (all from the 3rd folder):

What is weird is that these 4 items are in fact one single item (test) which was moved out of the ‘src’ folder. Consequently, its subdir “custom” was moved too, and the same in the 'out dir. So it’s not a big deal, but probably they should not be detected as “out of sync” still…

I just deleted the ‘test’ item from ‘out’ folder on the NAS by hand and it immediately disappeared from the “out of sync” list. I can do the same with the second one in ‘src’, but do you want me to perform a test to understand why they still show up ?

Kind regards,

Vincent

imsodin · June 5, 2020, 8:37am

For clarification (you could also post full screenshots from both sides instead of answering the following ): The out of sync items in the screenshot appear in the web UI of windows on the remote devices section for the nas, right? What does the “Documents Vincent” folder show on the nas - up-to-date, out-of-sync, failed items, local/global state?

vdeconinck · June 5, 2020, 9:03am

As Confucius said, a screenshot is worth a thousand posts, so:

NAS side:

when clicking on “out of sync items” in the “Folders” section:

when clicking on “Failed items”:

Win side:

when clicking on “out of sync items” in the “Remote Devices” section:

KR,

Vincent

imsodin · June 5, 2020, 10:36am

Tadaaa, we have a winner, in the form of the second error message: In your directory there is a file (or several) which is ignored. Thus Syncthing can’t delete it, because it doesn’t know whether you want to keep the ignored files or not. If you never want to keep those, you should add a (?d) prefix to the relevant ignore pattern to allow Syncthing to delete the file.

vdeconinck · June 5, 2020, 6:57pm

Hi Simon,

Oh, yeah, right !

admin@NAS:/volume1/docs/Documents Vincent/Programmation/Java/Ginj/src/info/ginj/test$ find .
.
./custom
./custom/@eaDir
./custom/@eaDir/Leaf.png@SynoEAStream

So if I understand correctly, this folder got sync’d from Windows to the NAS, then Synology created its bloody @eaDir folder which is in the Syncthing ignore list, but when the folder was deleted on Windows, Syncthing didn’t delete the remote folder because the ignored file could be important. Makes sense !

Didn’t know about the (?d) prefix. Maybe it should be added on https://docs.kastelo.net/synology/sharing/ . Do you want me to contact the author ?

Anyway, it did the trick!:

On NAS:

On Win:

100% in sync everywhere, I like that. I’m going to add more folders.

Thanks for your help and patience and keep on the good work!

Vincent

imsodin · June 5, 2020, 7:09pm

Nice to see everything green

That would be @calmh

vdeconinck · June 26, 2020, 9:28am

Hi.

Before this topic auto closes, I just wanted to chime in to express my satisfaction and gratefulness for Syncthing and the help I got here. Version 1.6.1 works like a champ and I decided to switch all my synced folders to Syncthing.

Keep on the good work and have a nice day,

Vincent

Andy · June 27, 2020, 5:03pm

I created some basic ignore patterns, using in all the peers on all my devices, means Synologys, Windows computers Android Smartphones. Since this time all is running smoothly:

(?d)@eaDir  
(?d)(?i)@SynoResource  
(?d)(?i).@__thumb  
(?d)(?i)Thumbs.db  
(?d)(?i).thumbnails  
(?d)(?i)ehthumbs.db  
(?d)(?i)desktop.ini  
conflict

system · July 27, 2020, 5:03pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.