Ok - can you please run these two commands and post the results:
ls -lah /var/packages/syncthing/target/bin
and
ls -lah /var/packages/syncthing/target/var
I suspect stindex isn’t executable - chmod a+x /var/packages/syncthing/target/bin/stindex would fix that.
root@Synology:/var/packages/syncthing/target/bin# ls -lah /var/packages/syncthing/target/bin
total 75M
drwxr-xr-x 1 sc-syncthing root 234 Mar 5 14:38 .
drwxr-xr-x 1 sc-syncthing root 18 Oct 22 07:58 ..
-rwxr-xr-x 8 sc-syncthing root 32K Oct 21 23:13 addgroup
-rwxr-xr-x 8 sc-syncthing root 32K Oct 21 23:13 adduser
-rwxr-xr-x 8 sc-syncthing root 32K Oct 21 23:13 busybox
-rwxr-xr-x 8 sc-syncthing root 32K Oct 21 23:13 delgroup
-rwxr-xr-x 8 sc-syncthing root 32K Oct 21 23:13 deluser
-rwxr-xr-x 8 sc-syncthing root 32K Oct 21 23:13 nice
-rwxr-xr-x 8 sc-syncthing root 32K Oct 21 23:13 renice
-rwxr-xr-x 8 sc-syncthing root 32K Oct 21 23:13 start-stop-daemon
-rw-r--r-- 1 root root 13M Mar 5 14:31 stindex
-rwxr-xr-x 1 sc-syncthing 104555 21M Feb 29 20:47 syncthing
-rwxr-xr-x 1 sc-syncthing 104555 21M Feb 28 17:19 syncthing.old
-rwxr-xr-x 1 sc-syncthing root 21M Feb 22 11:10 syncthing.rc6_2020-02-22
root@Synology:/var/packages/syncthing/target/bin#
root@Synology:/var/packages/syncthing/target/bin# ls -lah /var/packages/syncthing/target/var
total 168K
drwx------ 1 sc-syncthing root 374 Mar 5 14:38 .
drwxr-xr-x 1 sc-syncthing root 18 Oct 22 07:58 ..
-rwxr-xr-x 1 sc-syncthing root 615 Sep 20 2017 cert.pem
-rw------- 1 sc-syncthing 104555 66K Feb 28 22:23 config.xml
-rwxr-xr-x 1 sc-syncthing root 62K Feb 14 15:03 config.xml.v29
-rw------- 1 sc-syncthing 104555 825 Mar 5 10:48 csrftokens.txt
drwxr-xr-x 1 sc-syncthing root 86 Sep 26 2017 gui_stop
-rwxr-xr-x 1 sc-syncthing root 1.1K Sep 20 2017 https-cert.pem
-rwxr-xr-x 1 sc-syncthing root 1.7K Sep 20 2017 https-key.pem
drwxr-xr-x 1 sc-syncthing root 374 Mar 5 11:40 index-v0.14.0.db
-rwxr-xr-x 1 sc-syncthing root 288 Sep 20 2017 key.pem
-rwxr-xr-x 1 sc-syncthing root 229 Mar 4 2017 options.conf
-rwxr-xr-x 1 sc-syncthing root 832 Oct 22 08:00 syncthing_install.log
-rwxr-xr-x 1 sc-syncthing root 139 Mar 5 05:19 syncthing.log
-rwxr-xr-x 1 sc-syncthing root 6 Mar 5 05:19 syncthing.pid
-rwxr-xr-x 1 sc-syncthing root 0 Jul 8 2019 .syncthing.tmp.389677925
root@Synology:/var/packages/syncthing/target/bin#root@Synology:/var/packages/syncthing/target/bin# chmod a+x /var/packages/syncthing/target/bin/stindex
root@Synology:/var/packages/syncthing/target/bin# ls -lah /var/packages/syncthing/target/bin
total 75M
drwxr-xr-x 1 sc-syncthing root 234 Mar 5 14:38 .
drwxr-xr-x 1 sc-syncthing root 18 Oct 22 07:58 ..
-rwxr-xr-x 8 sc-syncthing root 32K Oct 21 23:13 addgroup
-rwxr-xr-x 8 sc-syncthing root 32K Oct 21 23:13 adduser
-rwxr-xr-x 8 sc-syncthing root 32K Oct 21 23:13 busybox
-rwxr-xr-x 8 sc-syncthing root 32K Oct 21 23:13 delgroup
-rwxr-xr-x 8 sc-syncthing root 32K Oct 21 23:13 deluser
-rwxr-xr-x 8 sc-syncthing root 32K Oct 21 23:13 nice
-rwxr-xr-x 8 sc-syncthing root 32K Oct 21 23:13 renice
-rwxr-xr-x 8 sc-syncthing root 32K Oct 21 23:13 start-stop-daemon
-rwxr-xr-x 1 root root 13M Mar 5 14:31 stindex
-rwxr-xr-x 1 sc-syncthing 104555 21M Feb 29 20:47 syncthing
-rwxr-xr-x 1 sc-syncthing 104555 21M Feb 28 17:19 syncthing.old
-rwxr-xr-x 1 sc-syncthing root 21M Feb 22 11:10 syncthing.rc6_2020-02-22
root@Synology:/var/packages/syncthing/target/bin#Synology 1815
root@Synology:/var/packages/syncthing/target/bin# /var/packages/syncthing/target/bin/stindex -mode idxck /var/packages/syncthing/target/var/index-v0.14.0.db/
172453 block list entries out of 943247 needs GC
root@Synology:/var/packages/syncthing/target/bin#Run it on the other side.
I started in the same time and toke longer. I got a lot of entrances “Missing need entry for needed file …” as follow:
.......
Missing need entry for needed file "Anwendungsdaten/ProfiCash/Druckaufträge/2013/Kontroll-Liste 364002 2013-02-20.pdf", folder "qt39u-sn675"
Missing need entry for needed file "Anwendungsdaten/ProfiCash/Druckaufträge/2014/Kontroll-Liste 404004 2014-11-25.pdf", folder "qt39u-sn675"
Missing need entry for needed file "Anwendungsdaten/ProfiCash/Auswertungen/U+S 2014-06-01 bis 2015-11-30 Konto 364002.TXT", folder "qt39u-sn675"
Missing need entry for needed file "Anwendungsdaten/ProfiCash/Druckaufträge/2013/Kontroll-Liste 404004 2013-02-06.pdf", folder "qt39u-sn675"
Missing need entry for needed file "Anwendungsdaten/ProfiCash - Kopie/ProfiCash/mikogo-viewer.exe", folder "qt39u-sn675"
Missing need entry for needed file "Anwendungsdaten/ProfiCash/DATEN/3_71.DBF", folder "qt39u-sn675"
Missing need entry for needed file "Anwendungsdaten/ProfiCash/DATEN/0005.ptk", folder "qt39u-sn675"
Missing need entry for needed file "Anwendungsdaten/ProfiCash/Keys/23794021.KEY", folder "qt39u-sn675"
Missing need entry for needed file "NTN/allgemein/Wenger gross Potomac FKbD90WP.pdf_Zone.Conflict.Identifier", folder "nwztd-r3g4l"
Missing need entry for needed file "Anwendungsdaten/ProfiCash/Druckaufträge/2018/Commerzbank 401679600 2018-11-30.pdf", folder "qt39u-sn675"
731761 block list entries out of 926361 needs GC
root@Synology:/var/packages/syncthing/target/bin#
All the outlined files are in the filesystem available.
Yeah, this is the device with a screwed database.
Suggest you set the healthy side to send only for all folders listed in the error messages, and run -reset-database on the unhealthy device.
Perhaps try reset-deltas first.
Are those files ignored?
No, these files are not ignored. I running now ./syncthing -reset-deltas, because of no function of ./syncthing -reset-database
But is not the first time tu run ./syncthing -reset-deltas up to now with no effects or changes.
Every time I use this command, I get
root@Synology:/var/packages/syncthing/target/bin# ./syncthing -reset-deltas
[start] 18:33:24 INFO: syncthing v1.4.0-rc.10 "Fermium Flea" (go1.13.8 linux-amd64) teamcity@build.syncthing.net 2020-02-29 18:51:48 UTC
[start] 18:33:24 INFO: Automatic upgrade is always enabled for candidate releases.
[BBRAO] 18:33:24 INFO: My ID: BBRAOxxxxxxxxxxx
[BBRAO] 18:33:25 INFO: Single thread SHA256 performance is 75 MB/s using minio/sha256-simd (62 MB/s using crypto/sha256).
[BBRAO] 18:33:26 INFO: Hashing performance is 60.56 MB/s
[BBRAO] 18:33:26 INFO: Migrating database to schema version 1...
[BBRAO] 18:33:26 INFO: Migrating database to schema version 2...
[BBRAO] 18:33:26 INFO: Migrating database to schema version 3...
[BBRAO] 18:33:26 INFO: Migrating database to schema version 5...
[BBRAO] 18:33:26 INFO: Migrating database to schema version 6...
[BBRAO] 18:33:26 INFO: Migrating database to schema version 7...
[BBRAO] 18:33:26 INFO: Migrating database to schema version 9...
[BBRAO] 18:33:26 INFO: Compacting database after migration...
[BBRAO] 18:33:26 INFO: Reinitializing delta index IDs
[BBRAO] 18:33:26 INFO: Starting deadlock detector with 20m0s timeout
[BBRAO] 18:33:26 INFO: Overall send rate is unlimited, receive rate is unlimited
[BBRAO] 18:33:26 INFO: No stored folder metadata for "default"; recalculating
[BBRAO] 18:33:26 INFO: Using discovery server https://discovery.syncthing.net/v2/?noannounce&id=LYXKCxxxxxx
[BBRAO] 18:33:26 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) starting
[BBRAO] 18:33:26 INFO: Ready to synchronize "Default Folder" (default) (sendreceive)
[BBRAO] 18:33:26 INFO: TCP listener ([::]:34918) starting
[BBRAO] 18:33:26 INFO: Using discovery server https://discovery-v4.syncthing.net/v2/?nolookup&id=LYXKCxxxxxxxxxx
[BBRAO] 18:33:26 INFO: Using discovery server https://discovery-v6.syncthing.net/v2/?nolookup&id=LYXKCxxxxxxxxxx
[BBRAO] 18:33:26 INFO: Anonymous usage reporting is always enabled for candidate releases.
[BBRAO] 18:33:26 INFO: Completed initial scan of sendreceive folder "Default Folder" (default)
[BBRAO] 18:33:26 INFO: GUI and API listening on 127.0.0.1:8384
[BBRAO] 18:33:26 INFO: Access the GUI via the following URL: http://127.0.0.1:8384/
[BBRAO] 18:33:26 INFO: My name is "Synology"
[BBRAO] 18:33:26 WARNING: Syncthing should not run as a privileged or system user. Please consider using a normal user account.
[BBRAO] 18:33:37 INFO: New NAT port mapping: external TCP address 79.246.211.230:30147 to local address 0.0.0.0:34918.
[BBRAO] 18:33:37 INFO: Detected 3 NAT services
[BBRAO] 18:33:58 INFO: Joined relay relay://62.210.140.119:22067
[BBRAO] 19:03:28 INFO: Sent usage report (version 3)Since the problem has shifted to Synology 1815 because of the “Out of Sync Items”, I’ll give a brief outline of the things.
The allegedly not synchronized elements (10.391) should be contained in 6 peers (folders):
However, the individual peers are all up to date and do not show any “Out of Sync Items”:
And yet this deviation is still displayed today:
Why are the 10.391 elements not divided and displayed among these 6 peers?
Please query https://docs.syncthing.net/rest/db-file-get.html for one file that is displayed as out-of-sync both on “Synology-412” and on “Synology-1815-SynoComm”.
Also are all the folders with out-of-sync files in send/receive-only folders on one device? Did you switch any of those devices from send-/receive-only to send-receive or the other way around at some point?
Thanks for the stindex output. I am currently suspecting a logic error around determining needed files and switching to/from send-/receive-only folders (just circumstantial suspicions, nothing concrete unfortunately).
Everything is up to date on the Synology 412, peers and device:
Regarding the peers, only “Buchhaltung” and “Serpil” is on “Send only”. You can see it in the pictures of the peers. However, there doesn’t seem to be a pattern.
Meaning when you click on the number of out of sync sync items to get the list of files, they are from various different folders, right?
Could you run the rest command for one of those files please:
This will (hopfully) show what the difference in database is, that causes the inconsistent display.
As showed above…
I understand that “GET /rest/db/file” should run as a script.
Can the data to be entered export from the database?
It is sometimes difficult for me to store the correct data, e.g. the correct information for “"modified":"2018-08-18T12:21:13.836784059+02:00",” or “"modifiedBy":"SYNO4VL",” etc.
Yes.
Right, missed that sorry. Well, then my send-/receive-only gut feeling is wrong. Lets hope the /rest/db/file output contains some clues.
You don’t need that, the url part is something like 'http://localhost:8384/rest/db/file?folder=ccbah-xmdl5&file=path/to/an/out-of-sync/file'.