I’ve started afresh on Syncthing v1.18.1 release with the following setup.
DATA FLOW INTENTION: deviceA =====> (encryption by Syncthing) ====> deviceB
device B got a new SSD a few days ago and therefore everything set-up afresh. Even on device A, the shared-to-encrypt folder did not exist before, I’m first-time using this feature on both sides. After 3 days of initial syncing and getting to “up2date” state on both devices, some files on device A changed (not very much change in data amount). Today, I’ve found deviceB unexpectedly “out of sync”. The device was just sitting there, 24/7 connected, and was expected to follow device A. I have only debian 10 + syncthing on device B and no SMB shares/etc. so I’m 100% sure, no one changed any file in the receive encrypted device B file system meanwhilst.
Which logs do you need to troubleshoot this further. My feeling is some bug might have hit me over.
deviceA:
Debian 10.9 amd64 with Syncthing
folder “cf_so_pool0”, sendOnly, sharedWith deviceB using encryption pw.
deviceB’s log (what hit my eyes that could be an error):
2021-08-09 05:04:31 Device 5B65DNN-XXX client is "syncthing v1.18.1" named "cf_deviceA" at XXXX/tcp-client/TLS1.3-TLS_CHACHA20_POLY1305_SHA256
2021-08-09 05:04:32 Device 5B65DNN folder "cf_ro_pool0" (9jbn4-6ctbo) is delta index compatible, but seems out of sync with reality
This message repeats every 24 hours when my DSL line reconnects.
I don’t see anything special about them… is there a way (because of the concept securing data if sending encrypted) in the design to match the encrypted filename somehow to the original source path/file combination manually? I thought that wouldn’t be possible easily or at all.
Right, I still don’t see anything suspicious. I’m syncing “slowly” changing data,e.g. deleted or added files. No databases, no git or the like. Let’s say 10 files per day. text notes, pictures mainly. all this worked flawlessly for two years now when I didn’t use the new feature.
Another question if info is sufficient to tell: Is it “riskless” to press the delete unexpected items button or might it result in my encrypted copy being not the same as the original when I need it later.
If the two unexpected files look like encrypted files, then you could find out what actual files they are using syncthing decrypt. You can copy just those two files (including their parent dirs) to another directory and run syncthing decrypt on it, which then gives you the plain filenames which should be a better start to try and figure out how they ended up that way.
@imsodin
The items in question belong to my work time tracker tool which writes a backup of my work start / stop times I’ve logged once a day to the given folder on device A.
It’s interesting that the “unexpected items error” persists. Because I’ve now tried to re-export my work times and then hit the rescan button (to grab the updated file) on device A. I expected device B to pull the updated file “receiveEncrypted” from device A. This is what happened.
… so the two changed files came correctly from device A to device B under another filename (not the same as in the error). I *** feel *** it would be safe to hit the “delete unexpected” button. Do you agree or do you want more info (maybe file descriptor dumps from the database)?
I’d also expect it to be safe. Would be nice to know how these files got there, but then again I don’t have any theory to check and not really the time/inclination to go digging in the dark. We can have a closer look if it happens again.
I’m testing migration into full encrypted folders on a new device, quite similar to your setup: deviceA <–> Encrypted Folders <–> deviceB, except all folders are Send&Receive.
First attempt didn’t go well simply changing deviceA and deviceB settings via Syncthing Web UI (almost at the same time) by unmarking folders on the “old” and marking on the “new” syncthing device resulted in deviceA never finish syncing and “unexpected items” everywhere. So decided to scrap it.
Second attempt added deviceA and waited for a full sync, but when added deviceB still had quite a few “unexpected items” so decided to scrap it again.
Third attempt added deviceA to a fresh install and let it sync 100%,
deleted all folders on deviceB’s disk before adding it and received everything fresh from deviceA. This worked better and got no “unexpected items”, but still have one .DS_Store delete stuck on the encrypted side (long story ).
Noticed twice WhatsApp Database (backup) two files as “unexpected items” but that self-healed. Files msgstore.db.crypt14 (latest) and msgstore-YYYY-MM-DD.1.db.crypt14 (from “yesterday”, renamed from msgstore.db.crypt14).
I could also try to troubleshoot further if it helps.
Because the bug seems to be coming back, how about enabling debug logging on the remote device to find the actual culprit?
I would clear the current logfile (if there is any), then run Syncthing using STTRACE=db,model syncthing -logfile=<path-to-logfile> and wait for the bug to affect yet more files .
This is between two Windows 10 devices, and the folders were added and downloaded on the receiving side from scratch. I have no idea what might have caused those “locally changed items”. I plan to to do more testing though, this time with debug logging enabled.
Also, this is likely a different issue, but as I was trying to remove the receive encrypted folders on the remote device using the Web GUI, I kept getting these errors:
).
.
, will setup the trace&log shortly.
