Some files in subfolder not synced, syncthing claims "Up to Date" on BOTH sides

I have some files in subfolder of a synced folder which are not synced from machine A to machine B, and adding any more files into this subfolder won’t transfer them. However, this subfolder is not empty and exists on both machines and has synced in the past, now it just stopped. OTHER subfolders of the same synced folder still process new files fine. And here’s the weirdest part: syncthing claims “Up to Date” on BOTH sides.

I already tried -reset-database on machine A, the one with more files that it doesn’t want to hand over to machine B, to have it reindex that entire folder including the stuck subfolder, but it didn’t help.

What could possibly be the problem?

1 Like

Ignore patterns. Permission issues (especially if you’re on an odd NAS OS or similar). Show screenshots?

1 Like

Both sides are linux with ext4 and recent 5.x kernel. Wouldn’t permissions cause errors/failed items?

The ignore patterns for the folder with id importantdocuments (disk folder /home/ellie/Documents/ on both sides too) are on both sides:

/My Writing/
/My Writing Sync/

The subfolder that won’t sync anymore is docs/input (on disk /home/ellie/Documents/docs/input), all affected files have names beginning with 2020 or 2021 since they’re named by date followed by the paperwork item name I scanned and saved. However, files with the same naming scheme in the same subfolder have previously synced, and are still present, along with the newer files that won’t sync anymore.

Edit: is there something I can check in the UI for what syncthing thinks of these specific unsynced files on machine A (where they’re present)? Like if it thinks they’re synced, or what

Edit 2: the folder is also pretty giant as a whole (the Documents/id importantdocuments folder I mean), says 84366 files in the syncthing UI. In case that is somehow relevant

Edit 3: also the non syncing files reference medical documents etc in name, so if screenshots really would help still i guess i could direct message them or something but maybe not in a public post

There should indeed be errors in the UI if the files can’t be accessed. If there are no errors and both sides claim up to date and it’s not ignore patterns or permissions then I’m not sure.

1 Like

Screenshots of the web UI shouldn’t contain any filenames, and if folder path/label is sensitive, you can easily censor them. It helps as it gives an overview of your setup and state, plus there’s a lot of small bits of info which might hint at whats going on.

If you add a new file into a directory that doesn’t get synced, does that get synced?

If you add a new file into a directory that doesn’t get synced, does that get synced?

In Documents/docs/input no. In Documents/whatever yes. it appears always yes, but some of the still unsynced older files in docs/input just won’t transfer. I guess I’d have to manually rename them all and then back again so they get synced. Seems like a bug of sorts?

Machine A (with the not synced files):

Machine B (which misses the files):

Ok, new files syncing is as expected. Most likely if we look closer it’s going to missing indexes. That somehow comes up every now and then and still no clue why.

You can fix this by running with --reset-deltas on either device to resend indexes.

I have done --reset-deltas now on both sides, but it still hasn’t synced the affected files. Anything else worth trying or should I just go the rename route? (Which will likely get rid of any remaining clues to figure out why this is even happening)

Oh, cool. Turns out the reset deltas thing just led to syncthing deleting it without asking on both sides now, so the files are just gone now.

Is there any good reason why .stversions still doesn’t carry deleted files for cases like this? Data loss is the reason that constantly leads me to want to abandon syncthing for good. It works fine for a year or two, then something inexcusable like this happens.

That’s bad. It means Syncthing accounted those files as deleted on the device they were missing, but didn’t previously tell the other about those deletions. Now with --reset-deltas that information was propagated and acted upon. If you have versioning enabled on the device that previously had the files, they should be in .stversions. Querying /rest/db/file for one of the files will give some information on it’s history (GET /rest/db/file — Syncthing v1 documentation), but it likely won’t be able to explain why it was marked as deleted in the first place. Only logs from the time it happened (with --audit or db debug logging) could explain that, and that would require a way to reproduce. Which there usually isn’t for weird occurrences in production like this.

Yes it is bad, and .stversions has never, ever in my life had deleted files (just overwritten ones). It should though, ideally enabled by default with some short final deletion span of like 24 hours that should be configurable. I beg you, this would have saved me in all my significant, regular data loss events with syncthing.

It rolled back the entire folder by the way, every single file added in the last two months missing on one side was treated as a deletion. In any case, .stversions appears to be utterly useless for this, and why? I think this should be fixed.

The screenshots show now versioning being enabled? I understood that you want that to be enabled by default (Option for trashcan versioning to affect only deleted files, enable by default · Issue #6164 · syncthing/syncthing · GitHub). However it means there’s no indication that versioning failed - it simply wasn’t enabled. Correct?
And there’s now something that partially helps for this: There are configuration defaults. You can set your default for new folders to enable versioning.

As to the actual problem of why the deletions propagated: Did you check the logs? Is there anything out of the ordinary (messages about pulling/syncing or scanning in general, maybe even related to the paths in question)?

The above status show the same on both sides: 79607 files, 6830 folders, 34.1 GB. So that would indicate that various files are maybe not indexed, right? The question would then be, which signs are in the filenames that prevent a indexing?

It seems I have the same issue in this topic but I’m not ready to do --reset-deltas right now. Can I do something to debug such behavior?

Moreover, I have the third node on which another set of files has been missed. So, I will describe my case a little bit more:

There is the first node (Synology NAS) on which a set of files is created (about 5 80-kB files per second) in 10 minutes. Other two nodes are online in this time (Linux). As a result the folder is almost synced, each node in “Up to Date” state but some different files are missed on each linux node.

I will try to repeat this situation to catch some logs as it looks like a bug.

I have a similar behaviour. When I add a file to a folder (with rescan) on a device (device A) with paused remote devices, then this file will not be synced when the remote devices were unpaused and connected again. But when I look on the gui of the remote device it shows me a difference between global and local state of this folder. No ignore patterns are used that will prevent synchronizing. When I rename the same file on device A and do a rescan when the remote devices are connected then the file will be synced.

Please open a new topic for your problems. Even if the problem seems similar, they might be different. And the hardest thing about debugging is getting the information needed to understand a particular setup and circumstances under which a problem occurs. It doesn’t help having multiple of those mixed. Of course do absolutely link topics which you consider to be similar/the same to establish context.