Local Vs Global State updates (senders)

Hi there,

I had an issue with a global state being incorrect on an encrypted receiving (remote) device - it was using a prior global. I had to re-setup the sending folder on a local device, and with extra ignores, and so the global state was suppose to be reduced, but the encrypted device still had the old global state. What was strange was the encrypted receiver showed ‘Up to Date’ even after a re-sync, pause-resume, and even restarting Syncthing. I didn’t notice this until I tried to add another remote receive encrypted device which refused to sync correctly (even after overriding, which did nothing, restarting, etc.), since it appeared to have the wrong global state as well (it was from a fresh install). Both encrypts were initially sharing (due to connection speed differences to the sender, I wanted to ensure both synced between themselves even if the sender was offline). I never got anything about overriding on the sender. Similar to this: Encrypted peer sync stuck, global state mismatches

Anyhow, I wanted to know if a receive encrypted device ever updates its global state to the sender local (or even global) state? If so, how does this work? If not, is there a way to force a sender’s local state and override the global on all devices? Does a receiver device (in my case encrypted) ever check its global state with non-receiver devices?

Thanks

It sounds like your problem might actually be related to your ignore patterns. If a device scans its local state and finds 1000 files, they get announced to the peers and both sides end up with a shared global state of 1000 files in their database (and on disk). When you later add an ignore pattern that reduces the first device’s local state to 500 files, it also starts ignoring any announcements from other devices about the ignored files. So its global state will also drop to 500.

But ignoring is purely a local thing for each device. The ignore patterns are not applied to other devices automatically. In case of receive-encrypted folders, they probably cannot even apply. That’s why the remote end still knows about its global state of 1000 files.

To get the encrypted remote device to forget about and delete the files which should no longer sync, you probably need to set up the folder from scratch. Start on the device where you want the ignore patterns, add the folder with a new ID and this time, specify the patterns immediately. Then share it with the encrypted device.

Alternatively, you could add another device which receives the folder from the encrypted device, but has the password to decrypt it. Then on that device, you delete the files you don’t want synced. It will propagate to the encrypted device as a deletion. But the original source device will not actually lose the files because it has a matching ignore pattern and Syncthing won’t touch or delete ignored files.

Hope this helps to understand how your situation came to be. I don’t use encrypted folders myself, so not 100 % sure whether it will work this way.

Thank you for your response.

To me the behaviour of receive-only devices in relation to the global state seemed strange and I’m not sure if it is a once-off, design gap, or if intended for other reasons.

It seems receive-only devices should be slaves to the global state since they don’t actively contribute any changes, so I’m a little puzzled as to why they might maintain their own outdated global state, even if it was obtained from another receive-only device for the same folder.

Without an up-to-date global state (synchronized with the senders) on the Receive Only device folder, the system looks like it won’t even recognise any changes and so that may have been the reason it didn’t prompt me to override local changes on the receiving device folder.

While rebuilding the folder from scratch, as you suggested, resolves the immediate issue, it becomes impractical with large folders, especially when only a small portion of files have been ignored or removed. Additionally, the fact that the global state of a receive-only device doesn’t validate its own global state exacerbates the problem, especially with folders containing extensive data, much of which remains unchanged.

In essence, the current issue here in how the global state updates are handled on receiving devices does not represent an accurate current state of the folder. This is extremely concerning where large amounts of data are stored and manual intervention becomes cumbersome and time-consuming.

In essence, the issue here is that Syncthing is made to synchronize folders, making them look exactly the same on every involved device. Ignoring things on one device only, and not consistently on all involved devices, goes against that goal and will always lead to differing states between devices sooner or later, maybe even “Out of Sync” messages.

The possible design gap here is that for untrusted devices (receive-encrypted folder), the file names are not known and therefore cannot be matched by ignore patterns. Thus the only way to make them forget about files ignored somewhere else is to get an incoming deletion change for them. That is inherent to the fact that the file names are also encrypted. And the other design choice is that Syncthing will not propagate locally ignored files to be deleted on remote devices, since that has a large potential for unexpected data loss.

That’s all assuming that in fact the global states were communicated correctly. We’ve recently seen some reports about mismatching global states, so there could very well be a bug involved that leads to incorrect distribution of the index data. Just that nobody has identified it yet and found the source where it needs fixing.