Problem with Syncthing and Icloud

I use Syncthing and Icloud to synchronize obsidian knowledge base files On Windows computers these are

  1. SyncTrayzor 1.1.29
  2. Syncthing 1.27.5 I have one computer as a server and all other computers are syncthing from it.

On my macbook I have syncthing-macos-universal-v1.23.6 Also the folder on the macbook is synchronized with Icloud.

Periodically I lose files, usually they are images or pdf and I find them in the .stversions folder with a changed name. the file in the regular folder was photo1.png, and in the .stversions folder it became photo1~20240407-090830.png. And I have to copy it to the right folder and rename it by removing ~20240407-090830 Why is this happening ? who deleted my files ? and can it be fixed ?

You are deleting your file from a device, And since you are on the version control so a copy of file were saved in .stversion folder.

But I certainly didn’t delete anything

Using Syncthing with iCloud, it’s stacking a file sync app on top of another file sync app…

Desktop apps for iCloud, Dropbox, Box Drive, OneDrive, Google Drive and other cloud storage usually only locally cache a subset of the most recently used files. Then in concert with the operating system, makes it appear as though all of a user’s files are available via the local file browser even though most of the files are still in the cloud.

As a user touches a file within the cloud storage, the desktop app downloads/uploads files on demand. Then as a locally cached files age out due to non-use, they’re purged from the local cache to free up storage space and replaced with placeholders which may be only a few bytes in size but otherwise appears perfectly normal in a file browser, but to Syncthing there’s been a change.

For more about content caching in macOS: https://support.apple.com/guide/mac-help/change-content-caching-settings-on-mac-mchleaf1e61d/mac

To see if iCloud is the culprit, temporarily disable it for a day or two to see if the deleted file problem in Syncthing disappears.

2 Likes

So we can assume that this happens because icloud removes the file from quick access to leave it only in the cloud, and Syncthing picks up these changes and removes it everywhere?

I would certainly check, but this problem happens sometimes every two months, sometimes every half a year. Is it possible to change icloud behavior ? or maybe there is some other cloud service without this problem ?

Yup, and not just iCloud, but all of the popular desktop cloud storage services that use a selective file sync design.

There’s been quite a bit of interesting in the same feature for Syncthing: https://forum.syncthing.net/t/selective-sync-feature-most-important/16394/

I’m not aware of an option to disable the on-demand sync behavior in iCloud. Increasing the local cache size could be used as a workaround, but it’s not really a solution because the risk is still present.

As for using another cloud storage service, it depends on what you’re using iCloud for. Is it an off-site backup for your Syncthing folder? Storage requirements?

About the other cloud service, I thought it’s not suitable for me, I can’t give up icloud for certain reasons, so I’d like to set up Syncthing and icloud. Is it possible to customize Syncthing so that it does not delete certain types of files, but only changes them if there are any changes in them?

There’s a Syncthing option to ignore deletes altogether, but not to apply to only certain types of files.

On a related note, this isn’t unique to Syncthing, backup software has the same issue. It’s neither a problem with Syncthing and backup software, or a problem with iCloud and other cloud storage like it that use on-demand selective file sync. Other Syncthing users on this forum have tried to combine Syncthing with Google Drive, Dropbox, OneDrive and other cloud storage services similar to iCloud with the same stumbling block. The two types of file sync applications just aren’t intended to be used together on the same storage space.

One possible workaround is to move your Syncthing folder(s) outside of the iCloud folder path. Let Syncthing do it’s thing, and use a different tool to mirror those files to/from the iCloud folder path. Since iCloud keeps most files in the cloud, it won’t have a meaningful impact on disk space use.

Unfortunately, without a clearer picture of how you use iCloud I can’t offer any other suggestions.