Syncthing not syncing files in large directory (Windows)

[lazovskiy]

Hi there!

I’ve encountered an issue related to syncthing while using ksync. ksync is the tool that syncs local directory with the container in kubernetes cluster. It uses syncthing as the workhorse backend.

So we can assume that actual schema is: windows machine (C:/code) <-> linux machine (/var/www/code)

Initial sync works well, but there is an issue with file changes watcher.

When I create or change files on the linux box everything works as expected - all changes synchronized as soon as they were made. But when I try to do the same on windows strange things happen. When I create or delete directory - it syncs instanteniously, but when I create or modify file - no changes are detected and thus no synchronization is performed. File changes are synced only when full synchronization time comes. I’ve enabled debug logging along with file operations and filesystem watcher and it indirectly confirms my assumption: watcher reacts only to directories events.

I have about 18K files in various nested subdirs in this malfunctioning setup. When I tried to sync empty dirs - everything works as expected - file changes are detected from the both sides.

So I suppose, there is some limit in windows that prevents syncthing from functioning correct.

In linux we have some tunables like a fs.inotify.max_user_watches. Do we have something like that on windows?

Any suggestions?

Thanks!

[imsodin (Simon)]

@lazovskiy

I’ve encountered an issue related to syncthing while using ksync. ksync is the tool that syncs local directory with the container in kubernetes cluster. It uses syncthing as the workhorse backend.

Wait what?! Never heard of that and they seem to be integrating with Syncthings code rather tightly (well, actually importing some of Syncthings packages, not just spawning the process and accessing the web UI - I don’t know any more than that). Sounds cool and like a potential beneficiary of https://github.com/syncthing/syncthing/pull/5805. Anyway thanks for bringing this up, even if it might have been accidentally

@lazovskiy

So I suppose, there is some limit in windows that prevents syncthing from functioning correct.

In linux we have some tunables like a fs.inotify.max_user_watches. Do we have something like that on windows?

Not that I am aware: The nice thing on windows is that the system library used for notifications supports recursive watches - so there is no equivalent restriction to inotify.max_user_watches.

What is the root path of your folder? If it is a drive letter (e.g. C:) the problem might be Recursive events not triggering from root drive on Windows · Issue #148 · rjeczalik/notify · GitHub