File watcher regularly fails to detect some changes in complex folders

I’ve observed this problem especially on my main PC running Windows 10. I’ve got a lot of folders added to Syncthing. When it comes to the vast majority of them, it seems that the file watcher is able to detect all changes as soon as they happen. I can verify this by triggering a manual scan, which then doesn’t detect any changes inside those folders.

However, I’ve got maybe 2 or 3 folders that are quite complex with a lot of “stuff” happening inside them, i.e. subfolders/files are being created, modified, and deleted basically all the time (usually by other software). I’ve noticed that the file watcher regularly seems to fail to notice some of the changes, which I can confirm by running a manual scan, which then detects file changes that happened, e.g. a few hours ago (with the full scan interval set to 1 day instead of the default 1 hour).

I’m just wondering, is this something that can happen with a lot of constant changes taking place inside such folders? Is there some kind of a known “limit” of file events that the OS (i.e. Windows in this case) is able to report at one time? Would enabling watchaggregator in debug logging be useful to find out more about those missed files?

I don’t know the exact specifics, certainly not on Windows, but in general yes there is an event queue with a max size, both on the OS side and on the library side, so it’s entirely possible to miss events when there’s a lot going on. I think that if the queue is exceeded on “our” side of the API boundary we will detect that and trigger a full rescan, but events may well be dropped on the OS side as well if they’re being produced more rapidly than we’re processing them, for whatever reason.

I think it’s something in Syncthing. I’ve been testing this right now with watchaggregator enabled, and the log was filled with

[D4DZU] aggregator.go:334: DEBUG: aggregator/"xxx" (mv9pk-setpw): No old fs events
[D4DZU] aggregator.go:315: DEBUG: aggregator/"xxx" (mv9pk-setpw): Resetting notifyTimer to 10s

for all other folders, however the two folders in question were missing from it. Pausing and resuming them didn’t help either, so I’ve just restarted Syncthing completely, and now the aggregator does include them.

Basically, it seems that file watching just stops working at some point for those folders, and it doesn’t recover until a full Syncthing restart. I’m going to leave the log with watchaggregator enabled for a few hours and then see what the situation looks like. Is there any other debug logging that I should enable here for better understanding of what may be happening in the background?

Edit

I only want to note that I had Syncthing running for ~2 weeks non-stop before restarting it a moment ago, so I don’t really know when exactly the problem began to manifest during that time period. Because of that, I’m assuming that it may take a while, i.e. a few days or more, to encounter it again. At least with watchaggregator turned on, I can see in the log whether the folder is still being watched or not.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.