FS Watcher Status


(Simon) #21

Yes, the GUI and activation by default is yet to be released (in RC). The documentation is in place: https://docs.syncthing.net/users/syncing.html#scanning and https://docs.syncthing.net/users/config.html#folder-element


#22

Hello:

Thanks for the work on the FS watcher - really appreciate this inclusion.

However, I’m having an issue with the watcher failing to start up on large folders.

I’m running this on a Synology NAS, and have consequently raised the inotify max_user_watches as recommended (it defaults to 8192). So far I have this raised to 2097152 (confirmed by running cat /proc/sys/fs/inotify/max_user_watches ) - but I’m still running into failures on a couple of folders.

It does appear to be working on a folder with 25288 files, 646GB in size. It is not working on a folder of 131588 files, 238GB in size. Or one of 3908571 files, 21939GB in size.

Any ideas if this is to be expected for the quantity of files being watched, or if there are Bad Things to be expected for raising the limit even further? I tried to find out the limits of inotify elsewhere - but it doesn’t seem to be at all straightforward!

Many thanks,

Pants.


#23

Hello:

Still having issues with the FS watcher not starting up on large folders.

I tried doubling the limit even further (to ~4 million) - but that seemed to kill the watcher on even small folders.

I’ve reduced it back to the recommended value (1048576) - and the watcher has come back for my smaller folders. However, I’m still seeing Failed to setup, retrying on the larger folders.

I’m monitoring the logs in the watchaggregator category - but there doesn’t seem to be anything coming through in there - and I can’t see any other debug categories that might show problems in this area.

I’ve also been digging around for information on inotify - and on this page, it recommends running tail -f /var/log/dmesg to see if you’ve run out of fs watchers. I’ve done this - and it looks like I haven’t run out.

Unfortunately, as I’m running this on a NAS, I don’t have a full compliment of OS commands to try.

Any ideas where to look at next?

Thanks,

Pants.


(Simon) #24

The max_user_watches setting is definitely not the issue, neither is the size alone problematic (works fine here).

So far you haven’t posted the actual error message that is displayed as a notice/warning on the top, as a tooltip on the watcher status and in the logs - that’s what should explain the cause.


#25

Hi Simon:

Thanks for chiming in.

I can’t find anything in logs so far (any idea which debug log I should enable?) - but the tooltip on the watcher status says:

Periodic scanning at given interval and failed setting up watching for changes, retrying every 1m: not started

Thanks,

Pants.


(Simon) #26

So it hasn’t even tried to setup the watcher yet, i.e. the folder code is busy doing other stuff (initial scan?). UI screenshot with the affected folder expanded and/or logs could help shed some light.


#27

Hi Simon:

Ah - I’ve just seen that a couple of the folders are specifically referencing the inotify limits:

Periodic scanning at given interval and failed setting up watching for changes, retrying every 1m: failed to setup inotify handler. Please increase inotify limits, see blah blah blah

Sorry - I hadn’t noticed that those messages were specific to the inotify limit - I shall investigate further and report back!

Thanks,

Pants.