I’ve happily been using ST for some time now, but I’ve recently started getting this warning:
2023-03-24 06:40:25: Filesystem watching (kqueue) is enabled on "Folder1" (XXXX) with a lot of files/directories, and that requires a lot of resources and might slow down your system significantly
2023-03-24 06:40:25: Filesystem watching (kqueue) is enabled on "Folder2" (XXXX) with a lot of files/directories, and that requires a lot of resources and might slow down your system significantly
2023-03-24 06:40:25: Filesystem watching (kqueue) is enabled on "Folder3" (XXXX) with a lot of files/directories, and that requires a lot of resources and might slow down your system significantly
2023-03-24 06:40:25: Filesystem watching (kqueue) is enabled on "Folder4" (XXXX) with a lot of files/directories, and that requires a lot of resources and might slow down your system significantly
2023-03-24 06:40:25: Filesystem watching (kqueue) is enabled on "Folder5" (XXXX) with a lot of files/directories, and that requires a lot of resources and might slow down your system significantly
Some folders do have over 30,000 files so I can understand that, however others only have 5 small files in there (txt documents), so I’m wondering why this warning is appearing?
The machine is a low power 2C/4T CPU, so perhaps that is the issue, but I don’t see the CPU in any particular distress.
There’s around 45,000 in total. I would understand if the warning related to just ‘folder 1’ or ‘folder 2’, but it refers to folders that only have 4-5 files in them. Very much welcome your thoughts.
According to Syncthing’s source code, the threshold for the “Filesystem watching (kqueue) […]” warning message you’ve been seeing is when there are more than 10,000 files.
It’s been some time since I’ve actively used FreeBSD (and don’t do much software development on it), but I know that FreeBSD’s kqueue is a general purpose event notifier that’s also handling other system events in addition to any files being monitored for Syncthing. I’ve seen other software with the same 10K file threshold when working with kqueue, so it’s not unique to Syncthing. It might be related to kqueue’s original design goal of supporting up to 10,000 file descriptors at once.
In comparison, Linux’s inotify kernel subsystem is specifically designed just for filesystem monitoring. I’ve used it with backup/sync software involving a million or so files without issues.
Once I see enough stability I do aim to move to TrueNAS Scale, but in the meantime do you have any suggestions? Should I disable watching or is this more of a concern with the file numbers mentioned in another ST forum thread (300k+)? I doubt that my numbers will get to that, as I tend to archive closed projects to my server, it’s only used for active projects. I doubt the numbers would get any higher than 100k.
There is no problem unless you’re experiencing a problem. The warning is because watching too many files can cause the kernel to run out of file descriptors which affects the entire system.