# \[Linux\] Are periodic full rescans really needed when FS watcher is enabled?
**URL:** https://forum.syncthing.net/t/linux-are-periodic-full-rescans-really-needed-when-fs-watcher-is-enabled/12548
**Category:** Development
**Created:** [December 9, 2018, 2:47pm UTC](https://forum.syncthing.net/t/linux-are-periodic-full-rescans-really-needed-when-fs-watcher-is-enabled/12548 "2018-12-09T14:47:19Z")
**Posts on this page:** 4
**Page:** 2
### Author:  [@desbma](https://forum.syncthing.net/u/desbma)
#### Post date: [December 15, 2018, 9:16pm UTC](https://forum.syncthing.net/t/linux-are-periodic-full-rescans-really-needed-when-fs-watcher-is-enabled/12548/21 "2018-12-15T21:16:36Z")
Indeed, I can reproduce this on any filesystem with:
```
mkdir /tmp/test
cd /tmp/test
# create 2 directories
mkdir dir1 dir2
# create a dummy file
touch dir1/file
# create hard link to file in second directory
ln dir1/file dir2/file
# watch first directory
inotifywait -m dir1 &
# modify file from second directory
echo test >> dir2/file
# no event is generated...
```
For the curious, the rationale for this (intentional) behavior is described here: [https://lwn.net/Articles/605128/](https://lwn.net/Articles/605128/)
---
### Author:  [@desbma](https://forum.syncthing.net/u/desbma)
#### Post date: [January 8, 2019, 12:51pm UTC](https://forum.syncthing.net/t/linux-are-periodic-full-rescans-really-needed-when-fs-watcher-is-enabled/12548/22 "2019-01-08T12:51:57Z")
I have done a quick read of the Syncthing code around inotify, and here is what I found:
- At some place the code seem to detect and properly handle internal (go channel) overflow of events, and act accordingly by triggering a full rescan, ie: [https://github.com/syncthing/syncthing/blob/1e71b0093626a00867725813bebc733e967f5353/lib/fs/basicfs\_watch.go#L77-L78](https://github.com/syncthing/syncthing/blob/1e71b0093626a00867725813bebc733e967f5353/lib/fs/basicfs_watch.go#L77-L78) However in at least one other place it seems to be ignored: [https://github.com/syncthing/syncthing/blob/1e71b0093626a00867725813bebc733e967f5353/vendor/github.com/syncthing/notify/watchpoint.go#L90-L91](https://github.com/syncthing/syncthing/blob/1e71b0093626a00867725813bebc733e967f5353/vendor/github.com/syncthing/notify/watchpoint.go#L90-L91)
- The native inotify overflow events (IN\_Q\_OVERFLOW) are silently ignored: [https://github.com/syncthing/syncthing/blob/1e71b0093626a00867725813bebc733e967f5353/vendor/github.com/syncthing/notify/watcher\_inotify.go#L272-L274](https://github.com/syncthing/syncthing/blob/1e71b0093626a00867725813bebc733e967f5353/vendor/github.com/syncthing/notify/watcher_inotify.go#L272-L274) So the event is not propagated and a full rescan can not be triggered.
I am not familiar with the Go language or the Syncthing code base, so I may have wrongly interpreted the code.
Do you confirm my interpretation of the code, and are these issues fixable?
---
### Author:  [@imsodin](https://forum.syncthing.net/u/imsodin)
#### Post date: [January 8, 2019, 1:05pm UTC](https://forum.syncthing.net/t/linux-are-periodic-full-rescans-really-needed-when-fs-watcher-is-enabled/12548/23 "2019-01-08T13:05:30Z")
> [@desbma](#):
>
> Do you confirm my interpretation of the code, and are these issues fixable?
More or less: You did not do the distinction between actual syncthing code and notify code. The latter is a separate library, that we maintain a fork for which is identical with the original [GitHub - rjeczalik/notify: File system event notification library on steroids.](https://github.com/rjeczalik/notify) except for a few modifications we need. The notify library does send events without blocking, i.e. drops them - that’s a design choice. That’s why we use a buffered channel, so this never happens, unless this channel is full, which we detect (as you found out correctly). So that part is fine.
Inotify overflow events: Looks they are dropped, maybe that could be fixed. It’s in the notify library too, so you’d have to ask there, but I guess it’s not an issue because they send events without blocking, i.e. fast, so overflow shouldn’t be an issue. Just thinking out load here, I don’t know how inotify actually works.
---
### Author:  [@desbma](https://forum.syncthing.net/u/desbma)
#### Post date: [January 8, 2019, 1:38pm UTC](https://forum.syncthing.net/t/linux-are-periodic-full-rescans-really-needed-when-fs-watcher-is-enabled/12548/24 "2019-01-08T13:38:10Z")
Yeah I saw [GitHub - syncthing/notify: File system event notification library on steroids.](https://github.com/syncthing/notify) but got confused since what is in [https://github.com/syncthing/syncthing/tree/master/vendor/github.com/syncthing/notify](https://github.com/syncthing/syncthing/tree/master/vendor/github.com/syncthing/notify) is not a submodule pointing to this. Since you say it is the same, I suppose this is just a copy of the repository.
> [@imsodin](#):
>
> I guess it’s not an issue because they send events without blocking, i.e. fast, so overflow shouldn’t be an issue. Just thinking out load here, I don’t know how inotify actually works.
At least in theory it should be possible that the kernel pushes lots of events in the inotify queue, but the user space Syncthing process does not get scheduled often enough to consume the events (because the CPU is overloaded or because the process priority is very low) until the queue is full.
[Previous page](https://forum.syncthing.net/t/linux-are-periodic-full-rescans-really-needed-when-fs-watcher-is-enabled/12548.md?page=1)