now this apear in log, I assume auto rescan happen
syncthing[1036]: [72MO7] 2018/04/05 21:11:44.499995 walk.go:316: DEBUG: to hash: UK_ribon_projects/17-1c3fb04-fglass/float.png File{Name:"UK_ribon_projects/17-1c3fb04-fglass/ ........
syncthing[1036]: [72MO7] 2018/04/05 21:11:44.666353 walk.go:186: DEBUG: real to hash: UK_ribon_projects/17-1c3fb04-fglass/float.png
and file is transfered
my directory for sync is /home/jano/FORMING/Projects
but it is symlink /home/jano/FORMING -> /mnt/disk_hdd/FORMING
what do you think, can be a problem? because I see this in log and looks bit strange
syncthing[1036]: [72MO7] 2018/04/05 21:26:29.165149 walk.go:217: DEBUG: error: mnt <nil> lstat /home/jano/FORMING/Projects/mnt: no such file or directoryYeah, I think there is an issue there. There was one issue with symlinked folders plus the FS watcher fixed recently, but this may be another one. Basically, as I understand it, the filesystem watcher reports “real” paths, which is not what the rest of Syncthing expects when the folder is behind a symlink.
Also, symlinks suck.
This sounds plausible.
We don’t walk past symlinks, but symlinked root should be ok. At which point is the symlink here, what’s the folder path, etc?
The partial truncated/modified/selectively picked logs make it very hard to understand.
I’d be nice if you would write up a reproducer and opened a bug report on github with exact isolated steps, as in:
Also, if you can’ try the latest version/master builds to see if it was recently fixed.
well, some symlink was symlink to another symlink and some was rooted and some was not.
sorry, I was quick, trying to find problem fast.
I am quite busy these days and I know that symlink can be problem now. It is not problem to point syncthing to real directory instead symlink, so I did fix that and sync is working now.
Thank you for help
Everyone is busy, yet we still find time to provide support on the forum and find time to work on free software and not get paid for it. Be a team player.
You discovered a bug and regardless that you worked around it in can potentially break things for other people, please write it up on how we can reproduce it so we could fix it.
Until any hints to the contrary, I believe this is the same as https://github.com/syncthing/syncthing/issues/4867 and is already fixed in master by https://github.com/syncthing/syncthing/pull/4846
I just saw that there seems to be a new feature in the latest version.
In the folder it shows me an eye symbol. Is this the “watcher”? Is it documented?
What are the benefits and how can I use it?
I am using a NAS and it would be great if I could make the rescan intervals large to spare some energy 
I think the new thing is still in the release candidate.
Yes, the GUI and activation by default is yet to be released (in RC). The documentation is in place: https://docs.syncthing.net/users/syncing.html#scanning and https://docs.syncthing.net/users/config.html#folder-element
Hello:
Thanks for the work on the FS watcher - really appreciate this inclusion.
However, I’m having an issue with the watcher failing to start up on large folders.
I’m running this on a Synology NAS, and have consequently raised the inotify max_user_watches as recommended (it defaults to 8192). So far I have this raised to 2097152 (confirmed by running cat /proc/sys/fs/inotify/max_user_watches ) - but I’m still running into failures on a couple of folders.
It does appear to be working on a folder with 25288 files, 646GB in size. It is not working on a folder of 131588 files, 238GB in size. Or one of 3908571 files, 21939GB in size.
Any ideas if this is to be expected for the quantity of files being watched, or if there are Bad Things to be expected for raising the limit even further? I tried to find out the limits of inotify elsewhere - but it doesn’t seem to be at all straightforward!
Many thanks,
Pants.
Hello:
Still having issues with the FS watcher not starting up on large folders.
I tried doubling the limit even further (to ~4 million) - but that seemed to kill the watcher on even small folders.
I’ve reduced it back to the recommended value (1048576) - and the watcher has come back for my smaller folders. However, I’m still seeing Failed to setup, retrying on the larger folders.
I’m monitoring the logs in the watchaggregator category - but there doesn’t seem to be anything coming through in there - and I can’t see any other debug categories that might show problems in this area.
I’ve also been digging around for information on inotify - and on this page, it recommends running tail -f /var/log/dmesg to see if you’ve run out of fs watchers. I’ve done this - and it looks like I haven’t run out.
Unfortunately, as I’m running this on a NAS, I don’t have a full compliment of OS commands to try.
Any ideas where to look at next?
Thanks,
Pants.
The max_user_watches setting is definitely not the issue, neither is the size alone problematic (works fine here).
So far you haven’t posted the actual error message that is displayed as a notice/warning on the top, as a tooltip on the watcher status and in the logs - that’s what should explain the cause.
Hi Simon:
Thanks for chiming in.
I can’t find anything in logs so far (any idea which debug log I should enable?) - but the tooltip on the watcher status says:
Periodic scanning at given interval and failed setting up watching for changes, retrying every 1m: not started
Thanks,
Pants.
So it hasn’t even tried to setup the watcher yet, i.e. the folder code is busy doing other stuff (initial scan?). UI screenshot with the affected folder expanded and/or logs could help shed some light.
Hi Simon:
Ah - I’ve just seen that a couple of the folders are specifically referencing the inotify limits:
Periodic scanning at given interval and failed setting up watching for changes, retrying every 1m: failed to setup inotify handler. Please increase inotify limits, see blah blah blah
Sorry - I hadn’t noticed that those messages were specific to the inotify limit - I shall investigate further and report back!
Thanks,
Pants.
So it hasn’t even tried to setup the watcher yet, i.e. the folder code is busy doing other stuff (initial scan?).
Here is a quick example on my machine:
Folder ID f67cp-nrcdd
Folder Path /media/syncthing/mik
Global State 1 0 ~22.84 KiB
Local State 1 0 ~22.84 KiB
Rescans 1h Failed to setup, retrying
Shared With Miks-machine.local
Last Scan 2018-07-09 11:58:17
Not a pretty GUI dump, but you get the idea. A single file in the folder, yet the fswatcher isn’t running. I’ve turned on the debug option watchaggregator and I’m seeing events on other folders, but nothing about that folder.
Then, magically, it started working, but the ‘online’ logs don’t go back far enough for me to see what happened, and nothing has appeared in the syslog so I can’t pull it from there either. I’d guess over an hour had passed before it managed to start.
Sadly this doesn’t help in anyway to understand the issue. Syncthing logs to stdout, you should check if that is preserved somewhere (systemd et all).
I’ve got everything thrown into rsyslog, but there isn’t anything useful in there.
e.g.
$ grep f67cp-nrcdd *syncthing
Jul 9 11:35:49 syncthing syncthing[4952]: [ER2BS] INFO: Ready to synchronize "mik-films" (f67cp-nrcdd) (readwrite)
Jul 9 11:35:50 syncthing syncthing[4952]: [ER2BS] INFO: Completed initial scan of readwrite folder "mik-films" (f67cp-nrcdd)
Jul 9 11:51:40 syncthing syncthing[4952]: [ER2BS] 2018/07/09 11:51:40.094779 aggregator.go:158: DEBUG: aggregator/"mik-films" (f67cp-nrcdd): Stopped
Jul 9 11:51:43 syncthing syncthing[4994]: [ER2BS] INFO: Ready to synchronize "mik-films" (f67cp-nrcdd) (readwrite)
Jul 9 11:51:43 syncthing syncthing[4994]: [ER2BS] INFO: Completed initial scan of readwrite folder "mik-films" (f67cp-nrcdd)
$
Similarly, if I grep for mik-films, same list of entries. If there is a particular pattern you think I should be looking for, happy to look.
Nothing logged at all for ‘error’, ‘warning’ or similar.