Hi all! I’m moving from Resilio Sync to Syncthing primarily due to the much more robust .stignore features. I want to sync my files between two Macbooks and a Synology NAS (for backup). The most convenient for me is to simply sync one folder – my user home directory. This allows me to sync stuff like ~/.bashrc and ~/.ssh. Since there’s a lot of stuff under the home directory, I’ve set up my .stignore file to ignore everything but a few specific root files and directories:
// Force syncing specific files & folders
!/.bashrc
!/.ssh
!/Library/Services
!/Photos
!/Tools
// Ignore everything else
**
This setup seems to work – easy and flexible. While it syncs only a part of my home directory, the reported global state is
163,231 files
22,434 directories
~303 GB
I’m not sure this is the right way to go, however. Here’s why:
Scanning takes a long time: around 20 min. Is the scanning process smart enough, in the sense that it will only enter directories that are actually included by .stignore or will it brute-force check every single file in the entire home dir and match it against .stignore? If it’s the latter, this can become very inefficient if I sync only a small fraction of all files in the home dir.
File system watching efficiency: Does syncthing simply subscribe to the root folder or is it being smarter and only subscribes to files/dirs included by the patterns in .stignore? I presume it’s the former, which could lead to unnecessary syncthing CPU usage when unsynched files are modified?
Is there any other efficient way to sync stuff like .bashrc in my home folder without making these symlinks to files/folders that reside elsewhere and are synched?
Does it matter whether the last, ignore-all pattern in my .stignore file is * or **?
Syncthing tries to optimize by not scanning ignored directories. However, when there are excludes (!...) it doesn’t do this - it might run into excluded files, that should not be ignored, deeper in the tree. In your case this won’t happen because your excludes are slash-anchored, but Syncthing doesn’t optimize for this. So yes, it scans everything in your setup.
Question 2 I don’t know the answer to.
I guess you could !/.bashrc. In my case I do the symlink thing because I have my dotfiles in a git repo, and I think that’s quite painless too. (I have a script in that same repo that creates the symlinks for all the files in the repo, adding the dot at the front.)
A single star at the end should be fine, they’re equivalent in this case.
Thanks for the quick reply, Jakob! I see what you’re saying about the scan optimization and how in my case syncthing will index everything in my home directory recursively, due to the presence of include patterns. Unfortunate but can’t blame syncthing for not optimizing for my special case.
So I guess a more computationally efficient alternative would be to explicitly ignore every file and directory in my home dir that I don’t want synched. Which is doable, of course, though apps are (annoyingly) free to create stuff in the home dir. I guess I could have a script to re-generate the .stignore on occasion, though this solution seems a little contrived
Question though. I read somewhere that .stignore is translated to regular expressions internally. I wonder if I could convert my .stignore to a regex that ignores everything that doesn’t match my folders. Something like ^(/.bashrc), which would hopefully still trigger the optimizations you were referring to.
Oh, one more thing. You said the folder scan optimization is disabled when .stignore has include patterns, and that this optimization should still be easily possible when all include patterns are root-anchored. Can you please point me to the code that makes these decisions? Maybe I can come up with a patch to enable this special-case optimization.
