Some thoughts on ignores syncing, includes, and defaults.

So I disagree. The user can pre-create both the ignores and ignores global file. I don’t see how clicking some checkbox is less than having to type something in a file.

The cool thing for me would be adding a new node to an existing network. Today you need to somehow transfer that global ignore file ahead or retype it or whatever, with this feature it would just be checking a box that you want to honor global ignore patterns.

@nekr0z
If I get it correctly you want to designate a file named stignore-global (or similar) as folder specific global ignore patterns, which is synced through regular means. If it exists, apply it, if it doesn’t, just neglect it. Potentially have some control of whether to apply it at all and for editing in the web UI. Right?
If so the behaviour when adding a new device with such global ignores is still not very clean: At the start there is none, so Syncthing starts pulling all files. During that pull it encounters that global ignores file, but it’s just a regular file, so proceed. That could be changed to stop pulling, reread ignore patterns and start again. There is still no guarantee that will happen first. So you could end up with pulling in some files, then immediately ignoring them. So you’d also need something like I mentioned before, that checks whether that file exists on a remote and pulls it in ahead of pulling (and scanning ideally).

As a new user, this seems like a good idea. The ignore system seems a little too static. I recently noticed a behavior that seems problematic but in the other direction. I created a test directory with two JPEG files and two PNG files. I then installed Syncthing on a second machine and synced that directory. all four files came over. Then, on the second machine, I added *.png to the ignore pattern file. The two PNG files remained, so I deleted them manually. So far so good, but then I removed *.png from the ignore file. Syncthing could then “see” the missing PNG files and propagated that to the other system as a delete, resulting in the PNG files disappearing from the first system. That’s probably not a desired behavior in general.

Perhaps it could benefit from being a little more dynamic in both directions. If something is added to ignore, then all memory of those files ever being synced in the first place should be erased, so that unintended deletes don’t get propagated. Likewise, if something is added to include, then an immediate attempt could be made to sync that file if it exists on the other system, which would result in either a transfer or a conflict right away, but not a surprise later on.

1 Like

The user can pre-create both the ignores and ignores global file. I don’t see how clicking some checkbox is less than having to type something in a file.

But what am I typing in that file? It it’s a checkbox I don’t need to keep a copy of my ignore patterns somewhere else so that I can refer to what on earth I’m supposed to be typing/copying into those files. A checkbox would make Syncthing more self contained. As it is, I use Google Drive to remind me what is supposed to go into new Syncthing ignore files. This is a silly dependency to have and ways to improve this would be welcome. If I’m doing it wrong then Syncthing is clearly not as understandable as it could be. I feel a checkbox that I could toggle to grab ignore patterns from elsewhere in the cluster would improve this whole situation by reducing external dependencies, or by making it more understandable. Either outcome is an improvement to my mind.

1 Like

Basically, yes. Initial implementation would have it off by default, with a per-folder config option to enable. It can be enabled by default further down the road, if deemed appropriate.

Yes, that could be a sane idea after all. I haven’t yet looked at that part of the code, but seeing as how we have options for file syncing order (alphabetical vs random) it probably can be done.

I can potentially envision corner cases where the behaviour would still be unclean, with stignore-global itself being ignored by local ignore patterns or accidentaly deleted. Also, a decision should be made how to handle conflicts on stignore-global.

I really like the idea of having a syncing sync-ignore-list.

This is what I fiddled together to have a “global” and a “specific” ignore list.

.stignore:

!.stignore_global
!.stignore_specific

#include .stignore_global
#include .stignore_specific

.stignore_global:

(?d)(?i).DS_Store
(?d)(?i).Spotlight-V100
(?d)(?i).Trashes
(?d)(?i)desktop.ini
(?d)(?i)Thumbs.db

.stignore_specific:

/Test
/iCloud Photos

This concept works out for me but it took me quite a while to arrange.

I support the idea of inviting & helping a user to sync their ignore lists innately, because non-technical users have a hard time understanding and creating those. I know, because I’m one of them!

With additional nice comments and some examples in the .stglobalignore file, the user could get the idea behind it even quicker/better than reading the forum and the docs only.

Four years in a row now I’ve gone to my parents’ place for Christmas and fixed their computers; 4 years I’ve ended up reinstalling the OS for some reason or another, which results in reinstalling syncthing - and 4 years now that I’ve shot myself in the foot by forgettting to go into the advanced settings to type #include .stglobalignore each time I sync a new folder…

I see there’s a bountysource bounty for the “Next Gen Ignores” task which has a bunch of other stuff I don’t care about, and this one feature that I DO care about is a “maybe” :frowning: If there’s some way to me to donate towards this specific feature (“Act as if .stignore contained #include .stglobalignore by default”), I would love to do so <3

Initial implementation would have it off by default, with a per-folder config option to enable

I’m not sure why anyone would want it off by default (If you don’t want your repo to have global ignores, then don’t create a .stglobalignore file?) - but if this opt-in approach DOES get taken, it’d be nice to detect the presence of a .stglobalignore file and ask the user if they would like to use it - otherwise we’re just replacing “I forgot to go into the advanced settings and type #include .stglobalignore into the ignore patterns box” with “I forgot to go into the advanced settings and tick the Use .stglobalignore checkbox”…

1 Like

I guess you symlink this file as it appears to be in the folder root? And don’t have any other things in the ignore file as this is the only thing you need to remember to type in? So you could just symlink .stignore instead?

What about a default ignore file for new folders? Can exist in the config dir and also be edited in the UI. You can still forget to create that, but at least you only need to do it once.

1 Like

Sounds workable, though I prefer a config field over a new file.

I kinda expected dislike for a file based solution :slight_smile:

The advantage for me is i can just copy and edit a plain text file - simpler than interacting with a UI or xml. Anyway it doesn’t actually matter.

Yeah and it has its own format (stignores), which is nasty to embed in another format (xml). <ignores><![CDATA[asdf\n!gronk]]></ignores> vs <ignores><pattern>asdf</pattern><pattern exclude="true">gronk</pattern></ignores>? :slight_smile:

I’m not sure what would be symlinked to what here? I could symlink .stignore to .stglobalignore instead of using #include, but I’m not sure how that relates to the issue of “each time I set up a new PC, I need to remember to go and twiddle some extra settings somewhere else to enable my standard ignore list”

And don’t have any other things in the ignore file as this is the only thing you need to remember to type in?

That is correct - I’ve not yet found a use-case for local ignores, but pretty much all of my folders have global ignores in them (Code folder ignores .cache, Downloads folder ignores *.partial, Photos folder ignores Thumbs.db and .DS_Store, etc)

#include'ing a synced file is the best way I’ve found of making that happen - The only pain-point with this approach is that I consistently forget to enable it :frowning:

(With the consequence that junk which is supposed to be ignored gets into my metadata index - so I go back and add the ignores, delete the files… and then the ghosts of these files haunt me because of file names lingering forever - #5 by shish and my repos are marked as “out of sync” until I nuke the whole setup and configure from scratch with a new folder ID)

Your example is #include .stglobalignore. That presumes the existence of the .stglobalignore file, which is in the folder root. Either you then copy this into each folder from somewhere, or you create a symlink pointing to it. Regardless of which it is, it seems you could do this directly for .stignore instead, since you don’t use any folder-local ignore patterns.

Either you then copy this into each folder from somewhere, or you create a symlink pointing to it

Eh? .stglobalignore is part of the folder being sync’ed, so syncthing copies it from my fully-configured NAS to my freshly-installed desktop

it seems you could do this directly for .stignore instead

I wish I could! That would make my use-case so much simpler, but: https://github.com/syncthing/syncthing/blob/486230768e5bea25fc758c119f9b9a9c6d529af3/lib/fs/filesystem.go#L207

Ah, ok, that makes sense. However it also results in the usual chicken and egg problem of not having the ignores available until after the folder has synced. I guess this is fine in your case if the ignores are always symmetrical. Regardless, it sounds like imsodins suggestion would be helpful.

1 Like

Just set up a new laptop; synced my files from my NAS; made sure everything was sync’ed correctly, all looks good. A week later, I notice there’s some weirdness where files that are in my ignore list aren’t being ignored… guess what I forgot to enable AGAIN? T_T

For everyone in this thread, who ‘forget’ to add the .stignore file on re-configuration and reconnect: Do sync this .stignore manually! Best to copy/rename it to stignore (remove the dot) so you can see it.

best idea, I’m always having a stignore.zip in the folder root as a reminder and apply it on all nodes manually.

I find these discussions about wanting to automate something just because you are forgetful yourself, a bit excessive.

The #include function alone makes things easier to be able to transmit the ignores via the peers that are also to be transmitted. Sure, a standard entry with e.g. “#include .stglobalignore” would be helpful in order to at least have a cross-peer design. Further entries with “#include .something” are also possible.

Nevertheless, a few tools can also be used to build a script that copies a basic stignore from a central directory to wherever it is wanted, at least within one’s own reach, whether locally or externally. And then circulating a .stglobalignore for each peer, if necessary individually, shouldn’t be not so difficult anymore.

What I mean by that, dealing with such elements is basically a conscious process that also entails responsibility, as you have to decide separately for each peer which profile should ultimately be considered ignore patterns.

1 Like