Wildcard #include, and Relative #include

syrys · July 30, 2016, 5:39am

Let me first explain the 2 requests im asking for. and then i will give a usecase.

1. Wildcard `#include`. In current implementation, you can use `#include` on your `.stignore` file to attach additional ignore rulesets. However, it seems like each line of `#include` can only include 1 file. instead (or in addition?), i suggest allowing for wildcard `#include`'s, which means that you can targate multiple rulesets from a single line. for example: `#include **/.project-stignore` would targate any files named `.project-stignore` in ever folder apply those rules.

2. Relative #include As we all know, the files attached using #include, the rules of those files are applied as if the file was in the root of the share folder. for example, if i had #include /project1/.project-ignore and this file contains a rule /myfile.txt, then that will exclude a myfile.txt from the root of the share folder. if the rule was myfile.txt instead, then this would ignore any file in any folder named myfile.txt. But if i wanted to targate /project1/myfile.txt, i literally have to spell out exactly that path.

But instead, if the included ruleset could retain context of the folder it was in (you can simply append the folder path dynamically to every rule/pattern within the file, or have a character that you could add to the rules to inject the folder in which the rules were fetched) that would help a lot of people using this for development projects.

Here are 2 methods i suggest: A: in addition to #include, you can have #includeRelative or something similar. This would work the same as #include does, but instead, every line on the ruleset attached using #includeRelative would get the directory appended to the start. ie. #includeRelative /project1/.project-ignore on the .stignore, and the .project-ignore states /myfile.txt, then the rules would see /project1/myfile.txt thus the only file that would be ignored would be just that.

B: introduce a new character to the ignore rules files (similar to *, ? etc) where the character would be replaced by the folder in which the rules file exist. Lets say the character is @. This means: #include /project1/.project-ignore on the .stignore, and the .project-ignore states @/myfile.txt, then the rules would see /project1/myfile.txt thus the only file that would be ignored would be just that.

Use case (for both requests)

Im a developer. I work on lot of web and mobile development projects. Projects get added, removed, renamed reasonably often. I have multiple computers that i do development on, which i would like to have my project folders synced in. my projects are stored in the following way (fail directory tree to text attempt):

/MyProjects
-/CoolProject
-/NotSoCoolProject
---/ActualDevFiles
-/CatPics

I would like to “share” my MyProjects folder (which is the root of all the projects). Most of the projects have build scripts and such, which creates build, bin, dist folders on the fly, which would be horrible for syncthing (because constant syncing while im developing would be too much). Obviously, i cannot add these folders as wildcards since the project might have dependencies (like node_modules which might also have bin/build/dist folders). So them my ignore rules are going to look like this:

/CoolProject/bin
/CoolProject/build
/CoolProject/dist
/NotSoCoolProject/ActualDevFiles/bin
/NotSoCoolProject/ActualDevFiles/build
/NotSoCoolProject/ActualDevFiles/dist

…you see where this is going right?

There are lot of other use cases too. Specially if you can do wildcard #include, then adding a new project means you simply copy a rule file from a similar project into your projects main folder (thats all, see how simple it is?), and renaming a project folder (assuming you already have the rules file in there) means you dont have to do anything at all.

In the above example i only showed a ignore ruleset of 3 items, but in reality a project has a huge amount of files that should be ignored from syncing, being developers you guys should be more than familar. Here is an example of a Seed Project ignore file im dealing with: https://github.com/NathanWalker/angular2-seed-advanced/blob/master/.gitignore. Imagine if it was the Syncthing Project: https://github.com/syncthing/syncthing/blob/master/.gitignore.

Keep in mind, what im asking for should not affect any existing users, its simply extended functionality for advanced users.

If you guys think the feature request is legit, but dont have the ability to do it, but would allow a random developer (me) to try it, i might even be able to take a crack at it. If someone could give me some rundown on how i would run/test/debug and give me a quick overview, i would be happy to take a crack at it at somepoint

Cheers

calmh · July 30, 2016, 6:43am

I’m not super keen on adding this to the current ignore system. It’s already quite complicated and hard to use as is, and adding more complexity on top of it would make it worse. Better to work towards a unified redesign. There are also some performance issues with the current ignore system that would probably get worse with wildcard includes. It would necessitate scanning the entire folder first for ignore files, then again for actual files. So yeah, no, not sure about this one.

This sounds like git with a central server of some kind to me. At least that’s what I do. GitHub with private repos is fairly cheap for an individual, if you lack another well connected server to use.

syrys · July 30, 2016, 9:05pm

Fair enough. Ill leave it here with you guys if you change your mind. Personally i believe a product should have as much advance features as it can possibly offer its users, but not make it too complicated for non-advanced users (thus leaving the advanced features behind a door of sorts). So personally, the ignore system being “hard to use” sounds like a UI/UX problem where the solution isnt to reduce features but to improve how the features are implemented (i dont mean any offence by this).

I had a little look into the ignore system a bit (code wise as well as the docs), and i dont find it hard to use (but im an “advanced user”), but yes, i can see a “noob” would find it hard, but then again they can not use it? I mean, i dont see any other alternative solution (even with cloud hosting or not) having a system with a “easier to use” ignore system. Saying that though, it surprised me how there is no synced ignore file (i have seen couple of requests/threads making good claims and they makes perfect sense to me as i can relate to the use cases), and that the #included file’s rules act as if it was in the root of the share folder (although this is documented, this is not logical for a lot of people).

Saying all that though, what i say is completely invalid if it causes any performance issues. Obviously you shouldn’t consider implementing a feature that greatly hurts performance if not many people want it.

sorry, dont mean any disrespect or anything, great work on the project so far, im just pointing out what i see from purely my perspective.

Most of my project folders are git repo’s, yes. I tend to use bitbucket for personal projects, and client projects might use anything (bitbucket, gitlab, github, even svn) which is out of my control.