node specific / introducer based stignore per share

Feature
1

There are some thoughts/ideas I wanted to share and discuss with the synchting community. Parts of them may have been discussed here and there, but - I think - not in the following context.

I am running a kind of top down hierarchy of syncthing nodes by having one node that is added by all others as introducer (as described earlier here). The problem/thing I’d love to simplify is that there are some nodes syncing a whole directory (the fileservers) and nodes that should only sync parts of that directory (laptops, smartphone, … - let’s call them client-nodes). So right now I have stignore files set up on each client-node. Easier to administer would be a central, per peer-node stignore set up on the introducer-node.

I’ll try to explain that with the following use-case: Sharing a directory, i.e. photos, with somebody else.

In the current setup I have to seperate the folder to be shared from the main share - by putting the directory outside of the main directory or seting up a stignore-pattern to exclude the directory from the main share and setting up a new share.

If there would be a per node stignore for a share, I could just leave everything as it is. I would just add the new node to the introducer, share the main share with that new node and set a stignore-pattern on the share for the new node (on the introducer) that excludes everything except for the photos-directory.

Also it would be nice to maintain the stignores of your syncthing network centralized this way. You just need to open the UI of the introducer to see or adopt the ignore patterns of the connected nodes. Additionally it would be more comfortable to add an android node. Instead of fiddling on the touch screen to type in the ignore patterns you could just sit on a PC, open the UI of the introducer, add the android node and set the stignore-patterns for it.

It could also add some security. Right now I can sync parts of the big, allover share to trusted nodes via stignore on the trusted-node or to untrusted nodes only by separating them from the allover share as described above (what could mean to have to go through UIs of several nodes). If the per-node stignore gets pushed from the introducer to all other nodes, it might not be necessary anymore to seperate a directory from the allover-share in order to share it with an untrusted machine.

I hope you can follow my thoughts. I know, everything quite vague. Maybe something somebody else also thought of? Maybe an inspiration for a new feature?

Thanks for reading.

1 Like
2

Ignore patterns are not really fit to restrict access for untrusted nodes, as they are applied on the client side and therefore easy to work around.

Having pre-defined ignore patterns per node could be implemented by substitution in the include directive. Say I put “include .%h.stignore” into each node’s .stignore file and it gets expanded to each node’s local ID, that allows easy per node, synchronized patterns.

Would still need a way to synchronize the generic .stignore file, as discussed elsewhere. Automatically including an ignore file named after the node ID would solve that problem for this case. Just my two cents…

3

This sounds easy, but is really hard to implement, because ignores are applied at scan time, not at “send stuff to the other device” time.

Furthermore, information about all items that are not globally ignored need to be sent to the other side, because items are numbered as we detect changes during the scan, so you can’t just skip item 10 because its per-device ignored, as we have delta indexes.

So we when devices send information to each other, they say “I’ve seen everything upto item number X”, and we send only items higher than X, now lets say you skipped item 10, went past to item 20, and decide per-device unignore item 10, we have no way of discovering that that was never sent.

What you suggest needs pretty much per-device index/scanning, which would be balooning the database into n^2 territory.

I think you either need to break down the problem, have more granular folders that you share selectively, or use something like arigi, which is commercial software for centralised syncthing management.

Also, perhaps you don’t need bidirectional continous sync and there is a better software product for your problem.

4

Basically this is a request for a Syncthing cluster management system. Other than the commercial arigi, I’m not aware of anything. It would be great if there was a FOSS alternative to arigi.