Will Syncthing support Selective Sync like Dropbox does, which lets you exclude certain subfolders from syncing?
For example, you might have a small laptop and a large backup server. You would sometimes want to remove content from the laptop to free up space, but not delete it from the server. The laptop user is essentially checking out a rotating subset of the server’s content depending on what they’re working on.
It’s possible to kind of do this right now using only top-level directories, but that’s really not ideal. For example, currently if you want to be able to stop syncing some of the artist folders from your music collection you’d have to set up a top-level sync for each of the thousands of folders there.
Doing this on the receiver side (i.e. electing to simply ignore a tree of files) is fairly straight forward, the GUI is probably the most work. I would expect it to work on a directory level (i.e. one can exclude a directory, but not individual files), to start with at least.
Hi is there any progress on this? If I understand correctly ignore patters are the opposite - they define which files will not be sent out to other nodes. But I need to ignore on my device (laptop, phone) to exclude which subfolders not to download. Exactly as it works in Dropbox and their “selective sync”.
Will it work like this? The wiki page[1] says “not be synchronized to other nodes” which I understand only as outcoming changes from local machine, but not preventing to download files from other machines. If it can work like this, than OK, it should be enough.
And another thing, I am not sure about the syntax? How to say ignore (dont download) everything by default, only download folders and subfolders that I specify? Sorry, I am not geeky enough. Are there some more examples for this? Would be great to have some “checkable” treeview GUI like dropbox has…
Like the examples in the link you mention yourself, first set the folders you DO want to sync with the ! marker and then ignore everything. I do it the following on my android in the .stignore:
// DON't ignore the following dirs (i.e. DO sync them):
!/DCIM
!/TitaniumBackup
// Ignore all the rest
*
Dropbox doesn’t do it, Google Drive doesn’t do it, OneDrive doesn’t do it.
It’s too much micromanagement. Having it on a per directory basis should be more than enough.
You can always deal with this by moving stuff around.
I know that I can do it (I’ve been doing this with BTS for a while), but it’s a bit of a hassle, especially if you are trying to share content with friends. Moreover, moving files to a shared folder affects the content index of my DLNA server, not to mention that I have to remember to make the content available in the first place.
Here’s the share-flow (to coin a term):
folder of files (videos) is shared between multiple people; each person adds content
a zero-byte file is created on each peer that matches the name of the original
a file is sync’d on demand (from the source) when a peer requests it
an ‘unsync’ option would truncate the file to 0 bytes
This is merely a suggestion, I certainly recognize the effort that it takes to develop and maintain software.
I don’t think this is a viable option, given someone is handling (or still trying to handle) 3.5 million files with syncthing.
We have to construct the folder tree to be able to prompt the user with selections, and a tree with 3.5 million leaves will take very long to generate, transport it over the socket to the browser, and then get Javascript to parse it and display it.
Obviously this has scaling limitations with folders too (as you can have crazy amounts of folders and no files), but this is usually the less common case, hence I hope that this will work in an acceptable fashion.
Plus this gives us a nice property, you will have the tree like this:
[o] Subdirectory
[x] All files within "Subdirectory"
[ ] AnotherSubDir
[x] SomeSpecificSubdir
Which means if a new file gets added to Subdirectory, it’s always synced.
Where as if you break it down per file, a newly added file would have to be explicitly selected on remote peers after it becomes available.
You could probably combine selective sync + ignores to more or less achieve exactly what you want (selectively sync all files in X, ignore all, except X/thefileIwant).
I don’t think selective sync is feasible beyond on a ‘folder level’ … Anything more starts getting really complex really fast…
Some thing may be possible though, like ‘dont download files larger than X MB’ … It might also be possible to stream file ‘on demand’, and that’s a way off, but it could happen.
With regards to crypto-enforced-permissions and selective sync: A heavyweight library I read about the other week is called like CryptTree (cryptographicaly enforced distributed access control)… It’s a really interesting read, but it’s too heavyweight for this particular system. It’s just massive in terms of feature set – as well as learning curve to program and manage.
It would be helpful and I think straight forward to have a setting with which to prevent the sync, but allows to see e.g. as a list what can be synced and to start actually the sync of an object on demand, i.e. by clicking on it.
I am new to syncthing and have replaced other solutions with syncthing for my use cases. I think this feature would be really nice. But, for what I understand the only lacking part is a gui that helps the user update the .stignore file, am I right? I have tested a bit by manually updating the .stignore file with different directories and it seems to work as I expect. (e.g. adding “/test/” will ignore any updates in the test directory, both incomming updates and outgoing updates)
The GUI should be a tree with selectable directories, similar to e.g. Dropbox. Default all directories are checked = selected. If I uncheck a directory the .stignore should be updated with one single line for this directory.
If I am right, this feature is probably quite easy doable and do not need any update in the core, just in the GUI?
You do, as the way it is now, ignoring everything apart from a/b/c/d does not work, as a gets ignored because the pattern does not explicitly match a or a/b and they both get ignored.
Yes, but that was not my suggestion. The suggestion was to ignore the directories not checked. This means that the default is that a directory is synced but it is possible to not sync it.
It is of course possible to do it the other way around, to ignore all by default and adding selected directories only. However in that case that you select only a directory deep down, as in your example above where the user only wants the directory /a/b/c/d to be synced, the .stignore file needs to look like:
That is really cool! Sorry if I am to naive now, but one approach would still be the suggestion above, using the rest api to update the stignore according to the suggestion. Of course a user can update the stignore manually and destroy it, but a warning in the docs might suffice to at least be a proof of concept and get feedback from other users - since the code is close to finalized?
.stignore is already updatable from the web UI, it’s just in a form of a text box.
The problem using the tree selector ui with the current setup is that we’d have to do a lot of pattern expansion which could potentially result in a millions of patterns which would be hard to consume.
I did not get why there would be millions? The number of excluded directories would never be millions would they? Each excluded directory will have exactly one pattern, and in some rare cases like including a deep down directory only, a couple of extra “always include” patterns? I do think we misunderstand each other, or maybe I don’t see the problem due to beeing new at this.
Anyway, if I can be of any help, let me know. This feature is vital imho.
