Awesome software, and this appears to be an extremely common request. This is definitely my personal #1 request. Something like an option to go into on-demand-only mode when starting up an account. Everything is ignored at first, but I can see structure and files shared. Then I can select which folders/files to sync.
This would be great, especially to download large media files. I would love to be able to sync my files like this that are hosted at multiple locations. Then I can stream large files on-demand only, but still sync my other files as chosen.
Does SyncThing have a first/last pieces and sequential order feature like popular bittorrent clients? Add those two things and wow you would get millions of new users fast.
With āpopular bitorrent clientsā Resilio is probably meant and when creating a peer it is decided if you wants to select or not. If so, the structure of the directories, the files are transferred as 0 bytes with a special extension. Files or complete directories can then be selected on each client. Directory contents are automatically updated accordingly when changes are made.
A good thing for clients who donāt have enough memory. I have that e.g. already used in projects that are a few hundred GB or TB in size but do not fit on a client etc.
So far, I had primarily used this on smartphones. Since I am not currently using the Resilio APP because it freezes or only synchronizes once when started manually, this is not possible, so there is only a whitelist. Of course, this is not so convenient and you may not even notice that someone else has created or changed new directories, etc.
Opppsā¦ Iām sorry, Iāve deleted my first post by stupid mistake. Fortunately, I had a chance to save the text:
Hi all! I see Selective Sync feature is really demanded! Moreover, I would like to try one use case with this feature but I do not want to use commercial Resilio as Iām going to expand my solution to at least 10 persons. So I decided to start a small project to realize Selective Sync for Syncthing. I write this message just to get any feedback from anyone interested in.
Right now I have the first version of my app that should have a lot of bug and looks pretty ugly. But it can read global folder tree from Syncthing and form ignore file by checked treeview items. Iām not a professional programmer, it is just my hobby. So I can not promise anything, but I believe I will be able to write a few versions more to make it useful.
The main idea is to make a GUI for .stignore file. Iāve read some discussion here and on github, where people say that it could be a wrong way. But Iād like to try to find limitations of this approach.
I just like to notify that a new version of my third-party app for preparing ignore patterns for selective sync just released
here!
Iāve added a lookup of local files and context menu to remove unsynced items. Itās also possible to easy share new files in partially synced folders.
Dear Andreas
I forgot to mention that a special initial pattern is required because I parse the file by special headers. So the next lines starting with // must be kept:
//* Global ignores *//
!/.stignoreglobal
#include .stignoreglobal
//* User defined *//
//* Selective sync (generated by pyselective) *//
!/folder
//* ignore all except selected *//
**
It is necessary to be sure that pyselective works only with self generated patterns framed between //* Selective sync ā¦ and //* ignore all ā¦
It should be noted that .stignoreglobal must be synced first as Syncthing had a bug some time ago with including nonexistent files.
Oh, I just recognize that the name differs in your case!
Yes, it will work but you will see it in the file treeā¦ It would be better to hide all included (or hidden?) files in the UI I think.
Everything Ok right now?
Note: it is impossible to remove files under synchronization to avoid accidental loss of data. If youād like to do it, you need to use your file manager.
Bump up on this feature.
Iām currently working on a feature film where weāll have around 50TB of data, thousands of files and needs to distribute the files needed to people around the world.
We have ~2000 files that takes up 40TB (video files are BIG) and the rest is a lot of smaller files, image sequences and so on.
To sync up all 50TB for each computer is an absolute no.
Using a git-ignore would take away a lot of time for all the artists as they would need to edit it A LOT as they donāt need all the files in each folder. For example our video library. They might need a single rain element ~500mb and shouldnāt sync the whole 20gb folder of rain footage. This is one example but it will happen to a lot of files and folders through out the project.
A virtual file system/selective sync is the only logical approach as all artists can go though all files and download only the once they need. Maybe even automatic downloading when other softwares request the files.
As mentioned before, there seems to only be a handful of softwares out there that has this feature. In our case itās between two: Nextcloud and Resilio.
Nextcloud does exactly what we want. We can see all the files, it downloads automatically when requested and as a bonus we can see the size of the files before they are being downloaded.
BUT Nextcloud is suuuuuper slow on large amount of small files. As weāre working on image sequences we will have hundreds of thousands of files and in my tests it will just not work.
Next is Resilio. Super fast, easy to use, selective sync. The breaking part tho is that they use their own kind of āvirtual filesā. Weāre getting a bunch of .rsls files representing all files. So they wonāt download by request of other softwares as the original filepath doesnāt exists. This could be solved by some CLI commands or an API but that does not exists for resilio sadly.
So there is a big hole in the market for places like ours that host a lot of big files that needs to be synced to a bunch of places but canāt sync them all.
Syncthing would be the perfect fit for this kind of situation if virtual files would be a thing.
Have you tried Seafile? In my experience itās much faster than Nextcloud. Especially with lots of small files. It should support selective sync too and also has virtual file system support.
Unfortunately (for my use case) Seafile stores the data on the server as blocks. So files arenāt directly accessible from the server filesystem. With Syncthing I donāt have this issue.
So Iām currently hoping Nextcloud will fix the bad performance with lots of small files, Seafile will implement read/write access to all files directly from the server filesystem (donāt think this will ever happen) or Syncthing will get a user friendly way to do selective sync.
On Github there is a special Windows tool for Syncthing that can give a selective sync.
The base for this tool is the ignore list, which is filled by entries of it. In contrast to Resilio, there are no placeholder files. Basically it is an ignore list that can be created individually for each folder.
However, and this is the difference to Resilio, no device-related selection can be made with it in a easy way. The only way is that the respective recipient of the files must also store the individual ignore list. If he deletes them, heāll get all the files. Should that in turn represent a criterion, that tool is not usable.
I have bin looking at that tool for a while but not being able to see what files/folders exists without going to another place to search, copy over paths, pasting and so on for each and every folder to sync is just not feasible sadly.
For a single user/family that syncs a couple of folders that tool looks good enough but sadly not for us.
I havenāt found anything equal to what Iām searching for, free or commercial. If I find something commercial that does it, sure that can be of interest.
If you have the name of the software youāre thinking of, could you PM me it?
Any how, I wouldnāt say it isnāt a too specialised usecase as most smaller studios I have worked on simply go into each others computers/a nas to manually fetch files. But Iām also in the center of it so thatās why I might see a lot of usecases like mine
@danell couldnāt find a way to PM you, but it sounds like youāre looking for Resilio Connect which is commercial Resilio Sync with API. Have a look through the case studies and documentation, it sounds like you need a level of file control that Syncthing canāt provide you (being a general purpose sync tool) but for which Resilio Connect is designed.
