Just a heads up to @kozec that /rest/tree now exists, given he has any interest in this ;).
Selective sync to follow shortly.
3 Likes
[quote=“AudriusButkevicius, post:43, topic:1779”] Selective sync to follow shortly. [/quote]Does that means you are going to implement it as feature in Syncthing, so it wouldn’t be required to do hacky things with .stingore?
Yes. I have basic prototype for the UI, need to actually plumb it into the scanner etc. I am sure feature requests will follow asking to sync selections etc, but that’s a different story.
2 Likes
This sounds excellent. I look forward to it. 
To clarify what I meant in my most recent post. And this is one idea which attracted me to come start developing on this project, to learn if this is possible, and how long it might take… It’s the same sort of main idea as ‘selective sync’ that people like in Dropbox…
My inspiration is instead from another Bittorrent like file sharing protocol called Bitcasa. (Closed source and costs money , company is being sued, etc which is why I dont use it)
What I was talking about is to have Syncthing have ‘streaming’ behavior like Bitcasa. As an option. Suspend disbelief for a second, and imagine this is what I’d like to happen… ie.
(1) i have syncthing runnng on my home computers and a VPS or EC2. And they are all sync’d up.
(2) I go to my girlfriend’s computer. I install Syncthing We want to watch some movies / tv shows that are on syncthing. But I don’t want to (A) download all the shared files, nor do I (B) want to download the movie prior to watching.
(3) So essentially, the behavior in my pretend example works like this… Syncthing creates ‘placeholder’ files for the entire filesystem tree on my girlfriend’s computer, but doens’t populate them with any data. I see them as drive E: in windows or whatever.
(4) I naviate to E:\Movies\ and click on IronMan.mp4 or whatever. At this point, VLC issues a blocking read to E:\Movies\IronMap.mp4 . Syncthing either detects or intercepts this (just like Bitcasa does) ,and starts downloading the file. After a certain amount of pre-defined read-ahead occurrs (essentially a video buffer), then VLC’s blocking call returns, and IronMan.mp4 starts playing.
(5) Thus, two benefits overall (A) I can see an entire massive shared filesystem and browse it without downloading anything other than Filesystem metadata (how thing’s are layed out , what the files and folders are, etc). … and (B) There is some sort of mechanism or driver that allows me to ‘stream’ files.
(6) In the ideal implementation , #5 doesn’t just start ‘streaming’ the file from Block 0. It starts streaming the file from the point of fseek(File *fp, 0L, SEEK_CUR) … Or in other words, the current positon . So if I click on IronMan.mp4, let it buffer and load, then I skip ahead to the halfway mark… Then Syncthing will detect a read halfway through and start populating the file from there, or start writing the file at the zero mark from there, etc.
Okay, so I’m not familiar enough with video formats to know whether this is possible with .avi and . mp4 formats, or whether this is a pipe dream that only Netflix and Amazon Video can achieve. But the basic idea is to emulate the behavior of Bitcasa…
Meaning I want the E:\ drive to act as a ‘download-on-demand’ drive at the very least. Where Syncthing only downloads a file when I try to use it (meaning open(), read(), etc. And even better, Syncthing somehow only downloads the relavent portions of the file.
Just the same way that Synchting already only shares the parts of the file that have changes on write(), I’d like a similar efficient process for read() – at least as an option .
That’s my dream for ‘really cool feature that’s difficult to implement’ – it’s essentially like turning Syncthing into the mailman for a Network Share… Like how NFS or Samba behaves when mounted as a Drive letter… It only downloads on demand.
2 Likes
Wow! This is interesting, @cydron, but I’m afraid is too close to what Microsoft did on SkyDrive and fail: the users don’t have a simple way to know if a file is “real” or just a placeholder.
This is the reason I suggested an easier to understand and, thanks to the explanations of the developers, easier to implement. BTSync implemented it well, for what I can see.
May this could be a evolution of the placeholder files? So, we could watch how users react to the feature.
Oh really? Man I better go read up on Skydrive. Good call. That idea is on hold until I figure out why it failed for microsoft. Wouldn’t want to waste time on a feature that has no demand.
Sure, maybe copying BitSync’s style is better. They are the competiton , as I see it anyway. Thx for the feedback.
IMO, just making syncthing not require full syncing, preferably in a better way than btsync (currently you have to wade through the folder hierarchy of the source being synced) would make syncthing more useful for sharing with devices that have storage issues like mobile. Having the ability to view your synced files in a cache rather than the source hierarchy would be a killer option.
Not even sure what you mean by that. Not familliar with how btsync does it.
Hi,
Sorry for being unclear in my description. Here’s what I was trying to say:
syncthing and btsync work almost the same up to the point where the shared folders have been defined.
From there on, syncthing starts syncing the shared folders - it does this by starting to copy all files. Of course, on a target device with less free space than the source size, you eventually run out of space.
btsync, on the other hand, only syncs the folder structure of the source folders, leaving it to you to pick the files or folders that you actually want on the target device, thus avoiding the syncthing problem. Files that are downloaded have check marks against them to indicate local availability. This is nice, but the problem is that in order to see files that you may have synced, you have to browse through the folder structure, folder by folder - which is a pain.
So while we have to maintain the same directory structure on the target device in order to allow full syncing, it would be nice if syncthing could start by only syncing the folder structure (like btsync) and then add the feature where the files that are already synced are listed separately for easy access.
Not sure if I’m any clearer, but hope that helps.
I think that OneDrive has solved this problem properly. You simply choose which folders to synchronize, end of story.
I think that what @kinleyd wants is to be able to see a list of already synced files on Syncthing interface and to open them from there. Am I right, @kinleyd?
1 Like
Actually, if we were to be really careful and select only folders that fit the space available on the target device, then syncthing has also already solved the problem. I guess the functionality I’m looking for is simply having access to, say my whole collection of music and videos, but selectively downloading what I want without worrying too much about space availability nor having to go back and create new shared folders on the client and the master in syncthing everytime I want something.
1 Like
Well selective sync solves this doesn’t it? You select folders you want. Once you are done, you unselect them (and perhaps part of that we delete the unselected files) and select something else?
How is seeing files which are synced in Syncthing any more beneficial than using the explorer?
Sorry, I’m not sure if I’m missing something here. My point is that if I were to share, say my folder called Music, I can’t share it to my phone if Music contains my entire music collection (as it does, at 170GB) because my phone only has a max capacity of 16GB. I could then create different folder shares from within my Music folder so that it works within the constraints of my phone. That would work, but I would be less inclined to use syncthing as I could do pretty much the same thing with adb push or similar utility.
1 Like
I think you don’t understand what selective sync is. You choose which subfolders within a folder you want to sync.
OK. Let me work through the syncthing sharing process again. My comments so far are based mostly on the default behaviour of syncthing, which if I’m correct to say, is to copy over all files. For btsync the default behaviour is to only copy over the directory structure. I’ll get back again. Thanks for your time. 
UPDATE: I’ve got syncthing for Android version 0.5.18 on my phone. I have advanced Folder Picker selected in settings, but beyond that I don’t see any selective folder picking options. Is it possible this version of the app doesn’t yet support selective folder picking?
UPDATE 2: I’ve gone through the process of sharing a smaller collection of folders again. I don’t see any where I can selectively download folders, and all folders are downloaded on the target device by default.
Android app version: 0.5.18 Desktop version: 0.10.26
Is useful if the file is toooooooooo deep down on file tree.
But this is something that have to wait for the new interface. This feature requires to see the files of the folder on the interface in two different modes:
- file tree;
- flat list.
For the moment, I think it’s too complex. The new interface is just a draft yet.
1 Like
This is a future feature we are talking about…
1 Like