To clarify what I meant in my most recent post. And this is one idea which attracted me to come start developing on this project, to learn if this is possible, and how long it might take… It’s the same sort of main idea as ‘selective sync’ that people like in Dropbox…
My inspiration is instead from another Bittorrent like file sharing protocol called Bitcasa. (Closed source and costs money , company is being sued, etc which is why I dont use it)
What I was talking about is to have Syncthing have ‘streaming’ behavior like Bitcasa. As an option. Suspend disbelief for a second, and imagine this is what I’d like to happen… ie.
(1) i have syncthing runnng on my home computers and a VPS or EC2. And they are all sync’d up.
(2) I go to my girlfriend’s computer. I install Syncthing We want to watch some movies / tv shows that are on syncthing. But I don’t want to (A) download all the shared files, nor do I (B) want to download the movie prior to watching.
(3) So essentially, the behavior in my pretend example works like this… Syncthing creates ‘placeholder’ files for the entire filesystem tree on my girlfriend’s computer, but doens’t populate them with any data. I see them as drive E: in windows or whatever.
(4) I naviate to E:\Movies\ and click on IronMan.mp4 or whatever. At this point, VLC issues a blocking read to E:\Movies\IronMap.mp4 . Syncthing either detects or intercepts this (just like Bitcasa does) ,and starts downloading the file. After a certain amount of pre-defined read-ahead occurrs (essentially a video buffer), then VLC’s blocking call returns, and IronMan.mp4 starts playing.
(5) Thus, two benefits overall (A) I can see an entire massive shared filesystem and browse it without downloading anything other than Filesystem metadata (how thing’s are layed out , what the files and folders are, etc). … and (B) There is some sort of mechanism or driver that allows me to ‘stream’ files.
(6) In the ideal implementation , #5 doesn’t just start ‘streaming’ the file from Block 0. It starts streaming the file from the point of fseek(File *fp, 0L, SEEK_CUR) … Or in other words, the current positon . So if I click on IronMan.mp4, let it buffer and load, then I skip ahead to the halfway mark… Then Syncthing will detect a read halfway through and start populating the file from there, or start writing the file at the zero mark from there, etc.
Okay, so I’m not familiar enough with video formats to know whether this is possible with .avi and . mp4 formats, or whether this is a pipe dream that only Netflix and Amazon Video can achieve. But the basic idea is to emulate the behavior of Bitcasa…
Meaning I want the E:\ drive to act as a ‘download-on-demand’ drive at the very least. Where Syncthing only downloads a file when I try to use it (meaning open(), read(), etc. And even better, Syncthing somehow only downloads the relavent portions of the file.
Just the same way that Synchting already only shares the parts of the file that have changes on write(), I’d like a similar efficient process for read() – at least as an option .
That’s my dream for ‘really cool feature that’s difficult to implement’ – it’s essentially like turning Syncthing into the mailman for a Network Share… Like how NFS or Samba behaves when mounted as a Drive letter… It only downloads on demand.
