I get the feeling the BEP was just designed to fit this purpose (Syncing program), at least so far. But it could probably be used for so much more.
Imagine a protocol designed very similar to the current BEP (or BitTorrent) except that it’s designed not to download ordinary files from client to client in a mesh, but rather tweaked to download an HD video stream instead to be played right then in an ordinary player (VLC, maybe some propritary player). But instead of getting it from one server (as the traditional method works) it would get it from a P2P mesh. And likewise everyone streaming would also be an uploading participant to others. This should allow it to scale very well to large groups, maybe even millions wanting to stream very popular data (major sporting events, Netflix/HBO.Go TV shows that have just come out, etc…).
Based on some thinking I’ve done on this for a bit, I imagine some requirements for this to work might be:
10 second buffer (at least) from a completed data block to “moment of play” data block.
Of course 10 seconds (or whatever the user configured buffer time) would need to be programatically converted to a buffer of data size to fulfill that request, and that size would have to be calculated based on bitrate and measured bandwidth the user has available with they current connection.
Downloading client prefers closer clients
The protocol would need to measure latency or hop count and prefer closer clients to download from and vice versa with upload. This might aleviate some peering demand between 2 ISP’s if pieces can be downloaded mostly from other clients within the same ISP or at least topologically near the downloader on the internet. For efficiency instead of using traceroute to measure hop count, maybe set all UDP packets with a fixed TTL size and measure the lease decremented (I’m not even sure the application layer can ask these stats of a lower network layer can it?).
Strongly prefer data that will play soon
Much stronger preference for data pieces that are closer to the “moment of play” than further away (in the forward direction anyway). Bittorrent already has something like this in its protocol I think to prefer certain data or “opportunistically unchoke” certain things. I dunno maybe I’m getting the terms mixed up.
Retention of already played data
This should be a given. Client would retain past played data to upload to others. At least until client is closed by user, and perhaps for a user configurable time after that.
.
So I’ve obviously spent more than just a few mins of time thinking about this subject. I feel like a protocol that did this could really change the way video streaming is done, At least for popular video. I’m interested to hear what everyones thoughts are on this. Is this something that the BEP could actually be tweaked to do some day? Or do you think I’m just crazy.
I know BEP can be used for this because I’m already doing a variation of it with SyncthingFUSE. I am able to watch some compressed video files located on a residential internet connection from another residential internet connection with VLC.
The “10 second buffer” and “prefer[ing] data that will play soon” already happens through naively prefetching blocks.
I did think about considering close clients. I don’t believe the closeness is actually the metric you want, though. I think the likelihood of a peer delivering the data block rapidly is the real metric. Closeness plays in to that, but so does the peers capacity and current requests. I haven’t done any optimization around this yet, but I think a combination of past performance plus current requests will be my first foray.
SyncthingFUSE devices are not currently able to upload to peers. First, I didn’t really need to implement it. Second, BEP doesn’t currently have an effective ability to say “I have some blocks for this file, but not all of them.” That might be coming soon, though.
Serious streaming services have multiple encodings of a single stream. This allows them to show the stream in a variety of bandwidth environments. This would be a bit more difficult to coordinate with Syncthing BEP alone. For a bigger operation, I’d want some additional metadata managing which encoding is getting used.
Finally, IPFS might be a useful tool for media streaming. I was recently dreaming about a service where I could prove I own a movie (through some signature like hash of DVD content) and then be able to stream it with all the fancy encodings described above. During that dream, IPFS seemed more applicable than Syncthing and BEP.
I was unaware other competing protocols were out, except one that BitTorrent Labs had produced. But they were trying to license that out for profit. Which is a most excellent way to ensure that a protocol dies a slow death in obscurity. They seem to have forgotten that BitTorrent only spread because it was free (as in beer) for others to use back in the Napster-RIAA days.
I’m wondering if these protocols are ready for things like the Super bowl or championship league football or the world cup. Because I imagine that if everyone on earth stopped watching these on TV and started watching them on their mobile phones, computers or IPTV’s, there wouldn’t be a data center on earth or network with the bandwidth to handle it. Not unless the burden of upload was shared almost as widely as download. And that’s where the P2P part would need to come in.
That’s why I feel like media streaming is something that eventually only P2P would be up to the task of handling.