I have a ‘shared secret’ from a friend who wants to use BTSync to share stuff over the WAN. Is there any way I can use this to connect to him? It seems like the closed program must have a hard coded node discovery server. Do you think that’s true? If so, will Syncthing support shared secrets and node discovery (trackers?) at some point too?
I have no idea how BTsync works, but I’m sure the same use case could be supported somehow.
I have to say advertising it as a bittorrent sync replacement may mislead people.
BitTorrent sync can locate the other node by just entering the id of the folder (shared secret) because it uses BitTorrent trackers for this.
If right now Syncthing can’t locate a node over the Internet without having a fixed ip address, then the setup process is much more complex than what BitTorrent Sync is aiming to.
btw, the Clearskies is already using a tracker:
However, in terms of usability, they are far behind Syncthing. The approach of Syncthing with the daemon and web-gui embedded in it is more close to btsync than Clearskies.
There should be somewhere a list with differences to btsync.
btw. this thread is similar to: Can I use synthing to distribute content like btsync?
Of course it can, where did you get this idea?
Really? How does it work?
Found the answer myself, https://github.com/calmh/syncthing/issues/72#issuecomment-35866460
BTSync currently has 5 BT servers PREWIRED (for trackers and relays). Basically, they are trying to monopolize the entertainment industry “bundle” distribution market, which means they probably get the percentage of all sales based on a count of connections to some share. But you can ask them themselves about it.
This implies that you can ONLY use the BT trackers with BTSync even though it looks like they use standard uTorrent tracker algorithms, probably to get their count as high as possible. You can not use your own trackers with BTSync, nor you can use any standard torrent trackers. This is not a user configurable option.
On the top of that, it has been observed that even if you turn off all tracker related options for the share, including the DHT discovery, relays and local discovery and do not even use the preconfigured host(s), the nodes STILL can discover each other. Which means, if this is certainly the case, that you can not disable the BT tracker no matter what you do.
Interestingly enough, there has been a discussion about alleged cooperation of BT and NSA. One guy even stated that he did observe certain things that indicated some unusual interference. Furthermore, the thread that began to discuss the NSA related issues and was one of the more popular threads on their forums has been simply removed with “no further comment”.
Basically, the way it looks, as soon as syncthing implements the equivalent of public r/o key so that anyone could connect the a public share without any user intervention and transfer performance is improved, that might be the beginning of the end of BTSync.
Their claims as to traffic encryption are worthless
I don’t exactly know how the shared secret in BTsync works (does anyone?). For a shared secret thing to work, as I see it (and I haven’t fully thought this through yet so I might have missed something obvious) we would need to stuff in something like
The node ID (or perhaps something like a “cluster ID”) to connect to to get access. Perhaps this doesn’t need to be a full node ID in the current sense (256 bits), but it must be something unique enough that it can be global without risk for collisions (a GUID is 288 bits for reference, so 256 bits isn’t wildly “out there” for something globally unique). This is what we look up somewhere so we know where to connect.
A pre-shared key (password) so that not simply connecting to a cluster by blind luck gets you access. Again, AES256 and similar are the current state of the art (without being overly paranoid), so that’s another 256 bits of key material. This is what we use when connected to decrypt data and prove that we are allowed to join the cluster.
That gets us to 512 bits we need to encode and share. Using base32 (like current node IDs) that ends up being 512/5 = 103 characters worth of secret key. Doable, but annoying.
The BTsync secret key looks like base32 (roughly A-Z and 0-9) and is 32 characters. That’s 160 bits of data. The Kademlia DHT network uses 128 bit keys, so perhaps 128 of those are used to search the DHT (and are thus public), leaving 32 bits for the secret key? That would obviously be absurd so hopefully I’ve misrepresented how their stuff works, but the point is I have a hard time seeing how to get something secure out of such a small blob of data.
Bittorrent’s Mainline DHT uses 160-bit keys, so that might explain the size of the secret key in BTSync.
Yeah, but hopefully not, as the key used to lookup stuff in the DHT is public – it’s sent to the DHT network, after all…
There was an effort to reverse engineer the bt sync protocol. It never got far.
Here’s some info:
and some code:
At least it did find out a few things about how the secret works. As far as I understood it the secret as such is just a random number. The forum article linked above explains how the read-only secret is generated from that.
What is BTSync share key (secret)
Well, from what I understand, it is nothing more than a GUID. It has nothing to do with privacy or security, since, in case of BTSync, it will be passed through their trackers and relays. Also, if you do not wish to let BT have any information about your shares/repos, and disable the BT trackers and relays, then the node discovery could be done either by using the DHT, or by using the “predefined hosts”. Predefined hosts are simply the host names/IPs:ports of the hosts on the list of predefined hosts that do have a physical copy of this share/repo. In case of BTSync, predefined hosts can not be used as general purpose trackers. That is, it MUST contain the physical copy of the share in order for you to discover the other nodes via then, which is the idea I do not particularly like.
It seems to me that predefined host should be a full equivalent of a tracker regardless of physical presence of the data of the share. But this is arguable. Because in that case those hosts may potentially have to carry way too much traffic and for the shares it does not even know about. But, on the other hand, since “predefined hosts” is per share option, you may simply be able to click on a checkbox in a repo edit screen that says “use as a tracker only”. In this case, no physical data of the share is needed and all the tracker functionality will be available. This means that predefined hosts can be used as trackers for only those shares they know about and not just for any arbitrary share, like in DHT. So, when you pass the share key and provide at least one predefined host for initial node discovery, it assumed that hosts knows about this share and has it on its list, even though it does not have to have a physical data of the share.
And, since DHT is public by definition and DHT netowork can be joined by anyone, it implies that the repo key isn’t even meant to be a “secret” of ANY kind. It is merely a truncated version of share ID “GUID”.
But, via this key you have a mechanism of identification of other nodes that carry this particular repo, again, either via DHT or “present hosts”.
So now, once you have that key, all you need is to specify the folder where this share is to be stored in order to create your node for this share.
Once you specify these things, then, if you have DHT enabled, the other nodes that carry this share and have DHT enabled also will respond to your requests to join the share. Alternatively, when distributing the share key you may also provide the information for the “guaranteed” shared hosts. One of them is enough to trigger the process.
Interestingly enough, if at least one node has DHT or “use predefined hosts” enabled and a new node comes it on the this share, it will provide you the list of all the known nodes on this share. That means that “predefined host” essentially functions as a tracker and it knows about all and any nodes on this share currently on line or were on line some time ago within your preset limit of time when you keep them on “seen list”.
Furthermore, any node also behaves like a tracker, if you can discover it. Meaning that upon request it provides the list of all the nodes on this share that it knows about.