Can I use synthing to distribute content like btsync?

Support
1

Hi. I’ve been using btsync and I heard about syncthing as an opensource alternative. I’ve been using btsync to distribute some semi-static content by announcing my btsync key.

Is there a way to accomplish something similar with syncthing? I won’t know who are going to be sharing my contents (it could be hundreds/thousands of users) and I will be updating the content about once a month.

Or… is syncthing primarily a tool to share between a set of known hosts (more like a traditional Dropbox setup)?

Thank you!

2

The latter; syncthing won’t (currently) work for your use case.

3

Well, this is probably one of the most important issues and there are some things to consider and implement to make things easier and more compatible with other approaches.

I am just beginning to look at syncthing, so I don’t have much idea about it. But being able to sync based on a key (hash) of a share (top sync folder) seems like one of the most desirable things to implement.

Also, using regular torrents “under the hood” could probably help the whole thing greatly and make it more compatible with other existing approaches that are well developed by now and open source code is available.

Then come the issues of peer discovery via standard DHT protocol and using the standard uTorrent type trackers. This way, you can use hundreds of regular torrent tracker servers and, in combination with DHT, you become a part of the DHT global network and carry some DHT request traffic.

User configurable tracker list is highly desirable, if such is not implemented.

If regular and temporary torrents are utilized, like it is done in BTSync, then the whole thing greatly simplifies because there is a lot of open source quality code available in C++, Java and so on.

For example, here is the qBittorrent source:

http://sourceforge.net/projects/qbittorrent/files/qbittorrent/qbittorrent-3.1.9.2/qbittorrent-3.1.9.2.tar.gz/download

It is written in C++ which should be pretty easy to implement and port, and it includes a fairly complete set of functionality, including the DHT, trackers and so on.

There is basically no point of “reinventing the wheel” because things, methods, policies and mechanisms were already developed and source code is freely available.

The KEY word is torrents.

So, I am not familiar enough with the Syncthing, but it seems that the current approach of node discovery is good for the purpose of using Syncthing for private networks, which, security-wise is pretty good, except there is but.

But you simply must have to implement the general public distribution via SHARE (top sync folder) key, and not only via node key and allow automatic joining of swarms for all public shares without any user interaction or config file modifications, manual or programmatic. These mechanisms may work in parallel. But they are probably quite different and different sets of rules and principles apply.

The program simply must be BTSync compatible functionality-wise, even though to make it fully compatible is problematic because of some weird hashing/semi-encryption algorithms and mechanisms in BTSync, which is FULL of bugs. But you don’t have to be totally compatible with them. All you need is FUNCTIONAL compatibility so you could do the same things they do, and, probably, utilize the existing GUI status and progress reporting like it is done, for example in torrent programs, such as uTorrent. For example, it is highly desirable to show a dynamic picture of transfers, such as which exact files are being transferred right now, what is the progress of transfer, such as speed, percentage done and things like that. Otherwise, you just sit there and have not much of an idea of what is currently going on.

Not sure how this fits into current model, but it looks like Syncthing project is already something working, something functional, and I have tested a few things and, considering the stage of development, it is pretty good “as is”, except of this “but”.

Second issue I have noticed is pretty poor transfer speed performance. Not sure what the problem is. Could be a result of encryption overhead or whatever. It seems that the traffic should pump at the rate of 10 mbits without any problems. BTSync claims that on a LAN they can go at 90 mbits. What I have observed on some sample shares in Syncthing is somewhere around 200-400 kbits max, and that is not such a great performance.

Not sure if multithreading is utilized or design is single threaded, in which case there will be all sorts of problems, such as delays in transmissions while data is being encrypted. Everything in transfer engine has to be fully asynchronous and multi-threaded.

I have never worked with Go language, but it might be possible to use some C++/Java code in a library or via some language interface, which provides the DHT, torrent, tracking and all sorts of standard mechanisms for torrents. That would probably make it pretty easy to wire-in.

Hope this helps.

1 Like
4

There is also Vuze/Azureus source code (written in Java):

http://pkgs.fedoraproject.org/repo/pkgs/azureus/Vuze_5300_source.zip/

It also has all the necessary mechanicms for DHT, trackers and the rest of it.

Actually, Java is probably a better choice, especially from the standpoint of code clarity, synchronization and portability issues.

So, all the critical code is already there in open source version.

5

So, build something awesome on top of that then?

Look, sorry for the curt answer here, but I get kind of weary of the “I haven’t actually looked at how syncthing works or what it does, but here’s what I think you should do instead” kind of posts.

2 Likes
6

Hi all,

I would like to second this wish.

Given the usecase that I want to distribute content via Pulse to selected persons (or everyone) I need to have a feature which:

  1. makes it a one click for people to sync with a special folder on my computer
  2. makes it easy for me to verify if I want to give these people access to my data.

If I understand Pulse correctly this is a multi step process now:

  1. I publish on my website that I want to share something via Pulse.
  2. People send their device ID to me via email or via another channel and ask for access to my data
  3. I send my device ID to selected people or publish it on my website
  4. I add all devices to my instance
  5. They add my device to their instance
  6. I share my folder with their device
  7. They connect my share with a folder on their devices

This is a very long and complicated process.

In BitTorrent Sync the process is much more easier:

  1. I publish an url on my website
  2. People click on the url and get connected to the folder I share
  3. I have to acknowledge access or give everyone access by default

This is much more convenient. And the URL-thing is a good instrument to make it more popular. If BTSyc is not installed people see a download link.

Thanks a lot for making this programme. I am currently evaluating, what I can do with it – and what not.

I hope the new umbrella gives much more momentum to Pulse (aka Syncthing) Cheers juh

7