Scenario, I have SeedBox and would like to sync the contents to 10 different syncthing nodes. The SeedBox is the Repository Master and of course, all of content is coming from there. When syncthing is transmitting data, would it be sending sequential pieces to all 10 nodes simultaneously, or would it be sending different pieces to each node and they would all work together to fill in the gaps, thus sharing the load.
ie. I have a 600mb file being sent to 6 nodes, would it send different 100 mb parts to each node, or would it start from scratch on each one?
Even though individual blocks (128 KiB) are the unit of “diffing” and transferring, any given node only announces whole files. So in this case, you have one node that has the file and six nodes that don’t. Each of those six nodes want the file, and have only one other node to ask for it so all of them are going to transfer the entire file from the first node.
If one of the six other nodes gets done before the others, it’ll announce the file to the others and they have two sources they can load balance from, etc.
When all you have to transfer is one 600 MB file to many peers, this is going to be quite inefficient. If you had 600 * 1MB files, not all nodes will start pulling the same file. So when a given node wants one of those files, odds are some other nodes have already downloaded it and the spread is nicer.
Could limiting the amount of nodes the file is anounced to help with this issue? It will result in the file being transferred from the other nodes that will announce it
Note that a syncthing cluster doesn’t need to be full mesh either. You can connect A to B and C, B to D and E, and C to F and G to get a tree structure for example, if that makes sense with the available bandwidth and so on. Data will flow along the tree branches…
A tree structure could work, however the seedbox is the fastest upload of the bunch. Unfortunately with just one connection is will max out at around 10mbit regardless of how great their connection is. Ideally I would like each node to be working together with the seedbox, as that would allow the file to propagate faster, rather than just between eachother further down the tree.