Big files sharing efficiency

Support
1

Hi, maybe I have the wrong idea from Syncthing but isn’t it supposed to share the file transmission?

I have a cluster of 20 machines with a shared synced folder on all of them. When I add a file in one of the nodes, it is correctly replicated in the rest of the nodes. But what I have noticed is that no node shares the file before it have received it completely.

So if the file is “big” (in GBs level), my original computer is the only computer sharing the file to the other 19 computers and, as my network is a 10Mbs network, it takes forever to share a single file. Actually it is faster to copy the file to every other computer in the traditional way.

My idea was that the shared file has to get divided in small chunks and every chunk shared separately. And all the computers in the cluster can share the chunks they have already received although they don’t have yet the whole file. It’s my understanding this is the way all P2P protocols works (at least Bittorrent).

Am I wrong then? Am I doing something wrong? Can someone give some light on this?

Thanks a lot!

2

Currently only completely synced files can be sent to other devices, it is planned to change that, see https://github.com/syncthing/syncthing/pull/2252

As a workaround you can pause some devices so a few devices get the full file faster and they can also distribute the file…

3

P2P is peer to peer (opposed to centralised) but that doesn’t define how the transfers happen, etc, wheter its on parts on whole files etc.

Syncthing currently does not support sharing partial files, but I hope to finish that this year.

4

Hi,

Thanks a lot to both! I am looking forward to this functionality to be implemented. Sad that I cannot contribute at this moment to the project but I hope that at some point I can help somehow.

5

You can help testing once @AudriusButkevicius and other devs finish this feature (see #950) and support others in the forum who have questions about Syncthing.

6

Actually I came back to say just that! :smiley:

This week I have still time to test something. Then I am two weeks away. After that I can help again.

I will try to help in the forum but probably I don’t know still enough about Syncthing to be useful. :smile:

7

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.