Improved Block Distribution?

I’ve recently migrated from BTSync to Syncthing and I’ve noticed some unexpected behavior in how files are synced among many nodes. Let me pose an example to illustrate:

Let’s say I have 4 nodes, R1, R2 (remote), L1, L2 (local). They are all sharing a folder. The connection between R1 and R2 is very fast, as is the connection between L1 and L2. When a file appears on R1, R2 will get the file very quickly. This leads to L1 downloading the file from both R1 and R2, so the traffic is split. When I add L2 to the mix, I’m noticing it copy the same blocks from R1 and R2 instead of getting the blocks L1 has already downloaded. What I would like to see is L1 and L2 each downloading different halves of a file, then L1 and L2 share those halves to get the whole file.

I’m perplexed to see hardly any traffic between L1 and L2, even though they both want the same file and L1 may already have parts of it. Can anyone explain the existing behavior and are there any plans for better “swarming” for the network?

This may seem like an odd question, but my purpose is to defeat what appears to be connection-specific throttling. I’ve noticed my ISP throttle individual connections, such that doubling the number of connections roughly doubles the throughput. Increasing the number of BTSync clients had the intended effect when I was still using it. Increasing the number of pullers doesn’t seem to have helped, though.

Just on mobile phone, will edit later. There are two major issues about that:

Temp indexes is blocked by:

3 Likes

Basically, a device has to have an entire file before it shares any of it. There is a fix in the works but is waiting on a couple of other things before it is merged.

If you have multiple files that change at around the same time it might help to set one device’s pull order to newest and the other to oldest.

1 Like

That is exactly what audrius’ temp indexes is about, yes.