I have come up with the idea of device priorities and I would like to propose this here. My common Syncthing setups suffer from a very annoying problem. The setup is as follows:
One/two boxes are a server with reasonable bandwith.
Other nodes are laptops/desktops with high downstream and a very low upstream bandwidth.
When a lot of nodes are online and I drop a large file into my Syncthing folder, Syncthing uploads the blocks to each node at the same time. This is very annoying when you have low upstream bandwith as the time to complete doubles/triples… Maybe audrius’ “temp index” stuff would solve the problem anyway; but let me complete my braindump for the time being.
A potential solution would be to upload to the server node first and then let the blocks spread from the server. In my mind something like device priorities (or just a configflag “strong device”) would improve the situation. A device with high priority should be prefered when uploading blocks; devices with low priority should be fed with blocks when all blocks have been uploaded to high priority devices.
The idea is fine. We had some discussion about something similar way back and the protocol actually has something defined for this (that is not in use). It allows a device to announce a priority for itself, advising other devices to “please download from me”, “please download from someone else” or “do whatever”. The primary intention was to deprioritize mobile devices and direct downloads towards other devices. It doesn’t cover exactly what you need, but something similar. All that’s missing is some code to implement it.
I think the sharing of partially downloaded file being worked on by @AudriusButkevicius is the best fix for this.
Also, the nodes request the file from sources that have it so you would basically be ignoring the request or responding with not now. You could achieve something similar in the interim by pausing the other devices when sending large files.
The problem is when one node with low upstream bandwitdh tries to upload its blocks into the cluster. It does not affect the system as a whole, you are right. But it affects one node which wants to e.g. upload a VM image to the cluster, as it tries to upload to as many as possible nodes at the same time.
Potential candidate for the “unplanned - contributions welcome” milestone?
I think the scenario that does affect the system as a whole, and is worth avoiding, is when the latest version of a file lives behind a slow uplink. With the current setup you may have three devices that want they file, and they’ll all pull it from the one that has the file available - total time: three times the time it takes to upload the file over the slow uplink. With some sort of prioritization we could get the file to a better connected device and reduce the total time to one time the slow uplink and two times some faster link.
But there’s certainly overlap with the temp indexes feature here - with random block pull order that may even cover the case fully.
Also, I’m at 10 kilometers in the air looking down at clouds and that’s pretty cool.
That’s exactly what I am talking about. Thanks for clarifying my somewhat confusing wording (I talked about uploading, but Syncthing actually pulls blocks, and so on)!
The temp index feature only would partially solve the problem I think. For example three nodes pull the file (which is behind a slow uplink); once a block is available in the cluster it spreads via the temporary index feature. So,
three nodes pull one block from the slow device (time to complete is three times worse),
once the block has been pulled from a random device, it spreads.
There is some overlap with temp indexes, but the bottleneck still stays at the beginning of the chain: the initial pull from the device with slow uplink bandwidth. Some prioritization could improve the situation (“nobody but serverX is allowed to pull from me”).
Yeah, Syncthing is even being discussed 10 km in the air. The term “uploading into the cloud[s]” gets a new meaning.
Well everyone pulls a random block, and from then onwards we pull from the device with least outstanding requests, so the device with a slow upload will probably be less likely to get requests.
The only problem is that when we pull random blocks, the only peer that might have the block might be the original peel, hence there is an optimization which we could make, such as pulling the block which has been seen in the least peers to make it spread better.
About nobody but serverX is allowed to pull from me - just share your folder with serverX and nobody else, and then serverX and everyone else.
I am also looking for a similar feature - I have a similar setup as the thread opener and would also like to have something like “device priorities” for uploading files.
Was there any advance on this topic in current 1.18.2?
There are per device rate limits nowadays, which can be used as form of priority control, as we always prioritise the device with least outstanding in flight requests, so setting a rate limit will cause more requests to stay in flight.
But no, there is no priorities, as the goal is always to get the data as fast as possible.
