We have about 13 remote devices in our cluster, and 2 of them have have slow network access. Is there a way I can tell Syncthing to not use those slow clients, instead favoring the other that are way faster? These 2 slow device connections are bottlenecking all the other devices when they encounter multiple large files. All devices are send/receive and macOS.
I have tried using “smallest files first” and that might be helping to an extent.
Or if I could setup the slow devices to only push updates to one computer, instead of pushing updates to the entire cluster? We had a 1.5-2 hr delay yesterday where many files just weren’t getting through.
Any ideas?
We’re using our own relay server. The two slow connections have older cable internet at their location (~25 mbit).
In a normal Syncthing instance, you indeed can configure the slow devices to first push to a single device (let’s call it server) by sharing the folders on the slow devices only with the server. From there it can be distributed to the other devices by sharing the folders with others (the others have to be configured to not share with the slow devices and have to share with the server at the very least). This can be done in the GUI under folder settings. BUT i don’t know if that makes it any faster. It’s worth a try though.
As long as the faster devices on the other side of the server do not wait for the new data on the server to arrive and then in one fluid motion to be forwarded to them, building on @AudiiusButkevicius statement, they should rather connect to other faster devices, because the server still has outstanding requests, right?
Unless…the bottleneck is the relay-server?
Sorry, if i say something stupid. I don’t know the code.
I doubt it’s the relay-server. It’s dedicated just to my 13 devices and is a fast VM inside my organization’s infrastructure.
I have 12 devices, all reading/writing all folders, sharing all folders with all the others. The 13th is send/receive and capturing versions from all the other devices. It has a fast connection. So, following your theory, this 13th device could be the sole recipient from the slow device.
I saw there were some advanced settings related to this, but I hesitate to change them without confirmation that it’s worth the attempt
The large file originates on the slow device, right?
By bottlenecking, do you mean that large files being synced prevent smaller files from being synced? That’s possible because by default at most 2 files are pulled concurrently. Syncing to a single device won’t change that. And I am not sure it is problematic in general: The overall sync time doesn’t change regardless of whether large files are pulled before small ones or not. And Syncthing can’t know what is more important to you: The large or small files. That’s what the pull order setting is fore, but even that is limited: If new small files are created while syncing, the large files still need to go through before a new sync is initiated and the new small files are synced.
So the solution depends on your requirements. E.g. if you have higher prio files and lower prio large files, you could separate them into two folders so they never interact/wait for a large file to be synced.
Thank you, @imsodin. I follow what you’re saying; good points. And, yes, the large file originates on the slow device. I think what I’ll do is have the person with the slow device create large files outside of the sync folder, and then slowly drop in the files throughout the day, instead of all at once. After giving it some thought, that works better than splitting our folders into two groups based on priority.
Somewhere there’s an advanced setting to allow a larger number of concurrent file syncs. (Default is 2, I would try 3). My thought is that if only 1 large file being synced, then two smaller ones could still get by. I found the setting under Advanced Configuration, under the folder’s settings, the field called “Max Concurrent Writes” - is that it?
