Everything about my setup seems fine. The issue is, if I drop a large number of files, about 1000, into a sync folder, the file copy works well for a while, then the rate drops dramatically. For example, after about 300 files, it just crawls along from there. Everything seems fine otherwise.
I am sourcing the files from a Macbook Pro (2021 model) at home via router/cable internet to an Ubuntu VM destination at Digital Ocean. I’ve checked all the obvious things, e.g. the Syncthing version matches, the remote address is not a “relay”, there is no logged errors, the CPU and memory loads are negligible, and so on.
FYI, the files are image files of modest sizes, typically 3MB each. The network speed shown on the Macbook Activity Monitor roughly matches the speed shown in the Syncthing GUI, roughly averaging 50 KB/sec which seems incredibly slow to me. Testing separately at the same time by manually copying files (with “scp”), I get roughly 5 times the copy speed using scp. It seems to me that there’s a rate limiter active within Syncthing to slow down the operation.
The attached screenshot shows the performance on the VM (the Digital Ocean “droplet”). You can see the hump followed by the flatline.
I tried this, for science. I set up a small VM on DigitalOcean (Basic, Premium AMD, 1 vCPU, 1GB), installed Syncthing from our APT repo, and shared a folder with a couple of thousand photos from my MacBook. Then I went for a walk with the dog, which took about 20 minutes. When I came back and checked, the rate had dropped to about zero but then immediately picked up again. The computer wasn’t asleep, the connection wasn’t dropped, but I suspect macOS had throttled down Syncthing (as a background process) to almost no CPU time.
You could also check whether similar issues show up for other file transfer programs. Home routers sometimes have a weak CPU or bad thermal management. So especially if it has to do port forwarding, the router might just be hitting some limit and killing your bandwidth. Maybe do a speed test right after you notice the transfer rate dropping?
I was particularly suspicious of item 3 I mentioned, the additional fan-out nodes.
I did another test this morning. I set up two subscribers to the DO node, both being local laptops. Then I turned off one of the laptops so that it’s disconnected. Then, dropping a lot of files into the remaining laptop, they transferred to the DO node very quickly. I was expecting that the DO node would have assigned all available threads to pushing files to the down laptop, get tied up with timeouts, and slow to a crawl. That didn’t happen.