Transfer rates drops dramatically after 15 minutes

Everything about my setup seems fine. The issue is, if I drop a large number of files, about 1000, into a sync folder, the file copy works well for a while, then the rate drops dramatically. For example, after about 300 files, it just crawls along from there. Everything seems fine otherwise.

I am sourcing the files from a Macbook Pro (2021 model) at home via router/cable internet to an Ubuntu VM destination at Digital Ocean. I’ve checked all the obvious things, e.g. the Syncthing version matches, the remote address is not a “relay”, there is no logged errors, the CPU and memory loads are negligible, and so on.

FYI, the files are image files of modest sizes, typically 3MB each. The network speed shown on the Macbook Activity Monitor roughly matches the speed shown in the Syncthing GUI, roughly averaging 50 KB/sec which seems incredibly slow to me. Testing separately at the same time by manually copying files (with “scp”), I get roughly 5 times the copy speed using scp. It seems to me that there’s a rate limiter active within Syncthing to slow down the operation.

The attached screenshot shows the performance on the VM (the Digital Ocean “droplet”). You can see the hump followed by the flatline.

1 Like

I tried this, for science. I set up a small VM on DigitalOcean (Basic, Premium AMD, 1 vCPU, 1GB), installed Syncthing from our APT repo, and shared a folder with a couple of thousand photos from my MacBook. Then I went for a walk with the dog, which took about 20 minutes. When I came back and checked, the rate had dropped to about zero but then immediately picked up again. The computer wasn’t asleep, the connection wasn’t dropped, but I suspect macOS had throttled down Syncthing (as a background process) to almost no CPU time.

I’ll let it run for a while longer while I’m using the computer, but I’m not really expecting to see a problem, apart from maybe macOS power management, to be honest…

You could also check whether similar issues show up for other file transfer programs. Home routers sometimes have a weak CPU or bad thermal management. So especially if it has to do port forwarding, the router might just be hitting some limit and killing your bandwidth. Maybe do a speed test right after you notice the transfer rate dropping?

Update: :person_shrugging:

I don’t think it’s a systematic problem. (The sawtooths in the graph are just DO jankyness, reloading moves them around but doesn’t make them go away…)

Thank you, all, for your time spent on this. This morning, I can’t repro it. I was able to transfer 1223 image files in about 20 seconds. There are a few differences in the setup.

  • I tested with a new sync folder.
  • I disabled all cron jobs on both ends.
  • On the DO end, I don’t have additional fan-out nodes. The previous sync folder had a number of nodes which it was trying to push to, half of them offline or possibly sleeping.
  • The syncthings have possibly been restarted without “nice” (but not 100% sure).

As you can see, I have a few variables to try to isolate.

1 Like

I was particularly suspicious of item 3 I mentioned, the additional fan-out nodes.

I did another test this morning. I set up two subscribers to the DO node, both being local laptops. Then I turned off one of the laptops so that it’s disconnected. Then, dropping a lot of files into the remaining laptop, they transferred to the DO node very quickly. I was expecting that the DO node would have assigned all available threads to pushing files to the down laptop, get tied up with timeouts, and slow to a crawl. That didn’t happen.

2 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.