Syncthing hogs system resources

Hi,

I’m using SyncTrazor version 1.0.16 with Syncthing version 0.11.2 (should be the latest) on a Windows 7 Pro x64 machine. I’ve been using it for quite a while now to synchronize several hundred GB of files between my two computers.

Even though I marked “run syncthing as a low priority process” in SyncTrazor’s syncthing settings, and even though the program has long finished scanning my drive for the first time, it keeps using a lot of I/O, as well as a lot of RAM. It’s memory usage is comparable to that of Firefox with several open tabs, as can be seen below. These cause my machine to be noticeably less responsive whenever syncthing is working.

How could it’s RAM and disk usage be reduced?

Thanks,

Shlomi

PS I set Rescan Interval=600s on all synced directories.

There should be no I/O unless it’s scanning or syncing, so that’s unclear. Likewise RAM usage should shrink when idle…

I had that issue with SyncTrazor as well. Because it was on my wife’s computer, I couldn’t spend time troubleshooting, but installing syncthing-gtk and using the ‘low’ priority option seemed to make it usable again for a ~260Gb folder.

Scott

Interesting.

@shlomiag can you confirm whether it happens when syncthing.exe is run on its own, without SyncTrayzor? (You’ll find syncthing.exe in C:\Users\<You>\AppData\Roaming\SyncTrayzor).

The process’s priority relates to how much CPU time it will get, not to how much memory it’s allowed to use.

Hi,

Thank you all for your responses. I’ve closed SyncTrazor and executed syncthing.exe as @canton7 suggested. After completing it’s initial directory scan, the program reported RAM Utilization to be 372 MiB (!), having low CPU & I/O as expected. Process explorer reports similar figures.

Why does syncthing keep using so much RAM? Is there anything I could do to help analyzing this problem? e.g., uploading syncthing logs for the community to review?

Shlomi

Give it a few minutes and I expect it will not. Close the gui / browser for good measure.

Nope. About an hour and still around 300 MB of memory.

Are you looking at what Syncthing reports itself, the “Working set” column in your screenshot, or something else? There’s a million different ways to measure “memory usage”, and most of them are not relevant. :wink:

I’m looking at all of them, and they’re all pretty similar. syncthing.exe consumes 297 MiB per it’s own report, 380 MB of “private bytes” and 317 MB of “working set” (as per process explorer), after over an hour of (nearly idle) uptime.

What’s the total data set syncthing is handling? In terms of GB and number of files?

(For comparison, my syncthings idle at around 80 MB with ~520 GB in 150k files being synced. Loading the GUI brings it up to 130 MB. Leaving it alone for 5-10 minutes takes it down again, when Go’s garbage collector + scavenger has run.)

I’m syncing about 500GB in about 157k files, very similar to @calmh. They’re divided between 5 synchronized directories.

I’ve done this again, just to clarify:

  1. I verified that SyncTrazor is down.
  2. Launched syncthing.exe from command line, and closed the browser window it opens.
  3. After about half an hour (synthing long being idle), the program itself reports 295 to 323 MiB under “RAM Utilization”, while process explorer reports numbers between 300 MB to over 400 MB, depending on the exact time and column you’re looking at.

How much memory should syncthing be consuming? How could one calculate it, given the directory & files count & sizes?

I’ve just copied 3.5 GB of personal information from my cellphone to my computer. Below is what ProcessExplorer reports in the following minutes about syncthing resource usage, and about it only.

I think it is little more complicated and depends also on:

  • architecture (windows, linux, …)
  • instruction set (32bit, 64bit, …)
  • size of RAM - Adaptive database cache size (#1742)

My config:

NAS: Intel Atom, 1 GiB RAM, Debian 32bit: 67 GiB in 27,5k files in 15 folders: Database block cache capacity 15891 KiB, GUI reports 122 MiB RAM PC: Intel Core2Duo, 4 GiB RAM, Windows 7 64bit: 42,5 GiB in 27k files in 13 folders: Database block cache capacity 64964 KiB, GUI reports 157 MiB RAM

NAS share basically same folders as PC only with 2 more containing bigger files.

Well, the machine I’m using is a laptop with Intel i5-2410M, 4 GB of RAM, Windows 7 x64. @chucic, does that make the figures I mentioned earlier seem reasonable from your perspective?

From the perspective of a user relatively new to syncthing, I think that it’s a great software. But one might expect a software that’s supposed to “sit in the background and do nothing” (when there’s nothing to sync) to require a little less resources in it’s idle time.

No, it doesn’t. There is big difference between amount of shared data, but calmh’s ram usage blow that from table. I guess you would have to do some profiling to figure this out.

Guess you meant that there is not a big difference between amounts of data shared.

And how could I do that?

To profile, set the environment variable STHEAPPROFILE=1. Syncthing will run (very) slowly, and keep a file heap-<process-id>.pprof with a running profile of allocations. Send this in for analysis. :)

The graphs above show lots of CPU usage. That implies syncthing is doing something. My memory measurements are from idle state - i.e. everything in sync, not scanning.

Hm, I wrote it really badly. I meant that there is big difference between my dataset and yours, but because calmh has similar dataset as you, any comparison from that perspective is misleading.

To see if syncthing is “idle” or not according to itself, you can run it as syncthing -verbose. It’ll report things like Folder "Photos" is now scanning and ... is now idle etc to the log/console.