Alright. Here it is, after ~3.5 hours that my computer has been supposedly doing “nothing”. Lowest RAM usage values SyncThing itself reported are 117-155 MiB (or at least did so, till it started scanning again). Below are the performance graphs reported by ProcessExplorer for the last minutes.
I’m looking forward to heard your opinion.
Heap profile file: https://drive.google.com/file/d/0B6oiRZUrM7KNdE43ZUFJbERLaGc/view?usp=sharing
@calmh and @chucic, I’d be happy to hear any insights you may have, following the heap profile data I uploaded above.
Heap profile looks fine and mostly shows buffers and caches allocated for the database. However it accounts for 61 MiB, which could represent up to 120-130 megs (times two for GC plus some overhead for stacks etc). That’s a fair bit less than the 270 MiB reported by Windows above, so I’m not sure what the difference is.
Looks like fairly frequent CPU activity in the graph above - there are no axis labels (that’s … useful?) but you say “last few minutes” and I’m guessing the bar on the left has the same scale as the area chart to the right… If this is with Synctrazor we should be using filesystem notifications, so if the activity is from scanning, increase the scan interval?
jb@syno:~/Downloads $ go tool pprof syncthing-windows-amd64-v0.11.5/syncthing.exe heap-05308.pprof
Entering interactive mode (type "help" for commands)
(pprof) top
60042.12kB of 60982.84kB total (98.46%)
Dropped 1225 nodes (cum <= 304.91kB)
Showing top 10 nodes out of 72 (cum >= 26850.98kB)
flat flat% sum% cum cum%
34342.28kB 56.31% 56.31% 34342.28kB 56.31% github.com/syndtr/goleveldb/leveldb/util.(*BufferPool).Get
16032.23kB 26.29% 82.60% 16037.48kB 26.30% github.com/syndtr/goleveldb/leveldb/memdb.New
7832.06kB 12.84% 95.45% 7836.12kB 12.85% runtime/pprof.writeHeap
642.02kB 1.05% 96.50% 1407.14kB 2.31% github.com/syndtr/goleveldb/leveldb.(*dbIter).next
536.38kB 0.88% 97.38% 536.38kB 0.88% github.com/syndtr/goleveldb/leveldb/cache.(*mBucket).get
469.52kB 0.77% 98.15% 469.52kB 0.77% encoding/base64.(*Encoding).DecodeString
138.56kB 0.23% 98.38% 34480.84kB 56.54% github.com/syndtr/goleveldb/leveldb/table.(*Reader).readBlock
40.03kB 0.066% 98.44% 16077.74kB 26.36% github.com/syndtr/goleveldb/leveldb.(*DB).newMem
5.30kB 0.0087% 98.45% 474.81kB 0.78% github.com/syncthing/syncthing/internal/auto.Assets
3.75kB 0.0061% 98.46% 26850.98kB 44.03% main.folderSummary
(pprof)
Thanks @calmh!
The horizontal (time) axis in the above PE (Process Explorer) graph is about 10 minutes long. The side bars to the left indeed have the same scale as the rest of the graph, as you’ve noted.
I’m already using a Rescan Interval of 600 secs (=10 mins), instead of the default 60 secs. So I guess that the performance peek seen at the right of the graph must correspond to a rescan. I can increase this value again. However, this would not solve the large memory usage issue. What else could explain it? Is there anything else I could do to test?
How much memory does a “naked” SyncThing+Synctrazor installation require (no synced folders)?
If this is with Synctrazor we should be using filesystem notifications, so if the activity is from scanning, increase the scan interval?
If Synctrazor is monitoring the filesystem, why does it need re-scans at all?
In addition, I’ve noticed that my computer becomes less responsive when syncthing is scanning. Checking Run syncthing as a low-priority process in SyncTrazor’s settings makes it have a base priority if 6 instead of 8. But that’s a “cheap” trick, letting Windows decide when to schedule syncthing to work, instead of making syncthing itself aware to it’s surrounding. How about the following idea:
Why not check whether the system is idle or not? If it isn’t idle, use much resources than normally (without stopping syncthing’s altogether). Or checking the system load (should be fairly easy to do on Linux, donno about Windows). There must be a better way to make syncthing better aware of how much resources everybody else needs right now.
Synctrayzor does not modify the Syncthing config file, thus Syncthing will rescan as much as you tell it to. There is a new API that would allow the delay of scanning if a monitoring service is available. I think it is nearly integrated in Synctrayzor. Feel free to set the rescan interval to 86400.
It is not guaranteed that all events will be captured with the monitoring system. Therefore I’d advice to everything scan once a day.
Thanks! I’m still quite surprised though that syncthing’s memory usage (at least under Windows 7) is comparable with that of a web browser. It’s not what one might expect…
Patches are welcome, see https://github.com/syncthing/syncthing and https://github.com/syncthing/syncthing/wiki/Building The next release (v0.11.7) already contains a patch to reduce the memory usage.
Sure. Thanks! 