Hello there. I am trying to sync folder that have like 500k files, a lot of small ones, and about 200GB of data, and problem is that every scan of these files takes ages (hours) and a lot of cpu time, while i’ve seen that syncthing uses file data + size for it’s check, so should be a lot faster. It’s nvme ssd (NM790), and copying that folder/compressing etc. is very fast, in minutes. Heck, even winmerge does it’s scan in couple of minutes.
Thing that adds to that problem is that continous scanning of files does scan WHOLE folder on single file change - why? It’s like i touch one file and it goes for hours to rescan again…
Another thing is syncing - it also literally takes ages (again, hourse), while doing smb/sftp etc takes minutes - looks like small files are very cumbersome for syncthing - is there any reason for that?
Newest syncthing on all devices, which are on windows 10/11 without any special setup etc., mix of 1 and 10gbit nics, lan+wan syncing test. HW is on newest am5 platform with 9950x3d, 96gb of ddr5 ram.
If you need any more information - i can provide, just tell me what
Thanks in advance!
During a scan, Syncthing is hashing the files it is scanning. Moving files with SMB or SFTP doesn’t do that.
Please share the specific CPU. Sometimes the “latest” build of Syncthing turns out not to be — please share the specific Syncthing version on this Device.
What is your CPU load like during hashing?
Have you made any changes to Syncthing’s configuration for performance? Take a look at Configuration Tuning — Syncthing documentation . Consider testing performance after setting setLowPriority to false (allows Syncthing to use more CPU) and setting hashers to a larger figure — I’d start with the total number of CPU cores and see how that works.
It likes to eat whole cpu core, so i suppose it uses that cpu cycles. I mean hashing can be slow, but not THAT slow, like 10-20MB/s average, while i can read and hash in other software same folder with usually 1GB/s+ speeds (depends which files it hash in current time). Problem is also with “Watch for Changes” - whenever i touch one file it tries to hash check whole root folder - why? Touching a 1KB file shouldn’t invoke hasing 500000 other files.
i typed my specs in 1st post - 9950x3d. Syncthing version:
v2.1.2, Windows (64-bit Intel/AMD)
“Hafnium Hornet”
Build 2026-06-26
And yes - i understand how it works. I remember i tuned stuff, but that was AFTER i noticed such low performance in any set profile like scheduled scan/watch scan, so it’s not tied to that knobs i modified.
I suggest just putting 1m files with random sizes (from /dev/random i.e.) totaling 200GB and trying to synchronise between 2 pcs - you will see how slow it goes.
Thanks in advance!