Scanning folders is extremely slow

Hello everyone,

I wanted to switch over from BTSync to Syncthing but Syncthings scanning of folders takes forever, and i can’t find out why.

There are three folders that should be synced over two machines. (Windows Server 2016, Windows 10) The Server-Machine is the “Master”-Device, so i created the Syncthing-Folders as read-only. Alltogether there are 330 Files in those folders which contain 90GB of data. not that much, right?

… but for some reason scanning those folders takes forever. I waited over night to see how far it gets and this was the result:

  • Folder A: Completed (~320 Files, ~50MB each)
  • Folder B: 7% (5 Files, ~8,25 GB each)
  • Folder C: 4% (2 Files, ~18,5 GB each)

I dont know exactly what is going on. Allthough it seems that Syncthing has problems with large files, take note that also Folder A took a unusual long time to complete the scan.

I tested with the latest stable-build (x64) over night. but also tested latest RCs (rc1, rc2) and also x86-Versions in the short-term. But nothing changed. (I expect to finish the scan at least within 30-60 min)

Is anyone able to bring me to the right path? I really don’t know what to do, how to get to the root of the problem. Thank you for every tipp!


Maybe worth mentioning:

  • I already connected the two machines; but they are sepperated from the firewall. I only opend port 22000 and connected the machines via their addresses: “tcp://192.168.xxx.xxx:22000”
  • I turned everything off “Nat-Traversal”, “Global Discovery”, “Local Discovery” and “Enable Relaying”
  • NTFS FileSystem
  • CPU Load is not high at all
  • Using the “Dark” Skin (hopefully the GUI has not anything to do with the backend) :stuck_out_tongue:

[OCF6B] 11:55:48 INFO: syncthing v0.14.41-rc.2 "Dysprosium Dragonfly" (go1.9.2 windows-amd64) teamcity@build.syncthing.net 2017-11-17 14:46:45 UTC
[OCF6B] 11:55:48 INFO: My ID: XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX
[OCF6B] 11:55:49 INFO: Single thread SHA256 performance is 127 MB/s using minio/sha256-simd (110 MB/s using crypto/sha256).
[OCF6B] 11:55:50 INFO: Hashing performance with weak hash is 103.69 MB/s
[OCF6B] 11:55:50 INFO: Hashing performance without weak hash is 118.15 MB/s
[OCF6B] 11:55:50 INFO: Weak hash enabled, as it has an acceptable performance impact.
[OCF6B] 11:55:50 INFO: Starting deadlock detector with 20m0s timeout
[OCF6B] 11:55:51 INFO: Ready to synchronize "BACKUP_1" (7rzpf-qcnof) (readonly)
[OCF6B] 11:55:51 INFO: Ready to synchronize "BACKUP_2" (lryx5-jc5nv) (readonly)
[OCF6B] 11:55:51 INFO: Ready to synchronize "BACKUP_3" (f2jyc-zbtqu) (readonly)
[OCF6B] 11:55:51 INFO: Send rate is unlimited, receive rate is unlimited
[OCF6B] 11:55:51 INFO: Rate limits do not apply to LAN connections
[OCF6B] 11:55:51 INFO: TCP listener ([::]:22000) starting
[OCF6B] 11:55:51 INFO: KCP listener ([::]:22020) starting
[OCF6B] 11:55:51 INFO: Device XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX is "CLIENT" at [tcp://192.168.XXX.XXX:22000]
[OCF6B] 11:55:51 INFO: Device XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX is "SERVER" at [dynamic]
[OCF6B] 11:55:51 INFO: Anonymous usage reporting is always enabled for candidate releases.
[OCF6B] 11:55:51 INFO: GUI and API listening on [::]:8384
[OCF6B] 11:55:51 INFO: Access the GUI via the following URL: http://127.0.0.1:8384/
[OCF6B] 11:55:51 INFO: Automatic upgrade is always enabled for candidate releases.
[OCF6B] 11:55:51 INFO: Automatic upgrade: couldn't find a release to download
[OCF6B] 11:56:10 INFO: kcp://0.0.0.0:22020 detected NAT type: Port restricted NAT
[OCF6B] 11:56:10 INFO: kcp://0.0.0.0:22020 resolved external address kcp://XXX.XXX.XXX.XXX:22020 (via stun.counterpath.net:3478)
[OCF6B] 11:56:16 INFO: Completed initial scan of readonly folder "BACKUP_1" (7rzpf-qcnof)

Scanning will happen as fast as your CPU can hash, and as fast as your hard drive can read. So I suggest you check the usage of each of these and see which one is maxed out.

as mentioned: cpu is low … and I can’t imagine that a normal 7200rpm drive is that slow. (no other accesses to the hdd)

Can you check the load and read speeds of the hdd to make sure that isn’t the bottleneck? The times you report would indeed not be normal, but failing hard drives aren’t unheard of.

I can imagine all sorts of things, best if we talk about facts, and not imaginary things.

You should be able to see the drive utilization somewhere in Windows. In win10 it’s in the task manager, not sure where it’s in windows server 2016.

Note too that scanning multiple folders in parallel will mean a significant amount of disk seeks, so don’t expect just linear read speeds.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.