Non-Initial Folder Scans Seem Slow

Hello,

Continuing to love the Syncthing lifestyle, etc etc etc - thanks all for a wonderful tool!

I’m just looking at a rather silly setup I’ve got running: it’s running macOS on an (Intel) 2013 Mac Pro, with about 20 Syncthing shared folders defined.

Most of those folders live on a RAID 0 array of 11 SSDs, connected via Thunderbolt. Total Syncthing data size is about 30TB, in 670000 subfolders, across 5.1 million files.

Everything generally works lovely. Sync happens. Everyone’s happy.

However, when I need to restart the system, the scan time for these folders seems very high.

For example, Syncthing has now been running for about 4 hours; 6 of the Shared Folders are still doing their scan since Syncthing started. To pick an example of these folders: one is 1.7TB, 13000 subfolders and 120000 files - it’s 36% of the way through this scan, and says it has about 6h remaining.

The last completed scan on this folder was earlier this morning; no data will have changed in the folder since then. Syncthing is currently consuming about 230% CPU usage.

So…

Does Syncthing rehash all the data during non-initial scans? I assumed it was comparing sizes and timestamps to determine whether files need re-hashing - but the level of activity going on at the moment suggests it’s much more than a few million filesystem lookups…

Thanks!

1 Like

Since it gives you a percentage it is rehashing data, which it indeed only does when (it thinks) something has changed on disk. Is it a normal HFS/APFS filesystem or are you doing something bizarre and unusual?

1 Like

Hi Jakob,

Thanks for the quick reply. These folders are on a standard APFS filesystem (which itself is on top of Apple’s CoreStorage RAID - though I doubt that makes any difference to Syncthing). All six folders currently scanning are displaying a percentage progress.

I have performed a minor macOS update since this morning, and updated Syncthing from 1.22.0 to 1.23.1 - but otherwise, the folders should be largely unmodified.

1 Like

I’ll let the scan complete, then try restarting again and see what happens. It might just be a freak occurrence…

1 Like

Yeah, dunno, sorry.

1 Like

Ok - restarted Syncthing after the scan had completed - the subsequent scan worked as expected.

I then went to this system’s sister system, and performed the same macOS and Syncthing updates, checking the scan between each step - everything worked as expected.

Must have just been One of Those Things…

Thanks anyway!

2 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.