Initial scan stuck at 0%

Support
1

Hello Syncthing community!

I am using Syncthing (package from Safihre/SynoCommunity) on a Synology nas DS216.

Initial scan of a ~42GB folder full of stl files is stuck at 0%, remaining time estimated at ~1 month.

chrome_iPgUomNNzF
chrome_iPgUomNNzF1153×383 24.9 KB

The only change I applied is to the inotify limit according to this: https://docs.syncthing.net/users/faq.html#inotify-limits and proceeded to restart Syncthing.

Same folder, same size was scanned in less than 45 minutes on a Windows laptop with most rcent SyncTrayzor package and Syncthing update.

Is there a way to fix this and get it to work? A test with a different folder and few files worked flawlessly.

2

Check device resource usage (e.g. memory usage and swapping). If nothing stands out, enable scanner debug logging (Actions -> Logs) and post the logs.

3

Thank you for your fast reply. Here is what I get, I see nothing unusual on the screenshot.

Log:

2020-10-02 22:33:37 My ID: 
2020-10-02 22:33:38 Single thread SHA256 performance is 17 MB/s using crypto/sha256 (17 MB/s using minio/sha256-simd).
2020-10-02 22:33:39 Hashing performance is 16.04 MB/s
2020-10-02 22:33:39 Overall send rate is unlimited, receive rate is unlimited
2020-10-02 22:33:39 Using discovery mechanism: global discovery server https //discovery.syncthing.net/v2/?noannounce&id=
2020-10-02 22:33:39 Using discovery mechanism: IPv4 local broadcast discovery on port 21027
2020-10-02 22:33:39 Using discovery mechanism: IPv6 local multicast discovery on address xxx
2020-10-02 22:33:39 QUIC listener ([::]:22000) starting
2020-10-02 22:33:39 ...
2020-10-02 22:33:39 TCP listener ([::]:22000) starting
2020-10-02 22:33:39 Relay listener (dynamic+https //relays.syncthing.net/endpoint) starting
2020-10-02 22:33:39 Ready to synchronize "3D models" (qsayb-typhd) (sendreceive)
2020-10-02 22:33:39 GUI and API listening on [::]:8384
2020-10-02 22:33:39 Access the GUI via the following URL: http //127.0.0.1:8384/
2020-10-02 22:33:39 My name is "srvai-nas"
2020-10-02 22:33:39 Device xxx is "nakwada-legion" at [dynamic]
2020-10-02 22:33:39 Device xxx is "nakwada-newdesktop" at [dynamic]
2020-10-02 22:33:53 quic //0.0.0.0:22000 detected NAT type: Symmetric NAT
2020-10-02 22:33:53 quic //0.0.0.0:22000 resolved external address quic //xxx (via stun.syncthing.net:3478)
2020-10-02 22:33:54 New NAT port mapping: external TCP address xxx to local address 0.0.0.0:22000.
2020-10-02 22:33:54 Detected 2 NAT services
2020-10-02 22:34:54 Established secure connection to xxx at xxx:11703/relay-client/TLS1.3-TLS_AES_128_GCM_SHA256
2020-10-02 22:34:54 Device xxx client is "syncthing v1.9.0" named "nakwada-legion" at xxx:11703/relay-client/TLS1.3-TLS_AES_128_GCM_SHA256
2020-10-02 22:39:05 Joined relay relay //xxx
2020-10-02 22:44:18 quic //0.0.0.0:22000 resolved external address quic //xxx(via stun.syncthing.net:3478)
2020-10-02 22:46:10 Connection to xxx at xxx:11703/relay-client/TLS1.3-TLS_AES_128_GCM_SHA256 closed: reading length: EOF
2020-10-02 22:49:31 quic //0.0.0.0:22000 resolved external address quic //xxx (via stun.syncthing.net:3478)
2020-10-02 23:03:50 Sent usage report (version 3)
2020-10-03 13:13:35 Enabled debug data for "scanner"
2020-10-03 13:13:37 Walk qsayb-typhd [] current progress 56910874/42216853501 at 0.0 MiB/s (0%)
2020-10-03 13:13:38 Walk qsayb-typhd [] current progress 56910874/42216853501 at 0.0 MiB/s (0%)
2020-10-03 13:13:40 Walk qsayb-typhd [] current progress 56910874/42216853501 at 0.0 MiB/s (0%)
2020-10-03 13:13:42 Walk qsayb-typhd [] current progress 56910874/42216853501 at 0.0 MiB/s (0%)
2020-10-03 13:13:44 Walk qsayb-typhd [] current progress 56910874/42216853501 at 0.0 MiB/s (0%)
2020-10-03 13:13:46 Walk qsayb-typhd [] current progress 56910874/42216853501 at 0.0 MiB/s (0%)
2020-10-03 13:13:48 Walk qsayb-typhd [] current progress 56910874/42216853501 at 0.0 MiB/s (0%)
2020-10-03 13:13:50 Walk qsayb-typhd [] current progress 56910874/42216853501 at 0.0 MiB/s (0%)
2020-10-03 13:13:52 Walk qsayb-typhd [] current progress 56910874/42216853501 at 0.0 MiB/s (0%)
2020-10-03 13:13:54 real to hash: random file

chrome_cANDjpBsY4
chrome_cANDjpBsY4767×549 22.1 KB

4

Does the log go on and on with just stagnation progress lines or are there more “real to hash” lines? Which could answer if it’s extremely slow or completely stuck.

What kind of filesystem is the data on? It looks like you have a lot of small files, which means if filesystem calls were really slow, that might be significant.

5

Yes, it goes on with stagnation progress lines. There are more real to hash lines every now and then however.

The data is stored on a ext4 formated drive. I do have a significant amount of small files I could ignore. Is it possible to reject one specific subfolder and its entire content from scanning/syncing?

EDIT: I found how to do it, using .stignore file. Ignoring the folder containing loads of tiny files solved my issue, scanning is progressing and already at 7% as of now.

Thank you for your time and help, it pointed me in the right direction!

6

This is a not particularly fast system, and with both lots of small files and the database on the same (spinning disk) storage as those files I think terrible performance is expected.

7

May I ask what filesystem would be best in my usecase? I can move the db to a network drive (over Gigabit LAN), if that helps.

8

I don’t think a different filesystem will help, and I also don’t think you can keep the database on network storage.

9

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.