Syncthing is not synking folder with more than 1 TB

The company where i work have 1 NAS where collaborators work with many files and folders(Files: 2 393 126 | Folders: 498 576 | Size: 1,02 TB) I configured a server in a virtual machine with the NAS mounted and syncthing installed (srv-backup) Then i configure a second backup server in a raspberry pi 3B+ with syncthing installed and a external hard drive mounted on linux (ntsf).

All the folders are being synced but the folder with +1TB is not synced.

In the second screenshot the folder appears “Paused” but i just paused before taking the screenshot. It was syncing for 2 weeks.

what could be the problem?

Was there no progress in the number of out of sync items? Initial sync of such much data on a low-powered device like a pi can take a while (even if you “pre-synced” the data, it still needs to hash and compare all the data).

Thanks for your reply. I didn’t do a “pre-sync” and it did sync some files in the beginning.

it is best to do a pre-sync?

Not necessarily, no - that was just a precautionary note as sometimes people don’t understand why it isn’t immediately up-to-date after pre-syncing.

So it doesn’t do anything now, i.e. the number of out-of-sync items doesn’t change, there is no activity in the out-of-sync list (e.g. a huge file being downloaded or similar)?

There is no apparent activity in the raspberrypi for that particular folder

Then activate model debug facility (actions->logs) and then unpause the folder and post the ensuing log and check the resource usage.

2019-06-04 13:42:54 Enabled debug data for "model"
2019-06-04 13:43:00 progress emitter: bytes completed for d7397-fm6td: 0
2019-06-04 13:43:00 model@0xc0001562c0 NeedSize("d7397-fm6td"): {0 0 0 0 0 0 [] 0}
2019-06-04 13:43:00 progress emitter: bytes completed for fk2cp-dseky: 0
2019-06-04 13:43:00 model@0xc0001562c0 NeedSize("fk2cp-dseky"): {0 0 0 0 0 0 [] 0}
2019-06-04 13:43:00 progress emitter: bytes completed for fkssh-xxyqw: 0
2019-06-04 13:43:00 model@0xc0001562c0 NeedSize("fkssh-xxyqw"): {0 0 0 0 0 0 [] 0}
2019-06-04 13:43:00 progress emitter: bytes completed for hdvdl-vnipa: 0
2019-06-04 13:43:00 model@0xc0001562c0 NeedSize("hdvdl-vnipa"): {0 0 0 0 0 0 [] 0}
2019-06-04 13:43:00 progress emitter: bytes completed for xjczg-gghz2: 0
2019-06-04 13:43:00 model@0xc0001562c0 NeedSize("xjczg-gghz2"): {0 0 0 162 0 0 [] 0}
2019-06-04 13:43:09 Failed to exchange Hello messages with BBZYHNW-3E74IJN-XLBIJBR-DIHYIZJ-JNOOA6I-2KK5SSH-ZG2VPAK-XDTN2AC at 192.168.1.6:59240-192.168.1.104:22000/tcp-client/TLS1.2-TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305: read tcp 192.168.1.6:59240->192.168.1.104:22000: i/o timeout

RAM and CPU utilization as reported by syncthing:
RAM Utilization >> 87,8 MiB
CPU Utilization >> 30%

the server (raspberrypi) appears to be having problems comunicating with other server (srv-backup). So a did a restart on the raspberrypi

It’s possible the underlying problem is indeed about the connection, but have a little patience: That log covers less than 20s. On unpausing the folder, it needs to check the data for changes. While that is in general a fast process, when happening on a external HD on ntfs on a pi for 1TB, that will take a moment. You will get a debug log line containing “initial scan of”. Wait for that, then there should be something about pull and that’s when it gets interesting (if it’s indeed connection, it will say something about no device available).

So, After a few trys i finally decided to post the last result from pausing folder share and activating model log.

The result can be seen in the link: https://pastebin.com/raw/Py4fk2ZX

Probably i will give up from raspberry pi and use a more robust PC for the synchronization because raspberry pi apparently lacks the capability for large size folder synchronization.

It’s got nothing to do with capabilities and debug logging wouldn’t have been needed because the issue is simple: Permissions are missing. There’s tons of errors like this (which are logged at info level and show up as failed items in the web UI):

2019-06-06 17:02:34 Puller (folder "DISCO_IU" (...), file "..."): creating directory: chmod ...: operation not permitted

Ensure that the user Syncthing is running as has the required permissions on the shared directories and then try again.

1 Like

hmmm, this is strange because

The external hdd is monted in fstab file with full permissions:
UUID=94BA918FBA916F0C /data/backup/seagate_backup_plus_drive ntfs defaults,auto,umask=000,users,rw,nofail 0 0

And a simple ls -la confirm the full access:
pi@raspberrypi:/data/backup $ ls -la
total 12
drwxrwxrwx 3 pi root 4096 mai 21 15:45 .
drwxrwxrwx 3 pi root 4096 mai 21 15:45 ..
drwxrwxrwx 1 root root 4096 mai 21 16:07 seagate_backup_plus_drive

Only the owner of a file can change it’s permission, and Syncthing hopefully isn’t running as root. If you don’t want to sync permissions (and it looks like you don’t given that setup) you can check the ignore permission setting for all folders.

1 Like

You are referring to this option, right?

I will give this a try.

1 Like