Large files >190GB don't get synced with untrusted device

Hi Syncthing-team,

I have got a Syncthing on RasparryPI Syncing over internet to untrusted (password protected) QNAP syncthing partner. Everything works well, but one large file (over 190GB) keeps uploading without completing for a couple of weeks now. Both devices are running the latest version of Syncthing.

Since the expecting syncing time is over 24 hours, I increased

  • Keep Temporaries (hours)

up to 240 on both sides to no effect.

I couldn’t find anything related in the logs. Google also seems to know nothing about this issue.

Please let me know what other specific information I should provide.

Cheers, Boris

Keep Temporaries only applies when the affected files haven’t been synced for more than 24 hours (by default, e.g. because the remote device wasn’t online). It doesn’t sound like that’s the case here, so changing this particular setting probably won’t do anything.

What does the GUI say when the sync fails? Does it simply restart the synchronisation from scratch? Please post screenshots of the Syncthing Web GUI showing what the situation looks like on both sides.

1 Like

Thank you for the quick reply.

On both sides we see ~390GB syncing in process (the size of our big file). The actual upload progress is not shown (progress bar or reducing amount to be synced). I understood, that the amount to be synced is updated on the file base, not the actual progress.

The uploading rate is always ~ 5MB/s, thats almost the entire upload capability of my internet.

The small files on the screenshot get synced over the time. The big file always stays.

Screenshot from the one side. Other side to go.

image

You’ll probably need to check the logs for why the transfer fails. Enable -verbose to get some more information, potentially.

Once failed, for encrypted files, we can’t reuse blocks from the temporary. (Or, if we possibly could, we haven’t implemented doing so. The short reason is that we don’t have a record of the expected hash of the encrypted block.) So an encrypted file will sync in entirety or fail in entirety, which will be a large transfer again for you.

Ok, the disconnects were causing by my German internet provider 1&1 (huge thanks for that :frowning:). 1&1 disconnects my router every 24 hours to assign a new IP address. This process can’t be stopped or delayed I was told.

It looks like this problem can’t be solved right now if using encrypted (untrusted) hosts.

Now I’m using encfs to create an encrypted view of my files and then sync it without syncthing-encryption. Will see, if it will work out for me.

Thank you for the help!

1 Like