Prevent syncing partial files

Is there a way to delay syncing until the file is complete?

My setup:

  • I have a Pixel phone that’s in constant sync with a folder on my home server.
  • This phone automatically uploads photos to Google Photos.
  • I (manually) copy photos from my laptop to the server - sometimes from my local network, sometimes remotely.

The problem happens when I copy photos to the server remotely and my internet connection is slow. In this case Syncthing seems to copy the file to the phone before it’s been fully uploaded to the server. The phone then uploads the same file to Google Photos multiple times, at various stages of completion.

This is what the end result looks like on Google Photos:

Is there a way to tell Syncthing to wait until the file is complete before starting to sync? If not, can I maybe add a fixed (~10 minute) delay between file creation and sync start? Any other advice to solve this issue also appreciated!

The description sounds a little complicated, but from what I can understand, it doesn’t seem to be Syncthing but rather Google Photos that tries to upload the Syncthing’s tmp file (while still being copied from the computer to the phone) instead of the final image file. This is the only way I could try to explain the partial images seen in the screenshots.

This is just my speculation though. Maybe someone else will have a better idea on what’s exactly going on, or perhaps has experienced a similar problem themselves.

It sounds to me like you upload a .jpg to a server somewhere, using whatever non-Syncthing method, and then Syncthing picks it up from there and syncs it to your phone. If that’s the case, and your initial upload doesn’t use a temporary file (just a growing .jpg), there’s no way for Syncthing to know that it’s not “done” when it looks at the file and sees a .jpg of some kind.

Better to upload to a .tmp file, ignore *.tmp in Syncthing, and then rename the .tmp to .jpg when it’s done. This is roughly what Syncthing does when syncing files.

This is indeed exactly what I do yes.

What I’ve done as a workaround is to upload the files to a temporary directory, and then once they are all uploaded, move them to the Syncthing dir. This works, but it’s a bit of a hassle, that’s why I was wondering if there’s another solution.

Is it perhaps possible to delay syncing by a fixed amount of time, let’s say 10 minutes, from the moment a new file is created?

I was going to say “no, nothing like that”, but in fact there is an advanced folder option “fs watcher delay” that might work if you’re using the filesystem watcher (notifications). It sets the time in seconds from when we see a change to when we act on it. You might try increasing that significantly to see if it helps.

2 Likes

Or use periodic/mamual scans instead of watching.

2 Likes

Thanks, I’ll have a look into that!

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.