Temp files still kept despite keepTemporariesH set to 0

I have a problem with temporary files not being deleted despite setting keepTemporariesH to 0. I did not use this option before, so my understanding may be wrong, but I suppose that setting to 0 should disable keeping temporaries altogether. Is this correct?

Basically, the situation is as follows. Device A pushes some large files to Device B. While the files are still being uploaded, they get deleted/moved on Device A, which results in aborting the transfer. However, the ~syncthing~ temporary files created in the process have not been deleted.

I have tried to filter the log for relevant entries.

2021-02-07 19:48:10 Puller (folder "XXX" (gdwva-7bfwn), item "file1"): syncing: pull: no such file
2021-02-07 19:48:10 "XXX" (gdwva-7bfwn): Failed to sync 1 items
2021-02-07 19:48:10 Folder "XXX" (gdwva-7bfwn) isn't making sync progress - retrying in 1m36s.
2021-02-07 19:58:28 Puller (folder "XXX" (gdwva-7bfwn), item "file2"): syncing: pull: no such file
2021-02-07 19:58:28 "XXX" (gdwva-7bfwn): Failed to sync 1 items
2021-02-07 19:58:28 Folder "XXX" (gdwva-7bfwn) isn't making sync progress - retrying in 1m15s.
2021-02-07 19:58:32 Puller (folder "XXX" (gdwva-7bfwn), item "file2"): syncing: pull: no such file
2021-02-07 19:58:32 "XXX" (gdwva-7bfwn): Failed to sync 1 items
2021-02-07 19:58:32 Folder "XXX" (gdwva-7bfwn) isn't making sync progress - retrying in 1m4s.
2021-02-07 20:14:56 Puller (folder "XXX" (gdwva-7bfwn), item "file3"): syncing: no connected device has the required version of this file
2021-02-07 20:14:56 "XXX" (gdwva-7bfwn): Failed to sync 1 items
2021-02-07 20:14:56 Folder "XXX" (gdwva-7bfwn) isn't making sync progress - retrying in 3m5s.

Right now, it has been 12+ hours since then, and the folders themselves are 100% “Up to Date” with no issues, but the temporaries are still there with no idea when they are actually going to be deleted.

Edit: I have forced a manual re-scan, and the files are gone. This would indicate that deleting temporaries is tied to the periodic scan. Am I right? The folders are set to scan every 1 day, so this would make sense, although a potential problem could be that if periodic scanning was disabled completely, temporaries might stay untouched for a very long time, i.e. until the next restart/upgrade.

Basically, the problem that I want to solve here is that the non-tech-savvy user gets confused by visible temporary files. Because of that, I would like to either hide them, or force them to be deleted immediately on a failed download. If they are deleted only on scanning, then I would probably need to either set the scan interval to something extremely low (which is not the best solution, as we are using slow HDDs here), or maybe to use the Task Scheduler to hide/delete them periodically in addition to Syncthing.

In my personal builds, I have actually tried to modify the code, so that temporaries stay hidden even after a failure, but in this case I strongly prefer to keep using the official Syncthing binaries.

Deleting temporary files is tied to the periodic scan…

The periodic scan is still triggered when file watcher is in enabled to account for things like this.

Assuming it doesn’t fall out of sync for some reason the periodic scan shouldn’t be all that taxing on your system.

1 Like

I do not think that file watcher helps though… I have just tried to add a new file to a folder with a temp file present, and only the new file got added to Syncthing, while the temporary one was left alone. Only a full rescan seems to get rid of them.

It takes ~25 seconds to scan through the following folder.

It may not be very taxing, but setting it to 60s or similar (to make sure that the temp files are deleted almost immediately) would mean triggering the scan almost constantly, and on this system it would probably affect the overall performance too (as everything runs from the same HDD).

No it doesn’t. I was in a rush to finish my reply. By default, the periodic scans still run every hour when watcher is enabled.

So with the keep temp setting at 0, the longest it will hanger around for is an hour by default.

Is that too long for you purposes? I would not scan every minute on that folder but would set it to 10 minutes as a fair trade off if required.

Yeah, for myself, it does not matter at all, but for this user, I would prefer that they do not see temporary files ever. The best solution would be to get them either removed immediately on transfer interruption if keepTemporariesH is 0, or to somehow force them to stay hidden.

Edit: I think that I will go for a hybrid approach and just use the Task Scheduler to run a batch script to hide the temp files every minute or so.

This script

for /f delims^= %a in ('dir /a-h /b /s "~syncthing~*.tmp"') do attrib +h "%a"

takes only 1-2 seconds to run on the same folder, so there is a big difference.

I could also do

for /f delims^= %a in ('dir /a-h /b /s "~syncthing~*.tmp"') do del "%a"

or

del /a-h /q /s "~syncthing~*.tmp"

but is this 100% safe? I assume that being unhidden means that the temp file is no longer in use, but I would like to stay on the side of caution :sweat_smile:.

I just wanted to give the final update to this topic.

We have set up a task in Task Scheduler to run every minute that would delete all non-hidden Syncthing temp files. It has been working great, and only takes ~1 second to run, so the resource usage is minimal.

If anyone is interested, just set the task to do the following (extracted from the XML). Also, using the del command directly without a for loop is much faster.

<Exec>
  <Command>"%ComSpec%"</Command>
  <Arguments>/c "del /a-h /q /s "~syncthing~*.tmp""</Arguments>
  <WorkingDirectory>PATH-TO-SYNCTHING-FOLDER</WorkingDirectory>
</Exec>

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.