File renames, moves.

Feature
23

Is this basically what I suggested as the first option in this post? File renames, moves. - #11 by mraneri

24

Hi Tomasz,

See below:

Best, Jon

25

Hi Mike,

I guess it’s quite similar as the deletions wont be pushed out until the scan completes without error.

In our case the main issue we had was when a directory is renamed when a scan is in progress leading to the whole directory being discarded and synced from scratch.

Best, Jon

26

Thank you very much :relieved:. I’m going to test the commit in my personal builds, and also try to do some specific testing with renames and moves in my test environment when I find some time in the next few weeks. I’ve also been planning a bit of moves and renames in one of my real folders with large video files, so I’ll probably be able to see whether there’s any positive impact there as well.

1 Like
27

Does this handle Moves AND file renames at the same time? Ideally it would be great if syncthing can detect that one file was deleted and one file with the same hash was added in ANY folder, not only the same folder. Even if the filename is different.

28

Nothing is done cross folder

29

I mean within one syncthing folder but between subfolders… if so that’s a bummer. (It would also mean subfolder renames aren’t handled as renames.)

30

I mean cross syncthing folders.

31

Okay sure. Across syncthing folders may be across OS volumes or on different storage devices.

I don’t expect moves and renames across syncthing folders but it would be nice if move and rename at the same time could be detected and handled by syncthing if the move is among a subfolder inside the same syncthing folder.

32

That is the case, commented as “taking rename shortcut” in debug for model but, in our testing, it won’t work reliably if

— the file system events for the changes span a couple of batches (presumably a timing issue anyway, I believe it’s been previously discussed)

— the directory or the files change again during the scan causing the deletion events to hit and then the changes to be discovered afterwards.

Most of the time, pausing before renaming a large directory will work and the rename shortcuts will be taken (except in the case of point 2 above)

33

I’ve been testing the code for a few hours, but I’m actually observing quite a big problem with it. Basically, it seems if there is a file that is locked in Windows due to being in use, then Syncthing will not be able to sync deletions in the folder at all until the particular file becomes unlocked.

This is a huge problem in my case, because in my systems, I’ve got a few such files that are locked most of the time due to being used by a particular piece of software. They only sync when the software isn’t running. Normally, Syncthing just fails to access them when they’re in the locked state, yet it still can sync the rest of the folder with no issues.

34

Hey Tomasz

That sounds about right as no deletions will be processed until the scans pass without error. This is a pretty hacky workaround but I can’t see a better way to hold on to the files to guarantee the rename shortcuts are taken than to hang it off a successful scan. I guess particular errors (ie file in use) could be checked and excluded instead of just using the total count. In our case we’re not that concerned with the speed of delete propagation.

I’m sure there’s a better way to do this, will take another look.

35

Understood :slightly_smiling_face:. When I was testing the code yesterday, the files were unable to sync for 5+ hours, so I eventually had to revert the changes.

36

The same experience, leading me to go back to Synology Drive Server/Client solution, which must have a Central Server…

I like the Synching decentralization structure, but the .stfolder, the .stignore and the flaw of remote rename problem have driven me away.

37

I see these as the main deficiencies of syncthing. For me the benefits still outweigh the shortcomings but yeah the moves/renames problem is real and the selective sync thing doesn’t exist.

38

And from what I understood about version 2 of Syncthing, the current and a5 least partially working renaming will be disabled:

  • Rolling hash detection of moving data is no longer supported as this effectively never helped. Instead, scanning and syncing is faster and more efficient without it.
39

No, that has never had anything to do with renaming.

40

Is anyone looking at improving the rename/move situation? Just moved 20 GB of files from one subfolder to another within the same syncthing folder and again duplicate copies were made in the new location and the files in the source path were moved to stversions. In this specific case it appeared the data was copied from the local disk and not retransmitted, but still, the files were copied, and not moved.

This topic comes up repeatedly. It’s challenged that the rename process works fine and requests for exact steps to reproduce. Exact steps are provided, and then generally there’s no response.

Anyone have ideas on how to improve this? The logic still doesn’t work right…

41

This is expected and as designed, a file moving still produces a new version, even if it’s the same.

Reason for this is that folders are not always shared just among you, and people would get confused when files disappear because someone else has moved them, but no versions were produced.

2 Likes
42

Interesting.

In all the years this topic has come up I don’t ever recall seeing that explanation.

Anyway it’s a problem for me and unwanted behavior as I rename a directory with 500GB of data and I end up with 500GB of duplicated data in my versions folder.

It’s also a little disappointing if I may be honest that so many times this has come up and people say “renames work, it shouldn’t be doing that. Write up the steps to reproduce”. And now the answer is suddenly “this is as designed expected behavior.“ Also no OS’s “trash can” works this way.

Anyway.