Doing a test with a large folder of books, and when new books are added to the folder and subfolders, the main folder is renamed so we know the date of the current library.
Now this could be for anything really, but when the main folder is renamed, it looks like Syncthing treats that has a delete and create of the files, so the whole structure is re-synced again. Now when the size of the main folder is around 10G even over the wifi it can take a while.
Maybe when doing the scanning of the new folder, checking the hashs they could be compared to what is already known, to see if all that has to happen is the file needs to be moved not recopied over the network.
Ok. With just one of the machines connected (server) I have run the syncthing with the option provided.
Now on the command line on the folder doing a mv source dest
Did the rename and got flooded with messages like.
[67ZEU] 12:06:38 INFO: Puller (folder “Books”, file “Library2015.3.28a/Susan’s Books/Momotaro.pdf”): pull: no available source device
Where Library2015.3.28a is the old folder. Renamed it to Library2015.3.28b. This is version 0.11 under Debian Linux.
Then gets this
[67ZEU] 12:08:17 INFO: Puller (folder “Books”, file “Library2015.3.28a/Fiction2015.3.28/R/Robin Hobb/Golden fool - Robin Hobb.epub”): pull: no available source device
[67ZEU] 12:08:17 WARNING: Folder “Books” isn’t making progress - check logs for possible root cause. Pausing puller for 1m0s.
I’ll leave it to finish the scanning, and will check back soon… approx 10,000 files to scan and 8G
Here is some more now that it has finished scanning…
[67ZEU] 12:22:38 INFO: Puller (folder “Books”, dir “Library2015.3.28a/Fiction2015.3.28/D”): delete: remove /Users/fran/Books/Library2015.3.28a/Fiction2015.3.28/D: directory not empty
Also I notice that while it is scanning the number of files and size goes up.
I’m re-running it again as I had a client on my Mac that was still running in the before run, and this time I have renamed from b to c. This time when there is only the master on line, there are no messages on STDOUT.
Starting up the client for the 6.8G and 8649 files the following happens
[67ZEU] 12:46:40 INFO: Puller (folder “Books”, dir “Library2015.3.28b/Fiction2015.3.28/R/R Karl Largent/The Witch of SixKill (multi-html) - R Karl Largent/files”): delete: remove /Users/fran/Books/Library2015.3.28b/Fiction2015.3.28/R/R Karl Largent/The Witch of SixKill (multi-html) - R Karl Largent/files: directory not empty
[67ZEU] 12:46:40 INFO: Puller (folder “Books”, dir “Library2015.3.28b/Fiction2015.3.28/R/R Karl Largent/The Witch of SixKill (multi-html) - R Karl Largent”): delete: remove /Users/fran/Books/Library2015.3.28b/Fiction2015.3.28/R/R Karl Largent/The Witch of SixKill (multi-html) - R Karl Largent: directory not empty
[67ZEU] 12:46:40 INFO: Puller (folder “Books”, dir “Library2015.3.28b/Fiction2015.3.28/R/R Karl Largent”): delete: remove /Users/fran/Books/Library2015.3.28b/Fiction2015.3.28/R/R Karl Largent: directory not empty
and this is from the web interface on the client
So you can see that it wants to re-sync all of the files.
Not sure if it would help this case, cause I don’t get the errors the poststarted sees, but I took the liberty to do a similar test, but then with 10GB of Pictures (JPG and RAW). On both sides I started syncthing with STTRACE=model and renamed a directory on one node (node A).
What I do see is that node A and B syncs their databases but on node B I see it copying ALL the files into the .stversions directory as if the files where deleted/changed, then it takes a rename shortcut for all the files (from old directory to the new one) and finally it deletes all the pictures in the original directory (including the directory).
So not much network traffic here, but when it’s capable of seeing the rename, a copy off all the contents is just a waste of time and diskspace (unless you turn off versioning). Shouldn’t it detect first if it’s a rename before copying? If it’s a rename than don’t bother copying it in the .stversions directory.
edit: What version are you using Fran? I am using the latest v0.11.1 on Ubuntu 15.04 and Kubuntu 15.04
I have the same issue than Fran_Firman on synology arm V5 dsm 5.2 syncthing 12.15 and on another synology x64 dsm 5.2 syncthing 12.15.
When you rename a file an other one is created on the other side and all the files inside are downloded again! And the content of the old file is deleted.
excuse me for my english i’m french.
The first synology who syncthing installed, has a master record that is reflected in the second synology folder that has the slave … and when I rename a folder on syno A the folder is created on syno B but the content is not just moved,but re-downloded again and the files who should rename whas deleted.