I have a 2TB drive filled with music. I have an identical 2TB drive in a separate computer on the network that I use as a backup. I manually copied all of the contents of the first drive to the backup a few weeks ago. Since then the only changes made to the original involved deleting a few hundred MB of folders and files.
Today I decided to try out Syncthing and set it up to sync these two drives as a test. I set the first drive as the master so I would not lose any changes. Unfortunately, Syncthing was not smart enough to realize the only difference between the two drives was a few hundred MB of deleted files and folders. It detected almost every directory was deleted, so it began deleting them and recreating them all. The result was an attempt to transfer several hundred GB which was taking several hours over 100Mbps LAN, so I had to cancel it.
I will be happy to provide any logs or information I can to help debug this problem. Both computers are running Fedora 21 Workstation Syncthing v0.11.1, Linux (64 bit).
That shouldn’t happen (and normally doesn’t, in my testing). The exception would be something like if the rename fails for whatever reason, resulting in a delete and a following transfer.
I can’t think of a reason the non changed files would be transferred.
Note though that it’ll display the full amount of data as “out of sync” - that doesn’t mean it’ll get sent over the network.
It showed “out of sync” by a few hundred GB on the backup server. On the primary it showed out of sync by only a few MB. However, when I clicked on the list of out of sync files it showed folders and files deleted and recreated that were not changed. That is why I was confused.
It showed upload and download network activity, although I don’t remember how much because I did not screen shot it. Both drives were still in the “Scanning” state. Does that mean it is still hashing all of the drive contents? I assume that is going to take a very long time for large drives like this to complete. Is it okay to share folders before they are done scanning or should I wait for the scanning process to complete?
I should also mention these drives were recently moved from Windows systems, so they are both formatted in NTFS. The files and folders that were deleted were a few music albums, but also the now unnecessary Windows system files: thumbs.db, desktop.ini and $RECYCLE.BIN, System Volume Information and msdownload.temp folders.
Syncthing will scan the folders and during that time it might/will show that vast amounts are out of sync - don’t be concerned. It will compare the content and most likely won’t transfer any actual data from your set if both sides are the same.
You can let it finish scanning first, or share it from the beginning… I personally liked to wait for both sides to stop scanning and compare the total file count/size myself before setting up the share.
Depending on your machine the initial scan for a few hundred GB could take hours (decent machine) or days (RaspberryPi).