Synchronizing Millions of files


I want to synchronize my Hard drive to another computer THere are Millions of files. This take a long time, eventhough both hard drives are the same. Once scanned synchronized, will this go faster? Or does the tool always take so long for scanning? Or once scanned it will go much faster and changes will be protocoled?

The first scan will need to read the contents of every single file, and process all of the data (“hashing”). This is much more time consuming than subsequent scans.

ok, is there a value I can work with? Like it will be 10 times, or 50 times, 100 times faster?

Is there a data base, that will be built?

The file metadata is all held in a database, but I can’t really say how much faster it will be. That depends on the software, the hardware, how it’s set up and how busy it is. Most of the architectures that Syncthing supports come with a file system notify feature which means a folder doesn’t need to be scanned often, because Syncthing gets notified when a file has changed.

It will be <total data you have>/<total data that has changed since last scan> times faster.

Also, protocoled is not a word, so not aure what you mean.

As we scan the files we build an index (ala database) for each file, cutting it into 128kb chunks, and holding a hash for each chunk.

All of this is explained in the protocol documentation.

why I ask, I use goodsync, but I wanna change to syncthing. The difference in the performance is big. And I am not sure why. Goodthing is very fast compared to Syncthing. ANyone any idea why.

I synchronized the same harddrive. 4gb of data.

Goodsync, 30 min

Syncthing, 12 hours

Based on 4GB taking 12 hours it sounds like you might have been connected through a relay that was limited to 100KB/s

Did you mean both drives were already synchronised through another tool and the initail scan took 12 hours? Unless this is a very underpowered processor this seems wrong. My phone can index a couple of GB in a few minutes.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.