I am new to Syncthing, it seems generally awesome and what I am looking for, and I am also looking at 10TB replication on a LAN - but I’ve been having some difficulties.
The speed of scanning the first node is great, it builds the index at the read speed of the drive (160MByte/sec - 17 hours), the index is on a separate SSD, but after that, it is impossibly slow at getting that data onto the next node, going at maybe 12MByte/sec (would take 10 days) - even slower on a low end NAS (3Mb/sec - 38 days)
An OS file copy is much faster (110MByte/sec - 1 day) - or faster if direct attached, but you then have the problem described in this thread, of the second node scanning the OS copied files, and the indexes fighting with each other.
Is there a way to put the second node in a ‘seeding’ or ‘write only’ state (the opposite of master), where it does not automatically scan it’s files, and expects the files it is pulling to already be on the local storage, then it can scan the local file and keep it if it matches, or download / replace it if it doesn’t. The index could then match the order of the first node, eventually delivering a copy of the source index to match the copy of the files.
Copying the index over seems complex, and if there is already another share, you would have to merge the indexes. Also, is it an OS neutral format - can I copy the index from Windows to Linux?
Searching these forums for ‘speed’ brings up lots of posts about quite slow performance of Syncthing, I think this is in part due to the multi-tasking that ends up thrashing magnetic disk’s into their worst possible access pattern of small random reads for large files.
I’ve tried playing with the number of copiers and pullers with no real improvement on the seeding performance.
Once it is seeded, I think the performance will be fine, but getting it setup is proving quite challenging.