Syncthing indexing of 16TB Datas

Support
1

Hi,

We have two machines that we want to synchronize using Syncthing. Each machine contains roughly 16 TB of data, consisting primarily of small PDF files—potentially billions of individual files.

We have started the Syncthing service and added the folder we want to share, but Syncthing estimates that it will take around 6–7 days to index all the data on each machine.

Is there any way to accelerate this process, or are there specific settings that could help complete the indexing faster?

Thank you for your guidance.

2

Is the index database on fast SSD? Are there indicators of disk, CPU or RAM being a performance bottleneck?

3

hi chaos,

The machines are VMs and have 8 vCPUs and 32GB of RAM. The storage is an NFS Share on SSDs with high performance.

the CPUs are not even 20% used and RAM is about 20GB from 32GB used.

4

I should also have asked about network-related bottlenecks.

Take a look at Large NFS mount sync to another NFS mount .

It’s not clear to me that you can speed this up a whole lot based on what I know so far. Ideally the database would be on a local SSH rather than on an NFS mount.