3-way sync

I have a large (1.5TB) send-only sync folder on a PC (“W”) at work, and a fully synchronised copy on a PC (“H1”) at home. I want to migrate this to a third PC (“H2”) on my LAN, so I have configured H2 as a remote device on both the other W & H1, and shared the folder from W and H1 to H2.

It is slowly synchronising, but if I look at “Remote Devices” on H2, it says it is 90% syncing with W but up to date with H1, which doesn’t seem to make sense?

If I look at the Remote Device on H1, it says it’s up to date with W (which is correct) and at 6% syncing with H2, which also sounds about right. The status on W matches this (H1 up to date, H2 at 6%).

As H1 & H2 are on my LAN I would expect most of the sync traffic to be between those two but it feels like it’s mostly pulling it across the DSL link to my office (which will take forever!) Indeed it’s been syncing for ~12h which should be long enough to transfer 1.5TB over a gigabit LAN (both H1 and H2 are connected to the same gigabit switch).

Syncthing adds overhead, as in the local bandwidth might not be the limiting factor for the sync (might be any one of cpu, disk syncing, db writes - you need to check on H2). It should definitely pull most things from the local device when the bandwidth is much higher there. You can check that looking at the up/download rates on the remote devices.

That’s not what those statuses mean: This is saying that from the point of view of H2, W is 90% in sync with the global state - as in W is out-of-sync. That likely means it’s still exchanging metadata with W, i.e. H2 hasn’t yet the full picture of what W has and has not. That again points at database updates being a potential limiting factor here. What’s the Syncthing db stored on (SSD, rotating disk)?

You can check that looking at the up/download rates on the remote devices.

OK, they vary considerably as I watch them but they do indicate that at least some of the time a decent amount of data is coming from H1 rather than W

What’s the Syncthing db stored on (SSD, rotating disk)?

W: HDD (data also on HDD, but in that case RAID5) H1: SSD (data on external HDD) H2 (a Pi): SD (data on external HDD)

So presumably what I need to do is stop the sync, rsync the sync folder from H1 to H2, then resume the sync?

No that won’t change much, it still needs to hash everything, compare metadata and update the db. If H2 is a pi, it’s entirely expected that it takes a good while to process the initial 1.5TB.

How about rsyncing both the data and the Syncthing metadata?

Copying the Syncthing database. Might work, but risky. Do you really need it? If you do go ahead with it, definitely do set H2 to receive-only to prevent anything from propagating. Also just to be clear a disclaimer: I am not recommending doing that.

No worries, I think I’ll just wait it out!

1 Like

I’ve moved .config/syncthing to the external drive (it doesn’t do the SD card any good to have the DB there anyway) and that seems to have speeded things up - hard to quantify but maybe twice as fast? Thought it worth mentioning in case anyone else finds the same issue.

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.