Hi,
I gave up on maintaining my fork with db maintenance tunings for large folders (at least for the forseeable future). I’ve recently hit three problems that forced me to use an alternative :
- there’s a bug at least in my rebase on the 2.1.1 version that makes Syncthing allocate huge amounts of RAM (>100Go) after some time. I’m not sure if the bug is in my patch or the original code (see second problem),
- it seems there was some refactoring and my patches now touch code that recently changed and that I didn’t have time to properly review/understand,
- 1.29.5 is running on two servers where the Go toolchain will eventually migrate to at least Go 1.26 but 1.29.5 doesn’t work with it (just had to downgrade to Go 1.25 on one server).
I always had a backup plan for our needs using btrbk (as our main server to be backed up stores its data on BTRFS subvolumes). Having Syncthing work for us would have been ideal but we can’t spend months on it. Early on I had already one folder synced with btrbk with promising results.
btrbk isn’t as flexible as Syncthing even with our simple “one source - two replicas” needs (we will miss the exclude patterns and might have to work around this by moving directories). But it is very lightweight (uses almost no CPU/RAM/IO) and it seems we can get replication latency in the 2-5 minutes range by using it to transfer snapshot diffs at this rate without generating noticeable load.
With the problems above switching to btrbk seems the safest route for us.
I leave the fork (GitHub - jtek/syncthing at db_maintenance_performance · GitHub) accessible if others want to have a look at it. If needed I can answer questions about it too.