best way of using syncthing with 100 nodes

Support
1

Hello,

I am working on trying to improve a client-server setup on 90+ windows machines with syncthing.

Currently I run a batch script that connects to each machine (using SMB), runs robocopy to mirror each folder, then moves on to the next machine. It has a minimum run time of roughly 1 minute per machine, so I end up needing 90+ minutes to sync.

On Syncthing, I noticed the introducer feature isn’t able to delete nodes (from the other machines) if I delete them from the introducer computer. Is there a way around this? (ex. a command-line to delete the database (except for self & introducer) and force a re-introduction upon startup?)

Would it be better to run each client machine to the server without using an introducer? (ex. PC1 <=> SERVER, PC2 <=> SERVER, etc.)

My end goal is to be able to sync all the machines in under 5 minutes. What would be the best way to accomplish this? Is this the correct use of this software?

Thanks

edit: I would use DFS for this but due to security requirements I am unable to join the machines on a windows domain (so, no active directory). Also, eventually the number of computers might increase to ~300 within the next few years.

2

I guess Syncthing is meant for individuals, and not for 100 machine setups. If you want to use it on a 100 machine setup, it’s natural to have to put in some effort into managing this. Write something on top of syncthing that deals with this and manages node addition and removal.

As for topology, if you have a big cluster, it might not be the best idea to connect everyone to everyone, but probably work towards a snowflake topology.

Also, your end goal of 5 minutes is meaningless, as it doesn’t say anything about sizes, link speeds and so on, furthermore, the more machines you will have the longer things will take to propagate, at which point 5 minutes might become impossible.

1 Like
3

I’ll see what I can put together. I’ll experiment with the snowflake layout a bit.

Thanks

4

There seems to be nothing wrong with your original approach, and it has some nice advantages: you know for certain that a deployment to a particular computer had finished, for instance.

The problem is that it’s too slow, but it sounds like it wouldn’t be hard at all to make it deploy to multiple machines at once, which would speed it up enormously with very little effort…

5

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.