Configure 1999 SC machines to replicate from one source holding machine _and_ between them

Support
1

Hi,

I am looking for the correct settings on the one central machine (node) that holds a folder of files that shall be replicated to 1999 other machines (nodes) one way, top down, so not bidirectional. However, all of the 1999 machines shall know about the other machines so that even if some machines do not have contact to the central mode would get updates from a peer machine that holds newer data.

How would I configure the central node? How would I configure the 1999 other nodes?

PS: surely, the users do not want to click “allow” for all 1998 machines connecting…

2

I would use whatever other config management tooling you’d use to roll something else out to thousands of machines. I’d rethink the full mesh of 2000 nodes though, I don’t think that will scale well. You might also run into some scaling changes on the central node, but you can test and see.

3

I’m spitballing here — my Syncthing cluster consists of three Devices. But here’s how I would start to think about it:

  • Central repository Device, Send Only
  • Ten intermediate repository Devices, Send & Recieve
  • Break the 1999 Devices into 10 chunks, and have 100 Devices connecting to each of the intermediate repositories, Receive Only.

But I sure would test this carefully before going into production.

4

So you basically scale the problem down factor 10 and then combine 10x those chunkts into one. That would not help in my use case as I could not predict which of those maichens are actually physically meeting in one LAN segment (cut off from the internet). Hence, if they would not be under the same of the 10 machines, they would not know of each other.

It is just strange to me that this is so different between syncthing and resilio. In Resilio you basically have an owner of a share and if you use the newer flavor of resilio you one time have to accept the new machine to join but only on the owner node. If you chose the older flavor you never have to accept. Who has the key can have access. In Syncthing you have to accept on any node that there is a new node that wants to share. So basicylla 2000x2000 =4.000.000 acceptances.

5

Like @calmh said, this gets a lot easier if you push a configuration to each of the Devices. The users don’t even have to know about Syncthing, depending on your use case.

6

and wholly unnecessary.

You would have 2000 machines each trying to establish connections with 2000 other machines. That’s 4 million connection attempts.

Probably can create 45 groups of 44-45 machines. Each group meshes among itself and perhaps each of the 45 machines in the mesh meshes with one additional machine from another group.

If the “server” goes down each group could learn of changes from another group and distribute the changes among themselves.

Or, just have each machine connect with 45 other machines (selected randomly)

Statistically you would need a lot of machines to be down for the changes not to propagate everywhere.

7

On the flip side, if one of the keys needs to be revoked, every node needs to be updated. With Syncthing, a single node can be kicked out (optionally done via the REST API) without impacting the others.

8

Hi Jakob,

thanks for taking the time to answer this one. What would be a realistic, usable amount of nodes to n:n mesh? Like 10, 100, 500, 1000? And also: what is a realistic, usable amount 1:n mesh?

Thanks!

9

Hard to give exact figures, as it really depends on available resources and how much of them you will want to invest per machine.

Roughly, each sharing device requires some memory, file descriptors and network bandwidth while connected. And the database needs extra storage per device, keeping up with what index data has been exchanged with it or is outstanding. So maybe you should try to narrow down the answer from looking at each client’s available hardware specs, then design from each one’s local perspective.