I am planning to use Syncthing in a big network of around 100 devices.
I have several master nodes which distribute their data every day to varying clusters of receivers. These clusters can change on a daily basis and the whole system has to be able to dynamically adapt to adjust for missing devices in case of failures.
I plan on writing a script which sets the configuration and ignore files on each device every day to adapt to the requirements.
My question is, how I should set up my network topology. I could use a completely interconnected network with connections between each of the 100 nodes, adjust the network daily based on the clusters or try to find a different topology.
Do I have any disadvantages by choosing the interconnected network? I heard the index.db size can increase significantly in big networks, but is this also the case if each device only shares 1 or 2 folders with up to 15 devices and all other folders are ignored in the ignore file?
These clusters should also be able to overlap.
I looked into Arigi to centralize the management of the shared folders and configurations, but this would only make my scripting a bit easier and does not help me with my topology questions.
Has anybody worked on a similar system with Arigi or Syncthing and can share some insight?
It’s not clear to me if you mean changing as in “new devices are set up, old devices disappear never to be seen again” or just as in “devices are intermittently connected”. The latter wouldn’t require any special scripting.
Generally speaking, don’t go for full mesh for clusters of that size.
It will mostly be disappearing for a day or two due to failures, but the bigger change is that all folders will be deleted everyday at a set time and after a reconfiguration syncing to different sized clusters will start.
I want to sync images that are built on the master modes to several devices each for testing. I want to use syncthing because of the incremental changes during the built which can be synced while the software is building to reduce the time that fully copying the images after building would need otherwise.
The system should be able to be configured before the build phase to adapt to the expected testing requirements of the different builds.
I believe that small clusters with full mesh should be connected via some clients to other clusters.
See Maximum number of devices and folders . But I did not setup a config like this myself and I am not sure how easy it is going to be to maintain a topology like this.
Be aware that you can link all devices to each other and only make the sharing topology by folder configuration. (Please report back what is working well, I am quite interested)
if it is possible I would prefer the fully connected network (in terms of connections).
This would be set permanently and only adjusted for new devices.
The shared folders would be changed everyday.
I have read a few times that big networks should not be a full mesh, but I am not sure if this counts only if folders are shared between all devices or if this counts for connections as well.
There is some overhead per connection, although the amount of work per connection is roughly proportional to the amount of data shared over it. A connection with no folders shared over it doesn’t cost much, although it is also pointless. A hundred connections sharing one folder is a hundred updates to send for every change, etc.
Ok. This is what I expected. If it is possible without a huge overhead I would try a full mesh first, because it decreases the complexity of my script. I only have to manage new devices and do not have to find an algorithm for the perfect topology.
Since you are working on Arigi as well: Do you think that it is a good option to manage the shared folders dynamically by updating the tags of the devices? I am interested in the centralized management, but I am not sure if it will be worth the cost since I will have to script the dynamically changing configurations anyways.
I’m not sure why exactly you need to juggle the folders like that, but if you do that is how I would do it, yes. You can of course script it yourself as well for the same effect.
Thanks! The reason for the juggling is a bit complicated and I do not want to share it here.
Ok. We will think about the opportunities Arigi offers for us and then we might contact Kastelo about it. The monitoring looks interesting and the management seems to make it a bit easier.