Easily boostrap a peer-to-peer server cluster?

Use case: we run about 5 servers in a private network with private IPs in Docker Swarm. We need to share some config and data files among the servers. Because a dedicated file server can still fail, we would prefer to have all files locally on every server, just sync them on rare updates. Don’t need a GUI.

I researched a lot, built a complicated Syncthing in Swarm proof-of-concept 7 months ago. Now I want to go at it again, as Ceph and GlusterFS are just “too much” and I don’t want to use rsync with cron.

Syncthing seems to be the right tool for the job, only the dynamic setup seems missing.

WISH 1: Peer-Discovery by DNS

I would love to start Syncthing via Docker Swarm, only pass a secret token and its internal DNS name. Docker will respond with all IPs of the service containers to the DNS request, so Syncthing could connect automatically and auth with token.

WISH 2: Auto-Update by DNS

In intervals, Syncthing should check again via DNS if all returned server IPs are already known. If not, add them, auth with token.

WISH 3: Default shared folder

Enable a default shared “sendreceive” folder, which I can mount into the Docker container, which then automagically has the same synced content on all servers.

Result:

Imaginary docker-compose.yml to use with docker stack deploy:

version: '3.9'

services:
  syncthing:
    image: syncthing/syncthing:1.3
    hostname: '{{.Node.Hostname}}'
    volumes:
      - /var/syncthing:/var/syncthing
    networks:
      - sync-net
    environment:
      - ST_DEVICE_ID={{.Task.ID}}
      - ST_AUTH_TOKEN=A1B2C3
      - ST_DISCOVERY_DNS=syncthing
      - ST_DISCOVERY_INTERVAL=5s
      - ST_AUTO_SHARED_FOLDER=/default
    deploy:
      mode: global
      endpoint_mode: dnsrr

networks:
  sync-net:
    name: sync-net
    driver: overlay

Is there a way that Syncthing will ever be that tool supporting mobile devices and small server setups alike? What will it take? Where to start?

You cannot set the device ID. It is a hash of a cryptographic certificate used in the TLS connections. Thus you need to give each instance their (possibly pre-generated) cert.pem and key.pem files. I don’t know what you mean with the ST_AUTH_TOKEN. If it should be for the management GUI, that can already be set via the STGUIAPIKEY environment variable.

You can set DNS names for each remote device in the configuration. They are looked up when dialing the remote device. Changing these and adding / removing remote devices can all be done through the API, so you could regularly update the other devices’ addresses.

Same thing for sharing a folder. Either do it via API when Syncthing is running for the first time, or give it a partial config.xml file with the desired folder pre-configured before starting it for the first time. Syncthing will respect the file and add any missing config options with default values. Have a look at the syncthing generate command documentation to learn how this can be prepared using a separate home directory on a different host, so you only need to drop the generated files into your container.

1 Like

Thanks for your response. I don’t know SyncThing too well.

How are SyncThing peers identified and accepted by the others? I would like to automate that, therefore the idea to start every instance with a common token/secret.

Can I have a little script just adding all DNS-provided IPs as new peers to each SyncThing instance every 10 seconds? Will SyncThing automatically de-duplicate? How to auto-accept them?

With the config.xml I can create a default folder that is shared with every other peer, even if the peer is newly added?

For device A and B to connect, A has to add B and B has to add A.

Where A and B are device ids. Device ids are hashes of the certificate for the device. They prove that they have that device id by proving that they have the private key for a certificate who’s hash matches the device id.

You can do most of the things you suggested via the api/cli.

You cannot share folders with devices that don’t exist in your device list.

Syncthing will not deduplicate ids.

Seems you want a lot of automation, which suggests you’ll have to write a lot of code.

1 Like

Thanks for your response!

I already created complicated code for this to work, it takes an extra container to orchestrate the other SyncThing containers.

async function run() {
    const ips = await getDockerServiceIPs('syncthing_syncthing')
    console.log('Got IPs from Docker Swarm', ips)
    const ids = await getAllSyncthingIDs(ips)
    console.log('Got IDs from Syncthing', ids)
    const updatedDevices = await addMissingSyncthingDevices(ips, ids)
    const updatedFolders = await addDevicesToFolder(ips, ids, 'default')
}

(Source)

Never put it into production because I thought it has to many moving parts and was afraid something could break.

Just wanted to check again if I have overlooked some simplification or there has been some further developments to make it easier.

Hi @bluepuma77,

Did you manage to set up Syncthing on docker swarm? We’re in the same situation and trying to find the best tool for it. Please inform me if you were able to set up Syncthing successfully.

Thank you