Programmatically setup Syncthing server cluster via REST API?

Feature
1

I would like to use Syncthing on a few servers to share some files, like the Wordpress upload folder. Instead of putting the files on a remote shared drive, I would like to have them locally on every server, synced with Syncthing. When one file is added or changed on one server, the others are updated.

This should run in Docker Swarm. My idea is to have Syncthing running headless in a container on every server. Then add a single Syncthing-controller container, that fetches all internal Syncthing IPs every 5 minutes via Swarm API, checks via Syncthing REST API with all servers if all IPs are already registered as peers, and adds them and a default shared folder if necessary.

I setup my first Syncthing server “cluster” via GUI, got the token, added it on one server, confirmed it on the other. Same for folder, added it on one server, confirmed it on the others. So quite a few steps that probably need to be replicated via REST API.

Is it even possible to setup a Syncthing instance without any GUI, just using the REST API? Can an initial token be supplied via env or set via CLI?

Has this been done before? I prefer not to re-invent the wheel and rather use something proven then try to code myself. Internet search didn’t show me any viable projects.

2

Certainly this is possible. It’s not clear to me if you mean “API key” or “device ID” when you say token. For the API key, it can be set from the command line when starting, so you could use the same for all your instances and it would be known beforehand by your management system. As for tying the devices together, you need the device IDs which you can get via REST, and push the corresponding device entries to the config on both sides.

I’m not aware of any ready-made solution. I tried building one, but it turns out it’s quite hard to build something generic that fulfils all use cases and doesn’t devolve into just being programming. (And also, nobody wants to pay for this. :))

3

@calmh How can I set the REST API key from CLI?

I really tried hard to find that answer, to no avail.

4

https://docs.syncthing.net/users/syncthing.html?highlight=gui%20apikey#cmdoption-gui-apikey

–gui-apikey=

Override the API key needed to access the GUI / REST API.

This should be the argument you’re looking for:)

5

Thanks, I guess I was irritated by the “GUI” when I don’t want to use the GUI :wink:

6

Is there a simple way to disable CSRF security when using REST API?

/ # curl http://localhost:8384/rest/noauth/health
{
  "status": "OK"
}

/ # curl -H "X-API-Key: A1B2C" http://localhost:8384/rest/system/ping
CSRF Error
7

I’m not sure why you would want to do so (and no, not that I’m aware of:)), since using the right API-key would be the solution here (“X-API-Key: …”).

8

As Eric says, there is no CSRF protection when using an API key. Unfortunately, the error becomes misleading: read it as “bad API key” instead, in this case.

9

Moving forward, will post final result here.

I can fetch all Syncthing instance IPs from Docker Swarm, then get their myID via GET /rest/system/status and add it to the others with POST /rest/config/devices:

{ 
  deviceID: device, 
  autoAcceptFolders: true 
}

Next I would like to add a single folder with POST /rest/config/folders to the first Syncthing instance, adding own and other myIDs:

{
  "id": "/var/syncthing/data",
  "label": "/var/syncthing/data",
  "path": "/var/syncthing/data",
  "type": "sendreceive",
  "devices": [ { "deviceID": "A1B2C3-..." }, { "deviceID": "B2C3D4-..." }]
}

Problem: the other Syncthing instance changes the path by itself:

{
  id: '/var/syncthing/data',
  label: '/var/syncthing/data',
  filesystemType: 'basic',
  path: '~/var syncthing data', <= ###
  type: 'sendreceive',
  ...

What’s the correct way to add a single synced folder to all Syncthing instances? Should I use autoAcceptFolders or add the single folder manually to every instance? Do I need to manually set/update the path on every instance?

Or should I better just stick with the single default folder and make sure on every instance the folder has all other deviceIDs set?

10

Auto-accepted folders will have Syncthing make up a path by itself, as it has to. If you want to control it manually you need to add the folder there yourself via the API.

11

So on every node I need to create the folder with a common ID and the correct path first. Only when all nodes know about the folder, I add the other deviceIDs to it?

12

Here we go, a proof-of-concept docker-swarm-syncthing setup.

One “syncthing-controller” container for a simple work flow:

  1. Get all Syncthing container IPs from Docker Swarm
  2. Get Syncthing IDs from all containers via IP
  3. Get Syncthing “devices” from each container and add IDs which are missing
  4. Get Syncthing default folder from each container and update “devices” with IDs which are missing
  5. Sleep and repeat

With this another service/container can use /var/sycthing/Sync/myfolder for files which are automatically replicated across all Syncthing nodes. This can be used for configuration files, LetsEncrypt certificates or folders like Wordpress upload.

2 Likes