Simplifying a particular use-case


while I do adore syncthing (it’s the bee’s knees - and the only file sync utility I personally use) I sometimes wonder if there is some potential to simplify a particular use-case. I am looking for…

  1. opinions on this idea
  2. pointers into the source code since I’m not familiar with the syncthing code base

Ok, I will first discuss the use-case, then my pain points with it and lastly a rough idea about how to simplify this.

The use-case is this: I need to share folders only among machines under my direct control. There is no third party involved. Additionally I do run a star topology. So there is one syncthing client that is on machine that is always on and serves as the “central hub”. Also I use quite a lot of machines, some of them just transient VMs, etc., quick throw-away installs.

The pain point:

Setting up is a bit of a hassle. I need to…

  1. Add the device “manually” by copying one device ID
  2. Select folders to share on my central hub (I need to remote login to that machine for that, or alternatively forward ports to the greater internet and care about publically visible IP addresses, etc.)

Albeit while this is not too much work it sometimes is enough friction that I forego it and use some other mechanism like emailing a file to myself. Especially as I use many machines, some are transient VMs, etc., so it does get annoying at times.

The possible remedy:

Since all machines are under my direct control it should be possible to skip both steps above and replace them with a mechanism which is based on sharing two pieces of knowledge (a key and a device name) which is used to derive all other keys, certificates, etc:

On the central hub we generate a private key - the master key and we choose a name for the central hub syncthing client. Using key derivation we derive a new private key (by some sort of mixing in the name) which is then used to create certificates, IDs, etc (I’m not sure at all about the crypto implementation in syncthing and how it all fits together.)

On a spoke node we use the same private master key (this needs to be transferred to each new device, but that’s just a scp invocation - or some other mode of transfer. Alternatively we could even derive the master key from a sufficiently strong passphrase alleviating the need for even this one transfer). We chose a name for the spoke node and use that name to derive a new private key from which then the certificate, IDs, etc. are derived. Then, since we also know the central hub’s name, we can rederive its ID and immediately send a connection request to it (sending along our name).

The central hub can then check if the spoke node’s certificates, etc, are derived from the same private master key. If that’s the case automatically accept the connection and then immedialtely offer all folders to the spoke node.

The setup then reduces to a couple of steps which can all be performed on the spoke node.

  1. Acquire the private master key (or derive it from a passphrase)
  2. Run syncthing and tell it to connect to the central hub

Note: This is not really limited to the hub-and-spoke topology. By sharing a secret and rederiving keys, certificates, IDs, from it allows skipping the manual connection step between any pair of devices since it establishes trust.

Note: I implemented something a little bit similar for managing my VPN/wireguard setup: GitHub - fps/0vpn: An experiment in making a wireguard VPN setup super easy (almost 0-conf) (very hacky collection of shell scripts and one trivial go program). The details are, of course, completely different, but the general idea is the same.

Sounds like what you want out of syncthing is mostly a shared filesystem, as you seem to want the same files on every device, etc. Perhaps that is what you should use?

Also, whats preventing you from pre-generating certs (hence ids), adding them on the hub, and then just picking up an off the shelf pre-generated identity for your new device?

I’m not sure things relating to certificates could be done easily, as the device id is derived from raw certificate bytes and there can only be a single cert in the chain, so you don’t want to change that.

Perhaps you could add an option for devices to trust certs that the device has signed with it’s own private key, then getting an identity is done via a CSR from the hub.

A long while back I added Hello message types to the protocol to allow unauthenticated devices to present some content that makes them authenticated, but I never spent too much time thinking of what that content would be and how to do something like that securely, but I don’t think some crazy key derivation shenanigans is the way for that. More like generate a ecdsa pair, share it with someone via email or something, present the public part and evidence that you have the private part when connecting in Hello message.


Hi! Thanks for your quick reply!

Is there a shared filesystem that is available on Windows, Linux and Android which supports NAT-traversal? Syncthing does check all these boxes (and more.) Well, besides not really being a filesystem. Besides I do like the offline availability of the files…

Regarding the rest of your comments: I don’t feel qualified (yet) to address them, since I know next to nothing about the implementation of syncthing. I guess I’ll start by reading the BEP protocol documentation.