Latest encrypted devices proposal, what & how

For discussion of https://github.com/syncthing/syncthing/pull/6214 without overwhelming that PR

3 Likes

So what my proposal current does/offers:

  • Lets you set an encryption password on a folder shared with a device. The data for that folder will be encrypted with the password when sent to that device.

  • Lets such “untrusted” devices exchange this encrypted data between each other in the normal manner, without knowing any passwords.

  • Lets a new “trusted” device bootstrap from such an untrusted device, by adding it normally with the same password as in step one.

  • Untrusted devices cannot change or introduce new files / directories, as all index entries from them are expected to be encrypted and they can’t produce such index entries without the password. At worst, changing an encrypted file will cause a sync failure when someone trusted tries to read that data and fails to decrypt it.

  • All encryption is AES-GCM.

Draft docs:

https://docs.syncthing.net/branch/untrusted/html/users/untrusted.html

What is secured:

  • File data
  • File (and directory etc) names including file structure (that is, you don’t see which files belong together in a directory)
  • Other file metadata (timestamps, version history, permissions, symlink target)

What is not secured:

  • File size. It changes slightly because each block gets a header with some overhead in it, and the file gets a trailer with some metadata, but it’s still easy to derive the original file size from just the encrypted file.

Problems and limitations:

  • The untrusted device can’t verify the correctness of the data because it doesn’t get hashes for it. Doing that would require hashing the data at source multiple times, once for each encryption key in use anyway, and extending the file metadata with these multiple block lists.

  • Changing the folder password is not possible once a device is set up. Or rather, doing so means all existing data is invalid and must be wiped. We’ll probably want a mechanism to do just that, for when there is suspicion that the key might have leaked.

  • Folders on untrusted devices should be set up as “receive only” to avoid confusion, and should ideally be smart enough to not scan any new files that might pop up as these will just cause problems. This can be enforced from the other side when it sends the “encrypted” bit in the initial handshake.

What one might imagine which is not currently on offer:

  • Elegant end to end security with signed updates and some sort of public/private key hierarchy.

  • Having a single device accept some folders as normal/trusted ones and others as untrusted. That works now.

9 Likes

As far I understood the draft concept the “receiveOnly” encrypted node would fit my use case pretty well. I’m currently seeking for something easier and more lightweight than encfs.

I don’t know if it would work well on Windows regarding the 255 chars file path limit. One app could ignore the limit and still work fine accessing the files, but the user could get into trouble when attempting to move/copy that folder for some reason. Alternatively, there’s a registry switch to allow more chars in a path on windows - not very user friendly as one would have to read the docs then.

I’m super excited to have this feature, thank you for taking another stab at this! I will contain my excitement long enough to pretend to have “serious discussions” about this, but the TL;DR is that for my use case, the code can be merged as is and will immediately be put to use.

For the use case of ‘me family and friends who all live within 10-100 miles of each other and want to make their files house-fire resistant without paying for cloud storage’, the as-is solution works out just fine.

We each have always-on devices with 500-2500 GB of space on slow drives sitting there for serving trusted syncthing shares and K3S clusters etc as home labs. Adding another syncthing instance with an untrusted device ID so we can each be yet another backup node for each other’s devices doesn’t require any more functionality than what the PR currently offers.

This also enables some amount of syncing to a VPS or other shared internet resource without worries of hacks / network admins ‘reading your mail’. There is certainly room for the size-hiding and additional obfuscation for resisting a certain amount of active scanning or forensic analysis, but for those of us who just don’t want our monthly budget files opened by a bored CSR, this encryption should work just fine.

2 Likes

I’m really excited to see this feature being developed!

I have not been able to find all discussions about this solution, so pls excuse me if I’m just repeating something that has been discussed before.

Has it been considered to always encrypt the data, and to send it encrypted to all devices? (I understand this would be an encryption layer on top of the already encrypted communication)

On a high level, what would need to be changed in this architecture:

  1. A way to store & communicate the secret key to trusted devices. 1 secret key per folder would be used.

  2. When reading/writing data to disk:

  • On trusted devices: encrypt/decrypt the data before storage.
  • On untrusted devices: nothing (but do enforce the folders without secret key to be “Receive Only” folders)

None of the hashing/syncing/etc algorithms would need to be changed. I think this would drastically reduce the complexity of this new feature, while still offering a robust solution.

I don’t think that would reduce the complexity to do it always … but moving the key from per-remote-device to per-folder would do something similar I guess. Those who have the folder key can decrypt it, those who do not can’t, as it’s always sent encrypted.

It adds some extra encryption/decryption overhead for devices that don’t need it today.

It also adds a key management problem, in that it will be very tricky to change the encryption key. When it’s per device instead, we can throw away the key when we throw away that device. When it’s per folder, everyone sharing the folder must agree on the change.

3 Likes

Thanks :santa: for working on this feature, it will be helpful to many people!
I’m sure you know Restic (Go) and Dupliati (C#), which you could maybe use for code inspiration as both these programs use encryption.

I just cross readed this topic but every time I read some specific implementation detail, I asked my self; how does the use case looks like, which scenarios gets addressed by which implementation detail. Is there an overview of the high level requirements available? Just to understand which scenarios are taken into account and which not …?

That’s what I tried to flesh out in the first post. What scenarios it fits is dependent on what it protects and how to set it up. Hopefully if this matures we’ll make a couple of howtos.

2 Concerns:

  1. Would the single encryption pwd per device work with an untrusted endpoint supporting multiple devices + folders? Example: Untrusted NODE1, should be able to host 10 different devices + folders inbound and maintain them separately.

  2. I have concerns about your approach of a device “bootstramping” from the untrusted node SINGLE encryption password. EXAMPLE: if one of the ten hosted folder devices is destroyed, and needs to recover from the untrusted node1 data, adding that single password + the folder ID would be enough? The use of just an encryption password is a concern.

Thanks

Things are evolved slightly from the above (I’ve updated the first post), password is now per folder. So it’s possible to share several folders with different encryption keys, or some without encryption. Multiple trusted devices each sharing different stuff with the untrusted device using different passwords is also fine.

If two trusted devices want to sync “through” the untrusted one, or bootstrap from it, they need to use the same password for those folders. It is a shared key for those trusted devices.

3 Likes

Those two changes are appreciated. I didn’t see them discussed on github, but if you saw those are planned to be addressed, that would be good.

I added some draft docs to explain what’s going on;

https://docs.syncthing.net/branch/untrusted/html/users/untrusted.html

2 Likes

Why would the an untrusted device list the encryption password in the config.xml? I’m not following the purpose of that. I could see that on the “trusted devices”. Otherwise, can’t a user change the config.xml and then see the trusted data.

When one creates a new folder, do you need to specific at that point if it’s a trusted folder? Or can one decide later on that you want to add an untrusted node?

Allow the ability to manually specify rescans period on untrusted nodes.

I’m planning to use untrusted nodes on NAS devices, so please make sure the encryption while very robust is not brutally CPU intensive.

What is the purpose of trying to mirror the directory structure? isn’t this going to seriously complicate the implementation. What happens if you have 10k folders and 2m files. I don’t understand the logic of hiding directory structure. Encrypting folder/file names is enough.

Thanks!

There is no password in the config on untrusted devices.

No need to decide at folder creation time, the decision needs to be made at sharing time.

The only purpose a scan would have on an untrusted device would be to mend data that had been accidentally deleted or modified. I’m undecided on how to handle that yet.

Encryption is AES which is often hardware accelerated.

I’m not sure what you mean exactly about the directory structure. There are basically two alternatives. Either the encrypted side has the same structure as the plaintext side, although the names as scrambled, or it does not. The latter is in fact easier and has some security advantage, so I’m going with that. (It’s easier, because when encrypting the names become longer than their clear text counterparts. This means they might need to be split up to not exceed max path component lengths. Doing that while retaining the original structure is tricky. It does however mean there will be more directories created on the encrypted side than on the original.)

I added a section to the draft docs (linked above) explaining the directory structure.

3 Likes

What are your thoughts behind using password over a generated key that can be shared as a file?

The former relies on the user setting an adequately strong password, which is easy, but in practice still often doesn’t happen. The latter usually has the disadvantage of having to transfer a file. However given it’s used for a Syncthing folder on mutually trusted devices, that can be done using Syncthing. Or one could imagine storing it in .stfolder and adding a BEP message to request and send the key (though that’s deep into bikeshedding territory).

There’s no reason to assume the two trusted devices can communicate except via the untrusted one, so sharing the key via Syncthing won’t work I think. Although it does beg the question of what the introducer does with untrusted devices. Share the password, if it knows it?

I went with password because it’s the simplest, from a usage point of view. The scrypt key derivation makes sure we get a good encryption key regardless and should make brute forcing at least difficult. I’d much rather keep it as simple as possible for now. :slight_smile: Currently I’m looking at how to tweak the sharing dialog to make this user friendly, and I don’t see that getting better if it requires uploading a key as well…

Currently it’s at least theoretically possible to guide someone into configuring a Syncthing device over the phone. The device ID is annoying to communicate but designed for that to be possible. A large key file wouldn’t help…

There’s nothing to prevent having a Generate Strong Password button in the sharing dialog, though. Like the API key…

3 Likes

Just to chime in from user perspective: I like calmh’s simple approach, especially the password to remember in desaster recovery cases, very much. If this fits the dev aims, please continue it :slight_smile:.

1 Like

I didn’t consider the case where devices only sync via the untrusted device (in hindsight that’s probably pretty common for the use-case of untrusted devices). Anyway I need to read and understand the used crypto before asking questions about it…

1 Like

Looks good!
How many different “modes” are there? If there are just “trusted” and “untrusted” I suggest using a checkbox instead of dropdown for better UX.
And maybe simple text “encrypted” or “encrypt?” + checkbox (if checked, pw field becomes active) is even more clear for the average user without a technical background?