Encryption for remote syncthing device

anarchos · May 22, 2015, 8:22am

Hi, I don’t think this is possible so this is more of a feature request, unless there’s a way to do this using 3rd party apps…I’d like to run a syncthing device on my VPS cloud server (digital ocean in my case) but I’d like to have the entire folder encrypted since it will not be in my physical control. I know traffic between devices is encrypted via TLS but the files themselves will be sitting there unencrypted. Any way to have seamless encryption of files on a device? I’m not too concerned about my PC or phone, since I control physical access to them.

pgrm · May 22, 2015, 8:33am

You could technically encrypt the file system.

Not sure if that would solve it for you, or you’d rather want to use the VPS as a relay and backup server only, so you won’t be accessing the files directly on that VPS. I think this would be a great feature if Syncthing would support that. You could even offer hosted backup and relay servers, while the users could be sure, the server and anybody who has access to it, wouldn’t be able to see the actual content of their files.

bigbear2nd · May 22, 2015, 9:34am

Hi!

There is a long long Github issue for that:

github.com/syncthing/syncthing

Support for file encryption (e.g. non-trusted servers)

opened 03:07PM - 04 Apr 14 UTC

closed 08:13AM - 13 Apr 21 UTC

Natanji

enhancement

So I have had a look at BitTorrent sync, syncthing and alternatives and what I a…lways wondered about was the possibility to not only sync between resources I own and trust, but also external resources/servers which I do NOT trust with my data, up to a certain extent. One way to do this is using ecryptfs or encfs, but this has many obvious downsides: it is not an interoperable solution (only works on Linux), the files are actually stored in encrypted form on the disk (even if the resource is trusted and this is not necessary, for instance because of the file system being encrypted already), etc. What I propose is somehow configuring nodes which are only sent the files in an encrypted format, with all file contents (and potentially file/directory names as well; or even permissions) being encrypted. This way, if I want to store my private files on a fast server in a datacenter to access them from anywhere, I could do this with syncthing without essentially giving up ownership of those files. I could also prevent that particular sync node from being allowed/able to make any changes to the files without me noticing. I realize that this requires a LOT of additional effort, but it would be a killer feature that seems to not be available in any other "private cloud" solution so far. What are your thoughts on this feature? EDIT: BitTorrent sync mentions a feature like this in their API docs: "Encryption secret API users can generate folder secrets with encrypted peer support. Encryption secrets are read-only. They make Sync data encrypted on the receiver’s side. Recipients can sync files, but they can’t see file content, and they can’t modify the files. Encryption secrets come in handy if you need to sync to an untrusted location." (from http://www.bittorrent.com/intl/de/sync/developers/api)

But I guess some more discussion will not hurt.

@calmh requested that we do this in the forum:

calmh · May 22, 2015, 10:32am

Yeah, because here I don’t get notifications that need to be cleared every time someone says “yeah that’d be cool” or “I have this broken workaround here that doesn’t do the same thing that you can try…”

stevenroose · May 30, 2015, 10:27pm

There’s a long standing issue for that too, FWIW

stevenroose · May 30, 2015, 10:28pm

But, on topic now, I don’t want to read the complete issue thread.

What is the main reason why encrypted peers is not possible with Syncthing?

AudriusButkevicius · May 30, 2015, 10:58pm

Because nobody implemented it yet.

stevenroose · May 30, 2015, 11:47pm

I’ve just taken a look at how device linking and discovery works. To me it seems hard to implement this using the current system.

In a cluster of three devices of which one is a non-trusted one, the other two have to come to an agreement about which key to encrypt all outgoing data to the third device with.

I’d think about a solution like this: (I think its actually the opposite of how introducers work.)

When you init the first device in the cluster, it is the only master device. It creates a private key to use for encrypting data for untrusted clients (you want to use the same key for all untrusted devices because otherwise more information than necessary is given away when there are a lot of untrusted devices)
Whenever you add a new device to a master device, you check a box to indicate if it is trusted or not. If it is trusted, you hand it the key so that it becomes a master device as well.
Introducers of course indicate to devices that the new device is untrusted.

A similar thing can be done to solve #62 (which I just stumbled upon), where source devices sign their changes with the master key required to make changes.

(I literally just minutes ago first looked at how syncthing works so this might be complete nonsense.)

Just thinking further here, this whole idea could actually be taken further to changing the device-based syncing towards cluster-based syncing/discovery. Imagine that a syncthing cluster would be built around a private key (which could be backed up offline). The cluster would be identified by (a hash of) its public key.

All trusted devices hold this key.
Whenever you want to setup a new device, you enter the identifier and it will announce itself, asking for permission to the cluster.
All trusted devices get a notification, asking what to do with the device. You can then decide what tp do with it
1. Trust it and hand it the key.
2. Make it a read-only device, handing it a signed message saying that it is a read-only device.
3. Make it an encrypted device and hand it a signed message saying that it is an encrypted device.
The new device then messages all other devices, proving her “role” in the cluster that has been given to her by another trusted device (this prove is either a self-made signed message with the key she received or the signed message she received)
Now, when a device is advanced by a new device, the new device can authenticate itself using a signed message.
The cool thing is, when all the trusted devices die are get stolen, you still have the backup of the key and can use it to find the untrusted devices that can hand you all the files in encrypted form. Of course validity checks have to be made, but that is certainly possible.

I’m probably driving it too far for the simple use cases syncthing is intended for, but such a system would allow for a lot new interesting use cases. It actually goes in the direction of a more distributed cloud solution, like Storj, f.e. (Storj is a really cool project, you should definitely take a look!)

calmh · May 31, 2015, 5:47am

So to summarize, you’ve envisioned something totally unlike Syncthing in operation, protocol and security model. That’s fine, but doesn’t actually move anywhere towards adding this functionality to Syncthing.

There are ways to retrofit the needed functionality on the current protocol (basically encrypting hashes and file names, adding an encrypted-hash field, and encrypting the actual file blocks of course), but it’s tricky and ideally something a crypto expert should look at as it’s not completely obvious that it’s correct. It would be worthwhile for someone to actually spend the time trying to implement that though, as it is something we could in fact include and ship. Me and Audrius talked about it and there’s a half done protocol proposal somewhere. I should dig that out and clean it up so people can look at it.

But further visions of a different future based on something fundamentally not-Syncthing aren’t useful, in the context of things we want Syncthing to do.

calmh · May 31, 2015, 7:25am

This is my current best guess/recollection:

gist.github.com

https://gist.github.com/calmh/e9976f6248d54917ebd6

encryption.md

# Goals

Syncthing devices that store data in encrypted format. Such a device can participate fully in a cluster, but the information on the device is not directly usable.

# Assumptions and Definitions

 - There exists a secret key per folder, known only by devices that can access the unencrypted data. We call these "secure devices".

 - Devices that do not know the secret key can only access encrypted data. We call these "insecure devices".

This file has been truncated. show original

sneak · June 19, 2015, 12:04am

Added another $50 to the bounty[1]. If I weren’t so swamped with work this quarter I’d take a crack at implementing this myself. Perhaps just sticking a .stfolderkey.pem in the root of a shared folder on the trusted peers, and having each block stored on the untrusted ones (each with its own file key, encrypted to the folder key, in a header) might work with the least amount of implementation. I’m not sure because I haven’t had a chance to dig into the existing st design yet.

<3 you all. thank you.

best, -jp

[1] https://www.bountysource.com/issues/1474343-support-for-file-encryption-e-g-non-trusted-servers

liefde · July 9, 2015, 4:44pm

Isn’t this missing the point? What you want is what services like tresorit and spideroak deliver, i.e. encryption before files leave the client, and decryption when it enters the client. TLS is just an extra layer, but as soon as it leaves the tunnel, it’s all vulnerable again when stored on a remote server or cheap NAS device. You would want a local switch serverside to receive data encrypted or not, and then the clients (the other ends) need to encrypt the data before transmission. Sounds more complex than it is. The code is already out there, it’s more a question of CPU usage and weight.

generalmanager · July 9, 2015, 4:56pm

@liefde Please read the github issue, it’s not about https and your proposal unfortunately doesn’t cut it.

Homomorphic encryption and special databases could help ST accomplish this tough.

Here is an article + discussion about this: https://news.ycombinator.com/item?id=9839438

liefde · July 14, 2015, 8:21am

You haven’t even read what I wrote. And it wasn’t a “proposal” either, it’s the only way to make this work securely. As it stands, syncthing only secures the transfer, while in transit, the transport. You need client side storage decryption, and server side storage encryption.

calmh · July 14, 2015, 8:31am

Look, guys. There’s an embryo to a protocol change proposal above. If you have energy to spend, why not look it over and suggest improvements or something. Obviously this is about encrypting data before it leaves the secure device; reiterating that doesn’t move anything forward. The code isn’t “out there” in this case - it’s a matter of implementing it in the context of the protocol we already speak, or as changes to that protocol.

NickPyz · January 22, 2016, 5:14pm

In the news:

The “other” (closed source) peer-to-peer syncing application released encryption for untrusted devices on January 21st.

Previously, the other product required a manual hack to derive Read-Write or Read-Only encryption keys. It’s now a mainstream feature on both their paid and free versions.

FYI …

lfam · January 22, 2016, 8:51pm

Since it’s closed source it’s impossible to audit. In that case it’s really not very different from using your friend’s server as an unencrypted node and just trusting them not to read your files, or from any of the “other” closed-source file sync programs.

I guess if they claim to use a standardized crypto-system you could capture the packets as they left your machine and compare them against the results of an open implementation of that crypto-system. But you could never be sure there wasn’t some un-triggered bug or backdoor that would be activated later.

NickPyz · January 22, 2016, 9:11pm

Yes, I agree. That’s why I am here and not there.

I have faith that when the encryption issue is resolved in Syncthing, it will be based on an open and peer-reviewed system. So in that sense, it’s better to wait.

Meanwhile, those who have untrusted devices in their cluster (in my case, a 3rd party VPS) can always use TrueCrypt or VeraCrypt containers for any private data. Bit of a pain in the a$$ … but the VPS administrators theoretically can’t see what is being stored on their node.

ong · March 6, 2016, 11:43am

Hi

I want to upvote the feature request (as https://github.com/syncthing/syncthing/issues/109 )! I see the optional encryption as big bonus to SyncThing. Personally I am using Crashplan Free right now to sync - as I am backing up to peple I do not want to be able to read all my files. It SyncThing would enable encryption on a distant system … GREAT!

ong

kc1212 · March 8, 2016, 11:03am

I’m also interested in having the encryption feature.

Regarding the Crypto Proposal https://github.com/syncthing/syncthing/wiki/Crypto-Proposal, is this the direction that Syncthing is planning to take, or is it still an open question?

I’m asking because I saw a lot of interesting discussion here https://github.com/syncthing/syncthing/issues/109 regarding convergent encryption. But that was not mentioned in the Crypto Proposal.