Is syncthing right for me?

Support
1

I’ve been looking at syncthing, which seems pretty cool. What I’d like to do is list how I want my chosen syncing thing to work, and ask the question: is syncthing the tool I want? This is not a list of complaints that syncthing doesn’t do this stuff: it’s more a list of questions about how I’d best do this stuff with syncthing. “We don’t do that; we’re not going to do that; go find another tool” is a perfectly acceptable answer here! So, a list:

Sync multiple folders from my desktop machine to my server

I have a desktop Ubuntu machine, and I care a lot about the Documents, Pictures, and Work folders in my home directory. I’d like those to all be synced to my server, which is on the same LAN in my flat as the desktop machine; when I change a file on my desktop, that file should be copied to the server.

This seems to be doable, especially now that 0.7x supports multiple repositories, so I can make each of those folders be its own repository. I don’t really want the symlink approach, because it means that I have to set up the symlink farm on any other machines that I set up as well, but obviously I could do that if required.

Get at individual files from my phone

I’d like to be able to, from my phone, browse a list of the files that syncthing knows about, and explicitly choose one and download it to the phone. I don’t want to sync everything to the phone always. That way, if I’m on the road with just my phone, I can still browse and view any of the files I have synced. I don’t need to be able to add files from the phone to syncthing’s repositories – it’s download-to-phone only, not upload-from-phone.

This… doesn’t seem doable, as far as I can tell. Syncthing doesn’t have mobile clients, which is mostly OK (I can always write one which talks to the HTTP API, and I’d be happy to contribute that), but there are two other things there: the first is that there doesn’t seem to be an API way to list the contents of a repository and fetch them. There’s only the ability to change the metadata about a repository; I can add new repos through the web UI, but not see the contents of a repo. Maybe to do this I should be adding some other service which exposes the contents of a syncthing repository to the outside world – a webdav server? don’t know – but then having the syncthing repos ID isn’t all that useful. Does the global syncthing announce server let me ask it about a repos ID and then get a list of nodes and their global IPs so a mobile client can know where this webdav server is?

Secondly… as far as I can tell I’d need to expose port 22000 and the web UI port 8080 to the outside world in order to do this: that means that it’ll get hammered by scripts and so on. Is syncthing robust against this sort of thing? How do I, for example, password-protect it? Quite possibly this isn’t viewed as syncthing’s problem – if I want to expose syncthing to the outside world then I should be putting it in a chroot, putting a password-checking SSL proxy in front of it, and so on – but that sounds worrying.

Graphical view on sync status

As an Ubuntu person, I’d like to have an indicator in the menu bar showing when syncing’s happening, what’s been synced and what hasn’t, and so on. Obviously I can always open the web UI for a detailed view, but it’d be nice to have something there in my status menus. This seems like a fairly obvious client of the API, and that sounds like something I’d be fine with writing, if it’s doable.

I’d be interested in thoughts on these points, whether they’re “no you don’t want syncthing” thoughts or “yes! this is how you should set things up” thoughts… :smile:

1 Like
2

Hi! First of all, I think this is an awesome discussion to have (what problems do we want solved, really?) and it’s great to hear. For the shortest possible answer, I think synching plus something is what you want, or possibly syncthing in the future. I think all your use cases are valid and are things that I would like to see happen… Looking at the points one by one,

Sync multiple folders from my desktop machine to my server

Yep. This is use case 1 A for syncthing. Works fine as it is today, and in v0.8 (not released, look again next week) the GUI is even reasonable for multiple repositories and I think it’s pretty usable.

Get at individual files from my phone

Trickier! My first thought was what you had already identified as a solution (if maybe a half-assed one): using syncthing to sync the files, then exposing them via a web server or similar.

As it is today, syncthing runs out of the box on Ubuntu phones, for what it’s worth. It will probably never ever run on an iOS device (Go not welcome). It could potentially maybe run on Android? Windows Phone is out of the question, probably.

But syncthing itself isn’t optimal for running on a phone and there are other things going on here. I talked the other day to a guy who had started writing an implementation of the protocol (BEP) in Java to be able to run natively on Android. Things he was figuring out was those you mention; not syncing all of a repository but only select files for example. The protocol supports this.

Implementing a light weight protocol over HTTP - maybe more of an extended API - to list, get and put files wouldn’t be very difficult. This would enable a lighter weight client running on a phone to talk to a “master” instance running on a server, for example. Perhaps that would be useful, perhaps not…

Graphical view on sync status

Yep. The same would be needed for Mac (menu item) and Windows (tray thing). I might possibly write one of those for Mac, but I won’t for Linux (because I don’t run it on the desktop) or Windows (because I don’t run it at all). That said, that is the long term plan as far as I see it (some lightweight native GUI tool that starts syncthing in the background and talks to it over the REST API to get status and tell it what to do) so I really hope someone wants to write those thingies. :smile:

3

The BEP protocol (port 22000) should be very resistant against hammering. Basically, it’s a regular TLS connection, authenticated by the certificates on both end. If the certificates don’t match or something is wrong with the handshake the connection is dropped, and this is all handled by the Go TLS stack which is pretty good. If you get past that step it means you are a node we have explicitly allowed so the security requirements are a little lower but it should still handle corrupt protocol messages. It hasn’t been fully hardened though so there might be issues lurking.

As of v0.8 (…) the web GUI (port 8080) has Basic Authentication support. Without HTTPS this is roughly equivalent to hiding the key under the rock by the door… HTTPS is coming though, sometime in v0.8.x - we already have the certificates etc in place for BEP use, so all that’s needed are a couple of config options and a few lines of code.

4

In v0.8, symlinks are no longer followed on the source at all. It was always just a hack to enable multiple repositories without actually supporting multiple repositories. :slight_smile: At some point, we will need support to optionally synk symlinks “verbatim” but this needs thinking about, for how to handle it cross platform etc.

5

Excellent set of answers. Responses below, along with One More Requirement That I Forgot :smile:

First, hooray, symlinks are a hack, I hate them, totally agreed, I loved that Ubuntu One didn’t follow them, and syncthing working that way as well is excellent news.

Indeed not. Hence why I’d like, as you note, a lightweight client which talks to a (not-yet-existing) files REST API rather than full-on syncthing itself. However, that means exposing both the BEP port and the REST API to the internet. I’m quite prepared to believe that BEP is well-secured and safe to expose, but the stock web API… maybe a bit less so? I’m not thinking just about SSL and auth here but about injection attacks and so on, which it may not be hugely tested for because You’re Not Really Supposed To Give Untrusted People Access To It At All. Happy to be corrected here, of course :smile:

If the API exists and is documented (and I end up using syncthing) then I’m happy to write an Ubuntu indicator. Mac stuff can be your job; Windows and Linux desktops can be someone else’s :wink:

Is BEP stuff talked over HTTP? Or is it a custom protocol? Specifically, can I talk it from browser JavaScript?

Also, the new requirement which I forgot to mention before:

Syncing between NATs

I have a third computer – actually, it’s my dad’s computer – at his house, on his LAN, behind his router. I would like him to be able to sync a folder from that computer onto my server, which is on my LAN behind my router. Is that doable? Would I need to have my server’s BEP port open to the world to do that?

6

I don’t think injection attacks etc will be able to accomplish anything more destructive in the interface than you can already do whenever you have access - i.e. reconfigure syncthing…

BEP itself is binary and not carried over HTTP. It could be implemented in Javascript, but not I think in a browser since I don’t think you get access to send bits over raw TLS connections…

Regarding NAT, that’s a known pain point. :slight_smile: https://github.com/calmh/syncthing/issues/79 tracks the implementation of UPnP (will come some day), until then you’ll need to set up a manual port forward of port 22000 from the NAT router to the internal computer.

I get from your questions that you are worried about having the BEP port “open to the world”; I don’t think you should be. Having it that way is a requirement and is what happens even when there is UPnP etc implemented. All my stuff is set up that way. But if you prefer a different port, simply set the listen address in the GUI and forward that port instead and things should wire up themselves.

7

I’m not, really. Your notes above about how BEP works and that it’s secured are convincing. What I’m worried about is having the web port open to the world (so that a mobile app can talk to it), because it’d be quite reasonable for you to say “no, the web stuff isn’t particularly designed to be secure against attackers, because you’re only supposed to access it from localhost as a control interface, not expose it to the world”. If you’re not saying that – if you’re confident that the web UI should be exposeable to the world without problems – then OK, no worries. Note that if I can reconfigure your copy of syncthing then I can drop arbitrary files onto your computer, which would be a bit of a security problem :wink:

8

duh.

It has just occurred to me that I don’t need a mobile app which can browse files over some sort of clever not-yet-written REST API. I can just use sftp. And there are zillions of apps which can do that. So that solves the access-my-files-from-my-phone problem :slight_smile:

9

Clarifying this: my dad’s machine is behind his NAT, and I want folders on his machine to be synced with my server, behind my NAT. Does this mean that I need to open the BEP port to the world on his machine and mine, or do I just need to do it on my router and not his?

10

One side is enough, so one can reach the other.

11

At the moment, yeah, kinda like that. At least because it’s currently unencrypted and someone can man-in-the-middle it and grab your password, once set. But if a password is set and the attacker doesn’t know it, I fully expect it to be secure. If it’s not, it’s a very serious bug indeed and not your own fault for exposing it.