[Docker] Syncthing and Syncthing Discovery behind Nginx reverse proxy with Let's Encrypt


(Scott Hansen) #1

I recently finished setting up a completely Dockerized setup of Syncthing and the Syncthing Discovery server (among some other services) behind an Nginx reverse-proxy. The information here is mostly the same as in the docker-letsencrypt-nginx-proxy-companion README, but there are a few gotchas.

What do we get out of this?

  1. Easy SSL-secured subdomain name access to our Syncthing GUI and Syncthing discovery server. syncthing.domain.com and discovery.domain.com
  2. Easy to move the whole setup to a different host
  3. Easy to add additional reverse-proxied containers without having to restart or reconfigure the nginx stack.
  4. Client containers can be easily stopped/restarted without affecting the rest of the clients.
  5. Automatic SSL certificate generation and renewal through the Let’s Encrypt service.

Requirements

  1. A host with Docker installed and domain name/subdomains setup.

  2. Docker images (these can be pulled at runtime or beforehand, except Syncthing needs some setup first). Note that I don’t provide images for syncthing or discosrv through the Docker Hub…you will need to build your own.

  1. Nginx template for docker-gen. We will modify it later.

Setup

  1. The nginx template needs to be modified to add the following:
  • The X-SSL-Cert headers proxy_set_header X-SSL-Cert $ssl_client_cert;
  • ssl_verify_client directive. ssl_verify_client optional_no_ca; Notice the if statement bracketing this directive. Make sure to update the (sub)domain name that you will be using for the discovery server. The discovery server container is the only one that should need this.

Partial nginx.tmpl:

....
proxy_set_header Connection $proxy_connection;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $proxy_x_forwarded_proto;
proxy_set_header X-SSL-Cert $ssl_client_cert;
{{ end }}

server {
	server_name _; # This is just an invalid value which will never trigger on a real hostname.
	listen 80;

.......

	{{ if (exists (printf "/etc/nginx/certs/%s.dhparam.pem" $cert)) }}
	ssl_dhparam {{ printf "/etc/nginx/certs/%s.dhparam.pem" $cert }};
	{{ end }}

	add_header Strict-Transport-Security "max-age=31536000";

    {{ if (eq $host "discovery.domain.com") }}
    ssl_verify_client optional_no_ca;
    {{ end }}
	{{ if (exists (printf "/etc/nginx/vhost.d/%s" $host)) }}
......
  1. Create directories on your filesystem (you could also just use volumes) to store the downloaded certificates and the nginx.tmpl file. For our purposes here, I’m storing certificates in /usr/local/etc/nginex/certs and the nginx.tmpl file in /usr/local/etc/nginx/nginx.tmpl

Running

We will end up with five running containers: nginx_run, nginx_gen_run, nginx_letsencrypt_run, syncthing_run and syncthing_discovery_run. How you manage your running containers is dependent on your platform. I’m currently using Ansible to start the containers for this particular set, but I use systemd service files on another platform. To make it easy for me (and it’s fairly readable as well), I’ll show the Ansible playbook snippets to run the various containers. These should be easily translated into regular docker run statements or Docker Compose stanzas.

  • Nginx
- name: run Nginx container
  docker:
    image: nginx
    name: nginx_run
    state: running
    ports: 80:80,443:443
    stdin_open: True
    tty: True
    volumes: 
      - /etc/localtime:/etc/localtime:ro
      - /etc/nginx/conf.d
      - /etc/nginx/vhost.d
      - /usr/share/nginx/html
      - /usr/local/etc/nginx/certs:/etc/nginx/certs:ro

Equivalent docker run syntax (I’ll just put this here for the first example):

docker run -d --name nginx_run \
   -p 80:80 -p 443:443
   -v /etc/localtime:/etc/localtime:ro \
   -v /etc/nginx/conf.d \
   -v /etc/nginx/vhost.d \
   -v /usr/share/nginx/html \
   -v /usr/local/etc/nginx/certs:/etc/nginx/certs:ro \
   nginx
  • The docker-gen container. Automatically generates reverse-proxy nginx configuration for attached container.
- name: run Nginx-gen container
  docker:
    image: jwilder/docker-gen
    command: -notify-sighup nginx_run -watch -only-exposed -wait 5s:30s /etc/docker-gen/templates/nginx.tmpl /etc/nginx/conf.d/default.conf
    name: nginx_gen_run
    state: running
    stdin_open: True
    tty: True
    volumes_from: nginx_run
    volumes: 
      - /etc/localtime:/etc/localtime:ro
      - /usr/local/etc/nginx/nginx.tmpl:/etc/docker-gen/templates/nginx.tmpl:ro"
      - /var/run/docker.sock:/tmp/docker.sock:ro
  • The Let’s Encrypt nginx client container. Automatically requests and downloads SSL certificates for client containers.
- name: run Nginx-Letsencrypt container
  docker:
    image: jrcs/letsencrypt-nginx-proxy-companion
    name: nginx_letsencrypt_run
    state: running
    stdin_open: True
    tty: True
    volumes_from: nginx_run
    volumes: 
      - /etc/localtime:/etc/localtime:ro
      - /usr/local/etc/nginx/certs:/etc/nginx/certs:rw
      - /var/run/docker.sock:/var/run/docker.sock:ro
    env:
      NGINX_DOCKER_GEN_CONTAINER: nginx_gen_run
  • Syncthing container (if desired).

    • Note that although the reverse proxy forwards the GUI port 8384, it does not forward the working ports 22000 or 21027/udp. Therefore if you add this client address to another client for direct access, it would be tcp://<domain.name>:22000 instead of tcp://<sub.domain.name>:22000.

    • Another caveat to running Syncthing in a container is that Local discovery won’t work unless you use host-only networking. This typically won’t be done on the same host as a Discovery server anyways.

    • Also note that I’m keeping my Syncthing configuration in a data-only volume syncthing_config.

    • The VIRTUAL_HOST, VIRTUAL_PORT and VIRUAL_PROTO are the environment variables that get passed to the docker-gen container to generate the reverse-proxy config. Make sure you set the subdomain you will be using.

    • The LETSENCRYPT_EMAIL and LETSENCRYPT_HOST are for the nginx_letsencrypt container. The LETSENCRYPT_HOST should match the VIRTUAL_HOST.

- name: run Syncthing
  docker:
    image: syncthing
    expose: 22000,21027/udp,8384
    name: syncthing_run
    ports: 22000:22000,21027:21027/udp
    state: running
    stdin_open: True
    tty: True
    volumes: 
      - /home/username:/home/username
      - /etc/localtime:/etc/localtime:ro
    volumes_from: syncthing_config
    env: 
      VIRTUAL_HOST: syncthing.domain.com
      VIRTUAL_PORT: 8483
      VIRTUAL_PROTO: https
      LETSENCRYPT_HOST: syncthing.domain.com
      LETSENCRYPT_EMAIL: user@email.com
  • Syncthing Discovery server. Same comments apply for the environment variables here. Notice the -http flag to run the discovery server in proxy mode. This feature is only available (as of today) in the git version, not the v0.12.2 release.
- name: run Syncthing Discovery Server
  docker:
    image: syncthing_discovery
    name: syncthing_discovery_run
    command: -http
    expose: 8443/tcp
    state: running
    stdin_open: True
    tty: True
    volumes:
      - /etc/localtime:/etc/localtime:ro
    env: 
      VIRTUAL_HOST: discovery.domain.com
      VIRTUAL_PORT: 8443
      LETSENCRYPT_HOST: discovery.domain.com
      LETSENCRYPT_EMAIL: user@email.com

You can add other containers as well using the same pattern, and they will get certificates (if requested) and a reverse-proxy configuration automatically assigned without having to restart the whole stack.

TODO

  1. Try using Docker Networking features from v1.10+ instead of the NGINX_DOCKER_GEN_CONTAINER env variable and legacy style links.

  2. Systemd service files to start/restart the containers in the correct order

  3. Docker Compose file to start/restart the containers in the correct order

Feedback and corrections are welcome!

Scott


discovery server with --http response 404 page not found.
(Antony Male) #2

A post was split to a new topic: Discovery server behind an Apache reverse proxy


(Antony Male) #3

A post was merged into an existing topic: Discovery server behind an Apache reverse proxy


(Audrius Butkevicius) #4

I would appreciate if you’d make a pull request to the docs explaining how to setup nginx as a reverse proxy with SSL handoff, as it’s a fairly complicated step.

I don’t think the letsencrypt stuff should go in the docs as the setup involves exposing docker socket to an unprivileged container which I consider an unsafe practice, plus it suggests providing this socket to a bunch on containers which haven’t been proven to do only the thing they advertise on the tin.

I guess we can link from the docs to this thread, and people can see this disclaimer.


#5

@AudriusButkevicius I’m looking around in the documentation before setting up a discovery an relay server, I’d assumed it would be ideal to use legitimate certificates for these services. Is there any documentation (other than what @firecat4153 has here) for proxying to discovery/relay services from a nginx?


(Audrius Butkevicius) #6

Relays don’t use HTTP so nginx has nothing todo with it (apart from status reporting to the pool but that’s optional anyway). For discovery, I don’t think there is anything in the docs, so this is probably as good as it gets. Also check -help on the discovery server.


(Jakob Borg) #7

Relaying can indeed not be proxied like that. Discovery can, however. This is what we’re using for the public servers:

Nginx with appropriate TLS config plus the following site definition, one for IPv4 and one for IPv6:

server {
	listen [2a03:b0c0:0:1010::4ed:3001]:443 ssl;
	server_name localhost;

	proxy_cache discosrv-v6;
	proxy_cache_valid 200 404 1m;
	proxy_cache_use_stale error timeout invalid_header updating http_500 http_502 http_503 http_504;
	access_log off;

	location /v2 {
		proxy_pass http://127.0.0.1:9093;
		proxy_set_header X-SSL-Cert $ssl_client_cert;
		proxy_set_header X-Forwarded-For $remote_addr;
	}
}

which then proxies to a discosrv instance:

#!/bin/sh

ulimit -n 10240

exec setuidgid discosrv /usr/local/bin/stdiscosrv \
  -db-backend=postgres \
  -db-dsn=postgres://discosrv:...@localhost/discosrv-v013-v6 \
  -limit-avg=5 \
  -limit-burst=10 \
  -limit-cache=40000 \
  -listen="127.0.0.1:9093" \
  -http

The magic part for discosrv is the -http one.

I’ve considered docker-composing the nginx plus postgresql plus discosrv setup, but it’s been too much work for too little so far.


x509: certificate signed by unknown authority
(Scott Hansen) #8

@AudriusButkevicius, sorry, I must have missed your post last year. I’ll go dredge up my memories of this thread and try to update and convert it into something suitable for the docs.


(Scott Hansen) #9

Oh wait…I did get a PR merged! I completely forget PR #149. Sigh. Getting old I guess…