Fehl:8 https://apt.syncthing.net syncthing Release
Certificate verification failed: The certificate is NOT trusted. The certificate chain uses expired certificate. Could not handshake: Fehler bei der Verifizierung des Zertifikats. [IP: 2a03:b0c0:0:1010::99:8001 443]
I believe apt (at least on Debian) uses GnuTLS not OpenSSL, and GnuTLS is sadly horribly broken (also this, plus a dozen more links to issues). I believe the GnuTLS version shipped by Ubuntu is simply unable to validate Let’s Encrypts new “expired” certificate chain.
We might need to ask @calmh to configure the server to serve Let’s Encrypts “alternate chain” (which doesn’t include the cross-sign up the expired DST Root CA X3), which will fix connections with broken OpenSSL/GnuTLS.
The cross-sign to the expired root is only useful for Android devices, and I have doubts that there are many users connecting to apt via Android.
I tried our installation instructions in Docker images of Ubuntu 14.04, 16.04, 18.04 and 20.04. In 14.04 apt-get fails with a handshake error, in 16.04 and newer it works just fine out of the box. Perhaps the Docker images are not representative somehow?
Yeah that are the backports I forgot about. Last month they shipped patches to all supported versions which mitigate this issue. It’s a bit embarrassing that I completly forgot the backports, because I actually talked to the devs making these backports… Let’s just blame it on not enough sleep.
It still holds that the cross-sign is only useful for Android devices, so the apt.s.n endpoint is probably better off without it - it will help all machines not getting these fixes.
apt.s.n runs a (very old) Caddy 1.something, which I tried upgrading to a recent Caddy yesterday but didn’t get all the nuances correctly ported to the new config format, and then I dropped it when I realized the problem wasn’t that Caddy was producing a too-old certificate chain. If there’s an incantation in new or old Caddy that helps then we could add that incantation, but otherwise it seems to me like an up to date system of even the old distributions should work fine with the current setup…
I’m also sorely tempted to outsource this problem and just throw the whole apt.s.n archive up into Azure/Netlify/GitHub pages or whatever and let them serve it… It’s just static files after all, with a few hundred gigs per month outgoing traffic.
Yeah ZeroSSL is a reseller for Sectigo and their roots still have some years left of lifetime (+ are also very old, so highly compatible). I do however consider switching the CA a “in emergency break glass” solution, and I don’t see the emergency right now (and I say this even though I’ve answered over 250 help topics about “all my systems broke just now, it says certificate expired” in the last ~2 days). ZeroSSL also has some questionable business practices. In addition, calmh said this is running Caddy v1, so it’s not possible anyway.