Nodes see each other through dynamic discovery but remain "disconnected"

I’m trying to figure out why my nodes can see each other on the LAN but won’t connect and (as a result) won’t sync. This may be a bug but I wanted to check the forums for help first in case there’s something I’ve missed. Hopefully something with my local setup is incorrect and there’s a solution to document. This problem is affecting all the nodes on my LAN.

As of three days ago (on the 19th) syncthing stopped syncing my files. I’ve tried deleting the local cache files and running syncthing manually using STTRACE=discover syncthing. The web GUI gives the correct LAN IP addresses of the nodes I’m hoping to connect to, but the status is still listed as disconnected.

Here’s some typical output from the log

DEBUG: discover: Reconstructed URL is &url.URL{Scheme:"tcp", Opaque:"", User:(*url.Userinfo)(nil), Host:"192.168.0.2:22000", Path:"", RawPath:"", ForceQuery:false, RawQuery:"", Fragment:""}
[PDR6Y] 2017/09/22 10:17:12.706861 local.go:240: DEBUG: discover: Replaced address 0.0.0.0 in tcp://0.0.0.0:22000 to get tcp://192.168.0.2:22000
[PDR6Y] 2017/09/22 10:17:12.745981 local.go:175: DEBUG: discover: Received local announcement from 192.168.0.4:41282 for LIYLNLQ-DHZ6JQE-GWQA5ED-UJR6UEV-5ZVTHOR-NSRVQEX-UL2ZUGA-WR2QIAW
[PDR6Y] 2017/09/22 10:17:12.746027 local.go:238: DEBUG: discover: Reconstructed URL is &url.URL{Scheme:"tcp", Opaque:"", User:(*url.Userinfo)(nil), Host:"192.168.0.4:22000", Path:"", RawPath:"", ForceQuery:false, RawQuery:"", Fragment:""}
[PDR6Y] 2017/09/22 10:17:12.746034 local.go:240: DEBUG: discover: Replaced address 0.0.0.0 in tcp://0.0.0.0:22000 to get tcp://192.168.0.4:22000
[PDR6Y] 2017/09/22 10:17:17.510246 local.go:175: DEBUG: discover: Received local announcement from 192.168.0.3:46044 for PDR6Y6N-ZPVLTRL-N2KGXPW-67AFT4Z-ES667G5-T26LP6P-ETR6EYO-BULDDQ7

So even the log looks like it’s able to “hear” the other nodes. But none of the nodes can actually synchronize between each other.

FWIW:

>: uname -a [10:13:59]

Linux redacted 4.12.13-1-ARCH #1 SMP PREEMPT Fri Sep 15 06:36:43 UTC 2017 x86_64 GNU/Linux

I’m running syncthing as a user through systemd systemctl start syncthing@me.service

The arch repo updated to the current version 0.14.38 on the 19th, the same day my files stopped syncing.

If it comes to it I can force-downgrade to 0.14.37 but would like to avoid that hassle.

I suggest you try our latest release. I know one of the distros somehow managed to produce a binary that failed to establish connections and panic. There is no panic in your case, but I suggest you try out our binary to verify.

Downgrading to 14.37 didn’t help, so I’ll try the binary direct from source next.

Firewall blocking incoming connections to port 22000 would be my other guess.

1 Like

I was trying to figure out today why 2 nodes could see each other and could not connect, this had been happening regularly to me for the past days (with 0.14.37 under Arch Linux).

They would only reconnect if I restarted the 2 processes (on each computer) but would not reconnect if I restarted only one side.

I have fixed it by opening 22000 TCP. I would be curious to understand why Syncthing was not opening the port because my PNP works fine and it is able to open other ports. Anyway what looks like a similar case was fixed opening manually 22000.

So I checked to see if port 22000 was open and ss reported it was. I could see the announcements hit each port by watching tcpdump on all of my LAN nodes. Still nothing. I ran iptables -I INPUT -p tcp --dport 22000 -j ACCEPT and the nodes started communicating again.

So in this case at least the solution was to manually open port 22000 on each node.

That leaves a few questions:

  1. Why do I have to open the port explicitly?
  2. Why do I still see the multicast packets arrive even if the port is NOT open?
  3. What changed since about last week when the same setup was working okay?

Oh well, at least it works now, again. Thanks to all for the input.

The firewall on your computer is yours to manage and we can’t guess what might have happened to it. You can still see announcements on port 21027 even if you block incoming connections on port 22000, because they are different ports and protocols.

I’m continuing this thread in hopes that if anybody in the future has a similar problem they’ll see how it was fixed here. I’d change the title to include [SOLVED] but that doesn’t seem to be an option.

When I ran tcpdump I was only looking for connections on 22000 and saw what looked like announcements there, even before I modified iptables. The docs list 21027 as a local announce port and I’m seeing a lot of multicasts there now that things are connecting.

So I’ll just amend my stated solution: Open up port 22000 (and possibly 21027) in the firewall explicitly even if you see activity on them.

Indeed, tcpdump looks at interface traffic - before it is filtered by iptables.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.