Strange device ID behaviour

Situation: two brand new installs on two workstations. Both on openSUSE Leap, ST v0.14.15

On both work stations:

rm -rf .config/syncthing
rm -rf Sync
syncthing -generate=.config/syncthing
/usr/bin/syncthing -no-browser -logflags=2 -home=/home/koos/.config/syncthing -verbose

Now the log file is flooded with unmatched device ids. And notice how the device ID NIE6MZ5 is repeated multiple times:

dec 19 17:39:34 specht syncthing[12729]: [7IBNY] 17:39:34 INFO: Could not connect to relay relay://146.185.129.232:443/?id=XP5SIGA-QZNLY3K-DCH2GUL-SEPITP6-W4G35B3-RQTYWRK-ZYJCBNA-YIIV5AV&pingInterval=1m0s&networkTimeout=2m0s&sessionLimitBps=0&globalLimitBps=0&statusAddr=:22070&providedBy=: relay id does not match. Expected XP5SIGA-QZNLY3K-DCH2GUL-SEPITP6-W4G35B3-RQTYWRK-ZYJCBNA-YIIV5AV got NIE6MZ5-3M6XIKV-PDO6NZK-KT7VHJE-IQ4UT7T-MCWRDAP-7QPM2WW-7AZUMQO
dec 19 17:39:34 specht syncthing[12729]: [7IBNY] 17:39:34 INFO: Could not connect to relay relay://37.187.196.165:22067/?id=CCQTHY4-6PKXSCR-62CEVZX-N67AN2Z-Q543WOU-DCXCZU4-KL5QWQG-EOH3LQA&pingInterval=1m0s&networkTimeout=2m0s&sessionLimitBps=0&globalLimitBps=0&statusAddr=:22070&providedBy=Arcanite Solutions: relay id does not match. Expected CCQTHY4-6PKXSCR-62CEVZX-N67AN2Z-Q543WOU-DCXCZU4-KL5QWQG-EOH3LQA got NIE6MZ5-3M6XIKV-PDO6NZK-KT7VHJE-IQ4UT7T-MCWRDAP-7QPM2WW-7AZUMQO
dec 19 17:39:34 specht syncthing[12729]: [7IBNY] 17:39:34 INFO: Could not connect to relay relay://176.9.54.201:22067/?id=VO3GBYK-BGNVTP7-DFQJZCP-6LGVZVU-SU364F4-EKHWXRV-W4FOPN5-X7T7NQM&pingInterval=1m0s&networkTimeout=2m0s&sessionLimitBps=0&globalLimitBps=41943040&statusAddr=:22070&providedBy=adrian AT blinkenlights.ch: relay id does not match. Expected VO3GBYK-BGNVTP7-DFQJZCP-6LGVZVU-SU364F4-EKHWXRV-W4FOPN5-X7T7NQM got NIE6MZ5-3M6XIKV-PDO6NZK-KT7VHJE-IQ4UT7T-MCWRDAP-7QPM2WW-7AZUMQO
dec 19 17:39:34 specht syncthing[12729]: [7IBNY] 17:39:34 INFO: Could not connect to relay relay://178.238.228.171:22067/?id=HRHITUZ-PECCMAK-JEKLFHZ-3FBH6ZI-VS6CX3U-LSAJRXX-B6PJT5N-C4M7QAO&pingInterval=1m0s&networkTimeout=2m0s&sessionLimitBps=0&globalLimitBps=3750000&statusAddr=:22070&providedBy=Stefano: relay id does not match. Expected HRHITUZ-PECCMAK-JEKLFHZ-3FBH6ZI-VS6CX3U-LSAJRXX-B6PJT5N-C4M7QAO got NIE6MZ5-3M6XIKV-PDO6NZK-KT7VHJE-IQ4UT7T-MCWRDAP-7QPM2WW-7AZUMQO

These a just a few of the dozens of entries.

On both workstations I go the the settings screen and disable:

  • NAT traversal
  • Global discovery
  • Relaying

From the console output I pick up the ID of my other workstation:

VERBOSE: Discovered device 7IA6W2D-QMB7RX6-VW3EWTX-FUQWPMI-WV5662H-CF2PFKK-H46IWUH-HO5N4AP at [tcp://192.168.1.240:22000]

Now the strange bit: when I pair the other endpoint using it’s new id 7IA6W2D, I get to see two new remote devices:

  1. my second workstation added manually with the ID 7IA6W2D
  2. the device making itself known through a popup, having the infamous device ID NIE6MZ5. Even more strange, the second device assume the fully qualified hostname and IP address of my second workstation.

Whatever I try:

  1. I can’t get rid of the second “ghost” remote device.
  2. Syncthig doesn’t work. Not even on the brand new default folder Sync.

For those wondering: my router is locked down (no PNP, and no incoming ports).

Also, v0.14.09 worked flawless for me. It is only after the upgrade to v0.14.15 that I am having these problems.

The repeated ID mismatch sounds like a https proxy trying to MITm. I suspect if you down grade it will most likely not work anyway as there is something on the network you are on.

Also, check for multiple instances of syncthing on the other end.

1 Like

Thanks for the reply!

Checked and double checked: no multiple instances running.

You should check what process owns that port as you see the incoming connection.

Starting to look like black magic:

The ghost device NIE6MZ5 is not the remote partner but the device 7IBNY itself!

dec 19 20:07:30 specht syncthing[12784]: [7IBNY] 20:07:30 INFO: Established secure connection to NIE6MZ5-3M6XIKV-PDO6NZK-KT7VHJE-IQ4UT7T-MCWRDAP-7QPM2WW-7AZUMQO at 192.168.1.213:41860-192.168.1.240:22000 (tcp-client)
dec 19 20:07:30 specht syncthing[12784]: [7IBNY] 20:07:30 INFO: Device NIE6MZ5-3M6XIKV-PDO6NZK-KT7VHJE-IQ4UT7T-MCWRDAP-7QPM2WW-7AZUMQO client is "syncthing v0.14.15" named "zwaluw.lan.pohw.nl"
dec 19 20:13:31 specht syncthing[12784]: [7IBNY] 20:13:31 VERBOSE: Disconnected from device NIE6MZ5-3M6XIKV-PDO6NZK-KT7VHJE-IQ4UT7T-MCWRDAP-7QPM2WW-7AZUMQO

In my lan: specht = .213 zwaluw = .240

Now watch how NIE6MZ5 is also .213, but pretends to be zwaluw

BTW, I checked with netstat, and the associated process belongs to the syncthing process I’ve fired up at specht itself (.213)

So I am not sure I understand the problem, NIE6MZ5 connects to 7IBNY , judging from the logs. The fact that is succeeds implies that it’s a trusted device.

As you run syncthing, you can see My ID: ... in the logs which explains who’s who.

The fact that all your relays and the device is getting the NIE6MZ5 device ID indicates something is man-in-the-middling your connections. Either a proxy or your antivirus, most likely.

No proxy, no virus scanner. Nothing local (unless I’m hacked) which can intercept my traffic.

Some further testing really got to me.

Eventually I’ve plugged my laptop right into the ISP line, straight into the big bad internet with just FW protection.

dec 21 20:50:30 zwaluw.lan.pohw.nl syncthing[5304]: [LCQ7W] 20:50:30 INFO: Could not connect to relay relay://188.32.25.120:22067/?id=MO54YEY-I5BBMWM-NMSVXY2-TIXDICJ-KORQIGM-D22ENBU-KUAKKLR-PJ6TXQS&pingInterval=1m0s&networkTimeout=2m0s&sessionLimitBps=212992&globalLimitBps=20971520&statusAddr=:22070&providedBy=https://keybase.io/ushkinaz: relay id does not match. Expected MO54YEY-I5BBMWM-NMSVXY2-TIXDICJ-KORQIGM-D22ENBU-KUAKKLR-PJ6TXQS got NIE6MZ5-3M6XIKV-PDO6NZK-KT7VHJE-IQ4UT7T-MCWRDAP-7QPM2WW-7AZUMQO

So again the notorious NIE6MZ5.

I took a different, almost clean other laptop (my VPN gateway) and installed fresh copy of ST v014.15.

F*ck me…:

[VYX2X] 20:58:39 INFO: Could not connect to relay relay://37.59.37.175:22067/?id=DWRGRTQ-X6NWNW6-FOZLH2N-HT4Q2O2-UQ7AJHD-CPWLGNO-UBULHR7-2AKIPQY&pingInterval=1m0s&networkTimeout=2m0s&sessionLimitBps=0&globalLimitBps=0&statusAddr=:22070&providedBy=: relay id does not match. Expected DWRGRTQ-X6NWNW6-FOZLH2N-HT4Q2O2-UQ7AJHD-CPWLGNO-UBULHR7-2AKIPQY got NIE6MZ5-3M6XIKV-PDO6NZK-KT7VHJE-IQ4UT7T-MCWRDAP-7QPM2WW-7AZUMQO

Again the NIE6MZ5

As soon as I disconnect the central ISP internet cable all errors are gone. In all cases.

At this point I have not further suggestion than that there is a bad/rogue node out there.

What don’t understand is how that interferes with my local LAN device-to-device communication. But perhaps that question remains unanswerable.

Something is trying to intercept your traffic, potentially on the ISP or on your box (due to malware), yet given laptop A happens on your ISP, laptop B on VPN I’d say it’s malware.

I’ve got the same problem with NIE6MZ5 device ID and OpenSuse Leap install my syncthing configuration is the following:

  • 2 Fedora PCs (f24 & f25) with syncthing working flawlessly between them
  • 1 OpenSuse Leap 42.2 with syncthing having the NIE6MZ5 syndrom All have the same 0.14.16 version According to what I read in the thread, I wonder if there is not a problem with syncthing distributed through OpenSuse repo

I’ve dug further but couldn’t find any source of malware. Just to get rid of the problem I’ve wiped my laptop did a fresh install of openSUSE Leap + Syncthing v0.14.9 (SUSE baseline).

The errors are now gone and it looks to be solved.

I’ll report back if there is anying special to report.

Many thanks for all your advice and effort! :relaxed:

Happy Xmas!

Wow!

That is what I’ve wondered myself as well. I did some investigation in that direction and downloaded the source rpm (v0.14.16). The Syncthing tarball in the src does match up (md5) with the tarball from the Syncthing website. Ofcourse that is not water tight proof. The binary can still be loaded with malware. Or have another malfuncion.

Now the v0.14.16 is not the openSUSE baseline. For the moment that’s why I stick to the openSUSE repo regarding Syncthing (v0.14.9)

This peaked my interest so I downloaded the latest (?) Syncthing from the OpenSUSE and extracted it on my Ubuntu VM. It appears to work fine so far as at least starting up and joining a relay which is more than it does for you guys. So I don’t think there is anything seriously wrong with the binary. Perhaps the OpenSUSE distribution itself uses a proxy or something and that could be the explanation.

(The binary is also shipped with auto upgrades enabled, which presumably doesn’t work for a normal user but seems like a packaging bug.)

openSUSE maintains 2 “baselines”:

  1. Rolling release (openSUSE Tumblweed)
  2. Stable release (openSUSE Leap)

You can find the available packages here for all combined openSUSE releases: http://software.opensuse.org/package/syncthing

That is including community contributions.

I’m out for the rest of the. Familily get together… :fork_and_knife:

v0.14.15 does not appear to be a packaged version:

So yeah, still no idea.

I solved my problem and it is related to a specific version. As noted by Ceaus, official version of syncthing in OpenSuse is v0.14.9. In order to keep close to versions of Fedora PCs, I chose to add the Network repository https://build.opensuse.org/package/show?project=network&package=syncthing which is available in the list given OpenSuse software search:

In the Network repo, syncthing is in v0.14.16.89 I then added ccgx, one of the four repositories providing syncthing in v0.14.16.90 (MasterPatricko is also providing v0.14.16.89 I have to admit that for the sake of correct test methodology I should have used this one) I worked straight out of install! I did not have time to investigate into the package coming from Network repository but will notify the repo maintainer.

v0.14.15 does not appear to be a packaged version:

That was last week. In the mean time they’ve moved forward to v04.14.16.

Right, but then you’re talking about some random persons personal or community build? Perhaps ask them what’s up?

I got word back from the maintainer. If I understood it correctly openSUSE uses different Go versions. The Syncthing maintainer is supplying a Go patch to support these multiple Go versions for Syncthing. And there was a problem with this patch. The issue with the patch is expecting to have this negative side effect. The maintainer has fixed it today, and we’ll see his new package on the next repo build.

Many thanks again for your help!