remote device missing in cluster config error

After I updated to 1.24 I can no longer sync files. On one of the machines I see the following error in the log file:

remote device missing in cluster config

I looked online but could not find out what to do to fix it.

closed: handling cluster-config: remote device missing in cluster config

I try a bunch of things but the error in v1.23.7 remains the same. On remote device it runs v1.25 but locally I rely on Fedora 37 and have to wait till the repository is updated.

The only thing I could see that would cause this is if it’s somehow trying to use multiple connections (supported in 1.25 but not in 1.23) which, of course, shouldn’t happen.

If you want to debug it, enable STTRACE=connections on both sides and post full logs from when the error happens, with some time before and after it visible.

I upgraded to Fedora 38 and now have 1.24 on the client-side and 1.25 on the server side.

The error persists. I see:

sent cluster-config without the device info for the remote on folder

Maybe on the server-side I did the upgrade twice by accident when trying to solve another problem. Once under root and once not. What would be the easy way to see if there run two instances of SyncThing on the same server?

To list all running processes for all users:

$ps aux

Or: $ps aux |grep -i syncthing

user1+ 1350 0.0 0.2 1250336 4944 ? Ssl Oct20 0:21 /usr/local/bin/syncthing -no-browser -gui-address=0.0.0.0:8384 -no-restart -logflags=0

user1+ 1355 0.2 7.4 1795972 157228 ? SNl Oct20 24:59 /usr/local/bin/syncthing -no-browser -gui-address=0.0.0.0:8384 -no-restart -logflags=0

root 15680 0.0 0.0 9044 832 pts/0 S+ 13:36 0:00 grep --color=auto -i syncthing

Looks like 3 processes.

I believe I did not run it under root before.

I was suffering from a bug introduced a short time back where it was complaining about untrusted devices and while trying to solve it I made the updates and I think I caused these multiple processes.

How can I see which is the one to keep? And then to try to remove the others.

No worries, it’s fine. The 3rd result is just from grep seeing its own process when you ran 'ps aux |grep -i syncthing` as root. :wink:

Instead of ps aux, try the following variation:

ps -ef | grep syncthing

Here’s the output from my system:

gadget      3335    3326  0 09:26 ?        00:00:00 /usr/bin/syncthing -no-browser
gadget      3340    3335  0 09:26 ?        00:00:46 /usr/bin/syncthing -no-browser

Note how Syncthing with process ID (PID) 3335 is the parent process of Syncthing with PID 3340.

From Syncthing’s FAQ:

Why do I see Syncthing twice in task manager?

One process manages the other, to capture logs and manage > restarts. This makes it easier to handle upgrades from within > Syncthing itself, and also ensures that we get a nice log > file to help us narrow down the cause for crashes and other bugs.

^

I see a little more now:

Fedora 38:

sent cluster-config without the device info for the remote on folder

Server side:

Added connection for JPA (now 1), they want 0 connections
Established secure connection to JPA 2/tcp-server/TLS1.3-TLS_AES_128_GCM_SHA256/WAN-P30-2
Device JPA client is "syncthing v1.24.0" named "fedora" at 1-12/tcp-server/TLS1.3-TLS_AES_128_GCM_SHA256/WAN-P30-2
Connection loop
Resolved device JPA addresses: []
Next connection loop in 1m0s
Removed connection for JPA (now [])
Connection loop
Resolved device JPA addresses: []
Next connection loop in 1m0s
Lost primary connection to JPA at 12/tcp-server/TLS1.3-TLS_AES_128_GCM_SHA256/WAN-P30-2: writing message: write tcp4 12: write: connection reset by peer (0 remain)

Is this sufficient?

No? I mean, it’s not even approximately what I asked for, is it?

I went to Actions > Logs > Debugging facilities and clicked ‘connections’ and it gave me a more extensive log.

It made a sync attempt with the machine in question and I copied a piece of the log when it tried to connect at both sides.

Then removed privacy related info such as machine names, IP addresses and such.

I am not sure what instead I suppose to do.

You removed too much, including the timestamps from each log entry. :smirk:

Instead of stripping the machine names, IP addresses, etc. replace them with fake info and leave everything else as-is.

The desire for privacy is understandable, but also understand that a heavily redacted tiny snippet from a log makes diagnosing a problem from a distance very difficult.

Fedora 38:

2023-11-14 03:08:01 Listen (BEP/tcp): connect from 111.11.114.409:11181
2023-11-14 03:08:02 Added connection for RJKRKF (now 1), they want 0 connections
2023-11-14 03:08:02 Established secure connection to RJKRKF at 120.484.38.22:57222-111.11.114.409:11181/tcp-server/TLS1.3-TLS_AES_128_GCM_SHA256/WAN-P30-SFDFHDFHDGHDFIU0N
2023-11-14 03:08:02 Device RJKRKF client is "syncthing v1.24.0" named "fedora" at 120.484.38.22:57222-111.11.114.409:11181/tcp-server/TLS1.3-TLS_AES_128_GCM_SHA256/WAN-P30-SFDFHDFHDGHDFIU0N
2023-11-14 03:08:03 Connection loop
2023-11-14 03:08:03 Resolved device RJKRKF addresses: []
2023-11-14 03:08:03 Next connection loop in 1m0s
2023-11-14 03:08:03 Removed connection for RJKRKF (now [])
2023-11-14 03:08:03 Connection loop
2023-11-14 03:08:03 Resolved device RJKRKF addresses: []
2023-11-14 03:08:03 Next connection loop in 1m0s
2023-11-14 03:08:03 Lost primary connection to RJKRKF at 120.484.38.22:57222-111.11.114.409:11181/tcp-server/TLS1.3-TLS_AES_128_GCM_SHA256/WAN-P30-SFDFHDFHDGHDFIU0N: writing message: write tcp4 120.484.38.22:57222->111.11.114.409:11181: write: connection reset by peer (0 remain)
2023-11-14 03:08:03 Connection to RJKRKF at 120.484.38.22:57222-111.11.114.409:11181/tcp-server/TLS1.3-TLS_AES_128_GCM_SHA256/WAN-P30-SFDFHDFHDGHDFIU0N closed: writing message: write tcp4 120.484.38.22:57222->111.11.114.409:11181: write: connection reset by peer

Server:

2023-11-13 21:08:02 Device RJFGHG-TKJFKFFK-FJFGNFF-RJFGHG-DJFDJFD- RJFGHG sent cluster-config without the device info for the remote on folder fjsdfdfs-fdjfjf9