Syncthing running on Linux Mint visible twice for other devices, device ID changes on reboot

Hi, I did my best to look through the FAQ, Docs and previous topics but I’m still confused about this.

  1. On Linux Mint 20.3 (based on Ubuntu LTS 20.04), installed Syncthing 1.22.2 as instructed here
  2. Created a systemd unit to run Syncthing as a daemon and on startup, by downloading this example unit from GitHub and running
sudo chown root syncthing@.service
sudo mv syncthing@.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable syncthing@myusername
sudo systemctl start synchthing@myusername

So far so good, I can now open the Web GUI:

My other devices see the Syncthing instance too:

  1. Paired it with other devices, synced some folders, was happy
  2. Later rebooted the Linux Mint desktop, opened Web GUI again
  3. The device ID has changed and all settings, shared folders are gone, it’s like a fresh install:
  4. My other devices now see both ID’s as nearby devices available for pairing:

Hence:

  • Is this normal? I saw in the FAQ the bit about seeing two syncthing processes in the task manager but to see two ID’s in remote devices as well?
  • If it’s an anomaly, what do you think is the proximate cause here and how to fix it?

I tried doing the install quite a few times, always ran

sudo apt purge syncthing
rm -rf ~/.config/syncthing/
sudo rm /etc/systemd/system/syncthing@.service
sudo rm /etc/systemd/system/multi-user.target.wants/syncthing@.service (symlink to previous)

in between tries for a (hopefully) complete uninstall. I should also mention that after rebooting, the ID always changes back to the same ID, 5OL3GL.

Thanks in advance.

No, not normal. If somehow Syncthing was started twice under the same user account, there would have been a warning.

There’s likely two separate Syncthing configurations in play.

Syncthing has a unique enough filename for its index database directory that the following command will should all occurrences of it:

sudo find / -type d -name "index-*.db"

If it were me, I’d do the following:

  1. (as root) apt purge syncthing
  2. (as root) find / -type d -name "index-*.db" (review and remove extraneous configurations as needed)
  3. (as root) rm /etc/systemd/system/syncthing@.service
  4. (as root) rm /etc/systemd/system/multi-user.target.wants/syncthing@.service
  5. (as root) apt install syncthing
  6. (as intended user) systemctl --user enable syncthing.service
  7. (as intended user) systemctl --user start syncthing.service
  8. (as intended user; optional) loginctl enable-linger

Notes:

  • Step 8 is optional. Use if Syncthing should start whenever the OS boots rather than only when the user logs on (when the user logs off, Syncthing also automatically shuts down).
  • Syncthing’s official DEB package from its APT repo already bundles a systemd unit file that’s installed at /usr/lib/systemd/user/syncthing.service.

Auto-starting via a system service (e.g., systemctl enable syncthing@myusername) is fine, but a user service (systemctl --user enable syncthing.service) has advantages:

  • Each user can enable/disable/start/stop a user service without requiring root/sudo.
  • On desktops/laptops, a user’s files often don’t change whenever a user isn’t logged on, so it can be useful to have Syncthing auto-start/stop when a user logs on/off.
  • Each user can decide if they want to auto-start Syncthing after a specified delay (useful if the source/target is a removable drive).

Thanks for the quick reply!

Blockquote If it were me, I’d do the following:

  1. (as root) apt purge syncthing
  2. (as root) find / -type d -name "index-*.db" (review and remove extraneous configurations as needed)
  3. (as root) rm /etc/systemd/system/syncthing@.service
  4. (as root) rm /etc/systemd/system/multi-user.target.wants/syncthing@.service
  5. (as root) apt install syncthing
  6. (as intended user) systemctl --user enable syncthing.service
  7. (as intended user) systemctl --user start syncthing.service
  8. (as intended user; optional) loginctl enable-linger

I followed these exact steps. The ID is now the one it was reverting back to 5OL3GL, on first startup (maybe that’s a clue to someone of what is going on). How it is always the same ID even between complete re-installs is beyond me, as Step 2) yielded only one config location, the usual one in ~/.config.

Now using the user service however there is another issue which is that after a reboot/logon the user service does not start up automatically. Any idea why those steps did not make it stick?

Thanks.

So 5OL3GL must be your original config and another one for MYEJ7AF that’s hiding somewhere.

As long as you don’t remove ~/.config/syncthing, the device ID won’t change for that user. It was auto-generated during the initial start of Syncthing.

So, systemctl --user enable syncthing.service (assuming the command returned no error) causes Syncthing to start up whenever the user the command was run under logs on. Syncthing will not start during a reboot because you’re enabling a system service.

If Syncthing doesn’t auto-start when the user logs on, check the status like this:

systemctl --user status syncthing.service

What’s the output from the status check?

As long as you don’t remove ~/.config/syncthing, the device ID won’t change for that user. It was auto-generated during the initial start of Syncthing.

That makes sense but as I said before, I explicitly did remove the whole ~/.config/syncthing folder before each re-install attempt.

So 5OL3GL must be your original config and another one for MYEJ7AF that’s hiding somewhere.

MYEJ7AF was one of the ID’s that the device had on one of the re-install attempts. Every time I’d re-install and start the service through the systemd unit, it would have a new ID (as expected), but after a reboot it would always be back to 5OL3GL (with folder definitions gone, greeted by usage statistics dialogue like in a fresh install).

What’s the output from the status check?

The output is

~$ systemctl --user status syncthing.service
● syncthing.service - Syncthing - Open Source Continuous File Synchronization
     Loaded: loaded (/usr/lib/systemd/user/syncthing.service; enabled; vendor preset: enabled)
     Active: inactive (dead)
       Docs: man:syncthing(1)

Edited one of my replies too many times so it got stuck in spam-suspect-limbo. Anyway it seems I’ve taken some steps forward:

I located the system service files under /lib/systemd/system and got rid of the system service. What got me there was

systemctl --user list-unit-files --type service --all | grep syncthing # list all user service units with "syncthing" in the service name
systemctl list-unit-files --type service --all | grep syncthing # same for system wide services
# found syncthing@.service and syncthing-resume.service in system wide service units
# then for each of those:
systemctl show "syncthing-resume.service" | grep systemd #find the path of the service file in the output
systemctl show "syncthing@.service" | grep systemd #find the path of the service file in the output
sudo systemctl stop <servicename> # for both/all system wide syncthing services found, running without --user option will correctly target the system wide services
sudo systemctl disable <servicename> # for both/all, same here
# now remove the system wide service files
sudo rm <the system wide service file paths found>
# finally
sudo systemctl daemon-reload
sudo systemctl reset-failed

Now running

systemctl list-unit-files --type service --all | grep syncthing

should show nothing and

systemctl --user list-unit-files --type service --all | grep syncthing

should show the user service if one has been set up.

Hope I got all that right and hope this helps anyone who installed a competing system wide Syncthing service and wants to get rid of it. To enable the user service instead if you haven’t already follow gadgets steps detailed above.

What remains to be solved in my case is to get auto-start of the user service to work on login. After login, checking the status of the user service yields:

$ systemctl --user status syncthing.service
● syncthing.service - Syncthing - Open Source Continuous File Synchronization
     Loaded: loaded (/usr/lib/systemd/user/syncthing.service; enabled; vendor preset: enabled)
     Active: inactive (dead)
       Docs: man:syncthing(1)

So enabled, but inactive. After some googling I thought home folder encryption might be causing the issue and tried adding

[Unit]
RequiresMountsFor=/home/myusername
After=network.target

into the service file, but alas it made no difference. Manually starting the service after login works by the way and everything works like a charm now. But: how to get the enabled user service to actually start on login?

The status output looks good as far as the Syncthing service being enabled.

systemd user services rely on a user’s home folder, so by the time the system has booted, your home folder would have already been unlocked (otherwise ~/.ssh/authorized_keys and other files necessary during password/key validation would be inaccessible).

Does the user account have a ~/.config/systemd/user/default.target.wants/syncthing.service symlinked to /usr/lib/systemd/user/syncthing.service?

Does the user account have a ~/.config/systemd/user/default.target.wants/syncthing.service symlinked to /usr/lib/systemd/user/syncthing.service?

Affirmative. I manually created that folder and link though, if that matters.

Btw what would the correct permissions be for the link?

Thanks!

Nope, same end result as with the systemctl command. The status output looks fine.

Was there a reason why you wanted/had to manually create the symlink versus using the command `systemctl --user enable syncthing.service"? (I’m wondering if there’s an issue with your systemd setup after all of the earlier manual addition and removal of unit files.)

Nothing special. It’s just a plain symlink for sysytemd to follow to find the unit file that needs to be loaded. By default, systemctl would have created the folder (if needed) and the symlink to be owned by the user issuing the command.

Syncthing would have auto-started when you logged on and shutdown when you logged off. If Syncthing isn’t starting immediately after logon, what happens when you issue the command systemctl --user start syncthing.service?

Nope, same end result as with the systemctl command.

Yes sure I can see that, I meant “Affirmative” as in yes the link exists.

If Syncthing isn’t starting immediately after logon, what happens when you issue the command systemctl --user start syncthing.service?

Manually starting the service by said command works no problem, as a workaround having that command at the end of ~/.profile works too and effectively gets the service started at login, but I would like to get it working as intended, without the line in .profile

Yup, I understand. :slightly_smiling_face:

The reason I asked is because if Syncthing fails to manually start via systemctl --user start syncthing.service, then it will also fail to auto-start.

Because several different things had been previously tried including manually installing, modifying, and/or removing systemd unit files (base on your earlier posts, I think at one point there were conflicting unit files), it’s important to make sure that there isn’t some other issue since systemd isn’t behaving normally on your Linux Mint system.

Step 1:

systemctl --user stop syncthing.service

Step 2:

killall -9 syncthing

Step 3:

systemctl --user disable syncthing.service

Step 4:

systemctl --user status syncthing.service

Step 5:

ls -lR ~/.config/systemd

What’s the output from step 4 and 5?

Output from 4:

➜  default.target.wants systemctl --user status syncthing.service
● syncthing.service - Syncthing - Open Source Continuous File Synchronization
     Loaded: loaded (/usr/lib/systemd/user/syncthing.service; disabled; vendor preset: enabled)
     Active: inactive (dead)
       Docs: man:syncthing(1)

Jan 11 21:30:11 myhostname syncthing[27898]: [KUECC] INFO: Established secure connection to 2V>
Jan 11 21:30:11 myhostname syncthing[27898]: [KUECC] INFO: Device 2VJMYWH-GPUEZJK-YEKHPWJ-HBIN>
Jan 12 20:12:58 myhostname syncthing[27898]: [monitor] INFO: Signal 15 received; exiting
Jan 12 20:12:58 myhostname syncthing[27898]: [KUECC] INFO: QUIC listener ([::]:22000) shutting>
Jan 12 20:12:58 myhostname syncthing[27898]: [KUECC] INFO: Connection to 2VJMYWH-GPUEZJK-YEKHP>
Jan 12 20:12:58 myhostname systemd[1088]: Stopping Syncthing - Open Source Continuous File Syn>
Jan 12 20:12:59 myhostname syncthing[27898]: [KUECC] INFO: TCP listener ([::]:22000) shutting >
Jan 12 20:12:59 myhostname syncthing[27898]: [KUECC] INFO: Exiting
Jan 12 20:12:59 myhostname systemd[1088]: syncthing.service: Succeeded.
Jan 12 20:12:59 myhostname systemd[1088]: Stopped Syncthing - Open Source Continuous File Sync>
lines 1-15/15 (END)

and 5:

➜  default.target.wants ls -lR ~/.config/systemd
/home/myusername/.config/systemd:
total 4
drwxr-x--- 2 myusername myusername 4096 Jan 12 20:13 user

/home/myusername/.config/systemd/user:
total 0

(just happened to be in the default.target.wants folder, that’s part of the prompt)

The output above looks good, and so far systemd seems to be working fine (I wanted to know if systemctl would remove the symlink under ~/.config/systemd/user/).

Try the following commands:

loginctl enable-linger
systemctl --user enable syncthing

Assuming that systemd hasn’t been configured to ignore user unit files, Syncthing will launch whenever Linux boots up.

If it doesn’t work as expected, we’ll have to take a closer look at your systemd configuration.

On a related note…

You might already be aware that the .service file extension is optional when referring to a service unit file. Other unit types include network and timer.

On my NAS, a timer starts Syncthing after a 15-minute delay because some of the storage volumes have filesystem consistency checks that run during the mount process at boot time.

It had been a while since I last used Linux Mint, so out of curiosity, I installed the 20.3 (“una”) release for a closer look.

After booting the new installation for the first time, I issued the following commands while logged on as user “gadget”:

  1. sudo curl -o /usr/share/keyrings/syncthing-archive-keyring.gpg https://syncthing.net/release-key.gpg
  2. echo "deb [signed-by=/usr/share/keyrings/syncthing-archive-keyring.gpg] https://apt.syncthing.net/ syncthing stable" | sudo tee /etc/apt/sources.list.d/syncthing.list
  3. printf "Package: *\nPin: origin apt.syncthing.net\nPin-Priority: 990\n" | sudo tee /etc/apt/preferences.d/syncthing
  4. sudo apt-get update
  5. sudo apt-get install syncthing
  6. systemctl --user enable syncthing
  7. loginctl enable-linger
  8. reboot

By the time I’d logged back on and opened a terminal to check, Syncthing had already auto-started.

A full upgrade (sudo apt-get dist-upgrade ; reboot) also didn’t turn up any issues.

Try the following commands:

loginctl enable-linger
systemctl --user enable syncthing

I did and got:

➜  default.target.wants loginctl enable-linger
➜  default.target.wants systemctl --user enable syncthing
Created symlink /home/myusername/.config/systemd/user/default.target.wants/syncthing.service → /usr/lib/systemd/user/syncthing.service.
➜  default.target.wants 

So the symlink got automatically and correctly created, but alas afte a reboot and relogin still:

➜  ~ systemctl --user status syncthing
● syncthing.service - Syncthing - Open Source Continuous File Synchronization
     Loaded: loaded (/usr/lib/systemd/user/syncthing.service; enabled; vendor preset: enabled)
     Active: inactive (dead)
       Docs: man:syncthing(1)

Thanks for taking the time for looking into the situation with a fresh Mint Una install! It appears that something is indeed different in the functioning of a fresh install and my current setup.

Could you post the output from the following command?

systemctl list-unit-files

Sure, to spare you from the 300+ listed units, I grepped the output for “syncthing”.

➜  ~ systemctl --user status syncthing
● syncthing.service - Syncthing - Open Source Continuous File Synchronization
     Loaded: loaded (/usr/lib/systemd/user/syncthing.service; enabled; vendor preset: enabled)
     Active: inactive (dead)
       Docs: man:syncthing(1)
➜  ~ systemctl list-unit-files | grep syncthing
syncthing-resume.service                                                  disabled        enabled      
syncthing@.service                                                        disabled        enabled

It’s still listing the disabled system wide services (as disabled).

Earlier I already deleted all files related to the system wide services, stopped and disabled the services etc and at that point

systemctl list-unit-files --type service --all | grep syncthing

returned nothing. How they got back there into the listing is beyond me.

systemctl --user list-unit-files --type service --all | grep syncthing

With the --user option shows the user service:

➜  ~ systemctl --user list-unit-files --type service --all   | grep syncthing
syncthing.service                     enabled  enabled    

the output for the user service looks exactly as it should right?

I was actually interested in the whole list to see what was different between your system and mine. :grin:

Yup, the output is what I expected.

Because there are interdependent pieces in play, any one of them could be disabled even while the Syncthing service is enabled.

For example, on my NAS, systemctl --user is-enabled syncthing returns the message “disabled” all while Syncthing auto-starts not long after system boot up. It’s because I have a unit file named “syncthing.timer” which I enabled via systemctl --user enable syncthing.timer (I wanted a 15-minute delay before Syncthing auto-started).

Can I email you the full output?

I’m probably overthinking but feels wrong to publish exactly what’s running on my system :smiley:

The output was too long for a PM.

Is this to be expected, this many lines for syncthing in htop:

This was after I started the user service manually