Starting syncthing from Systemd (take 2)

This isn’t a duplicate of Starting syncthing from systemd

So far, I am loving syncthing!

syncthing runs on the command line just fine, under my user:

$ syncthing
[monitor] 2015/05/24 21:13:18.466954 monitor.go:94: INFO: Starting syncthing
[XUBPK] 2015/05/24 21:13:18.751851 main.go:466: INFO: syncthing v0.11.5 (go1.4.2 linux-amd64 default) builduser@jara 2015-05-15 08:46:47 UTC
[XUBPK] 2015/05/24 21:13:18.751952 main.go:467: INFO: My ID: (...snip...)
[XUBPK] 2015/05/24 21:13:18.752439 main.go:740: INFO: Database block cache capacity 63778 KiB
[XUBPK] 2015/05/24 21:13:18.758490 main.go:795: INFO: Starting web GUI on http://127.0.0.1:8384/
[XUBPK] 2015/05/24 21:13:19.064706 main.go:870: INFO: Starting local discovery announcements
[XUBPK] 2015/05/24 21:13:19.065967 main.go:875: INFO: Starting global discovery announcements

I am trying to simply start syncthing as my user from systemd, as I do my other services. I’m using i3wm on Arch.

$ cp /usr/lib/systemd/system/syncthing* ~/.config/systemd/user

$ systemctl --user enable syncthing@eric.service
Created symlink from /home/eric/.config/systemd/user/multi-user.target.wants/syncthing@eric.service to /home/eric/.config/systemd/user/syncthing@.service.

$ systemctl --user start syncthing@eric.service

$ systemctl --user status syncthing@eric.service
● syncthing@eric.service - Syncthing - Open Source Continuous File Synchronization for eric
   Loaded: loaded (/home/eric/.config/systemd/user/syncthing@.service; enabled; vendor preset: enabled)
   Active: failed (Result: start-limit) since Sun 2015-05-24 21:17:25 EDT; 5s ago
     Docs: https://github.com/syncthing/syncthing/wiki
  Process: 20791 ExecStart=/usr/bin/syncthing -no-browser -logflags=0 (code=exited, status=216/GROUP)
 Main PID: 20791 (code=exited, status=216/GROUP)

May 24 21:17:24 wash systemd[3748]: Unit syncthing@eric.service entered failed state.
May 24 21:17:24 wash systemd[3748]: syncthing@eric.service failed.
May 24 21:17:25 wash systemd[3748]: syncthing@eric.service holdoff time over, scheduling restart.
May 24 21:17:25 wash systemd[3748]: start request repeated too quickly for syncthing@eric.service
May 24 21:17:25 wash systemd[3748]: Failed to start Syncthing - Open Source Continuous File Synchronization for eric.
May 24 21:17:25 wash systemd[3748]: Unit syncthing@eric.service entered failed state.
May 24 21:17:25 wash systemd[3748]: syncthing@eric.service failed.

And the systemctl file:

$ cat ~/.config/systemd/user/syncthing@.service 
[Unit]
Description=Syncthing - Open Source Continuous File Synchronization for %I
Documentation=https://github.com/syncthing/syncthing/wiki
After=network.target

[Service]
User=%i
Environment=STNORESTART=yes
ExecStart=/usr/bin/syncthing -no-browser -logflags=0
Restart=on-failure
SuccessExitStatus=2 3 4
RestartForceExitStatus=3 4

[Install]
WantedBy=multi-user.target

I believe I have the latest of everything:

$ syncthing -version
syncthing v0.11.5 (go1.4.2 linux-amd64 default) builduser@jara 2015-05-15 08:46:47 UTC

$ cat /proc/version
Linux version 4.0.4-2-ARCH (builduser@foutrelis) (gcc version 5.1.0 (GCC) ) #1 SMP PREEMPT Fri May 22 03:05:23 UTC 2015

EDIT: ps returns nada:

$ sudo ps aux | grep syncthing

I think I fixed it. It wasn’t documented anywhere that you have to modify the systemd file once you copy it to the user space. I’m new to Arch and Systemd, so that may have been a failing on my part.

Here’s the config that worked:

$ cat .config/systemd/user/syncthing@.service 
[Unit]
Description=Syncthing - Open Source Continuous File Synchronization for %I
Documentation=https://github.com/syncthing/syncthing/wiki
After=network.target

[Service]
#User=eric
#Type=forking
Environment=STNORESTART=yes
ExecStart=/usr/bin/syncthing -no-browser -logflags=0
Restart=on-failure
SuccessExitStatus=2 3 4
RestartForceExitStatus=3 4

[Install]
#WantedBy=multi-user.target
WantedBy=default.target

A few notes…

I had to remove “User=%i”. I tried “User=eric” to no available as well.

This got the service enabled, but not starting on login. Then I changed “WantedBy” to be “WantedBy=default.target” after reading the Arch wiki on Systemd/User that says once logged in, services for “default.target” as started. That got it to start on login.

1 Like

Well, Arch includes exactly these files, see here: https://projects.archlinux.org/svntogit/community.git/tree/trunk/PKGBUILD?h=packages/syncthing#n44

There is not need to adjust anything.

You have done the exact same mistake as I mentioned here: Starting syncthing from systemd - #16 by rumpelsepp You are mixing up the config files. You are using the templated unit for the user instance! On Arch Linux, just delete your custom units and file:

systemctl --user enable syncthing.service
systemctl --user start syncthing.service

or

systemctl enable syncthing@eric.service
systemctl start syncthing@eric.service

Next, a small hint which is very useful:

systemctl [--user] [--full] edit syncthing.service

This command does the edit magic behind the scenes and opens an editor for you. You can obtain the documentation from the manpage systemctl(1).

For example, you can specify the priority with such a drop in:

$ systemctl --user cat syncthing.service
# /usr/lib/systemd/user/syncthing.service
[Unit]
Description=Syncthing - Open Source Continuous File Synchronization
Documentation=https://github.com/syncthing/syncthing/wiki
After=network.target

[Service]
Environment=STNORESTART=yes
ExecStart=/usr/bin/syncthing -no-browser -logflags=0
Restart=on-failure
SuccessExitStatus=2 3 4
RestartForceExitStatus=3 4

[Install]
WantedBy=default.target

# /home/stefan/.config/systemd/user/syncthing.service.d/override.conf
[Service]
Nice=10

As I mentioned above, you are mixing up things. When you use the convenient files (on arch just use the shiped ones by the package), then everything is fine. :slight_smile:

Sorry if I didn’t mention it. Yes, I installed from the community repo and that is exactly what i am using:

suco pacman -S syncthing

I did find that thread, and I did as you mentioned in that thread by copying the “intended user instances” to my “.config/systemd/user/” directory, as instructed.

But straight starting with:

systemctl --user start syncthing.service

Kept returning an error in journalctl -xb that showed, if I can recall the error message, the GROUP was unable to start processes. I was able to get it to run by adding Forking, but I think that was starting to go down the wrong road.

I also did attempt to follow the instructions in your Wiki for auto starting, which was had even additional “enable” step:

systemctl --user enable syncthing.service
systemctl --user start syncthing.service

Again, this causes errors with the existing script as it was supplied. It always either wanted me to fork, or remove the user.

Again, this is exactly what i did. And the last two would not work without the --user param. Well, actually they would: you just get prompted for the user’s rights and password.

But again, same errors as before.

If you like, I can revert my working installation back to the error state and produce the errors for you. As long as User=%i was specified under [Service], it would not start without Type=forking added - which actually started to cause some login issues.

Type=forking was needed due to the errors produced.

And as I mentioned in my solution post, WantedBy=default.target was needed to get it to auto-start on login.

According to Arch wiki, Systemd/User only looks for targets with default.target to start on login:

https://wiki.archlinux.org/index.php/Systemd/User#How_it_works

multi-user.target is used by the root services.

Again, I am new to Systemd and I may have mis-interrupted multi-user.target. But again, changing it to default.target was the only way to get it to autostart. Otherwise, the service would be enabled but inactive (waiting to be started) on login.

I tried. I even attempted to follow the steps in the Arch wiki page:

https://wiki.archlinux.org/index.php/Syncthing

The inotify instructions suffer from the same won’t start issues as well following those same steps.

No. Since arch ships these files, you just don’t need to copy them somewhere. The wiki says: If your distro provides a systemd service file for syncthing you can skip step 2. The thread was about a debian package which does not ship the service files.

I think you did something wrong with copying the wrong service files into the wrong path.

Not exactly. :wink: You copied the system instance to the user directory in the first place, which does not work. The user instance takes the current user. When you try to start it as a system service you start it as root instead…

Just remove your wrong copies and enable the services which are shiped by arch. I use them on 5 machines and they work well.

Since syncthing stays in the foreground you don’t need a forking service type.

The user service has the default target! Again, you have mixed them up! See here: https://github.com/syncthing/syncthing/blob/master/etc/linux-systemd/user/syncthing.service#L14

Hmm. No.

$ systemctl --user start syncthing-inotify.service
$ systemctl --user status syncthing-inotify.service
● syncthing-inotify.service - Syncthing Inotify File Watcher
   Loaded: loaded (/usr/lib/systemd/user/syncthing-inotify.service; disabled; vendor preset: enabled)
  Drop-In: /home/stefan/.config/systemd/user/syncthing-inotify.service.d
           └─override.conf
   Active: active (running) since Mon 2015-05-25 16:39:14 CEST; 5s ago
     Docs: https://github.com/syncthing/syncthing-inotify/blob/master/README.md
 Main PID: 3379 (syncthing-inoti)
   CGroup: /user.slice/user-1000.slice/user@1000.service/syncthing-inotify.service
           └─3379 /usr/bin/syncthing-inotify -logflags=0

May 25 16:39:14 naboo systemd[1034]: Started Syncthing Inotify File Watcher.
May 25 16:39:14 naboo systemd[1034]: Starting Syncthing Inotify File Watcher...
May 25 16:39:16 naboo syncthing-inotify[3379]: [OK] Watching dokumente: /home/stefan/Documents

$ systemctl --user cat syncthing-inotify.service
# /usr/lib/systemd/user/syncthing-inotify.service
[Unit]
Description=Syncthing Inotify File Watcher
Documentation=https://github.com/syncthing/syncthing-inotify/blob/master/README.md
After=network.target syncthing.service
Requires=syncthing.service

[Service]
ExecStart=/usr/bin/syncthing-inotify -logflags=0
SuccessExitStatus=2
Restart=on-failure
ProtectSystem=full
ProtectHome=read-only

[Install]
WantedBy=default.target

# /home/stefan/.config/systemd/user/syncthing-inotify.service.d/override.conf
[Service]
Nice=10

edit: I can reproduce your issues. When you try to start the user unit with the system instance (or the other way round), then gnome asks you to enter the password. So as I said, remove the wrongly located stuff and reenable it. That should work.

Thank you for responding to every bit. Here’s some information that may help others that run into this:

I think you nailed it that I was copying the root/system-wide version to my local user.

This clearly shows exactly what i had to do to get it started: removing the User=%i and changing the target to default.target.

$ find / -name syncthing*
/usr/lib/systemd/system/syncthing@.service
/usr/lib/systemd/system/syncthing-inotify@.service
/usr/lib/systemd/user/syncthing-inotify.service
/usr/lib/systemd/user/syncthing.service

So yeah, I copied the wrong “system” ones over. I now see there is a “user” space I was unaware of. I only knew of the system/ directory from being new to Arch and setting up other services.

Also, as I mentioned, I am new to systemd. This tid bit also helps others running into this problem:

systemctl status syncthing@eric.service
!=
systemctl --user status syncthing@eric.service

The first will show dead, the second will show it running under my custom setup.

It was a good learning experience as it taught me more about Systemd. When I have more time to waste, I’ll attempt to revert to using syncthing.service instead of my user instances as mentioned.

1 Like

To sum up:

  • The user instance stuff with --user is intended to be used on a desktop computer. Syncthing starts when you log in to gnome or something; and it stops when you log out.
  • The system instance stuff is useful for a server. Syncthing then starts during the normal bootup sequence and stays running. To be able to specify the user, there is the templating stuff with @, e.g.: systemctl start syncthing@hans.service.
  • The user units are located in /usr/lib/systemd/user/ and can be overwritten in /etc/systemd/user/ and ~/.config/systemd/user/.
  • The system units are located in /usr/lib/systemd/system/ and can be overwritten in /etc/systemd/system/.

So, when you mix them up by accident, strange things happen.

Note: The paths are valid for arch linux and fedora and are documented in the systemd manpages. Debian (as usual) does its own thing.

1 Like

Mind adding this, verbatim, to the docs? There is a description there, but I think your summary is clearer than anything there currently.

That’s odd, I thought systemd worked exactly the same no matter what distro it was included in?

The service file I’ve been using from my post: Getting Syncthing running with a central file server running CentOS 7 minimal setup and windows clients

Allows me to use ps just fine:

[root@sync ~]# ps aux | grep syncthing
ataraxia   680  0.0  0.2 274672  4848 ?        Ssl  May11   6:24 /usr/local/bin/syncthing
ataraxia  6949  4.1  9.4 1310216 177652 ?      Sl   May11 1027:27 /usr/local/bin/syncthing
root     29231  0.0  0.0 112644   964 pts/0    S+   19:27   0:00 grep --color=auto syncthing

Yes, it does. But there are two systemd instances on every machine: user and system. He mixed up these two instances, which caused the problems. Your proposed service files work with the system instance.

1 Like

What puzzles me are the lines

SuccessExitStatus=2 3 4 RestartForceExitStatus=3 4

Exit codes 3 and 4 are both succes and fail?

My syncthing@.service has:

SuccessExitStatus=2 RestartForceExitStatus=3 4

which sounds more logical to me.

AFAIR these are necessary to avoid that systemd considers that an error has appeared:

When you e.g. restart sycthing you just tell systemd “hey, restart syncthing”. That’s not an error, thus SuccessExitStatus=2 3 4. Correct me if I’m wrong.

The cause of the confusion is actually a misplaced file on Github!! The USER-SYSTEMD file is located in the SYSTEM folder and vice versa:

Take a look, the path to the service file for user contains the content you would expect for system:

Apart from the above, I am now facing the exact same issue. I want system, so I changed the service file.

My syncthing.service file:

# /lib/systemd/system/syncthing.service
[Unit]
Description=Syncthing - Open Source Continuous File Synchronization
Documentation=http://docs.syncthing.net/
After=network.target
Wants=syncthing-inotify@.service

[Service]
Nice=7
Environment=STNORESTART=yes
ExecStart=/usr/bin/syncthing -no-browser -logflags=0
Restart=on-failure
SuccessExitStatus=2 3 4
RestartForceExitStatus=3 4

[Install]
WantedBy=default.target

I only added Nice=7. Now I run the following commands:

osmc@KodiTV:~$ sudo systemctl disable syncthing.service
Removed symlink /etc/systemd/system/default.target.wants/syncthing.service.
osmc@KodiTV:~$ sudo systemctl daemon-reload
osmc@KodiTV:~$ sudo systemctl enable syncthing.service
Created symlink from /etc/systemd/system/default.target.wants/syncthing.service to /lib/systemd/system/syncthing.service.

And finally I try to start it, check the status, it failed:

osmc@KodiTV:~$ sudo systemctl start syncthing.service
osmc@KodiTV:~$ sudo systemctl status syncthing.service -l
* syncthing.service - Syncthing - Open Source Continuous File Synchronization
   Loaded: loaded (/lib/systemd/system/syncthing.service; enabled)
   Active: failed (Result: start-limit) since Thu 2015-10-29 16:12:25 CET; 9s ago
     Docs: http://docs.syncthing.net/
  Process: 703 ExecStart=/usr/bin/syncthing -no-browser -logflags=0 (code=exited, status=1/FAILURE)
 Main PID: 703 (code=exited, status=1/FAILURE)

Oct 29 16:12:25 KodiTV systemd[1]: Unit syncthing.service entered failed state.
Oct 29 16:12:25 KodiTV systemd[1]: syncthing.service start request repeated too quickly, refusing to start.
Oct 29 16:12:25 KodiTV systemd[1]: Failed to start Syncthing - Open Source Continuous File Synchronization.
Oct 29 16:12:25 KodiTV systemd[1]: Unit syncthing.service entered failed state.
osmc@KodiTV:~$

I also tried a reboot and performed the steps again. I am running OSMC which is based on Debian upstream.

?

No, that is the correct file. Why do you think it is the wrong file?

Why? Just pick the shipped file and fire

systemctl enable user@syncthing.service

Your service file runs syncthing as root, which is not a good thing…

1 Like

Because this file:

shows:

    WantedBy=default.target

And has no “user=…”

But this file, which is meant for system instead of user:

does have "user=…" and wanted=multi-user.target

hence my confusion.

Just on the phone in the tube of munich…

That is just how it works. The system unit is a so called templated unit, that means you have to specify the user before the @ sign. You can thus run multiple instances under different users with one unit.

The user unit runs under the current user, thus there is no user configured within the service file.

I have to apologize, had to do more reading before posting. I read the documentation behind systemctl and finally understand now. Using user=osmc now and it works.

Thanks for taking time to reply and I’m sorry to waste your time.

2 Likes