"out of sync - permission denied" on one device only

Support
1

Hi,

so our setup at work is we have one “always-on” st-client (let’s call him Seymour) making sure everyone has the most recent version of a file at all times. Now Seymour is misbehaving, but only in one of the synced folders.

Whenever someone renames or removes a file in said folder, all other participants sync it correctly, but Seymour only creates the new copy (if it’s a rename) on his end. He does not delete anything, displays “Out of Sync” and “Failed Items: permission denied” for those files and stays there until we delete those files manually on his end.

Both the files and all the directories in the directory in question have permissions 660 and 770 respectively, Seymour is even running a script making sure this is always the case. All participants are on version 1.3.2 and all of Seymour’s other folders sync without trouble, no matter what.

I’m almost certain this is not a bug but I would really appreciate a hint where to look for the source of the problem, since I haven’t been able to find any differences in the st-folders configurations…

Thanks in advance!

rg

2

Generally: Log in as the user Syncthing runs as, and you should get the same permission error. Then take it from there.

Specifical guess: Check for ACLs.

For more specific pointers post ls -l output, give specifics about the system, …

3

Thanks for your suggestions.

That is exactly what I’ve done, the fact I’m not getting those errors is what is so baffling to me.

Here’s me checking and removing a failed item as the syncthing user. Syncthing should be able to do it on its own:

s1612@s1612:/data/gs/kfm$ ls -l
insgesamt 24
drwxrwx--- 10 s1612 mhi 4096 Jul 16 08:22 2014
drwxrwx--- 14 s1612 mhi 4096 Okt  2 16:45 2015
drwxrwx--- 11 s1612 mhi 4096 Jul 16 13:37 2016
drwxrwx---  9 s1612 mhi 4096 Apr 26  2017 2017
drwxrwx---  9 s1612 mhi 4096 Jan 12  2018 2018
drwxrwx---  9 s1612 mhi 4096 Okt  1 11:10 2019
-rw-rw----  1 s1612 mhi    0 Dez 17 14:31 something
s1612@s1612:/data/gs/kfm$ getfacl something 
file: something
owner: s1612
group: mhi
user::rw-
group::rw-
other::---

s1612@s1612:/data/gs/kfm$ rm something 
s1612@s1612:/data/gs/kfm$ ls -l
insgesamt 24
drwxrwx--- 10 s1612 mhi 4096 Jul 16 08:22 2014
drwxrwx--- 14 s1612 mhi 4096 Okt  2 16:45 2015
drwxrwx--- 11 s1612 mhi 4096 Jul 16 13:37 2016
drwxrwx---  9 s1612 mhi 4096 Apr 26  2017 2017
drwxrwx---  9 s1612 mhi 4096 Jan 12  2018 2018
drwxrwx---  9 s1612 mhi 4096 Okt  1 11:10 2019

We’re running Linux Mint 19.2 (Tina) with Cinnamon, 64Bit. Both on Seymour and other machines. Oh, and the source directory’s permissions are in order as well (makes sense, otherwise the conflict file couldn’t get written to the disk in the first place):

1612@s1612:/data/gs/kfm$ cd ../
s1612@s1612:/data/gs$ ls -l
drwxrwx--- 10 s1612 mhi 4096 Dez 17 14:40 kfm
s1612@s1612:/data/gs$ getfacl kfm
file: kfm
owner: s1612
group: mhi
user::rwx
group::rwx
other::
4

Yeah sorry, I am out of ideas then: Looks good and Mint isn’t on “my list of systems notoriously doing arcane stuff”.

Actually one more thing (doesn’t really fit, but when out of ideas anything goes): Hardening: Do you use snap (or anything else that isn’t apt)? And if not, try disabling the hardening options in the systemd service file.

1 Like
5

While we’re using apt exclusively for package-management, as far as I understand the hardening options only take effect if the service is run as root. This is not the case in this instance. Am I getting something wrong? If I am: I still don’t feel comfortable turning off security-measures without knowing exactly what they do. Could you elaborate on your suggestion?

Thanks again and sorry for the long delay, I really needed those long holidays :wink:

6

Actually looking at your path (/data/gs/kfm) I suspect systemd hardening to be the problem, as I think /data is non-standard (whatever that exactly means in general, in this context I suspect “something that systemd hardening doesn’t whitelist”).

The systemd hardening options apply not just when running as root. I do not know much about them, if you want to know a good place to start is the issue/PR adding it: https://github.com/syncthing/syncthing/pull/5351

And incidentally I just discovered that I ran without them on my system, as I have a drop-in replacement unit file in /etc (bad idea, use *.d dirs). When I removed that syncthing fails to start because of the private tmp stuff (I put my database into memory). Turns out I either need to also enable PrivateTmp for the unit that syncs the db to memory and join them together or disable it (I did the latter).

7

Alright, so I looked into those hardening options. I don’t think they’re to blame. For example ProtectSystem set to “full” explicitly only protects /usr, /boot and /etc from writing operations. I gave it a go anyway and deactivated all of them - to no avail.

If the fact that /data is not a standard-folder was responsible, that would mean all other syncthing-folders on the machine should show the same, faulty behavior (they’re all below /data) but they don’t.

8

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.