Sync xattr between different OS

Support
1

Hi !

First of all thanks for this awesome software. I’ve been moving out from Nextcloud with great success and am not looking back.

I’m mainly using MacOS, and I use the sync/send xAttr feature to sync my Finder tags between different machines. After a little bit of tweaking (some xAttr filters to set) it works well.

However, I also have one machine using Debian (a remote VPS). For various reasons I would like my Finder tags (which are in fact xAttr using the key com.apple.metadata:_kMDItemUserTags) to that machine. I’ve tried everything, including adding elevated capabilities to my system service etc… Still failing.

Then, I’ve read again carefully the docs and it says : https://docs.syncthing.net/v1.23.7/advanced/folder-sync-xattrs.html

Extended attributes are recorded and synced on a per-operating system basis. That is, if a file has an extended attribute on one operating system, it will not be synced to another operating system. For example, if a file has an extended attribute on Linux, it will not be synced to macOS.

:scream:

Does it mean that I have no chance of getting the com.apple.metadata:_kMDItemUserTags xAttr sent to my Debian VPS ? Is there any way to force this using some kind of advanced config ?

Moreover, how is iOS “recorded” by Syncthing ? Is it considered as MacOS ?

Many thanks in advance for your help !

2

Correct, and there is no such config.

3

Thanks for your reply. May I know the rationale behind that restriction ?

Is iOS considered as the same “OS” as MacOS ? (I’m currently trying to have my Finder tags synced to iOS through Synctrain with the help of @pixelspark)

Thanks :pray:

4

The naming and interpretation is platform specific, for example “com.apple.metadata:_kMDItemUserTags” is not a valid name for an extended attribute on Linux (it needs to begin with “user”, “system”, “trusted”, or “security”) and even if we rewrote the name into the user namespace there is nothing on the Linux side to interpret the contents.

Why do you want it synced to Linux?

I don’t know how iOS is handled.

5

Thank you very much for your reply

I didn’t want to be too specific about my usecase but I’ll describe it as you ask. To put things into context, all my “working” machines are under MacOS / iOS. The two linux entities linked to my syncthing network are my NAS and a remote Debian VPS. I want to use some specific Finder tags as triggers for remote actions / scripting done on the VPS. For example, have a shared Finder tag that would add the tagged file or folder path to a web sharing tool such as files.gallery.

I fully understand that my usecase is very personal (you could even call it weird).

However, the sync/send xAttrs feature from Syncthig is supposed to be a very advanced feature (= not for everyone, not by default) and I do not see the harm of making available xAttrs that would be silently ignored by the OS (but still could be used if that’s the user’s intention). It looks less dangerous than syncthing ownership, which Syncthing does already, but maybe I missed something !

In fact, it looks like “foreign” xAttr from another OS are the only piece that Syncthing doesn’t allow (yet) to sync.

Does this mean that this xAttrs Syncthing restriction / filtering by OS is not managed within Syncthig’s source code ? Are some dependencies playing a role here ?

6

I see; the use case makes sense, but at the moment it will indeed not work. Perhaps a workaround could be to ask the Syncthing API about the files, as that will show the attributes in question.

As for iOS I don’t know because I haven’t checked, I don’t use or build for iOS. Possibly it will identify as macOS since we don’t do anything more specific. Possibly something else will happen. :person_shrugging:

It’s possible we could apply some sort of mapping of attributes to the target OS, you could file it as a feature request.

1 Like
7

So we don’t synchronize xattrs from device A (MacOS) over B (Linux) to C (MacOS)?

I thought we did store them in the database and distribute along with the other index data. The middle node (B) just doesn’t apply them locally, but hands them over transparently?

8

As for iOS: syncing xattrs is not supported by the Synctrain app (but you can turn it on by using a custom configuration). I did not add any special code to Syncthing core for handling xattrs on iOS, so either it completely doesn’t do anything with xattrs on iOS, or it handles like macOS.

I did make a quick attempt at supporting xattrs in Synctrain on iOS but it didn’t work for reasons I could not understand at the time. See here. If I can find out the issue I am happy to add the feature.

That is my understanding as well - the attributes are in the index on all nodes but simply not applied on platforms other than the one they are for. In fact, the /rest/db/file API endpoint should give you the macOS xattrs on Linux as well (see here, it has a whole section on platform specific data). So perhaps the use case of @Limezy can be solved with that.

9

I quickly checked by the way, if iOS is seen as ‘Darwin’ (which it technically is, and also in Go’s build options darwin=macos||ios), then the macOS behaviour seems to be assumed on iOS. See here:

1 Like
10

Correct.

11

Thank you all for your replies !

  • About my usecase, I now have a great starting point and will try using the API to retrieve the “Darwin” tags on my Debian VPS
  • If I understand well, these are in the shared database so I do not even have to activate “sync xAttrs” on my Debian VPS client, nor bother to have my Syncthing executable working with elevated permissions for the API to be able to serve that data - which is even easier in a way
  • About syncing Finder tags with iOS not yet working, at least I can assume that this xAttrs OS filtering is not the culprit since the Syncthing client embedded into Synctrain should be seen as a “Darwin” type
12

So, in case someone reads this in the future and has a usecase similar to mine, I hereby confirm that I was able to access to my Finder tags on the Debian VPS client through the API. It’s still rough and needs a bit of work before scripting but at least it’s a proven method.

  • It doesn’t need send/syncxAttrs to be set on the VPS
  • It doesn’t need Syncthing executable to run with special permissions
  • Indeed, as stated by @acolomb, the data is distributed to all clients anyway, and that’s all we need

CLI to retrieve the data : sudo curl -s -H X-API-KEY:YOUR-API-KEY "http://localhost:8384/rest/db/file?folder=Documents" --url-query "file=Test"

The result comes out as a big JSON, from which you can extract the relevant part, for my case under the “Darwin” platform, with the com.apple.metadata:_kMDItemUserTags key ;

"global": {
    ...
    "platform": {
      ...
      "Darwin": {
        "Xattrs": [
          {
            "Name": "com.apple.FinderInfo",
            "Value": "AAAAAAAAAAAABgAAAAAAAAAAAAAAAAAAAAAAAAAAAAA="
          },
          {
            "Name": "com.apple.metadata:_kMDItemUserTags",
            "Value": "YnBsaXN0MDChAWwAQwBhAHQA6QBnAG8AcgBpAGUAcwAKADMICgAAAAAAAAEBAAAAAAAAAAIAAAAAAAAAAAAAAAAAAAAj"
          }
        ]
      },
      ...
    },
...

One last step, the value YnBsaXN0MDChAWwAQwBhAHQA6QBnAG8AcgBpAGUAcwAKADMICgAAAAAAAAEBAAAAAAAAAAIAAAAAAAAAAAAAAAAAAAAj extracted is a Base64 binary plist “bplist” Apple format. (refer to Understanding Apple’s binary property list format | by Christos Karaiskos | Medium)

Fortunately there is an apt package for Debian that you can install using apt install libplist-utils and then use to decode the bplist into a readable xml with a command similar to :

echo 'YnBsaXN0MDChAWwAQwBhAHQA6QBnAG8AcgBpAGUAcwAKADMICgAAAAAAAAEBAAAAAAAAAAIAAAAAAAAAAAAAAAAAAAAj' | base64 --decode | plistutil

In my case, the result shows :

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<array>
	<string>Catégories
3</string>
</array>
</plist>

So my Test file has a purple (color number 3) Finder tag called “Catégories” !

About the feature request for cross-OS xAttr mapping, I don’t need it for now. I’m afraid it would be very complex to handle because it’s more than just associating a key from an OS to a key from another OS : there needs to be an encoding / decoding step to accomodate with how OS are managing their data.

Thanks you again for your help !

PS: For those interested, other ressources on this rather obscure Finder tags technical subject :

2 Likes