I use syncthing to sync form Mac OS to Linux. Because extended attributes get lost I made my Mac “Folder Master” with the hope that some day extended attributes will make it to the other side for changes to go in any direction. Of-course Linux supports xattr as well, so the reverse problem may apply in other cases. So far this is not a critical issue for me, though I use tags a lot. This helps me keep a flatter, more manageable directory structure.
Because I use color tags a lot, I’m hoping that an eventual solution will allow to blacklist (or else whitelist) certain tags for a partial tags sync. That however is not as important as preserving the tags to begin with.
Keeping metadata as part of the filesystem is imo superior to non-standard solutions. So I’m considering something silly like implementing this filesharing idea with extended attributes. An alternative would be to use cozy cloud or something. With cozy though the files are in couchdb which is less convenient than straight fs.
Recently found this article which gave me some hope my wish has a chance. Samba and rsync are ok, but I rather use syncthing. Syncany doesn’t backup xattrs yet.
Extended attributes (xattrs) on all platforms are data. Infrequently used but still data.
Preserving them is difficult, complex and corner case prone but not preserving them is directly contradictory to stated primary goal of Syncthing. This has come up a number of times on the forums and I propose that it warrants a stronger warning than just the sole mention in the docs. https://docs.syncthing.net/users/faq.html
Syncthing should be:
Safe From Data Loss
Protecting the user’s data is paramount. We take every reasonable precaution to avoid corrupting the user’s files.
This is the overriding goal, without which synchronizing files becomes pointless. This means that we do not make unsafe trade offs for the sake of performance or, in some cases, even usability.
There are corner cases. For example, icon files traditionally hold all the icon data in the resource fork on Macs, because the icon is a “resource”. Syncthing will happily sync this as a zero byte file to everywhere else, including other Macs.
My personal opinion is that today it would be an extremely unwise choice to store critical data as alternate data streams, in resource forks, etc.
Nonetheless these things exist and it would be neat to preserve them. Pull requests welcome, as noted multiple times above.
Agreed. xattrs are an error prone, poorly engineered way of storing data. It is unfortunate that they are being used at all. Syncthing’s handling of xattrs is not a game changer for me but it does stop me from using some features of my OS since I know the changes will not be synced. I fear that Apple will keep adding more and more data to the xattrs in the future which will break all sorts of applications.
Syncthing has been a genuine blessing to my workflow. Thanks for the fantastic software.
Resource forks are not only a problem with syncthing. Also with SMB, NFS, AFP protocols. However, as forks can make it difficult to exchange files with other operating systems, this feature is not in common use. Even in OS X, resource forks are seldom used anymore. Source https://en.wikipedia.org/wiki/Resource_fork#Compatibility_problems
I think the cleanest implementation would be to add a string field alternate_stream to the FileInfo structure. The various alternate streams and attributes and stuff then become separate “files”. The actual puller would have to decide whether to treat changes as metadata changes (applied to the existing file) or content changes (file is copied with change data changed), depending on the specifics.
Just a point of feedback into the project: I look after a company’s IT, for which Syncthing is an ideal fit. We’ve installed a trial setup for them, and they’ve been using it for nearly six months.
Unfortunately they’ve requested to move over to another solution - probably Dropbox in their case - because Syncthing does not sync Mac OS X Extended Attributes. In their case, they heavily use tags/labels to manage the state of projects - and as these don’t get synced, they feel they’re not able to continue with Syncthing.
I know I know that you’d welcome a well-written pull-request for this; unfortunately I don’t (yet) have the time or skills to put something together myself. It’s on my radar to do so - but I can’t see myself getting to it in a useful way in the near future.
Please don’t bump dead threads, all that does is annoys people, and will definately have
zero impact on whether someone is interested in working on this or not. Best way to get features in syncthing, is to develop it yourself.
This is just a note for anyone who comes along and thinks about implementing this.
You should probably look at how Netatalk (the venerable Apple Fileshare Protocol server for Un*x systems) and Samba (the also venerable SMB server) do this. Because they changed quite recently from .AppleDouble to using extended attributes.
Of course it’s not necessary for Syncthing to store things in a compatible way with file sharing, but it would be really good if it worked. And no need to re-invent the wheel (and make it square).