Assuming no progress has been made since it was last discussed, can I ask to re-open and upvote the discussion (previous thread Sync file ownership and permissions) on “proper” multi-user syncing, supported by a single instance of Syncthing.
My use case isn’t remarkable. I have 4 or 5 PCs at home, plus a Debian server and a little old NAS box now used only for backups.
I want to maintain at least 3 copies of each users personal files, one on the server, on on the PC they use most, and a 3rd redundant copy. I also have a library of shared material (software, images etc.) I want to replicate across multiple machines.
I had hoped Synthing would be multi-user aware. The prospect of installing and configuring 20 or 30 separate point to point instances for each connection in this loose “mesh”, and trying to manage the permissions is not appealing.
I realise this means having at least part of the server running as root (or the equivalent on Windows), and mapping between Windows and Linux permissions, all of which is a pain to get right. However it’s not unique to Syncthing.
Samba for example has to solve exactly the same set of problems, and it does so by running the master server daemon as root, and then spawning a separate process for each user when that user ID connects. This might not perform brilliantly for the guys trying to support hundreds of users off one box, but for small and medium networks it should be fine.
I really hope this can be put on the Syncthing roadmap, because without it i’m either going to have to try to come up with some kind of bodge (kludge) for my server, like putting all user files in a “synthing” group, and having a cron job running chmod-R regularly to fix up the user permissions, or give up and carry on looking for a different solution.
Sorry to necro-bump this thread, but I thought it was better than starting a new one on the exact same subject. I also vote for this feature, so to speak. I would just add that it’s not only business environments that would find this useful. It would be highly useful for sync’ed NASes, even in home environments. We have a NAS, with 4 users’s accounts on it, that I keep bidirectionally synced with a similar NAS at one of the other users’s houses. The sync must be bidirectional, so rsync won’t do. I know numerous others who do this to handle both file serving and offsite backup at the same time.
The only tool out there which can really do this at a reasonable cost (free, in fact) is Unison, which works fine, but it’s an older, only lightly maintained and a bit clunky tool with some limitations. A modern tool like syncthing that can do this would be a major boon for small server syncing.
Since this feature would be most useful for server syncing, it really wouldn’t be a big limitation if this feature were only implemented on posix-style platforms, as others have mentioned. Hopefully, that would reduce the complexity.
There is a feature in recent syncthing that allows copying permissions from the parent dir. You need to run as root or as a user with magical permissions to do that, but it’s available in the current release under advanced settings.
"The following may be synchronized or not, depending:
File Permissions (When supported by file system. On Windows, only the read only bit is synchronized.)
Symbolic Links (synced, except on Windows, but never followed.)
The following are not synchronized;
File or Directory Owners and Groups (not preserved) "
This seems to imply that some permissions are indeed preserved, but owner and group are not. If you think the FAQ is not up to date, then I might download the software and try it out. If you’re pretty sure the FAQ is correct, then I wouldn’t need to.
@imsodin, that is always an option, as it is with BTSync, ownCloud, and just about every other sync solution that syncs everything except file ownership. There are a few issues with it, though. The biggest is that it’s just clumsy, requiring a lot of extra setup that shouldn’t be necessary. On a machine that I managed for a business up until a year ago or so, there were 23 users. Unison doesn’t require me to set up and maintain 23 sync processes and 23 configurations for all of those users. It’s probably workable in an environment of a couple users, but it definitely doesn’t scale well.
Unison has it’s own disadvantages, however, the biggest being the lack of discovery and dynamically tracking connections, etc. Syncthing has those modern features, and if it could just incorporate file owner and group, it would be a single-point solution for essentially all syncing needs.
That looks like an interesting approach and a good start. I think it covers my own use cases, though I’ve seen use cases where it wouldn’t work. In particular, I’ve seen people, especially in companies, create shared “project” directories for collaborative purposes where anyone with group access can place and edit files. Directories like that will have many files who’s owner isn’t the owner of the parent directory. I think that particular use case is probably declining, however, as that sort of approach is gradually getting replaced with more modern paradigms like repositories and revision control systems.
In other cases where random files do not have the same owner or group as the parent directory, it’s often inadvertent. Perhaps if this functionality is enabled, and syncthing encounters a file that has a different owner or group than the parent, it can sync the file according to this paradigm but log an entry somewhere so those files can be checked.
It looks close enough to what I personally would need that I’ll give it a try. Thanks…
EDIT-----------------
I apologize if this is confusing, but the system will not let me add another question, so I must add it here.
Hi,
I downloaded 1.1.0 and have been looking at it. Since this feature is not documented yet, I have a question or two.
The option seems to be available even if I run syncthing as a normal user.
If I’m running as a normal user, is the option silently ignored even though it still lets me turn it on? Or will it generate errors later on? (I haven’t actually tried to sync anything yet, just play with the settings so far).
It will generate errors. Not least because it might not even be able to create files in dirs it doesn’t own, depending, but otherwise when it tries to change owner/group.
I’ve been testing this and it seems to be working fine so far! Thanks for adding this feature. I think it should cover most common use cases of syncing multi-user NAS servers, though I haven’t tried scaling it up yet and migrating from Unison – that’s next after I rebuild the two NASes.
I suggest putting this on your main feature page. It is not a common feature and is often asked for. People searching for this feature should easily be able to find that Syncthing has it
This is something I will try in the coming days. I’m looking to sync home folders across a SAS connection, and preserving both chown and chmod file permissions is essential. Even copying extended setfacl permissions would be an added plus since not all people that belong to a group have write access, some have setfacl -rd u:750 to limit what they can do in a project folder and so on. I guess what I’m looking for is a bidirectional sync that preserves everything in the file environment so that users have the choice of running on one of two systems that have the same file permissions. Centralized shared NFS would be simpler, but would not be as fast and it is a single point of failure. Bidirectionally synced machines (both with zfs snapshots) provide better redundancy options and more available cores.
Maybe you could look at osync (https://github.com/deajan/osync), which is based on rsync plus bash scripts. I have not tried it but it claims to be able to sync ACLs. IIRC, it is possible to change ACLs without changing modification time, so change detection algorithms would need to be more complex.
This is generally an underserved area, as it is probably an uncommon use case at the moment. But with more people wanting to bidirectionally sync two NAS-type devices that are not both in a common UNIX environment (i.e., not accessing each other via NFS, etc.), maybe there’s a need for a tool that does just this.
The new feature implemented by Syncthing should cover a fair number of simpler use cases (including mine, fortunately, since we don’t use ACLs), but has some limitations in exchange for the simplicity.
I will give both osync and unison a try once the two new file servers are up, and Acl’s will be of little use if it is only for syncing /home. The mixed Acl’s will be more a concern later when dealing with shared project folders, that need read and write mixed group permissions.
I must now put some thought into how to connect the two file servers with SAS and if I will sync them over each other’s NFS mounted folders or over some ip based protocol like SSH.
I solved all my samba + Windows domain ACL / authentication issues by scrapping FreeBSD and Debian file servers and switching to OmniOS (Illumos). Has literally saved me 10s of work hours over the last year alone that would have gone to troubleshooting Samba.
Im doing this and running centospanel to help with this. Im using centospanel as a cloud solution and decentralized and centospanel fixs the permissions for me on the fly.
This is my full fallover and sync for panels files on a cloud all on many vps’s all over and no problems at all. all servers have to be the same. as far as spinnning new ones up. Centospanel installed first then restore database for centospanel from my cloud and then install sync with my installer script and let it take over syncing and then centospanel fix my permissions for me or can rebuild all permissions thru centospanel with a click of a button and all goes well from there.
That’s neat, I was toying with Cockpit and it looks pretty solid, but I haven’t explored it for any in-depth Samba work besides joining the domain yet. Will have to do some testing. Thanks for the info
I currently have a cron job that runs nightly to fix ownership issues. I have a multi-user NAS that Syncthing is running on (XigmaNAS). The user it runs on is called syncthing and it’s a member of the admin group so it has full access to all directories. Once a file is created, the ownership and permissions never change, it’s just new files that get created with the wrong owner and permissions.
The issue I am having with this feature is syncthing tries to run lchmod directly, without sudo, and it fails with “operation not permitted”. How can get syncthing to use sudo?
edit: syncthing can sudo without having to enter another password.
