I know this thread linked below is old but I was curious if use cases like this are achievable now with Syncthing:
I echo the comments in this thread in that Syncthing is my go-to because it just works, its secure, open source, and so forth.
I will try to explain what I am trying to do using Syncthing’s terminology:
My use case is very similar in that I need several Send Only nodes that cannot receive changes from any other node in any way. All the Send Only nodes have the exact same folder structure. There may be overlap in files that each of the Send Only nodes has with different mtimes, but the hashes for any given file itself would be consistent (The files themselves are part of a depot where the filename is the SHA1 hash of the file itself). The aggregate of these Send Only nodes would be considered the “Global State”. There would very likely never be a single Send Only node that has all of the files and as such there cannot be a single “Master”.
On the Receiving end, each node would be set in Receive Only mode. These nodes would each only receive the files they need using ignore rules. Effectively this would use ignore rules to pull the files needed from the aggregate Global State.
So this leads me to two problems that I can’t seem to work around unless there’s some special settings somewhere that would allow this:
Syncthing seems to require a single Master for any given Folder ID (which is really more like a folder “linkage” ID in my mind). I was hoping this wasn’t the case given the “Global State” terminology in the UI. But it does appear that any single Folder ID can only have one node act as the master at any given time. If I create several Send Only nodes with the same Folder ID they seem to just pass around Master status based on whomever last clicked the “Override Changes” button.
mtimes are synced. For this use case syncing of mtimes is unnecessary as long as the files are hash equal on the different Send Only nodes. Ideally I’d like to ignore mtimes completely because without that it would mean there must be metadata changes on Send Only nodes.
So basically I’m looking for creating a global state with no single master and also ignore mtimes. Is something like that possible? I cannot seem to find any application that would support something like that but Syncthing seemed closest.
I greatly appreciate any feedback or suggestions, and as always keep up the great work. I use Syncthing for several other use cases and I would love to make it work for this as well if at all possible.
Syncthing won’t do what you want it to. The global state for any given folder is made up of the latest version of each file announced by the participating peers… And you can have multiple send-only devices, but the in-sync status of each device will be complete nonsense.
The idea is that each device strives to sync up to the global state in question. Send-only means you don’t care what others announce, only your state is relevant and everyone else should fall in line. In that model it doesn’t make total sense to have more than one.
Syncthing in its base form is about keeping precisely identical sets of files on machines A, B and C. There are some exceptions that can be made (ignore patterns) and some protections against unexpected modifications (send-only/recv-only) but in the end identicalness is the goal. If identicalness is not your goal, Syncthing is probably going to fight you and end up being the wrong tool.
And you can have multiple send-only devices, but the in-sync status of each device will be complete nonsense.
Agreed. That’s what I observed in a proof of concept.
The idea is that each device strives to sync up to the global state in question.
Understood, I was just hoping the “Global State” could be composed of multiple nodes at a time and virtually merged based on the Folder ID instead of requiring a single master. But you confirmed Syncthing was simply not designed this way, even though it is possible to link folders that would suggest this is a valid configuration. (Simply manually using the same Folder ID)
Syncthing in its base form is about keeping precisely identical sets of files on machines A, B and C. There are some exceptions that can be made (ignore patterns) and some protections against unexpected modifications (send-only/recv-only) but in the end identicalness is the goal. If identicalness is not your goal, Syncthing is probably going to fight you and end up being the wrong tool.
Understood. Most of the pieces are there already, but this particular use case just doesn’t seem feasible.
However that raises the question of what is “identical” for the use case? In this case hash equal files are identical and the mtime does not matter. There’s already options to ignore other meta data and permissions so I was hoping there was a way to also ignore modify time for determining a valid “in-sync” status. In the model I outlined above obviously a Receive Only node would need to have some modify time with the file delivered… so even if this defaulted to the most recent mtime for that particular file amongst all the Send Only nodes that would be fine.
Either way, thanks for confirming Syncthing in its current form isn’t cut out for this sort of job. If you do have any creative ideas for how to solve this use case any suggestions are welcome.