Purge Delta Files/Folders Downstream

Hello,

For sake of clarity lets use this setup… I have 2 Devices and 1 Folder. One of the Devices has the Folder set as "master and the other set as “normal.” This gets me half way to my goal, where content isn’t being synced upstream to the “master” folder. My other requirement is that changes that put the master “out of sync” are removed from the “normal” Folder.

I.E. the “local state” and “global state” of the master Folder should always be identical.

Any guidance would be much appreciated!

My first suggestion would be to use filesystem permissions to prevent unauthorized modification on the “normal” side, where you do not want or expect modifications. That is, let the files be owned by a specific Syncthing user, run Syncthing as that user, and let other users only have read only access.

The second, worse, option is to keep pressing “override” on the master side. This will force the slave to resynchronize. You could do this automatically via the REST API if really required…

$ curl -X POST -H X-API-Key:abcd1234 http://127.0.0.1:8384/rest/db/override?folder=abc12-34def

https://docs.syncthing.net/rest/db-override-post.html

I understand what you’re suggesting… Personally don’t like the idea of having to manage it that way because it’s not truly autonomous because you’re relying on making OS configuration changes everytime there is a new Folder. I think it would be nice to have a feature that purges content from “normal” Folders automatically.

There are tons of examples where Linux software purges content in a master/slave configuration.

Do you have some examples? Of course rsync could be one, and might even suit your use case better, but that requires manual intervention on each “purge” so isn’t really the same thing.

The reason I’m asking is that while we get repeat requests for this, I’m deeply opposed to something that will automatically wipe out changes done by the user, in almost real time as they’re making them. This goes contrary to what Syncthing is all about, in my opinion.

The correct solution, which is implementable on all modern operating systems, is to prevent the user from being able to make the changes in the first place.

My request may conflict with some of the core goals of Syncthing. I just wanted to make it clear that there are scenarios where continuous overriding of “normal” Folders does exist.

My current plan is to make the “normal” Folders read-only for all users except the Syncthing user. Which solves 99% of my concern.

Out of curiosity. If a synced “master” Folder has content removed and while the “normal” folder is syncing it disconnects from the internet. When the “normal” folder connects again will it continue to erase what was removed from the “master” Folder?

What are those scenarios? I’d like to look up on it, perhaps it is doable in a non horrible way.

In principle yes, although I think you’ll be hard pressed to interrupt it in the middle in this fashion. The master side will send an index update saying “these five hundred files were removed”, which will likely arrive in a single chunk. Once it has arrived, the other side knows about it and will remove the files, whether connected or disconnected.

Thanks for the feedback and responsiveness!

What are those scenarios? I’d like to look up on it, perhaps it is doable in a non horrible way.

One example would be the synchronization of a central repository to a few load balanced repositories. Where content is added to the central repository and then consumed from the load balanced repositories.

In principle yes, although I think you’ll be hard pressed to interrupt it in the middle in this fashion. The master side will send an index update saying “these five hundred files were removed”, which will likely arrive in a single chunk. Once it has arrived, the other side knows about it and will remove the files, whether connected or disconnected.

Thanks for the explanation. I’m just trying to understand all the possibilities that might create an “Out Of Sync” status on the master Folder.

1 Like

This can be done with a normal folder setup

I’m not going to continue beating this horse, but just in case it was unclear I was looking for examples of

as that is what I don’t want to do, and if that’s actually common practice and people like it I would like to investigate.

1 Like

Honestly I don’t want to keep beating this horse either. It was a very simple case and point and it’s clear now that it’s too difficult for me to explain it to you.

Before this thread is closed, for your enlightenment… an example of this would be r10k. Where code is removed from slave compile masters if it’s not found on the master server.

Thanks,