File Versioning locally

If you could give an additional option to do file versioning on the local machine when a user deletes or changes a local file, it would allow Syncthing to be used as a local backup/versioning software (a whole new use), which may be immensely useful to a large number of users.

At present, if I delete/change a file on my local machine, only the remote machine that is syncing will save the versions. If you have this optional switch, the local machine would also save a version of the same file in .stversions folder (or another one).

Here’s a typical use case:

I have a desktop PC for CAD/video (high power apps) and a typical laptop for portability use. Some people have two computers - one home and one work.

I keep data synced between the two. However, if I don’t turn on my laptop for weeks, there is no versioning happening on my desktop PC, which makes me want to install another versioning software.

Thank you for creating this wonderful software and making it available to all for free. It is deeply appreciated.

2 Likes

There is certainly a case for local file versions, many OS:es offer this somehow out of the box (Windows shadow copies etc). But it’s not really part of file syncing and it’s difficult to do correctly in an external application so I feel this is out of scope for Syncthing.

I think there are two ways to solve you problem or ideally you combine both.

– Use a always on server. This is very convenient because no matter where and when - you can always get your data in sync. For example your laptop has not been online for a while. You grab it, head out and just need an internet connection to get back in sync, because the server is always up to date.

-Use dedicated backup software. Here you get all the advantages (like versioning) to back up remotely or locally. I can recommend https://restic.net/, I think completes syncthing rather well.

(I have a always on server with syncthing that regularly backups for me.)

1 Like

I second this feature request. I actually though that syncthing had me covered, when I realized this morning that it actually doesn’t do local file staggering. I understand that there is other software for this too, but it would be a nice option.

It’s simply not possible.

At the point you see a file deleted it’s already deleted, you can’t version it if its gone.

There are some technical solutions. We’re watching files for changes, so we can make a copy every time a file changes and already have it archived. I’m saying this because it’s the obvious response to your concern.

However we should not do this because 1) it would suck as we’d be in a race against user actions and we’d lose, 2) this is not what we are about, 3) there are good solutions for this already, properly implemented at the file system level.

So, no, again.

Just an idea: Would it be possible to sync the content of the stversions folder amongst sync partners?

A deletes file x1
A syncs delete to B
B moves x1 to stversions
B syncs stversions/X1 back to A

So versioning would become part of the synced share.

stversions is just a directory, you are free to add it as a syncthing folder and share it among your devices.

I wasn’t aware you could do that. That’s interesting, I will try it out sometime. Could lead to nasty feedback loops if you started deleting files in stversions though.

Keeping backup and versioning separate from syncing is still the sensible thing to do though.

Not really, as these are independant entities. New versions or version deletions does not mean new files in the parent folder.

Innovative idea! However, it wouldn’t solve the problem described in my first post in this thread. So at present, I am keeping both computers ON – my laptop and my desktop so that one of them keeps the file versioning while I work on the other. My goal was to not have to keep both of them on at the same time just for the purpose of file versioning.

Windows Shadow copies is not as elegant and powerful as the versioning you guys have implemented. With Trash Can, Simple and Staggered versioning, the user has the wonderful ability to use one or the other, for each different folder structure depending on how critical data it holds. The Staggering File Versioning is just amazingly implemented!

Thank you

As write Dominik, I suggest a always on server. In particular, simple a NAS, I suggest QNAP and WD RED HDD, with 2 or 3 HDD in RAID 1 Mirroring.


I am interesting if the solution to sync the stversions folder function, and not made some hidden problem in synchronization.

Well, I guess I am another one who (loves Syncthing, but …) wrongly understood the documented versioning capabilities BEFORE to decide to go for it, I read it there in the Syncthing doc :

Syncthing supports archiving the old version of a file when it is deleted or replaced with a newer version from the cluster.

Maybe one should amend the wording with

when it is deleted on a remote device

The issue, however, is that such wording would make Sincthing much less attractive compared to other modern options (none of us want that, do we).

To be consistent towards the users who did (or will) read that doc, at least a non-default, non-corruptive, elegant-enough solution would be fantastic.

I’m a techy kind of user who supports the need to keep the solution as simple, performing and straight-forward, for maintainers and users, as possible. So granted that redeveloping a versioning OS feature is not the goal here.

Instead, if there were an elegant way to allow remote versions to be synchronized back to the deleting device, I believe that the feature benefit would be worth the extra CPU+bandwitdth cost (at the condition that sync loops are prevented of course).

Such a solution might already work, but could be at the expense of tricky settings like ignore rules (have fought with them already, and lost, and I’m not the only one) that must cope with:

  1. possibly different versioning strategies on different devices for the same folder (I was surprised to see that this is allowed, might prove useful though I recognize)
  2. the need to have a visible, or not, “Version History” (or, …you name it) folder, on required devices (here too, I thought that would be an enforced consistent setting across the cluster, seems not)
  3. and more, possibly …

So: what would be great (if confirmed as required) is an option in the UI and/or protocol to strengthen the consistency of required folder settings (versioning folder name, ignore rules, versioning strategies, maybe) in a cluster for this locally available versioning purpose.

And hopefully that might be quicker for a developer to think it over and implement it than for me (us) to write and rewrite this plea (and the same developers to decrypt them.).

Hope this is useful.

And ultimately effective :wink:

The wording “from the cluster” is intended to convey precisely what you’re after, “on a remote device”.

I don’t think that’s an issue at all. We’re aiming to correctly describe the capabilities.

I confess I am no native English speaker or literate, but it seems to me that the current wording “from the cluster” would, to the reader, apply (almost) unambiguously to “a newer version” rather than to “deleted”. Which means that deletions are performed on the Syncthing side, which is the local client side by default to the mind of a user.

Total disambiguation could be done (but not sure how to best write that concisely in unambiguous English grammar).

I don’t think that’s an issue at all. We’re aiming to correctly describe the capabilities.

Fair enough to aim at doc precision.

For the issue, I meant something else, more about the adoption success of Syncthing:

if we clearly remove the historization/versioning from the product features doc, numerous potential users will either go away from Syncthing or use/develop alternatives or variants. I would have done so, and I am far from the only one to raise the question.

Preferably if, on the other hand, we allow users to benefit from local historization/versioning, they will keep coming and support the project.

As soon as that does not compromise the stability and maintainability of the product, I think it’s a very beneficial, tiny (hopefully) adjustment that is important to do.

If you have a suggestion for a tiny adjustment to keep versions of files deleted by the user, without preemptively keeping a second copy of all the user’s data, I’m all ears.

1 Like

Great, thank you !

Actually I recognize I would shoot more for @sicro’s solution than @4world’s request, the reason being that most of our users are connected to a server that we provide, with synced folder, so the “not connected” use case would not apply. And … maybe native OS features support that (would love to have a list of the Windows and Mac OS either native or free/OSS tools that do versioning without a installation hassle for mass users).

SO I understand from @AudriusButkevicius’s comment that it’s already supported, only cosmetic changes like enforcement of consistent folders settings would be required to prevent this use case to be broken by wrong stversions folder names, or wrong ignore rules, or conflicting versioning strategies.

That could actually even start without developement, just with a guide or tutorial how to configure settings to support this:

stversions is just a directory, you are free to add it as a syncthing folder and share it among your devices.

and this:

Not really, as these are independant entities. New versions or version deletions does not mean new files in the parent folder

Then for your use case the only interesting place to version is on the server, so that’s the version directory you need to expose to your clients. The clients don’t need to run any versioning.

So would this sketch of tutorial be correct (happy to amend if not):

HOWTO Syncthing local versioning based on server (connectivity required)

Use case:

  • Client 1 (create, modify, delete randomly, access to files older versions which are updated onlywhen connected)
  • Clients 2…x (create, modify, delete randomly, , access to files older versions which are updated only when connected)
  • Server (for historization/versioning purpose, always on)

Settings in GUI:

Clients side Web GUI:

  • Sharing: with server, and optionally other clients
  • Versioning: No
  • Ignore patterns: None
  • Versions path: None

Server side Web GUI:

  • Sharing: with clients
  • Versioning: Staggered
  • Ignore patterns: None
  • Versions path: _Older_Versions

THanks so much for pasting this post. SYNCTHING + RUSTIC was the missing thing in my life

1 Like