Incremental file versioning - not a full copy (RFE?)

Hi,

First to say - am amazed with how great Syncthing is.

The only item I found it’s missing is where large file (Outlook OST/PST) ~50GB is modified at just couple of locations then enabling versioning would prompt the other location where it is stored to keep multiple copies of this 50GB file.

I’ve looked at External option of versioning, though it raises two items: a) it talks about “removing” previous file from original location Question: Does it mean that the whole file is then transferred? If not, how is it built locally then if would be removed as per docs? b) Would it be possible to execute i.e. borg backup which itself does dedup, etc. as form of backup/versioning and sync from remote just the diff (thought it is default behavior)? I’m just getting confused with this requirement to “remove” file in case of external versioning - though similar logic seems to be applied using built-in mechanism.

The ultimate RFE would be - have versioning built on dedup/incremental backend, to save amount of data stored. This applies anywhere larger files are modified and versioning is required. This would be even for files with 20-30MB, just modified often - so normal office usage. And it would be amazing for guys like me syncing Outlook files or VM or anything larger in DTP/media, etc.

Not sure if it is easier to built in whole code into Syncthing, or just maybe use borg or other as backend performing all operations (easier) and just have parsing fronted or even drop that - and just have ability to not have to remove file in case of that versioning is used. The rest could be left to command line with borg (at least for me, though other users might want to have some UI built).

Thanks!

Like you mentioned, I’d expect external versioning with borg or similar to be exactly what you need.

Implementing typical backup functionality like dedup into Syncthing versioning would be a bad idea. Even just having versioning at all for remote changes makes people believe this counts as backup, which it doesn’t for countless reasons, adding functionality typically found in backups jsut makes that worse. Plus it’s a lot of complexity, that someone would need to test thoroughly and maintain long-term.

As for the external versioned which should remove (move) the original file, this happens after the transfer is complete but before the final rename.

1 Like