I’ve been tinkering with my backup solution lately, and now Syncthing has been used for long enough to be proven reliable (in my situation), I thought I would try and create the perfect solution.
I see that @kinsham was in a similar situation a few months ago. I used file versioning initially (in ST) and the system went a little squirrelly. So I tried turning that off and so far it seems to be running very smoothly, Backblaze and ST are playing nicely with each other.
Of course though, this means that a very helpful feature of ST can’t be used. For me, using TrueNAS, I’m falling back to Snapshots. If I delete a file or need to restore a previous version, I can go to TrueNAS and restore a cloned version of the snapshot that contains the correct file or version of the file.
As you can imagine, this is a bit of a pain. What’s more of a pain, as @kinsham said, was that Backblaze do not offer a “hold one version of the file” option. The Lifecycle options are only as shown below:
What do you mean exactly by this? Did you encounter any problems or malfunctions when using file versioning? As far as resource usage goes, file versioning should have no impact on that, with the exception of taking (possibly quite a lot of) disk space, of course .
No problem . You may want to be careful with simple versioning though, as the cleanup behaviour is buggy (see https://github.com/syncthing/syncthing/issues/7988) and can/will delete versioned copies too early. I’d recommend staggered versioning personally.
It also lets me replicate snapshots to other servers, so I have 3 copies locally on the LAN and my intention is to have an off-site one with BackBlaze. The cost is the only kick in the pants.
the replication feature is good though, I started a new replication last night (200GB) from one machine to another. It finished in 30 minutes.
I don’t think that’s true here, unless I am misunderstanding what the setup is: Syncthing makes sure the server has all the data from other devices, local snapshots and backblaze back that data up. That’s basically textbook usage of the right tools for the right job.
Ideally, the bug should just be fixed, especially since the cause and solution are known already. Someone only needs to roll up their sleeves and sit at the code. This includes myself too .
Many people already use the versioning, so it can’t just be “disabled”. The default settings are actually safe, as cleanout is deactivated as follows.
Thanks for that Simon, I admit doubts starting to set in about what I’m doing. I thought snapshots were at least sensible, in case SyncThing goes strange and starts deleting files. So far, the only times it’s gone wrong, is when the user (me) makes a mistake with the config.
Neither you or @calmh should beat yourself up about it, you and the other devs are doing a fantastic job and don’t receive the praise you should. I’ve looked at code and it’s mind boggling and that fact you maintain it to the level you have is already an amazing feat. As I’ll be eventually using it for business, I will be making monthly donations
One downside to running ST on TrueNAS Core is the delayed update schedule, iXsystems/FreeBSD tend to update plugins very slowly (for good reason). I have been meaning to do a manual install of ST to see if the update schedule is a little quicker. Because of this, it appears the default looks like this:
I’m curious, what was the required fiddling? My understanding is that it’s “just” Docker so should be fairly straightforward? Is there something we can improve on our side?
You and your colleagues have done a fantastic job, with TrueNAS Scale we’re still at the early stages of the product and many have advised not to trust for production workflows.
My issues were not the fault of ST, but rather the lack of info relating to Scale. I think part of the problem is a few things that are different between the Core ST and Scale ST, for instance the user id is different, which doesn’t help when trying to migrate between the two.
Do appreciate my background knowledge is fairly minimal, so don’t pay too much attention to my comments - I’m just an average Joe
That was the solution I eventually settled on. I didn’t know about Duplicati so used rclone in a cron script to update B2. I’ll take a look at that. Thanks for the tip.
Thanks for adding this Jean, out of interest are you using TrueNAS or something else?
At the moment, I’m really happy with using TrueNAS’s built in snapshots and having BackBlaze back them up. Allows me to roll back locally if there’s a problem (so no extra backblaze cost) and as replication goes to another machine, I have 2 lots of snapshots to locally lean on if things go a little wrong.
I use openmediavault on Debian, mainly for the convenient web interface. But the actual OS doesn’t matter much when everything is set up via containers.
I do both remote and local backups via Duplicati. I did use rsync with hard links initially for the local backups, but the large number of files made this impractical and wasted a ton of space. Thanks to deduplication, the Duplicati backups is about 10x smaller than the source data.
.
. You may want to be careful with simple versioning though, as the cleanup behaviour is buggy (see 
.
