I am reviewing how Syncthing works and wondering if it would be a good use case for my needs.
I have around 1.5TB of photos on my desktop computer (Windows 11) that I want to sync both ways with my Unraid storage. Here’s my plan:
Two-Way Sync with Syncthing: I will use Syncthing to sync the photos between my desktop and the Unraid storage. This means any changes I make on my desktop will be reflected on the Unraid storage, and vice versa.
Daily Backup with Duplicacy: I will use Duplicacy to back up the synced photos from Unraid to Backblaze B2 once a day.
The reason for the two-way sync is that I mainly interact with the photos from my desktop. However, there may be times when I access them on the SAN from my laptop and make edits. I want to ensure that any changes made on either device are synchronized.
Would this be an efficient and reliable setup? Any advice or recommendations would be greatly appreciated.
Yes, I believe so as long as you’re also backing up to local storage in case there’s ever an issue at Backblaze.
Unless your photo collection has a lot of duplicate files or consists of mostly uncompressed images, it’s not likely that there will be much storage savings from Duplicacy’s deduplication feature. It also means verifying/testing your backups to Backblaze B2 could require a lot of egress data usage, so a local backup provides a much cheaper and faster alternative, lessening the frequency of checking the backups on Backblaze B2.
My own setup combines Syncthing with Duplicacy covering a mix of desktops, laptops and mobile devices running Linux, Windows and Android. So far in the 6+ years of using Duplicacy, I haven’t encountered any reliability issues and have successfully restored files without fail after inadvertent file deletions, data corruption, etc.
Since I am syncing the desktop to the local NAS, your saying When I use duplicacy to send to Backblaze B2 , I should also send it to the local NAS outside of the sync ?
So if I send to both , I just only need to test local backup and for the backblaze B2 i can just testing random files vs the entire 1.5Tb
This all seems fine. The only thing I think that deserves additional consideration is the editing software you plan to use. Make sure the software can support saving sidecar files with the edits. Since syncthing only manages files and can’t sync databases if your edits are only stored, for instance in a Lightroom database then the edits won’t be synchronized.
I use DXO Photolab for RAW editing and have it configured to read and write sidecar files. So if I edit on a laptop they are synced through syncthing to the nas and also the desktop and if I edit on the desktop they are also synced. If your edits are only saved in a database then those won’t be copied over and it’s really not safe to use syncthing on the database itself.
Yes, outside of the Syncthing folder(s). On the same storage volume in Unraid is fine because you’ll have Backblaze B2 to fall back onto, but adding external storage such as a USB drive is even better.
As good as Backblaze is, it cannot guarantee that nothing bad will ever happen. And with 1.5TB of files, it’s a lot to download during disaster recovery.
Yes, exactly.
If we assume that after deduplication Duplicacy only needs a pool of data chunks 1/3 the size of your backup, that’s still potentially 500GB or more to download for an integrity check.
For egress on Backblaze B2, the first 3x the average volume of data backed up is free – i.e. if your backup is 1.5TB, you can download up to 4.5TB per month before being charged $0.01/GB. That works out to 9 integrity checks before paying $5 (500GB x $0.01/GB) for each additional check (it could easily cost more than the monthly storage fee of $6/TB/month).
Syncing the Lightroom database is fine, as long as you make sure to only open it on one computer at a time, and make sure it fully syncs between uses. It’s the OG Syncthing use case.