Setting up a simple backup server for my files, not so simple!

[dark_passages]

Planning to run Syncthing on my Windows 11 laptop and sync it with my TrueNAS as a backup.

What did I do:

1. Install latest version for windows for personal use (so it can access my documents folder): v1.27.6, Windows (64-bit Intel/AMD) “Gold Grasshopper” Build 2024-04-08

2. Install the latest version for TrueNAS Core: v1.27.6, Linux (64-bit Intel/AMD Container) “Gold Grasshopper” Build 2024-04-08 (noupgrade, stnoupgrade)

3. Connect /Data on the NAS to the SMB share on that same NAS. I first used a folder on the NAS that I created from my windows machine, but every file or folder added by Syncthing was unaccessable by any other user. Since a backup that I can’t access is not usefull, I connected Syncthing with the SMB share using the same user my windows machine uses to connect. So now both Syncthing and my Windows machine can access the files. (in case I ever need to restore a file from the backup.

4. Connected both Syncthing machines. They are on the same network but did not see each other, no problem, by using the device identification they could find each other no problem.

5. Created a Test folder on my windows machine and shares it with my NAS. This worked almost immediately with my document.txt file.

6. I switched on File versioning on the NAS side. I chose Staggered file versioning, but now when I edit a Word document, the backup file is copied (0kb) but the original document disappears form the NAS. It did still make versions when I modified the document on the windows machine, but in the actual folder I could not see a file. Restoring one of the backups did not work, still no file in the folder.

7. I created a filter **/~$. to stop syncthing from backing up temporary microsoft office files.

8. Adding a new Word document to the Windows folder no longer adds the document to the NAS folder. So I removed the filter, but no luck.

9. I removed the backup folder from the NAS (told syncthing to forget about it, then removed the actual folder) Then told syncthing on my windows machine the folder is no longer shared. (it does know, but does not inform me about anything unless I open the dialog) Then I reshared the folder to my NAS, and when I accept the share there, it complains that it can not create the folder.

10. I removed the backup folder from my NAS again (just told syncthing, since there was no folder to remove). Then I also told syncthing on my windows device to forget about the folder. After adding the folder on the windows machine and sharing it with the NAS again it could create the folder structure without a problem again. So there is for sure a problem with re-sharing a folder to a machine that already knew the folder at some time in the past.

11. Now even without setting up versioning on the NAS side, Word documents are not synced. I copied some word documents to my windows folder and none showed up on the NAS side. Then I added a test.txt file and that showed up almost immediately.

12. Since the NAS has only a backup of the files, I set up my windows machine as Send only and my NAS as Receive only. Now as soon as I add a test2.txt file on the windows side, it shows up on the NAS backup folder, but without doing anything myself, syncthing decides that the backup file has changed and is therefore out of sync with the windows folder. I’m 100% sure that the file has not been changed, no one had access to this folder and I didn’t do it either. I saw this behaviour when versioning was on, but even without it, it occurs.

13. Several things went wrong in the steps above, but syncthing status is always “up to date”?

Any suggestions on how to set up syncthing so it will make reliable backup’s of my files without skipping Office 365 files, preferably with versions kept on the NAS side?

[gadget]

@dark_passages

Any suggestions on how to set up syncthing so it will make reliable backup’s of my files without skipping Office 365 files, preferably with versions kept on the NAS side?

Technically speaking, Syncthing isn’t a backup tool. Although it can be part of a backup scheme, it’s best to use Syncthing as what it was designed for – a file synchronization tool.

A proper backup tool fulfills all of the requirements you listed above without the Rube Goldberg machine.

On a separate note regarding the permissions issue(s):

• Since running Syncthing in a container makes file watching more complicated to get working reliably (or at all), one option is to map the SMB share into the container. That way your Windows PC and the Syncthing container see the same “view”. (Search this forum for earlier posts about this.)
• Tweak the Samba configuration so that the Syncthing user (again, more complicated since it’s inside a container) has read/write access to the files used by your Windows user.

Depending on if your TrueNAS server is running FreeBSD or Linux there are additional ways to handle the permissions.

[JimD-aus (Jim D)]

I have had issues with truenas file permissions and eventually just decided that truenas can figure out it own permissions, so long as i could access the files from console, i was happy.

Fortunately (for me) i now longer work for that company and dont have access to it to tell you how i solved the problem other than just leaving it as standard.

[dark_passages]

Thank you for sharing, I hope you like your new job better.

[dark_passages]

Well as my experiments show, it’s not great at anything at the moment. Even without versioning I could not get it to cync properly. The website states “easy to set up” and it sure looks that way, but although it shows that everything is up to date, it’s not true.

Interestingly you offer good advise about the things I already got working, but not about the things that I could not get to work Guess everyone has the same problems.

You are right, I went back to FreeFileSync, it does not work when i’m outside of my own network, but since i’m working from home 4 out if 5 days a week, it should be good enough. Will check again in a year or so, hopefully syncthing has matured enough by then.

[gadget]

@dark_passages

Well as my experiments show, it’s not great at anything at the moment. Even without versioning I could not get it to cync properly. The website states “easy to set up” and it sure looks that way, but although it shows that everything is up to date, it’s not true.

All things considered, Syncthing is actually comparatively easy to set up for a typical user scenario of two or more devices, usually consisting of a desktop/laptop and a mobile device.

A couple years ago a work colleague wanted to use Syncthing to share files with me. He usually asks for step-by-step instructions if it’s anything beyond double-clicking on an installer and following a setup wizard. I emailed him the link to https://syncthing.net/downloads/ along with a few quick pointers. Less than 20 minutes later he emailed me back that he had Syncthing running on his Windows machine. I went to check my (Linux) machine and saw the pop-up dialog box asking for confirmation to accept the connection. The turnaround time was so fast that I hadn’t even decided where to put the files he was going to send me yet.

Many who ask for advice/help on this forum have much more elaborate setups that involve some combination of desktop/labtop, mobile device, NAS appliance, D.I.Y. NAS server, network file share, cloud storage, micro-server (e.g. some “fruity”-named SBC such as a Raspberry Pi or Orange Pi), e-readers, a VPS, a VPN, a VM, Docker/OCI container, multi-tiered firewalls, cellular links, reverse-proxies, bridged networks… the list goes on and on.

So to be fair, it’s really not possible for the Syncthing website to cover every possible user scenario.

Interestingly you offer good advise about the things I already got working, but not about the things that I could not get to work

If I understood correctly, my advice should have resolved at least up to step 11 of your timeline.

The core issue was getting the Docker container with Syncthing playing nice with the SMB shares. Resolve that, and most (if not all), of the problems you described disappear.

On a related note…

@dark_passages

7. I created a filter **/~$. to stop syncthing from backing up temporary microsoft office files.
8. Adding a new Word document to the Windows folder no longer adds the document to the NAS folder. So I removed the filter, but no luck.

**/~$ basically says “Match any file named ‘~$’”, which wouldn’t match any MS Word temp file that I’ve ever seen. There was something else going on that resulted in #8.

Guess everyone has the same problems.

Yup, more or less depending on how many moving parts are involved.

You’ve got to tame the file permissions in TrueNAS (FreeBSD or Linux), Samba, Docker – and last but not least – Windows before configuring Syncthing. Even if you’re already skilled with each individual component, a mashup of all of them is a very different beast.

You are right, I went back to FreeFileSync, it does not work when i’m outside of my own network, but since i’m working from home 4 out if 5 days a week, it should be good enough. Will check again in a year or so, hopefully syncthing has matured enough by then.

Depending on how your LAN is connected to the internet…

• DDNS and port forward to Syncthing on your TrueNAS server.
• Add a VPN service to TrueNAS and port forward to that (WireGuard is a good choice, lightweight, and readily available on multiple platforms).
• A mesh VPN service such as Hamachi, Hyperboria, Meshnet, TailScale, ZeroTier…

In order for Syncthing on your laptop to reach Syncthing on your TrueNAS server you would have had to use the community-run Syncthing relay servers or set up port forwarding on your network anyways.