Possible to use Syncthing as a backup/transfer tool?

Support
1

Hi, I’m currently trying to figure out how to back up my laptop. I lack an external drive. It’s a one-and-done thing, I just need to transfer a backup/copy of all my files to my main PC, which has a SATA drive with sufficient storage.

Can I use syncthing for this? I was considering just setting up C: as the syncthing folder, make it send only, and then have it transfer all files in realtime to my desktop as a backup. I ran into issus however, when windows started declining access to some folders.

2

Syncthing cannot access files, which are locked by the OS, so no, you cannot do that. You would need a tool that can use VSS (Volume Shadow Copy) for that.

Why are you trying to transfer the whole drive contents though? Windows copied that way won’t be usable anyway. Why not transfer only folders with your actual files (e.g. Desktop, Documents, Videos, etc.)?

If your goal is to actually move the OS in its current state and continue using it on a different machine, then you would need to clone the drive, not just transfer the files like that.

3

You could use macrium backup or veeam to backup the entire OS - but why would you want to do that?

If you’re looking for a better backup strategy for your data (also cross platform, in case you ever need your files outside of Windows) - look into organising your personal data and files in a format similar to this.

You could then store that ‘data’ folder outside of your Windows partition, so if you ever reinstall Windows, your files remain intact.

  • Uses less storage (also more cloud-friendly)
  • Easier and faster to backup
  • More portable (you can sync/backup only certain parts of that folder structure to other devices)
  • No OS/software has a proprietary hold of your data (windows/macrium/veeam, etc)
  • OS-agnostic - if you ever decide to use a mac or linux in the future, all you need is to copy that one folder over
  • Syncthing-friendly
4

the problem is I don’t know which folders contain my files. Sometimes I’d also need to access application data too, so that involves program files and appdata in the C drive.

I’m planning on wiping the whole windows install and lack any external drive to back it up to, hence this roundabout method.

5

it’s a one and done thing, I just need to have a copy of all the files in case I need something in the future, but I don’t have any external drive to back it up to.

I’ve done something similar with my PC, backing up windows to the HDD, and I’ve had to access the backup to grab some files a few times.

6

For a one and done thing you could just use robocopy when you need it; it already comes with windows.

1 Like
7

Robocopy won’t really do any better than Syncthing, because it also cannot access locked files. FreeFileSync can though, if you run it as administrator.

1 Like
8

Will give that a look-see, thanks. it seems like what I’m looking for.

9

looked into robocopy, besides what tomas said, it seems to be local-only. I’m planning on backing up the entire C: drive, and my laptop only has that single drive and single partition. I have no external drives to copy it to. The best I got is a PC with enough storage space to handle the backup.

Unless I misunderstood and robocopy can transfer the files over a network instead?

10

robocopy does work over the (local) network (and you don’t need to install anything extra which is why I suggested it initially), but freefilesync also comes with a little interface where you can see what’s happening and that may be useful to you.

As far as backing up OS locked files - not sure how much you’ll gain from it; like tomasz mentioned earlier - you will run into issues when restoring.

Unless you’re after a 1-click restore solution eg Macrium/Veeam for speed and simplicity - although you can create the backup with Macrium to a network location, I’m not sure how you would then restore it to your laptop if needed.

Other than the scenario above, it’s hard to justify backing up and maintaining things that can be easily downloaded again - like windows and programs/installers.

Depending on how many different programs you use and how flexible they are - you may have a better backup experience if they let you change their working directory or export the configuration/settings to a folder you control and just backup that folder to your PC (with syncthing).

You will see just how little data you actually need to back up as opposed to trying to copy the whole C drive when most of it will be of very little use to you.

1 Like
11

I’m not planning on restoring anything. It’s just in case. For example, my most recent use was trying to recover an autosaved .txt file from my text editor, as I never saved it proper. The autosave location was in the system files.

I just want the files to be accessible in a way where I can dig through and find individual files.

I guess it’s just because of how paranoid I am. I have lost many files over the years due to corrupt/incomplete backups and such.

edit: just to add more context, The plan for this time is to back up windows as a whole, so i can access the files, and switch to linux. Hence, why a single backup system like macrium won’t work, as I won’t be able to access files from that backup file.

12

Maybe in that case you can get away with just backing up your user profile (contains your desktop, documents, etc + app data folders) - autosaved txt files etc should not be locked by the OS, so I think you won’t have too many issues/warnings.

But maybe revisit what software you actually use, because lots of them are connected to an online account these days and backup your settings and configs to the cloud.

For the ones that don’t do cloud, many of them let you configure a working/export directory, which is a good practice to move to a location you control, monitor and backup.

Additionally, for things like VS code editor, cursor, etc - they tend to go in your user profile app data, so you could selectively backup just these few apps that you actually care about.

The reason I’m recommending your backup strategy to be a bit more intentional (as opposed to a massive data dump) is because you’ll then be able to easily use something like syncthing - while not a backup tool, it can be very powerful as a backup tool in your scenario - it provides a native file versioning feature, so you can configure it to keep multiple versions of your autosave.txt (for example). But if you backup a bunch of unnecessary and/or temporarily files, they will also get versioned which is.. not ideal.

The good news is that what you want can be achieved in many different ways (manual copy, robocopy, freefilesync, syncthing, etc) - and all of them should work for your use case.

I’m biased towards syncthing - very nice interface, low resource usage, easy to view and manage the data you share with your PC, works out of the box with no additional setup even when you’re not at home (if your PC is on), file versioning, etc.

13

I dislike using cloud-based systems, that’s why I’m so insistent on backing up the whole drive. I sometimes access application specific data in appdata and program files, too.

that’s true, and I’m now tracking what apps i install. Still, I have no knowledge on how many apps I have installed nor where they save the data, and I don’t really have the time to individually check the storage locations and back them up (I believe I have over 100 apps installed) I just find it easier to back up the whole drive, and I tend to remember/need data many months down the line when I notice I am missing something.

I am already doing that and am slowly transitioninng to syncthing. Shared applications are stored there (portable ones at least) instead of directly installing into the OS. (some games like minecraft and such) Still, my laptop which I’m wiping is many years old and has been used before I switched to such a system.

tbh the easiest thing would be to just buy an external hard drive and copy the files manually with windows explorer, but I lack the money to buy one, plus it isn’t worth it for just a single transfer of my c-drive, then never needing it again. That’s why I’m looking for a p2p system that can transfer all the files in realtime as my desktop has sufficient storage to hold the “backup”.

same. I’m really happy with syncthing and have it set up on all my devices (2 computers, 2 phones and an ipad), which is why I came here to ask as the first step after realizing it had trouble accessing system files, even if syncthing is set to send only.

14

Hmm how about using syncthing to send-only your C drive as you did originally, and use ignore filters to exclude OS locked paths and files since they’re probably not what you’re interested in backing up anyway?

1 Like
15

Hello!

As far as i understand this is what you want to achieve:

So this makes perfect sense:

Side note

The topic title mentions “backup” and it is also mentioned all over the comments. I would like to stick out my neck and say that this has nothing to do with backup at all. This is about having a copy of a certain set of files on another computer. Sure, “backup” is also about having a copy, but it is also so much more:

  • A strategy.
  • A process over time.
  • Etc.

This is, IMO, something you need to think about after you have done the wipe and re-install, to make sure that your data is safe in the event of theft, malware, accidental deletion etc. Syncthing is NOT a backup SW, but can be one part of a backup solution.

2 Likes
16

yeah that’s my bad. I use the word “backup” to include just copying a file somewhere else, and I label that copied file a “backup”. I should probably see if I can re-word the thread to make that clear. Thanks for the clarification.

that’s what I’m likely gonna do. I’m gonna have syncthing sync as much of the C drive as it can, then check the error logs and see what It missed, and copy them over manually if I can.

1 Like
17

Just keep in mind if you don’t enable some kind of versioning, as soon as a file is deleted on your laptop it will be deleted on your desktop. So if you want to try to recover a file that’s no longer on your laptop it would have been removed from the desktop too (unless you’re using versioning)

anyway there are many posts warning about using syncthing as a backup tool and those are mostly valid concerns. So please be aware.

18

I’ve got it set as send only on the laptop, and receive only on the desktop. Once the sync is complete, I’ll wipe the laptop and install the new OS. There should be minimal downtime.

I’m aware Syncthing isn’t meant to be a backup tool, and according to martin’s definition, what I’m trying to do technically isn’t one. It’s a one time transfer of all my files so I can wipe the PC and start over. the copy of all the files is there just in case I missed a file in my saving system (syncthing, forgot to save etc.) and need to recover it months after I wiped my storage.

19

Okay gotcha. Recommend you pause the folders perhaps on both ends before you start wiping the laptop. If you’re gonna wipe by reformatting from a boot USB stick then maybe you don’t need to but it doesn’t hurt.

20

yeah, I paused all the folders on my laptop, but since the desktop is receive only, I left it as-is. Syncthing is currently attempting to sync the around 250GiB worth of files, fingers crossed.