I’m panicking right now. I had all my some of my Plex media on a 5TB USB drive and it was syncing up with a copy on my unRAID server. The USB drive died a few months ago so I switched to using the library on the unRAID server (the USB was the “master” library that Plex used). Eventually I got a new USB drive (identical make, model, capacity, even the color) and hooked it up to my Mac mini that the Plex server runs on and started re-syncing to the unRAID copy. Which it’s doing right now.
I just checked the folder on the unRAID server and most of the folders are GONE. Most of my movie library is gone. What was almost 5 TB of movies and video are down to 140GB.
We’ll need some more information. How did you configure Syncthing once you connected the new USB drive? Did you do anything with the .stfolder Folder Marker or the configuration around that?
This is probably where I screwed up, on the .stfolder. I got an error about the folder marker missing and pointing me here for assistance. But after reading I still couldn’t make heads-or-tails on how to regenerate the marker. I checked the .stfolder on the unRAID and saw that it was just an empty folder so I just created a new .stfolder on the USB drive.
As far as backing up the unRAID server, well it’s part of a drive array but the array itself has no backup.
There’s a little bit more in that I have multiple folders being sync’d up with Syncthing on that USB drive that crashed. One folder has part of my Plex media and another folder has audiobooks and podcasts. I created new .stfolders for both but the audiobook folder seemed to be syncing up just fine so I was pretty freaking shocked when I saw what happened to the Plex media.
I suspect you connected the new USB drive, saw the warning that the folder marker was missing and mistakenly created the folder marker.
in this case syncthing believed you had reconnected the original drive (it doesn’t know the difference) and the scan shows you deleted all the files on the USB drive so it’s syncing those deletes with the unraid server. (This is why there’s a data loss warning in the missing marker warning.)
The proper way would have been to delete the folder in the syncthing UI instead of creating the folder marker on the empty USB drive. Then in the UI re-accept the share from Unraid and syncthing copies all the files back to the USB drive.
Hopefully you have a backup somewhere else or you have versions enabled on the unraid server.
I suspected as much. I intentionally avoided trying to re-configure all the folders because I had SO much trouble setting it up when I first started. Shows what happens when I try to take shortcuts.
One the bright side, I guess, there’s nothing in there that’s not irreplaceable with a little work, and it was only 5TB. If I’d accidentally wiped out a 40TB Plex library I’d definitely be smashing things.
The other feedback I would provide/suggestion when doing a major system change like replacing a drive would have been to set all of the folders on the machine with the USB connected drive to “receive Only”. In this case when you added the folder marker AFTER making that change, you would have seen “Local Additions” on the USB drive (which is actually all of the deletes) and you could have clicked a button in the UI to discard those deletes and resync with unraid. After that resync completed you could set back to send/receive.
I wonder if it would make sense for syncthing to have, at a folder level, a threshold for data changes that would cause the folder to pause.. (I.e. more than 50% of the data has changed, are you sure?)
Perhaps not so easy to implement. And it could create other problems. However this isn’t the first time someone has blown away a whole folder by recreating a folder marker they shouldn’t have recreated.
I’ll accept my own blunder in this matter. It’ll be time-consuming to get copies of all the video again but it’s not impossible. And to then re-do the Plex library after that.
A few weeks ago I started thinking that maybe Syncthing wasn’t what I should be using for my case and that maybe I should have been using a true backup solution like rclone. Probably should have gone down that road shrugs
You learned the hard way that a synchronized copy is NOT a backup. I feel your pain.
Sadly, a lot of people have not yet understood the difference between “backup” and “synchronized copy”. A synchronized copy fails in cases like yours, but also in case of e.g. ransomware.
I would also like to strongly recommend the “set all folders to Receive Only” method. It’s safe and much quicker than having to re-add and configure all of them from scratch. I use it all the time, e.g. after reinstalling the OS, and it has never failed me. Even if you do have the actual data on the disk, it’s still useful to have the folders set to Receive Only initially just in case, so that you are 100% sure that the state is correct and you’re not restoring older files, etc.