I have Syncthing on my PC and also on my android device. On Android the Syncthing folder is on a SD card which is encrypted in case I lose my phone.
After updating and restarting the smartphone the SD card takes some time to get decrypted again. However, the Syncthing app was faster in starting up than the phone decrypting the SD card. Syncthing seemed to think that having no files is the recent update and also deleted the files on my PC. I had file versioning on my smartphone but I couldn’t recover my files, probably due to an obviously missing storage device. After finish decrypting the SD card it just showed four empty “.stfolder”.
I tried Recuva and disc-drill without success, but could restore it with EaseUS Data Recovery, although costing $70. For me it became too risky that a backup system erases all my files and I will use a cloud system again.
Sorry to hear that, and we have a lot of things that try to prevent that from happening, but seems something went wrong and nothing caught it. Not sure why. Perhaps you are using the fork android app that does something foolish?
However, syncthing is not a backup system. It’s a system that makes two computer filesystems look the same, so by definition its not a backup system, and you’ve discovered first hand.
Sorry to hear that you almost lost your files and I’m glad you were able to recover them.
(I’m not involved with developing Syncthing, but I’m very interested in how the issue you described could have happened.)
Would you mind sharing the make/model of the Android device and also which version of Android?
What tool are you using to encrypt the SDcard?
Because the .stfolder marker folder is only created by Syncthing at the time a new sync folder is added. Anytime .stfolder is missing, syncing is automatically disabled.
It’s also odd that there were no .stversions folders (unless there hadn’t been any new file versions to retain).
The two most common types of encryption are full disk encryption and per-file encryption (there are also encrypted volumes that look like files but act as virtual disks).
With full disk encryption, there wouldn’t have been a usable FAT filesystem for Syncthing to access. As far as Syncthing is concerned, the SDcard would have looked as though it was unformatted. Syncthing wouldn’t have been able to create its .stfolder because Android would have rejected the request unless the virtual disk was ready to use.
Full disk encryption doesn’t actually decrypt an entire disk before it can be used. Instead, individual disk blocks are encrypted/decrypted on-demand via an interface that presents a virtual disk for applications to use.
Recuva works only on files that have been marked as deleted. But if any disk blocks occupied by a deleted file have been reused, full recovery is impossible. Recuva expects to see a FAT, exFAT or NTFS filesystem, which full disk encryption would not provide until the virtual disk is ready to use. (I’ve used Recuva on and off since it was released in 2007.)
disc-drill, or is it “Disk Drill”?, and EaseUS Data Recovery do what Recuva can do plus also a technique called “file carving”. (I’ve used EaseUS Data Recovery but not Disk Drill.)
But for all three data recovery utilities, recovering files from a full disk encrypted SDcard – especially for Recuva – isn’t normally possible even with file carving because there aren’t any recognizable files. And if there are, then there’s something wrong with the full disk encryption. (This is one area where per-file encryption is more advantageous compared to full disk encryption.)
Without more details, it’s difficult to offer a proper diagnosis, but I think the most likely explanation is data corruption on the SDcard.
To prevent this, couldn’t syncthing check if all files in a folder share including the folder are set to delete… And if so, it could prompt for human confirmation in the gui. Much in the same way it does right now for local additions.
This could be added as an optional setting for more critical data shares, and save people a lot of heartache and expense.
@timj, let’s not hijack this thread by converting into a feature request thread. (At least not until the thread is resolved.)
If you are quite sure that you have also experienced exactly this same problem, then feel free to share details, so that the problem can be better understood.
What I don’t understand is if the .stfolder was present but none of the other files were there, then everything would have been deleted on the remote devices… but when the files showed up after whatever delay caused them to be missing in the first place, shouldn’t they have copied over again?
I don’t know whether this is what happened; but a possibility is this:
User sets up syncthing and tells it to sync a given folder, say /mnt/foo
.stfolder marker is automatically made in (empty) folder foo
User decides that they need more space for foo, so they mount an encrypted sdcard on foo. Now /mnt/foo is empty again, and syncthing stops when it notices the missing .stfolder
User either adds the folder again, or they google the problem and are told that syncthing stops unless the folder .stfolder is present; so they manually create the .stfolder and restart syncthing. All is now well, and syncthing works as it should.
At some later point in time, for whatever reason, syncthing starts at a point in time where the encrypted sdcard is NOT mounted on foo.
But because the mountpoint already has a .stfolder marker from point #2, syncthing doesn’t sound the alarm, but instead simply proceeds to happily delete all files from the other synced devices since the .stfolder IS present but the files are not.
This kinda thing is hard to guard against. Only solution I can think of is something like making the .stfolder not be one standard for all installs, but instead somehow be created so as to be specific for the share, for example with the shareid as part of the folder-name, so that syncthing can detect not merely whether there’s a “.stfolder”, but indeed whether there’s the right .stfolder
Might not be worth the complexity and the hassle though; this kinda scenario should be rare.
The encrypted SD card is reinserted, and syncthing copies all the contents back onto the other devices…
But anyway if that never happens, then yes, everything is gone.
Regarding your proposal I can see a reasonable option where when any new folder is created, ST creates an folder called .stfolder_NNNNNNNN where N is some series of random numbers. This is noted in the folder config. And every folder is different. Whether that added complexity practically saves anything I think is not clear. In your specific use case, they have to recreate a folder in step 4 with the same number for ST to resume working in step 5. So presumably they go into the folder’s config, find the folder name with the random number and create the same random number.
Maybe a better safeguard would be putting a file inside .stfolder, like .stfolder/created with an ISO timestamp of when Syncthing automatically created the folder marker. The same value is recorded in the folder configuration. If the marker is found, but the timestamps don’t match, it stops until someone updates either the file or the config value. Could have a button to update both in one step.
With custom folder markers, the filesystem timestamp of the marker itself could be used. That is more fragile though, and not all systems (e.g. Android) have proper support for the modification time.
Overall a pretty niche problem though, so not sure it is worth the added complexity.
Right. This is particularly catastrophic when the reason a given device hasn’t been mounted is that this device is somehow BROKEN. And then the owner goes: “that’s okay, I mean this folder is synced to 2 other machines anyway, so I’m safe”.
And then they discover that actually, the device not getting mounted has, in this specific scenario, resulted in all those files being wiped in the two other locations too. Oooops.
I agree it’s pretty niche though; the current .stfolder approach prevents this EXCEPT in scenarios where for some reason that folder exists not only ON the mounted device, but also in the mountpoint-folder.
So it doesn’t affect Windows users because although NTFS supports mounting a volume to a folder, the target folder must be empty. It’s very unlikely that many Mac users use the “mount” command instead of the automounter. Same for Android.
On Un*x, it’s pretty unusual to store files under /mnt. And because it requires root privileges, a user would have had to done it deliberately.
Linux…
Most Linux users rely on the automounter (e.g., autofs, GVfs) which creates and deletes mount points under /media whenever removable devices are inserted and ejected.
Mount points are named based on the volume label if available, otherwise it might be the manufacturer name. Because there’s a chance of collisions due to more than one drive with the same volume label or manufacturer, a number is often automatically appended to a directory name as needed.
If a user manually creates a directory such as /media/myusbdrive/.stfolder, any removable drive with the volume label myusbdrive would be automatically assigned a different mount point (e.g., /media/myusbdrive1).
So a user could override the automounter, then use the mount command to attach a volume to a non-empty mount point under /mnt (possibly also adding entries to /etc/fstab).
However, what makes the scenario above even less likely is that most users format their removable drive volumes as NTFS or exFAT, or perhaps might choose drive encryption such as Cryptomator, EncFS, gocryptfs, securefs, etc. – all of those filesystem types on Linux are FUSE-based, and by default, FUSE will not mount a volume on top of a non-empty directory.
A user would have to go out of their way to circumvent multiple layers of safeguards including Syncthing’s sanity checks. So it just seem very unlikely.
(As of late-2021, Linux kernel 5.15 includes a native module for NTFS, but it’s not used by default on any distros I’ve checked.)