Folder marker missing, re-created it -- file mass deletion!

#1

Hi Forum,

the catastrophic case feared earlier has just hit me. Luckily I still seem to have a backup.

What happened? On the local computer, Synctrayzor this morning reported “path exists, but folder marker missing” for my synchronized folder “A” for some unknown reason. In fact, the .stfolder file (or is it a folder?) was missing from the folder.

So I stopped Sync locally, copied the .stfolder from another synchronized folder over to the local “A” folder and restarted Sync.

A minute later I accidentally saw on the remote (source) machine that Sync was about to mass-delete (thousands) of files from the source folder.

Thanks to File Versioning I seem to have a backup of the deleted files (knock on wood). However, the file names all contain a ~tilde and the modification (deletion?) date, so I first have to think about how to restore the original file names, and then restore the files back to the source PC…

Most of all of course I would like to understand what went wrong here, and how it could happen that Sync would start to delete all files on the source machine?

Thanks heaps already for help :o

Where to see recent Sync operations other than in the Windows tray?
Will sharing with more devices speed things up?
#2

What happened here is you’ve overridden the safety measures built in to prevent exactly this kind of situation, and got shot in the foot as the result.

Syncthing creates a folder marker (.stfolder) at the root of a directory it works with. This folder marker isn’t supposed to go anywhere, and if it disappears, Syncthing thinks something unexpected happened and stops. An example may be a folder on a removable device: if the device gets removed, the folder marker can no longer be seen, and that means no work should be done on this folder. When the device gets plugged in again, Syncthing sees the folder marker again and continues operation.

In your case it looks like something happened to your filesystem so that the directory containing your folder started appearing empty to Syncthing. Seeing no files in it, and no folder marker, Syncthing stopped its operation. Then you manually recreated the folder marker, so now Syncthing could see it, but still couldn’t see the files, so Syncthing interpreted the situation in the only possible logical way: there is the directory, it is accessible, but there are no files in it, which means the user (you) has deleted all the files while Syncthing was stopped. So, these files were reported as deleted by you to the other machine, and the other machine acted accordingly.

3 Likes
#3

Thank you Evgeny for the quick answer and for the very comprehensible explanation!

In fact, something has happened to the local file system/sync folder: while still all of its subfolders are there, almost all are empty…

In other words, something has started to delete all files locally, including (here, fortunately) the .stfolder file. And after I re-created the .stfolder, Sync naturally started replicating the deletions on the source…

Now I will try to restore the files deleted today on the source machine, and then start troubleshooting…

Damn, this kind of situation just can’t happen anymore (to me)…!

It would be great to have a feature that warns you of mass-deletions…

#4

The marker going missing is the feature that tells you it’s about to nuke everything. It’s a warning that something is screwed up and you need to investigate.

You just resorted to self invented remedy and shot yourself in the foot by not listening to the warning.

I don’t think there is any feature in the world that would prevent users from doing silly things.

#5

Or suggest an improved message / documentation. (https://github.com/syncthing/syncthing/issues/5207)

1 Like
#6

Well, yeah, but the complaint is really that the warning isn’t clear enough. You need to be aware of certain technical details to understand what it’s telling you; there is ample precedent for error messages spelling out such details when they are expected to be faced by non-technical users. This might be a case where that’s a good idea.

#7

Sure, I agree its not the best wording, and could be improved.

However nowhere does it say “go create the folder yourself, its totally fine”.

I think if you google that error message there is plenty of threads explaining the issue and how to deal with it.

#8

Ummmmmm :o

Where exactly does it tell me something like that?

:exclamation:

#9

Unfortunately, I’m already back with a question about file deletion.

Synctrayzor now has started to continuously show me the following on the local machine:

image
image600×700 48 KB

From this message, I am again unsure where these folders have been deleted, or are about to be deleted – on the local or on the remote computer?

Edit: d*mn it – Syncthing has again mass-deleted thousands of files from the source PC!

This is about to cost me my neck here

#10

Will the “Restore Versions” feature of Syncthing restore the files in their original state (i.e. automatically remove the ~[date] suffix that it made to the files during versioning? image

#11

I think so.

Well, after all the trouble you got in already, I hope that you already did a backup of your data (not as in Syncthing version, separately) before doing any more debugging/restoring (incidentally, you should do that anyway).

That error comes up when Syncthing got an update from another device, telling it to delete that directory. However there is still valid content therein, so it doesn’t delete it.

I suggest you stop Syncthing immediately, make sure that have all the files you want on both (all?) the devices (without Syncthing), then carefully reintroduce Syncthing. Part of being careful when doing that is for example setting the folders as send-only to see what differences there are before they get synced.

#12

Thank you. I really appreciate your quick help.

I’ll follow your advice, of course. Fortunately, I had activated versioning, after all (knock on wood that it caught all deletions).

A few years ago, the Syncthing file recovery feature didn’t seem to exist, right?

#13

Unfortunately, the “Restore Versions” feature of Syncthing doesn’t seem to work here.

It’s been sitting like that for an hour now:

image
image1007×181 9.04 KB

Maybe thousands of files and many GB of data are too much for it?

#14

Perhaps, check the browser developer console for clues.