Twice recently I’ve created a new folder, added some files, and then when I check back the folder is empty. I need some help figuring out what happened.
The setup:
~25 machines. a mix of windows, mac, linux, and 1-2 android devices
running for something like 4 years, adding machines occasionally
~20GB spread over 30 or so different shares
mostly Syncthing is up to date, but certainly a few machines are running older versions. the issues involved machines I know were up to date
some files never change, other are added or change daily
there is one “central” server that’s always on. it’s a VPS
our internet quality is, to put it mildly, terrible. most of the swarm is behind the Chinese GFW and there’s not much we can do to improve this
no file versioning from syncthing (there is a separate backup system that runs on the central VPS)
1st issue:
created a new folder in an existing share, added a few files to it, shared that with a relatively new machine, and (20 minutes?) later the files were gone. Luckily I had copies elsewhere so I added them again and everything seems OK.
2nd issue:
created a new share, put a couple different folders inside it, shared it with our central server as well as an existing machine in the swarm. now checking back ~10 hours later the share is completely empty except for a lone .stfolder (nothing in it). lost about an hour and a half of work
FWIW:
I don’t think there were any android devices involved in these particular folders.
The central machine has backups made every 4 hours. The new share folder from the second issue is empty in all available snapshots.
I would appreciate some guidance on how to get to the bottom of this. I’ve looked through logs previously for other issues and cumulatively probably spent most of a day on it without ever really learning anything other than an appreciation for the verbosity of the logging system. At this point I don’t think I will be able to solve it myself. I do have access to config files and (probably) logs on all the machines involved. I don’t remember exactly when the first issue happened, but could track down exactly when the second issue happened in the logs, if they are available that far back.
Sorry, I would be doing that, I simply have no idea how to narrow the logs down to what I need to see.
There isn’t an option for “recent changes” in the Debugging Facilities tab on the Logs dialog in the UI. Can you go into a little more detail about how I could, for example, see file events for a particular machine? The ‘fs’ checkbox seems to be the only thing close, and that’s not saying anything related to file creations or deletions from what I can see.
The information I need has scrolled off the end of the list in the UI. Looks like I’ll need to restart with -audit to see file changes.
I do have some raw logs… is it perhaps possible to re-construct this info from them? Ideally I’d like to avoid needing to lose data again before being able to debug this.
A few hours ago I re-created the files and added them to the folder again, so your suggested scenario isn’t available.
I can query the API. Is it possible to find any deletions in a particular folder? There should have been 0 deletions since it was created so learning that would be informative. I’m currently only seeing ways to list what’s there right now.
No, you can only query by file name, recent deletions etc would he in the recent changes dialog, for some definition of recent (and assuming you have not restarted the process).
I’m having a similar issue again. Can you point me to the logic that uses the API to generate the “Recent Changes” screen in the UI? I want to log that same data over a longer time frame. Searching for “recent changes” only finds the language translation files.