Db consistency check UI

So my change so far is to move the storage bits required for the check to use backend.Backend with a bunch of namespacedkv’s, as I’ve seen a bunch of people complaining that it takes tons of ram.

For the return value I started off creating a CheckResult with a bunch of ints, that track counts of errors in each category.

This is nice, as the output is concise and I am not going to return 1000s of strings into the UI. Essentially if any of the counts are non-zero, you’re probably stuffed.

But I suspect that might suck for a more detailed debugging session.

The thing that came to my head is to add a new logger specific for this purpose, which would log the errors if they are found, but always return counts to the UI.

Do you think this is good enough?

I am also contemplating adding -reset-deltas (and perhaps even -reset-db) to the UI given how much frustration it caused in one of the threads recently by users not having to care how the application got started and not knowing how to invoke that. I suspect this would be in a form of “set env var, restart, check env var on startup, if set, do the action, unset env var and restart”. We need the db closed for this, so I can’t think of a way to do this other than by chaining restarts. Any concerns?

The stuff that takes RAM is all the FileInfos and block lists, so I guess you are essentially creating a less efficient temporary copy of the database while doing the check? Might as well just access the original then?

I’m extremely skeptical towards adding UI for the -reset-* options. Under what circumstances are they proven useful? We should address those circumstances, not add a bunch of buttons to whack stuff that people will press when things seem out of order…

That’s actually a good point, yet there are some things that still need to be stored, like seen sequences and used blocklists, which I guess we’d still want to store somewhere.

Also, this does not answer my first part of the question regarding where do the error messages go (if they go anywhere).

It seems we routinely ask people to run -reset-deltas when remove devices are out of sync. Whether it’s because of previous bugs or whatnot, but it’s a fairly common suggestion.

Furthermore, if the user does run the db check, and the db is screwed, what do you suggest the user does about that? “Fix the cause” is not really a good answer when the database is already screwed.

I guess in that case they need to package it up and send it in, so we can forensic the shit out of it and solve the root cause. We found and solved a few such issues that way recently. But blowing away the database as a solution is extreme, and is anyway just an rm -rf or Shift-Del away so it’s hardly something we need to provide a button for I think.

The reset-deltas thing feels more like voodoo we try when metadata looks out of date, and I’m not sure it helps more often than it doesn’t. I suspect the root cause for the cases where it did help is the metadata fixes we just did… But it’s also harmless, so if there is a real case when we expect it helps for some reason (and I’d like to know what that is) then I could imagine a per-device “retransmit all indexes” button somehere in settings or whatever…

Not sure about the error messages, maybe a summary to the UI and details in a report file or something?

I would also take the approach of not including such functions in the GUI. Sometimes there are effects that a normal user cannot explain. I prefer for such issues a CLI based toolbox, which surely is more flexible. Problems and results is better to discuss here.

However, it would be good to extend the module parameters -reset-database and -reset-deltas with all additional functions so that the process can run. In the documentation there are no indications that all syncthing processes must be stopped. Finally a fully documentation with a how-to-do etc. would be the minimum.

e.g.:

ps -e | grep syncthing for checking the process status

killall syncthing to close all processes

etc…

The documentation is open source, so any contribution on things you believe are important are welcome.

1 Like

As idea for a GUI based toolset, I show you the WebAdmin of Kopano, I have running on my servers. In the GUI there is a CMD section, in which you can run all the commands you have in the CLI usage, all you can see by pushing the arrow down.

Such toolsets maybe good in parallel to the normal CLI, surely also in the future is needed.