Local folder state larger than global (randomly, with ignore patterns involved)

Recently, when testing various ignore patterns, at least a few times I’ve ended up with the following state, in multiple folders for that matter.

Local state larger than global shouldn’t be possible, right? I suspect this is caused by mixing negated and normal ignore patterns, e.g.

!/folder/*.txt
/folder

however I haven’t been able to reproduce the problem in my test environment despite numerous attempts.

The main problem with this situation is that the only way to get things back to normal seems to be through performing a full database reset :confused:.

Has anyone got any ideas what exactly can be going on here? I can provide debug information, etc. but I doubt it will be of any use without a reliable way to reproduce the problem at will…

Probably you’ve tickled a bug, but on that note, keep in mind those numbers are a counter updated in a fairly complicated manner and doesn’t reflect a “scan” of the database as such. The counters get dropped and recalculated from the actual database contents every 90 days or so, and there’s an env var you can set to force an earlier recalculation. Maybe set that for one startup and see if the numbers change.

1 Like

Do you mean STRECHECKDBEVERY? Unfortunately, it doesn’t make any difference :sweat:. I’ve tried doing --reset-deltas too but also to no effect. That’s why I believe that a database reset is the only way to fix things here.

If you have such a db, please make a copy! Though it might be a different envvar - on phone can’t check right now.

That’s the one. I’m also curious about the underlying reason, so do preserve it please.

That was quick! And I was just about to perform a reset… :upside_down_face:

There’s one issue though. Two folders are affected at the moment. One of them contains a lot of personal information but the other one does not. Would it be a problem if I removed all other folders first, and only shared the DB with the the second folder left in there?

Sounds fine

Here you go.

index-v0.14.0.db-wronglocalstate.zip (8.8 MB)

The final state looked like this.

There are two files that the database isn’t consistent about whether they’re deleted or not. Since the mismatch is two files this seems related, but I didn’t dig into it at all. The names of the files made me want to nope out of this completely:

VersionList ".stfolder/.stversions/.stfolder/~20221226-174916.stignoreglobal_versions", folder "abwgi-qj5iw", entry 0, FileInfo deleted mismatch, true (VersionList) != false (FileInfo)
VersionList ".stfolder/.stversions/.stfolder/~20221226-201229.stignoreglobal_versions", folder "abwgi-qj5iw", entry 0, FileInfo deleted mismatch, true (VersionList) != false (FileInfo)

I would say “explain yourself, you pervert/madman”, but I’m not sure I want to know…

I’d prefer the latter if I had to choose :sweat_smile:.

The explanation is rather simple though. These are remnants of https://forum.syncthing.net/t/is-there-any-danger-in-allowing-to-synchronise-the-folder-marker/19067 which I were going to take a step back on but haven’t managed to do yet (as this requires access to all devices at once which I actually have at the very moment so I may just do it right now). The .stversions is just file versioning set on another device using a custom folder path so that it can sync across all devices. No other devices have versioning enabled.

Ideally, the folder structure should look something like

.syncthing-customisations/.stignoreglobal_versions

with the custom versions path set to .syncthing-customisations/.stversions, and then the problematic path would become

.syncthing-customisations/.stversions/.syncthing-customisations/~20221226-174916.stignoreglobal_versions

What’s interesting though is that I weren’t suspecting these two files to be the culprit, especially since they do exist in the folder physically. I’ll now try revert the .stfolder sync, reset everything back to normal and see whether the problem comes back.

Thank you for looking into the database anyway :pensive:.

Edit:

Just for the record though, having reset the database, Syncthing did manage to properly recognise the two files as local additions.

I’ve got yet another example, this time without any .stfolder shenanigans.

Below is a Receive Only folder marked as “Local Additions”.

image

However, pressing the “Revert Local Additions” button actually adds the local file directory to the global state (!!!).

image

This is a part of a very large setup, so sharing the database would be complicated, but I personally think that this whole problem may be related to https://github.com/syncthing/syncthing/issues/8735 also.

Details to be figured out I guess, but I’m not surprised to see mismatches in directory count when there are un-ignore patterns matching files deep in the hierarchy.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.