Global state different from local state

[cosas]

Congratulations man! T’as pas lâché le morceau, this will help all us.

[brunod (Bruno)]

@cosas Thanks ! Wrong values are quite annoying (if not scaring) because it’s the only feedback the user has got to know if synchro has been done or not in the cluster. Could it be possible to correct/reset it, and/or to see where the difference comes from (device/folder/file) ? Merci ! Et bien le bonjour de Liège

[AudriusButkevicius]

@imsodin we should probably raise an issue for this?

[imsodin (Simon)]

@AudriusButkevicius

@imsodin we should probably raise an issue for this?

Sure. I am just confused about the circumstances of the problem. I mean I can see the inconsistent global state in the screenshots, but I do not follow what leads to it or what @brunod has found:

@brunod

I think I’ve found. Let’s have 2 directories, spread over the cluster; configure .stignore to limit access to one of them; rescan. The ignored folder will be removed from local, but it will stay on global and can’t be removed. If you remove it from the master, as it is not scanned according .stignore, it’s removal isn’t propagated.

When you ignore something that isn’t removed from global state until every device has ignored it, I don’t think send-only has anything to do with this. If you ignore it on one device and delete it on another, it should be removed from global on both devices.

[brunod (Bruno)]

IMHO yes, but if I’m the only one to think it would be usefull, maybe not. I’m maybe wrong and there are maybe other things that would solve this in a more global way, this problem being a side effect… You judge Thanks, BD

[brunod (Bruno)]

@imsodin

When you ignore something that isn’t removed from global state until every device has ignored it, I don’t think send-only has anything to do with this.

I think it’s related to the modified .stignore pattern, not the fact that it was read only. I presume (but din’t try) that if it was read write for all devices in the cluster, the same modification of .stignore would have led to the same display. I tryed to set the master read-write, hoping that if it was impossible to propagate it’s previous master state, it would received from others a correction to resync everything, but it didn’t happend. I had to cancel the .stignore to recover stability.

[AudriusButkevicius]

Can you get a small concise reproducible case that anyone can repro?

[brunod (Bruno)]

@AudriusButkevicius : more step by step than this ?

@brunod

Let’s have 2 directories, spread over the cluster; configure .stignore to limit access to one of them; rescan. The ignored folder will be removed from local, but it will stay on global and can’t be removed. If you remove it from the master, as it is not scanned according .stignore, it’s removal isn’t propagated.

then I have to test all the steps from scratch. I’ll post it when it’s ready

[imsodin (Simon)]

Yes please. From that description I don’t know what to do, e.g. what is master (you said send-only is not the problem, everything can be send-receive)? Ideally it is really a list of tasks, something like this:

1. Create folder with file dir/toBeIgnored, bring devices A and B in sync.
2. Add ignore patterns dir on device A.
3. Delete dir/toBeIgnored on device A.
4. …
   …

Result: Global on A shows x,y,z, on B x’,y’,z’.
Expected: Global on both A and B x,y,z

[brunod (Bruno)]

Here it is

Syncthing :

add TestDir,

save (No need to share or any special option)

Open file manager into the directory TestDir

create :

• NewDir
• file.txt
• file.pdf

copy file.* into NewDir

Syncthing :

Open TestDir,

check display to see global = local state : 4 files 1 folder

Edit share

Click button Ignore Patterns

enter :

(?i)!*.pdf
/*

Save,

save.

Check global state : 4/1; local 2/0 Vs reality 4/1

Open file manager into the directory TestDir/NewDir

delete file.txt

Syncthing :

rescan

Check global state : 4/1; local 2/0 Vs reality 3/1

Quite confusing as there is no share with other device already enabled; global should equal local and/or reality. At least, global shouldn’t be greater than reality. Thanks ! BD

[imsodin (Simon)]

Thanks @brunod , now I can reproduce and found the problem. Would you mind opening an issue on github? Just copy&past your reproducer.

[brunod (Bruno)]

Done,

Hope it’s fine

BD

[imsodin (Simon)]

Thanks again! PR is here: lib/db: Update global counts on invalidation (fixes #4701) by imsodin · Pull Request #4702 · syncthing/syncthing · GitHub

@brunod

Check global state : 4/1; local 2/0 Vs reality 3/1

The reality part I do not understand: You ignored everything except *.pdf, so 2/0 is correct, right? The result of my PR will be that global 2/0 and local 2/0.

There is also still the skipped directory on scan issue, due to which you might also argue that it should be 1/0, but that is not connected to the other issue and I intend to bring that up once https://github.com/syncthing/syncthing/pull/4584 is dealt with.

[brunod (Bruno)]

In reality, I mean when you go with the file manager and look what really is : you 've deleted NewDir/file.txt at previous step, so there are really 3 files left and 1 folder (and I mention it only as it is different of both global and local state display).

[imsodin (Simon)]

Well, then I was misinterpreting that - I assumed you mean that is what Syncthing should display. Syncthing only counts files which aren’t ignored, so if you do have ignored stuff it obviously differs from what you see in a file manager.

[brunod (Bruno)]

I 'm not the one who has to decide how to display, I just want to understand it if it’s usable But inconsistent data showing value greater than reality with/without exclude is wrong. That’s why I mentionned reality Thanks !