Syncing progress bar seemingly broken (Syncing 100% while not)

Check what that endpoint returns via chrome developer tools when you believe its broken and post the output.

It is now happening again, between core i5 laptop and a desktop. (and I have also gotten inconsistency on Mikhail-PC while moving the folder on Probook and not touching anything on Mikhail-PC)

I have checked the calls, but I didn’t see any /rest/db/status calls. When I tried to send GET /rest/db/status, it returned 404.

The 404 is because you need to specify the folder id (https://docs.syncthing.net/rest/db-status-get.html). Anyway that endpoint is only called when opening the web UI or changing config, otherwise status updates are event based. And in firefox’ network inspection tab I couldn’t find any indication of those. You’d need to put breakpoints in syncthingController.js (line 321) to detect those events. Or you could enable event debug logging in advanced->log to see if any folder summary events are dispatched at all.

It doesn’t show the syncing status of other devices consistently on mikhail-pc now. This is the /rest/db/status response for Desktop folder on mikhail-pc, although I couldn’t catch 100% 0B isuue:

{"errors":0,"globalBytes":19519051280,"globalDeleted":154231,"globalDirectories":2422,"globalFiles":50057,"globalSymlinks":0,"globalTotalItems":206710,"ignorePatterns":false,"inSyncBytes":19519051280,"inSyncFiles":50057,"invalid":"","localBytes":19519051280,"localDeleted":50161,"localDirectories":2422,"localFiles":50057,"localSymlinks":0,"localTotalItems":102640,"needBytes":0,"needDeletes":0,"needDirectories":0,"needFiles":0,"needSymlinks":0,"needTotalItems":0,"pullErrors":0,"sequence":154923,"state":"idle","stateChanged":"2020-06-21T15:12:37.1507496+02:00","version":154923}

And on probook: { "errors": 0, "globalBytes": 19519051280, "globalDeleted": 50227, "globalDirectories": 2422, "globalFiles": 50057, "globalSymlinks": 0, "globalTotalItems": 102706, "ignorePatterns": false, "inSyncBytes": 2086432222, "inSyncFiles": 5759, "invalid": "", "localBytes": 2084713068, "localDeleted": 911, "localDirectories": 2422, "localFiles": 5780, "localSymlinks": 0, "localTotalItems": 9113, "needBytes": 17432619058, "needDeletes": 0, "needDirectories": 0, "needFiles": 44298, "needSymlinks": 0, "needTotalItems": 44298, "pullErrors": 0, "sequence": 10046, "state": "syncing", "stateChanged": "2020-06-21T16:22:10.5044808+03:00", "version": 10046 }

The output is not useful if its not captured when you’re observing the issue, which if I understand was the case when you captured these.

Ok, but now I have other issue, where it shows that the device is Up to Date, while it’s actually not and is actively transferring data.

And the other side claims its syncing?

I agree, something looks off.

It seems to be syncing just fine, it shows Mikhail-PC to be up to date (which it is), and the folders are syncing and showing their progress as indented.

When the happens again, could you also query https://docs.syncthing.net/rest/db-completion-get.html for the device and folder showing the incorrect status (i.e. in the last example query it on mikhail-pc for Probook/Desktop). What I’d like to know is if the UI doesn’t display the correct respectively up-to-date values or if the underlying info in db is wrong.

This is the response I’ve received on mikhail-pc (http://localhost:8384/rest/db/completion?device=(probook’s id)&folder=kolwd-gp3fv):

{
  "completion": 100,
  "globalBytes": 19519051280,
  "needBytes": 0,
  "needDeletes": 0,
  "needItems": 0
} 

And on probook:

{
  "completion": 100,
  "globalBytes": 19519051280,
  "needBytes": 0,
  "needDeletes": 0,
  "needItems": 0
}

Edit: I have the same thing on RPi_RU:

{
  "completion": 100,
  "globalBytes": 19519051280,
  "needBytes": 0,
  "needDeletes": 0,
  "needItems": 0
}

So all devices (even probook on probook?) thinks that probook is 100 in sync but the folder is still syncing?

I’m not sure about the Probook itself, but it looks like all of the other devices think that it is 100 in sync and I can’t check now, because it went to sleep, restarted itself afterwards and now the issue is gone again :frowning:

Edit: I think I’ve triggered the problem. After deleting and re-adding the folder, other devices show, that it is Up to Date, but are syncing with it

Also, what does Preparing to Sync do exactly? I have now got some very strange and suspicious syncing after re-adding the folder, seemingly being originated from Probook, although it shows that file origin is 1.5.0 client (RPi4B).

There is a step where it builds the list of stuff that needs to be synced, does some house keeping before actually starting to sync, that’s what the state means.

Based on what you’re explaining, something definately feels off, but I am bit lost on how to proceed with this. Ideally if you can compile a set of steps that lead to this so that we could reproduce it - that would be ideal, because I think debugging via a forum post might prove to be ahrd.

Also, just to make sure, you don’t have anty syncthing related environment variables set, etc?

This is different folder now, since Desktop is now in-sync, but here is what Mikhail-PC thinks about Probook:

{
  "completion": 100,
  "globalBytes": 68040132917,
  "needBytes": 0,
  "needDeletes": 0,
  "needItems": 0
}

And here is what ProBook thinks about itself (and that folder is synced by 25%, so it shouldn’t be 0 I’d suppose):

{
  "completion": 0,
  "globalBytes": 68040132917,
  "needBytes": 68040132917,
  "needDeletes": 0,
  "needItems": 48549
}

I don’t have any variables, apart from the ones that may be by default in SyncTrayzor.

This seems to happen whenever I add folder, other devices go crazy and sync some random stuff even if there are no changes (although it might be normal, I don’t know) and just show that device is Up to Date, even when it’s not.

I think the 0 completion is a lie. I am not sure if you can ask A what it thinks it’s own completion is, hence it just returns garbage.

I guess if you can take a fresh setup, work out a step by step guide that causes this, it would be interesting to try and reproduce.

I can replicate it by running a fresh Syncthing instance, adding another node to it, accepting the folders, and then, if the folders are not in sync, the other node would not show the progress, even though it will be seemingly syncing just fine and transfer large amounts of data. And again, 1.5.0 reports the progress just fine.

So what are the “step by step” steps?

  1. Have 3 devices, A, B, C
  2. Connect A with B
  3. Start syncimg some data from A to B
  4. While it’s syncing, add C?

Does C have the data or not? Where is the fault observed?

My steps:

  1. Have device A (v1.7.0-rc.1 Windows) with the data and device B (v1.7.0-rc.1.dev.31.gdc145bfa linux) with fresh syncthing install.

  2. Connect A with B

  3. Add/merge existing folder from A to B and let it start syncing.

    –Problem occurs here–

  4. Add other devices with data, in sync with A, while it’s syncing

If it’s 1.6.1 or 1.7.0, then other devices would also not show the progress, but will still sync just fine (as device A does)

If it’s 1.5.0, then it will show the progress and work as expected

I have also seen it happen randomly on Raspberry Pi (1.6.1), although that might be because of the slow CPU and Web GUI not loading.

So I am not seeing the syncthing 100% 0b issue, but the device downloading the data weirdly does show up as “up to datE” on the device providing the data straight away, even while it’s clearly syncing.

I found some other dirt along the way too.

It’s getting late today, but I’ll try to look into it tomorrow.