Ingored files stay "out of sync"

The bug mentioned in #2991 is still there in v1.2 and the sync never completes. I have several folders on several computers (Ubuntu, Win10) in “syncing xx%” state for days, weeks, months without any data being transmitted since the files ouf of sync are on the ignore list on one end.

I was requested to open a new thread instead of necroing 4 years old thread (which I did already)

2 Likes

You were also requested to provide screenshots so we can see the issue :wink:

Screenshots, all from the same device & folder:

  1. Syncing never completes

  1. Out of sync items

  1. Ignore list

And from the device that is reported as syncing in your first screenshot? The folder pane on the left is also relevant.

Here is the receiving end (Win workstation):

Folder status:

Syncthing%20-%20Receiving%20end%20folders

counterpart status:

Receiving end’s ignored files:

Few words about the setup:

  1. Linux server #1
  2. Win workstation
  3. Win laptop #1
  4. Win laptop #2
  5. Linux server #2

Out of these:

  • 1-3 are in the same LAN
  • 4 is often in connection to 1 and 5 over SSH tunnel
  • 2-4 are often connection to 5 over SSH tunnel
  • 1 and 5 are never in connection to each other (until I have time to setup a VPN tunnel between the networks)

I have to post the screenshots again since the forum does not allow edits of old posts. These are all from the Win Laptop #1:

  1. Syncing never completes

  1. Folder list:

  1. Out of sync items

  1. Ignore list

Just so I get the nomenclature right:

Win Laptop 1 (L1) shows the workstation (W) as syncing. Both L1 and W show the folder as up-to-date. And W shows L1 as up-to-date. Any mistakes?

It seems like the ignore lists are equal from the screenshots, are they? (Asking because of the vastly differing local state numbers)

Can you query the /rest/db/file endpoint for one of the out-of-sync files on both on L1 and W? https://docs.syncthing.net/dev/rest.html

The sync has been in this “syncing”-state for weeks without any progress. Several restarts/reboots have occurred, without help. The ignore lists on LW/W1 are almost equal, but not the same. The both ends (L 1, W 1) have the files in “out-of-sync” state in their ignore list.

I do the DB reset next (on L 1). Btw. Until the sync handling is super-robust, it would be nice to have the DB reset as a feature e.g. under advanced settings.

Before you do a db reset, do an index reset (-reset-deltas). That’s much less resource intensive and might already clear things up.

As the folder on the other side says up to date, there is no actual syncing and thus no progress to be exptected - this is an incorrect state. And the information from /rest/db/file might give some clues as to what is wrong. However you can also just do the index reset and hopefully that will clear things up.

Regardless, it would be interesting to see the output as it smells like a bug.

What “output” should I send?

An intermediate report that doing -reset-deltas to all computers did not help.

It seems you have two folders, are both folders shared with both devices?

The output of the /rest/db/file API endpoint for one of the ignored files, from both sides, would be interesting.

I am strugging with the REST API:

curl -X POST -H “X-API-Key: KEYISHERE” ‘http://127.0.0.1:8385/rest/db/file?folder=FOLDER_ID&file=2%20WORK

Returns

404 page not found

Please note the syncthing server is running on port 8385 since I have two instances running under different users. API-KEY and folder (ID) have been masked.

I think its a get not a post

1 Like

Well. L1 says a file X is “out of sync” under the "Remote devices panel on Server 2. When I run the GET rest/db/file on that file (ULRencoded filename):

MOBILE SYNC\2019\07\20190712_075126444_iOS.jpg ==> MOBILE%20SYNC%5C2019%5C07%5C20190712_075126444_iOS.jpg

I get No such object in the index on Server 2 and L2 replies temporary redirect.

Use forward slashes for path separators. Regarding the redirect: Do you use https? Then you need the -k flag.

I managed the get the GET rest/db/file request to work on the Server 2 (Linux), but not on Laptop 2 (Ubuntu shell in Win10). The Laptop 2 keeps saying temporary redirect. I run the same command over remote SSH tunnel from Server #2 (Linux) and I got the same “temporary redirect” reply so therefore it must be a “legit” response, not a shell quotation issue.

Server #2 says:

{
  "availability": null,
  "global": {
    "deleted": false,
    "ignored": false,
    "invalid": true,
    "localFlags": 0,
    "modified": "2019-07-12T10:51:26+03:00",
    "modifiedBy": "WIN WORKSTATION",
    "mustRescan": false,
    "name": "MOBILE SYNC/2019/07/20190712_075126444_iOS.jpg",
    "noPermissions": true,
    "numBlocks": 0,
    "permissions": "0640",
    "sequence": 300193,
    "size": 2903726,
    "type": 0,
    "version": [
      "SERVER #1:1"
    ]
  },
  "local": {
    "deleted": false,
    "ignored": false,
    "invalid": false,
    "localFlags": 0,
    "modified": "1970-01-01T02:00:00+02:00",
    "modifiedBy": "",
    "mustRescan": false,
    "name": "",
    "noPermissions": false,
    "numBlocks": 0,
    "permissions": "0",
    "sequence": 0,
    "size": 0,
    "type": 0,
    "version": []
  }
}

And sorry for not knowing how to quote a block nicely

And sorry for sneakily correcting the formatting. Tripple backticks ``` before and after do the trick (or the preformatted text thingy in the “editor”).

Did you check the https thing (-k flag)? Otherwise on windows you can do it in power shell like this: Again, The remote device keeps displaying the synchronization(95%) It would be very helpfully, I have an idea about what it might be about (globally invalid files not being needed), but not yet what exactly is wrong - getting the info from the other side will hopefully help.

1 Like

Sorry for being slow here. The Windows machines are using SyncTrazor and I was unaware those Syncthing GUI servers actually are behind HTTPS. Finally got it working.

Here is GET rest/db/file for the very same file from Laptop #1 (the other end):

{
  "availability": [
    {
      "id": "SERVER #1",
      "fromTemporary": false
    }
  ],
  "global": {
    "deleted": false,
    "ignored": false,
    "invalid": false,
    "localFlags": 0,
    "modified": "2019-07-12T10:51:26+03:00",
    "modifiedBy": "SERVER #1",
    "mustRescan": false,
    "name": "MOBILE SYNC\\2019\\07\\20190712_075126444_iOS.jpg",
    "noPermissions": true,
    "numBlocks": 23,
    "permissions": "0640",
    "sequence": 513241,
    "size": 2903726,
    "type": 0,
    "version": [
      "SERVER #1:1"
    ]
  },
  "local": {
    "deleted": false,
    "ignored": true,
    "invalid": true,
    "localFlags": 2,
    "modified": "2019-07-12T10:51:26+03:00",
    "modifiedBy": "LAPTOP #1",
    "mustRescan": false,
    "name": "MOBILE SYNC\\2019\\07\\20190712_075126444_iOS.jpg",
    "noPermissions": true,
    "numBlocks": 0,
    "permissions": "0640",
    "sequence": 274445,
    "size": 2903726,
    "type": 0,
    "version": [
      "SERVER #1:1"
    ]
  }
}

So if I read these correctly both the Server #2 and Laptop #1 think there is even newer version of the file modified by WIN WORKSTATION that has the file under Ignored Files. I kind of think that remote computers should consider file modification by computers having the file on Ignored Files list, but this must be more complicated than that.

I have now set up a SSH tunnel between Servers 1/2 so those can finally sync each other. That may sort out my problem although I think Syncthing does not work here as it should.