Long time Syncthing user - some files missing on new system - SOLVED

TL/DR:

  • Use http://localhost:8384/rest/db/file to see what your database says about various files
  • syncthing --reset-database may fix files missing from new devices, especially if running syncthing for a long time
  • Always back-up first. Read the manual. Be careful.

I’ve been using Syncthing for several years across many versions, mostly from my primary “Server” machine, but also on my laptop, a VPS, phone, and a few tablets.

I recently setup a new workstation and I noticed some files were not syncing to the workstation… Very puzzling.

I used the REST API on both sides of the connection… On the “server” side, things looked fine…

`curl "http://127.0.0.1:8384/rest/db/file?folder=${MY_FOLDER}&file=${MY_FILE_NAME}" -H "X-API-Key: ${API_KEY}"`
{
  "availability": null,
  "global": {
    "deleted": false,
    "ignored": false,
    "invalid": false,
    "localFlags": 0,
    "modified": "2022-04-29T09:14:48.266327916+07:00",
    "modifiedBy": "574RMIJ",
    "mustRescan": false,
    "name": "${MY_FILE_NAME}",
    "noPermissions": false,
    "numBlocks": 1,
    "permissions": "0664",
    "sequence": 37526,
    "size": 160,
    "type": "FILE_INFO_TYPE_FILE",
    "version": [
      "574RMIJ:1655174150"
    ]
  },
  "local": {
    "deleted": false,
    "ignored": false,
    "invalid": false,
    "localFlags": 0,
    "modified": "2022-04-29T09:14:48.266327916+07:00",
    "modifiedBy": "574RMIJ",
    "mustRescan": false,
    "name": "${MY_FILE_NAME}",
    "noPermissions": false,
    "numBlocks": 1,
    "permissions": "0664",
    "sequence": 37526,
    "size": 160,
    "type": "FILE_INFO_TYPE_FILE",
    "version": [
      "574RMIJ:1655174150"
    ]
  },
  "mtime": {
    "err": null,
    "value": {
      "real": "0001-01-01T00:00:00Z",
      "virtual": "0001-01-01T00:00:00Z"
    }
  }
}

Yet on the “client” side of the connection, my new workstation, my file was not showing up…

`curl "http://127.0.0.1:8384/rest/db/file?folder=${MY_FOLDER}&file=${MY_FILE_NAME}" -H "X-API-Key: ${API_KEY}"`
{
  "availability": [
    {
      "id": "XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX",
      "fromTemporary": false
    }
  ],
  "global": {
    "deleted": false,
    "ignored": false,
    "invalid": false,
    "localFlags": 0,
    "modified": "2022-04-29T09:14:48.266327916+07:00",
    "modifiedBy": "574RMIJ",
    "mustRescan": false,
    "name": "${MY_FILE_NAME}",
    "noPermissions": false,
    "numBlocks": 1,
    "permissions": "0664",
    "sequence": 37526,
    "size": 160,
    "type": "FILE_INFO_TYPE_FILE",
    "version": [
      "574RMIJ:1655174150"
    ]
  },
  "local": {
    "deleted": false,
    "ignored": false,
    "invalid": false,
    "localFlags": 0,
    "modified": "1970-01-01T08:00:00+08:00",
    "modifiedBy": "",
    "mustRescan": false,
    "name": "",
    "noPermissions": false,
    "numBlocks": 0,
    "permissions": "0",
    "sequence": 0,
    "size": 0,
    "type": "FILE_INFO_TYPE_FILE",
    "version": []
  },
  "mtime": {
    "err": null,
    "value": {
      "real": "0001-01-01T00:00:00Z",
      "virtual": "0001-01-01T00:00:00Z"
    }
  }
}

If you check the output on the client machine, I think that availability with some ID may mean that it’s actually waiting for some other node to come online and provide that file…

There’s a long post about troubleshooting a few items not syncing

Since I’ve added and removed so many devices from syncthing over the years, I thought it’s probably something a bit out of sync on the server (my main host - of course it’s all really peer-to-peer) of my syncthing setup, the option to reset the database actually seems like a very good thing in my case. Especially considering so many version upgrades over the years.

So on my server side, I just stopped Syncthing, ran:

syncthing --reset-database

INFO: Successfully reset database - it will be rebuilt after next start.

Then started up syncthing again on my server machine, waited a while, and sure enough all of my missing files showed up.

In my case, I know the files I do want are the ones on the server, so I’ve also paused syncing to my laptop too and will do a reset there… Otherwise I think there’s a risk of files on the laptop wiping out files on the server once it comes back online.

Anyway, hope this isn’t bad advice, and I hope that it helps some other user that’s been running for a long time and suddenly has some files that aren’t showing up on a new device.

There’s also a --reset-deltas option that may be a better first step, but in my case, that database is so old a clean refresh is exactly what I wanted to try, and so far it worked perfectly.