Again, The remote device keeps displaying the synchronization(95%)

[cosas]

Hi all I get this (95% 0B (7 items)) displayed on a linux 0.14.44-rc2 (device A). There are 3 nodes involved in the folder share where the 7 listed files are. Other nodes are one linux 0.14.43 (device B) and one Win32 0.14.44-rc2 (device C).

All show same local=global=243 non-ignored regular files. A & B & C show Up to Date for the folder share when B & C show Up to Date with the other 2 devices. Device A shows the title message to the Device C only (Device B Up to Date).

List shows 3 filenames with Mod. Device A in folder root and 4 filenames spread in Mod. Device B root and a sub-folder.

IIUC this behaviour was fixed in 0.14.44-rc1 (bug #4668).

The question is : do all the nodes in the cluster need to upgrade to 44-rc1+ for the erroneous display to disappear?

Thank you whatever for this great soft.

[imsodin (Simon)]

The issue #4668 only fixed the inconsistency of displaying “out of sync items” together with “0 files”, so if you can actually see “7 files” and display the list of out of sync items, that particular issue was resolved. Updating other devices won’t change anything about the underlying issue of being out of sync.

Just to reiterate that I understand the situation correctly:
Device A shows device C as syncing 95%, and the out of sync files list has entries E for folder F. This folder F is up-to-date on all devices.

Do all files E not exist on device C and are not ignored on either A or C?

[cosas]

Just to reiterate that I understand the situation correctly:

Device A shows device C as syncing 95%, and the out of sync files list has entries E for folder F. This folder F is up-to-date on all devices.

YUC

Do all files E not exist on device C and are not ignored on either A or C?

True : not even any syncthing-regular-picture-filename.jpg.tmp pending in any 3 devices. Ignores templates on the 3 nodes is (?d)(?i)*.lnk, (?d)(?i)*.ini, (?d)(?i)Thumbs.db on the 3 nodes (no (?i) on C as it is a Win). C hosts a single shared folder.

Files E don’t exist in F anymore anywhere.

Files E may exist in an other folder F’ only shared by B to a 4th node D (this D device being a linux 0.14.43 showing everything OK)

All 4 devices knows each other but C & D unknowing each other. Layout.pdf (22.8 KB)

[EDIT] A was involved in F’ ignoring almost all, but I removed it ATM, restarted all nodes and the issue remain. All files E were removed from A some days ago with fslint-gui as detected as duplicates

[imsodin (Simon)]

Can you post the output of curl -X GET -H "X-API-Key: yourapikey" 'http://localhost:8384/rest/db/file?folder=F&file=pathRelToFolderRoot' | json_pp on both A and C for one of the files reported as out of sync. In windows powershell the command apparently looks like the following, I don’t know how to do it otherwise on windows: (Invoke-WebRequest -Method Get -Headers @{"X-API-Key"="API-Key"} -Uri "http://localhost:8384/rest/db/completion?device=DEVICE-ID&folder=FOLDER-ID").Content | ConvertFrom-Json

[cosas]

On A:

curl -X GET -H "X-API-Key: MyKeyHere" 'http://localhost:8384/rest/db/file?folder=Dog%20of%20three%20head&file=IMG_2499.jpg' | json_pp
malformed JSON string, neither array, object, number, string or atom, at character offset 0 (before "<a href="https://loc...") at /usr/bin/json_pp line 44.

Another try with othe file in subdir

curl -X GET -H "X-API-Key: MyKeyHere" 'http://localhost:8384/rest/db/file?folder=Dog%20of%20three%20head&file=Récup tél Marie Whatsap/IMG-20170806-WA0004.jpg' | json_pp
JSON text must be an object or array (but found number, string, true, false or null, use allow_nonref to allow this) at /usr/bin/json_pp line 44.

On Win side, I’ll have to find how to Power Shell (WinXP) if possible

[cosas]

For Windows, can I try remotely from a nux ? dealing with firewall, and gui listen IP ?

[imsodin (Simon)]

You need to replace MyKeyHere with your actual API key (Actions -> Settings).

For Windows, can I try remotely from a nux ? dealing with firewall, and gui listen IP ?

I guess that should work.

[cosas]

You need to replace MyKeyHere with your actual API key (Actions -> Settings).

I did, I only redacted the post. Is there an error in the output ?

I’m using https+User+pw for the GUI. Do I have to change the command accordingly?

Is there a space before MyKeyHere?

You can see 2 different kind of outputs. I gave several trials. For the 1st one I first renamed the folder to Doth, then reversed to original and retried replacing spaces in name with %20.

For the 2nd file can spaces or “é” or “/” in name be a problem ?

In the lap time, I searched & found some files from E list in B machine: they just were renamed (by me from within F’ on device A when A still belonged to F’). On B, I copied one to F and let it sync to A & C. Nothing bad happened, G=L went +1 on all 3 nodes . I let as is hoping ST would recognize the file hash on A and rebuild the same with original name, but it didn’t. Still on A I copied the file out of ST view, rename to original name, then moved it in F. A scanned it and pushed it to B & C. List then went from 7 to 6.

Then I did nearly the same with another file, but did the rename on place. GUI unsync list briefly jumped from 6 to 7 then immediately down to 5. All nice

I can try to do this for others, but that won’t help to dig the issue. Also, I’m pretty sure I won’t find exact same hash for the 5 remaining… I’ll give a try with a fake file with same name, just to see if ST remembers the genuine hash or don’t worry and only cares with name in this matter.

[cosas]

Without piping to json_pp:

curl -X GET -H "X-API-Key: MyKeyHere" 'http://localhost:8384/rest/db/file?folder=Dog%20of%20three%20head&file=IMG_2499.jpg'
<a href="https://localhost:8384/rest/db/file?folder=Dog%20of%20three%20head&amp;file=IMG_2499.jpg">Temporary Redirect</a>.

[imsodin (Simon)]

I did, I only redacted the post. Is there an error in the output ?

I’m using https+User+pw for the GUI. Do I have to change the command accordingly?

Is there a space before MyKeyHere?

I could have guessed that, sorry. I believe the API key is enough also with user/pw. Yes, there should be a apace before mykey.

The redirect message is probably due to useing http instead of https - try that.

[cosas]

I had to use curl -k with https and no piping to json_pp

curl -k -X GET -H "X-API-Key: MyKeyHere" 'https://localhost:8384/rest/db/file?folder=Dog%20of%20three%20head&file=IMG_2499.jpg'

Output:

No such object in the index

[cosas]

And on C machine, output :

curl -k -X GET -H "X-API-Key: RemoteDevKey" 'https://192.168.X.Y:8384/rest/db/file?folder=Dog%20of%20three%20head&file=IMG_2499.jpg'
No such object in the index

[imsodin (Simon)]

That means the problem is with the folder string, as the file can’t figure in the UI but not be in the (global) index at all. You did use the folder id, not the folder label? (sorry for asking )

[cosas]

Don’t be sorry man… as I’m not sure I understand your wording “…can’t figure in the UI but not be in…”

And no I never user the folder id. I thought about it but the command will surely be different … I should search in the doc around in the API section.

News : a 0B fake file with name listed in E dropped the list to 4 missing items

a non-0 B fake file with name remaining listed in E dropped the list to 3 missing items

So only names make ST happy here

[imsodin (Simon)]

The string after folder= needs to be the folder ID, which you find in the GUI when editing the folder.

[cosas]

Your command was OK about this part for PowerShel I noticed but did dare thinking you made a typo. Let’s try again with F-ID

On C (Win)

{
   "availability" : [
      {
         "id" : "A ID",
         "fromTemporary" : false
      },
      {
         "fromTemporary" : false,
         "id" : "B ID"
      }
   ],
   "local" : {
      "deleted" : true,
      "permissions" : "0",
      "modifiedBy" : "B-ID",
      "noPermissions" : false,
      "numBlocks" : 0,
      "version" : [
         "B-ID:3",
         "A-ID:1"
      ],
      "invalid" : false,
      "modified" : "2017-10-18T21:34:12+02:00",
      "type" : 0,
      "sequence" : 109056,
      "size" : 0,
      "name" : "IMG_2499.jpg"
   },
   "global" : {
      "modified" : "2017-10-18T21:34:12+02:00",
      "invalid" : false,
      "version" : [
         "B-ID:3",
         "A-ID:1"
      ],
      "name" : "IMG_2499.jpg",
      "size" : 0,
      "type" : 0,
      "sequence" : 45393,
      "modifiedBy" : "B-ID",
      "noPermissions" : false,
      "deleted" : true,
      "permissions" : "0",
      "numBlocks" : 0
   }
}

[cosas]

On A (nux wishing to push missing files to Win):

{
   "global" : {
      "modified" : "2017-10-18T21:34:12+02:00",
      "modifiedBy" : "B-ID",
      "deleted" : true,
      "sequence" : 45393,
      "type" : 0,
      "numBlocks" : 0,
      "name" : "IMG_2499.jpg",
      "invalid" : false,
      "noPermissions" : false,
      "size" : 0,
      "permissions" : "0",
      "version" : [
         "B-ID:3",
         "A-ID:1"
      ]
   },
   "local" : {
      "name" : "IMG_2499.jpg",
      "type" : 0,
      "numBlocks" : 0,
      "sequence" : 45393,
      "deleted" : true,
      "modified" : "2017-10-18T21:34:12+02:00",
      "modifiedBy" : "B-ID",
      "size" : 0,
      "permissions" : "0",
      "noPermissions" : false,
      "invalid" : false,
      "version" : [
         "B-ID:3",
         "A-ID:1"
      ]
   },
   "availability" : [
      {
         "fromTemporary" : false,
         "id" : "B-ID"
      }
   ]
}

Would you want the output from B ?

[imsodin (Simon)]

Would you want the output from B ?

No not necessary, this already contains relevant infos: They do fully agree on global and local versions, so both are in sync. However the index between A and C is somehow messed up, such that A believes C does not have the up to date version of the file (as seen in the availability part). I have no idea how that could have happened and/or how to find out, but resetting the index on A or C should resolve it (-reset-deltas option). Maybe wait with doing that until tomorrow, just in case @AudriusButkevicius or @calmh takes an interest in digging into this further.