I have a setup with two system only. I am writing a program where, when user clicks a button on local machine, i trigger scan via POST /rest/db/scan to capture the changes made since last scan.
Now, in my program, I need to wait till the local detected changes are synced on a remote machine. How can i reliably find that my remote device is updated to latest version.
Can i use ‘GET /rest/db/completion’ for this or can anyone suggest other way ?
There is an endpoint which returns completion of the remote end which should be good enough, though you have to make sure its connected.
Thanks for the prompt reply. Can you explain what is meant by endpoint ?
Are you referring to ‘GET /rest/db/completion’ or there is some other function which i can call.
Thats the right one, but you might want to call /rest/system/connections first to see if you are connected.
I tried the above approach but am facing an issue sometimes. File operations were done on local system. The files got synced from a local system to remote system. And for long duration even after the actual sync got complete and files were available at remote machine, call to ‘rest/db/completion’ on local system was returning completion:0.
Although synching logs show completion as 100,
[DNKY7] 2017/08/29 22:50:32.284649 events.go:231: DEBUG: log 51 FolderCompletion map[folder:b52441ad-5204-4221-86ec-c100be088ed0 device:O7IBMNW-WBGYCVC-VC4EZFK-FNOHEPF-RCP6BMP-27UXMID-ULE4BXW-LZ5CSQS completion:100 needBytes:0 globalBytes:677]
The call to ‘rest/db/completion’ on local system show completion as 0. Also since changes were made on local system, why does needBytes show 677 on local system?
DEBUG:urllib3.connectionpool:http://127.0.0.1:8384 “GET /rest/db/completion?folder=b52441ad-5204-4221-86ec-c100be088ed0 HTTP/1.1” 200 66
DEBUG:root:{‘completion’: 0, ‘globalBytes’: 677, ‘needBytes’: 677, ‘needDeletes’: 0}
You also need to give the device parameter to the completion call. Currently it’s answering the question for the all-zeroes device ID, probably.
Better yet, listen to the FolderCompletion event.
Yup it works correctly after I corrected my code to add device ID. An oversight on my part, apologies for the trouble.
The FolderCompletion event definitely seems like more efficient way to do it. Basically I make periodic calls to “/rest/events” with appropriate “since=lastSeenID” and filter the results for required ‘device’, ‘folder’ and FolderCompletion event.
Just wanted to confirm if this is also a reliable way to do it? I mean, is it possible that the event can be missed or not appear in the ‘rest/events’ results.
Yes, but you can do one better. Add a ...&events=FolderCompletion parameter and you will get only that event, and a specific subscription that buffers only those events. This makes it extremely unlikely that you’ll miss any events.
Thats awesome! Thanks for the tip. I was playing around with it and got some questions about event ID. I hit two URLS and compared the results
1)http://127.0.0.1:8384/rest/events 2)http://127.0.0.1:8384/rest/events?events=StateChanged
When I hit http://127.0.0.1:8384/rest/events, I am getting StateChanged events starting at 16:03 PM. { “id”: 2, “globalID”: 2, “time”: “2017-08-30T16:03:41.885027921+04:00”, “type”: “StateChanged”, “data”: { “folder”: “default”, “from”: “idle”, “to”: “scanning” } },
But when I hit http://127.0.0.1:8384/rest/events?events=StateChanged, I am getting events starting at 18:43 PM. { “id”: 1, “globalID”: 692, “time”: “2017-08-30T18:43:24.468212946+04:00”, “type”: “StateChanged”, “data”: { “duration”: 53.496939586, “folder”: “ae6eef13-bdf6-4b73-ac9a-57b8069559e4”, “from”: “idle”, “to”: “scanning” } },
Shouldn’t the second URL have returned events since 16:03 PM as no since=lastSeenID parameter was specified
Does the ‘since=lastSeenID’ refer to globalID or id field of the results
I couldn’t find reference for optional ‘&events=’ field at https://docs.syncthing.net/dev/events.html. where can I find it ?
It’s a new feature. The subscription gets created first time to make the call and stays alive for some time (as long as you keep polling every now and then it should stay alive).
There’s some stuff going on behind the scenes here.
You have event subscriptions which are sort of like buffered queues for events. There are a couple of them by default, and one of them is the subscription you are pulling from when loading /rest/events without any ...?events= parameter. This subscription sees most (but not all) event types.
Each subscription counts its own IDs, and these are the ones exposed as id in the JSON. This makes sense because on next poll you want to ask for new events on the same subscription with higher event IDs. There is also a globalID which is assigned incrementally to each event as its created, disregarding the subscription. A single event, with a given global ID, can end up in zero or more subscriptions.
So when looking at /rest/events?events=StateChanged for example you are looking at a subscription for only that type of event. The vast majority of events created are not StateChanged events, so expect large gaps in globalID. However, the id should be sequential for that specific subscription.
Non-default subscriptions (pretty much anything with an ...?events= parameter) are created when they are first used. They will start with id=1 and whatever globalID might be the latest when the next event of that type happens. The subscription then persists until Syncthing restarts.
Encountered a case where the above approach is failing. The syncing status was stuck at xx% but the files at both the end were in sync. Attaching images for the same:
when does this scenario occur and what can be done to recover ?
One of the folders is shared only one way.
And/or the ignore patterns are mismatched on the two sides.
Ok I might have misunderstood you but if if you are saying that folder was not shared with the peer device , i checked both the devices. Both sides have peer device added in it and both sides have shared the same folder-id with the peer device. Further both sides are configured for sharing in Send And Receive mode.
The contents of .stignore are not same on both sides. They both contain entries for different folders which I dont require to be synced. I have kept this configuration many times and it did not create trouble till now. Is it recommended to keep same .stignore file on both sides ? Will try that…
Further the files corresponding to .stignore dont have enough size to make syncthing get stuck at 56%, it should have gotten stuck at more than 90% if this was the issue though.
Also sharing what I found till now, it seems there has been data loss, files disappeared on both sides (probably before sync could complete) and so the globalbytes that were required for sync to complete are now nowhere available, causing it to be stuck. No manual delete operation was done, folder rename on one of the subfolder was done.
I have syncthing setup like this:
local-end : I have syncthing running on macOS with syncthing-inotify.
remote-end : I have syncthing running in ubuntu docker container on a remote server with the synced directory mounted inside the container.
I am looking into logs, will share here if i find anything.
UPDATE: I updated the .stgnore’s on both machines according to forum post of “Useful .stignore patterns” and the issue got resolved. Thanks!
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.
