Syncthing event API, intended usage and efficiency?

[immortal192]

I need a way to ensure sync is completed before e.g. shutting down the system or running certain applications. With the event API, /rest/events?events=StateChanged&since=0&limit=1 gets the latest ID and exits.

Disclaimer: no real programming experience but want to learn: Is the intended usage to keep listening for new events to do an infinite loop keeping track of the latest ID and incrementing by 1 to listen for the next ID? E.g. is the following the most efficient way to go about doing this and and how Syncthing (or even how most applications that require listening for updates like desktop messenger apps) does it?

ID=
while true; do
    # Setting timeout to "forever" is the correct approach to listen for events indefinitely?
    curl ... -X GET "https://127.0.0.1:8384/rest/events?events=StateChanged&since=S${ID}&limit=1&timeout=9999999999"
    ID=ID+1
    # sleep 1 <-- no need since timeout period can be set above
done

In particular I was wondering if this will have any measurable impact on battery life of a laptop. I suppose the only relevant part is the listening indefinitely part (otherwise a sleep 1 would be recommended for polling e.g. using the REST API?). If this is how Syncthing and “internet-facing” apps work internally or in the background, then I’m more inclined to run such a script as a module in my status bar on system start permanently. I know there are third-party apps for this but I would like something CLI and to implement my own as an exercise. Also instead of a shell script running external commands, would using a “real” programming language like Python or Go be more efficient?

Since Syncthing operates on intervals and not real time (even triggering a manual/forced sync does not update changes immediately, so I"m not sure how that’s really useful and I wish there’s a way to do that to avoid problem), there might be false positives such as the first immediate sync detected may not necessarily propagate the changes recently written on the disk (even if the comes after). This is probably for efficiency reasons. For a one-time check of sync status, I suppose the approach is to wait for X seconds and then check, where X seconds is the sync interval (and a few seconds more for good measure to account for latency or whatever)?

Much tips/suggestions is appreciated.

[Nummer378 (Max)]

@immortal192

E.g. is the following the most efficient way to go about doing this

The most efficient (and possibly also most robust) is to not limit the number of returned events at all and be prepared to handle events in batches. Algorithmically speaking:

1. Upon start, do an initial query /rest/events?since=0 (omitting irrelevant parmeters for brevity). Note that according to the docs this may block for an extended period of time (60 seconds by default) to wait for new events.
2. Parse all events that have been returned according to your program logic and remember the highest event ID that you have seen. This may not be linear, since events can be skipped.
3. Do a query /rest/events?since=highest-event-ID, where highest-event-ID is the highest event ID you have seen. Again, this call may block and return more than one event (as well as zero events).
4. Go to 2.

This allows you to process events in batches, which is more efficient than having to do one HTTP request-response dance per event. It might also reduce the chance of skipping events, because the API doesn’t keep them around forever.

To save resources, you can utilize the fact that the API blocks by default: If there are no new events, it waits for some time before returning anything. Generally, a HTTP client should be able to sleep (i.e. do almost nothing) while waiting for syncthing to return a response*. Therefore, not much energy is spent while nothing is going on.

Depending on the frequency of events, you might not need the most efficient way though. Going through events one by one is certainly possible.

*Internally, operating systems have event systems that allow applications to “wake up” when incoming data is received. A HTTP client can utilize this to conserve energy while waiting for a response. This is fairly low-level stuff though, generally already implemented for you.

[immortal192]

Perfect, thank you. Implemented as a shell script.