Technology for new API

Development
1

So I was playing with some new age hipster web tech stuff at work, namely angular material 10, as well as grpcweb (grpc for da browser), and was thinking about the foundations we’d build the next API and UI with.

There are a few “leading” standards for the web nowadays:

OpenAPI specs

Formerly known as Swagger specs.

The spec is a large (imho incredibly) verbose json (or yaml) file (see this), which effectively declares the rest endpoints, the schema for request/response with the types.

This can then be used to generate clients and servers for most languages, including javascript/typescript which could then be used in the new UI.

List of languages for generatic client/server components can be seen here: https://github.com/OpenAPITools/openapi-generator

This is purely rest, and atleast in Go (unlike Kotlin used at work) there is no trivial way to generate the spec from code of the API.

There are a few attempts which either require you to write additional code that acts as the declaration or extensive error-prone comment abuse:

The nice part about OpenAPI specs is that you can slap a nice swagger ui on top of it which makes it easy to interact with the API, example:

https://petstore.swagger.io/

Not so useful in our case, as we have UI, but incredibly useful for headless services which just expose APIs.

I guess in our scenario, we’d have to generate the server code from the spec, as otherwise I think it’s unlikely we’d be able to keep it in sync.

Pros:

  • Well supported for many languages
  • Good ol’ orthodox REST

Cons:

  • Verbose and IMHO painful to write specs, atleast compared to GRPC.
  • Only unary calls. No streaming support (looking at you events and all the things we use to poll)

GRPC gateway

This is what I believe Google are using for exposing their own services as rest services

Effectively it’s a set of annotations that you instrument your proto contract with (see this, which explains grpc-gateway how to turn the GRPC contract into a REST contract. The gateway can be mounted on the same mux as the GRPC service, so it’s not a separate component you run, just a library you have to use.

There is then a further plugin that allows you to generate an OpenAPI spec from the GRPC contract, which then allows you to generate typed clients for calling the gateway which then translates the calls to GRPC. This is obviously a generated contract and you probably would not want to modify that by hand.

You can/would use this OpenAPI spec to generate a strongly typed web client that can talk to your service via the gateway

Pros:

  • We gain GRPC support for non-web cases
  • Good ol’ orthodox REST for web cases
  • Well supported for many languages due to OpenAPI export
  • More sensible schema to write

Cons:

  • REST stuff exposed by gateway only supports unary calls.
  • Still have to write annotations on standard GRPC to make it work, yet again making it a bit verbose.

GRPC web

This is yet another "proxy" like solution, but this one is only targeted at the web, by generating javascript only. This is still a standard offering from the ecosystem, but it's main purpose is purely to address accessing GRPC from web browsers.

This is still “rest” behind the scenes, but there are a ton of weird headers flying around returning GRPC status codes and what not, so I don’t expect it to be inter-operable outside of it’s own ecosystem.

This does not require any extra annotations, and is able to generate strongly typed typescript (or js if you are a masochist) clients that make working on web stuff not miserable.

Pros:

  • GRPC support for non-web cases
  • Support for service to client streaming calls in the browser (hello events)
  • Trivial concise to write specs.

Cons:

  • Bye bye rest
  • Only support languages that support GRPC going forward (which is probably most languages we care about, as there exists grpcurl which allows calling GRPC from the shell)
  • Reverse proxy would require HTTP2 for real GRPC (HTTP1 for grpcweb)

I have an example, where I got grpc-web and grpc-gateway (hence transitively openapi specs) to work together, meaning we can have all of the following:

  1. Contract that declares the API (with annotations on how to map it to rest)
  2. Go code for the contract generated using Gogoproto
  3. A typescript client generated using grpcweb
  4. OpenAPI spec generated from the proto contract (can be used to generate rest clients)
  5. A single web server that serves all of this:
    • Serve GRPC
    • Serve GRPC gateway (convert rest traffic to GRPC traffic)
    • Serve GRPC web server
    • Serve OpenAPI spec on /swagger.json
    • Serve Swagger UI on /swagger-ui/

But this feels like an overkill, and I still have to use magical proto annotations to explain it how to map GPRC calls to rest calls.

My gut says that GRPC + grpcweb should be enough. We would use grpcweb for our own UI use-case. If others want to talk to our API, they would use GRPC or our grpcweb client if they are in the browser.

This is a bit of a middle-finger to people who run reverse proxies from 1980s, that don’t support HTTP2, but I don’t expect these sort of users want to poke at vanilla GRPC at the GRPC API anyway, and reverse proxying grpcweb is still ok.

This would also be an opportunity for the “let’s have the config as a proto contract so users can easily use it language of their choice” I suggested a while ago.

What do you guys think?

2

I haven’t used the fancy stuff much but a grpc spec that is used to generate server and client stubs have worked for me in another project. I didn’t try to make actual Json rest stuff on top of that though.

3

Well that is what I am saying, I’d like to abandon rest for the purpose of simplifying stuff, and be grpc (and grpcweb) only, yay, nay?

4

I quite frequently talk to the api with curl. We could have a client of sorts built into Syncthing which knows how to talk to remote instances but that’s a bit more engineering than currently.

There’s a lot of other api integrations out there as well. Being able to talk to Syncthing from powershell or whatnot without having to build a full client integration is a feature.

5

grpcurl is a thing, and it’s more structured way to talk to the api as the schema is enforced.

The cli could still be a thing.

6

I’m fairly strongly opposed to removing a plain text rest interface completely. Gpcurl may be a thing but it’s not going to be present on machines where I need to troubleshoot. However I can accept the server side having some manual shin methods to wrap rest over a grpc server side.

1 Like
7

The problem I see with that is that you have to maintain it manually, and when you maintain manually it’s hard to generate a client.

I guess the hybrid of grpc-web and grpc-gateway solves it, but it’s a lot of baggage, but better than having to maintain it manually.

Are we ok to carry all of that baggage?

8

I’d rather not. But I meant we could have a grpc definition and generated server side, and manual rest “wrapper” around the generated grpc server. Any real integration would be done around the grpc spec and I could still curl against the rest wrapper.

I guess this is a manual/light version of the gateway. Whether it’s less baggage I suppose I’m not sure.

Still not sure what grpc-web is precisely.

9

I really want to avoid the manual part as it’s a thing that has a lot of overhead and is bound to go out of sync. Grpc gateway is probably giving what you want, with a bit less overhead and harder to get out of sync

Grpcweb is purely exposing grpc to be called from the web browsers, I explained the benefits over gateway above, it’s pseudo rest, but you can’t generate clients for anything else than browsers.

10

I guess grpc-web talks binary grpc via a generated js client? Regardless we would need the gateway or something like it yes.

11

It’s base64 encoded binary proto with special headers for various aspects of transmitting grpc state.

Still grpc calling convention, aka /contract.ServiceName/Method via http1 but most of grpc baggage is wedged in headers, so you can’t trivially generate a client/use it for rest purposes.

12

Oh I’m open to something new but, please keep the rest api for the android wrapper. At least for a specific time so there would be time to migrate to grpc and probably get some help from the internet out there :-).

13

If anything, Android would be much easier to integrate using grpc than rest, obviously, this is not a binary thing. We’d keep the old api for a while, while we move our stuff to the new world, so there would be time to migrate, so I wouldn’t be worried about it.

1 Like
14

For me it’s a hard requirement that it’s possible to talk to the API with something standard. That is, a default Windows box where I can open a powershell and say invoke-webrequest, or a Unix box with curl or wget. At least for the most common API requests (status, config, etc). Beyond that I’ll reserve judgement until I see the code.

It would be very cool to have an API description as protobuf schema with grpc stuff though. So I agree with that.

3 Likes
15

gRPC is very nice but puts the bar quite high for thirdparties to interact with your service.

IMHO OpenAPI and websockets for events is the best approach and allows for easy tooling integration.

16

The problem I have with openapi is that it’s a complete bitch to maintain. It takes 30-60 lines of json to declare a single rest endpoint.

17

Yeah I don’t think that format was intended to be human writeable. But protobuf/grpc specs are, and as long as a frontend exists that speaks json that seems fine to me. I’m sure something exists to build decent API documentation (or even openapi files) from comments in that kind of file, otherwise we can build that tool.

18

I mean google’s proto annotations already lets you build OpenAPI specs from grpc specs, which is what grpc-gateway is. I’m just not happy that I have to annotate stuff on top of GRPC specs, oppose to just writing grpc and call it a day.

19

protoc-gen-doc is pretty handy. Bonus point: the comments also get used in the swagger documentation

1 Like
20

I can’t add anything meaningful on the specifics, this is new to me (thanks for the informative write-up :wink: ). On the high-level having a speced and generatable config and api sounds like a huge pro (and largish initial effort, with later payoff).

As for OpenAPI/REST vs gRPC: Apart from all the general shiny niceness of gRPC, streaming support for events does sound convincing. My (pretty useless) recent integration benchmark was controlled by events and flaky, which I could only explain by lost events (which isn’t that surprising given all it did was check and shortcut conflicts, generating tons of ItemStarted/ItemFinished).

And as for “standard tools” to debug: It should hold that any machine on which Syncthing is debugged, has Syncthing, shouldn’t it? Regardless of the topic discussed at hand, I was hoping to get around to integrating querying the API into the syncthing binary. That would be really nice for supporting, removing the need to find the right references to curl/cmd/powershell/… and troubleshoot everyone’s network to get e.g. the /rest/db/file info. That would thus solve the “generic debugability” requirement too.
The “cheap” alternative of just exposing the existing endpoints (yeii, backwards compatibility) or another limited (as in usability), but powerful (as in catch-all, like the current config one) set of endpoints via grpc-gateway annotations should do as well.