Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update README.md #63

Merged
merged 6 commits into from
Dec 15, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
170 changes: 115 additions & 55 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,77 +1,104 @@
![](docs/connect-rpc-scala-logo.png)

# Connect-RPC ↔ ScalaPB GRPC Bridge
# REST API for GRPC services with Connect protocol / GRPC Transcoding for Scala

This library provides a bridge between [Connect](https://connectrpc.com/docs/protocol) protocol and
[ScalaPB](https://scalapb.github.io) GRPC compiler for Scala.
It is inspired and takes ideas from [grpc-json-bridge](https://github.com/avast/grpc-json-bridge) library, which doesn't
seem to be supported anymore + the library doesn't follow a Connect-RPC standard (while being very close to it),
which makes using clients generated with ConnectRPC not possible.
![Maven Central](https://img.shields.io/maven-central/v/io.github.igor-vovk/connect-rpc-scala-core_3?style=flat-square&color=green)

Since integration happens on the foundational ScalaPB level, it works with all common GRPC code-generation projects for
Scala:
This library makes it easy to expose your GRPC services to the clients using Connect protocol (with JSON messages),
without Envoy or any other proxy.

* [ScalaPB](https://scalapb.github.io) services with `Future` monad
* [fs2-grpc](https://github.com/typelevel/fs2-grpc), built on top of `cats-effect` and `fs2`
* [ZIO gRPC](https://scalapb.github.io/zio-grpc/), built on top of `ZIO`
In essence, a service implementing the following protobuf definition:

*Note*: at the moment, only unary (non-streaming) methods are supported.
```protobuf
syntax = "proto3";

## Motivation
package example;

As a part of a GRPC adoption, there is usually a need to expose REST APIs.
Since GRPC is based on HTTP2, it's not the best option to expose it directly to the clients, which aren’t
natively supporting HTTP2.
So the most common approach is to expose REST APIs, which will be translated to GRPC on the server side.
There are two main protocols for this:
service ExampleService {
rpc GetExample(GetExampleRequest) returns (GetExampleResponse) {
}
}

* [GRPC-WEB](https://github.com/grpc/grpc-web)
* [Connect](https://connectrpc.com/docs/introduction)
message GetExampleRequest {
string id = 1;
}

They are similar, but GRPC-WEB target is to be as close to GRPC as possible, while Connect is more
web-friendly: it has better client libraries, better web semantics:
content-type is `application/json` instead of `application/grpc-web+json`, error codes are just normal http codes
instead of being sent in headers, errors are output in the body of the response JSON-encoded, it supports GET-requests,
etc (you can also read
this [blog post describing why Connect is better](https://buf.build/blog/connect-a-better-grpc)).
message GetExampleResponse {
string name = 1;
}
```

Both protocols support encoding data in Protobuf and JSON.
JSON is more web-friendly, but it requires having some component in the middle, providing JSON → Protobuf
conversion during the request phase and Protobuf → JSON conversion during the response phase.
Will be exposed to the clients as a REST API:

*And this can be tricky to set up*:
```http
POST /example.ExampleService/GetExample HTTP/1.1
Content-Type: application/json

The suggested approach in this case is to use a web-server ([Envoy](https://scalapb.github.io)) as a proxy,
supporting translation of both protocols to GRPC.
The general setup of the Envoy in this case allows proxying HTTP/1.1 requests to GRPC, while still having protobuf
messages in the body of the request.
{
"id": "123"
}

To support JSON, Envoy needs to be configured with Protobuf descriptors, which is not very convenient.
HTTP/1.1 200 OK

*That's where this library comes in*:
{
"name": "example"
}
```

It allows exposing GRPC services, built with [ScalaPB](https://scalapb.github.io), to the clients
using Connect protocol (with JSON messages), without Envoy or any other proxy, so a web service can expose
both GRPC and REST APIs at the same time on two ports.
It is compatible with Connect protocol clients (e.g., you can generate clients with [Connect RPC](https://connectrpc.com) `protoc` and
`buf` plugins instead of writing requests manually).

This simplifies overall setup: simpler CI, fewer network components, faster execution speed.
In addition, the library supports creating free-form REST APIs,
using [GRPC Transcoding](https://cloud.google.com/endpoints/docs/grpc/transcoding) approach
(full support of `google.api.http` annotations is in progress).:

## Features of the protocol supported by the library
```protobuf
syntax = "proto3";

```yaml
versions: [ HTTP_VERSION_1, HTTP_VERSION_2 ]
protocols: [ PROTOCOL_CONNECT ]
codecs: [ CODEC_JSON, CODEC_PROTO ]
stream_types: [ STREAM_TYPE_UNARY ]
supports_tls: false
supports_trailers: false
supports_connect_get: true
supports_message_receive_limit: false
package example;

import "google/api/annotations.proto";

service ExampleService {
rpc GetExample(GetExampleRequest) returns (GetExampleResponse) {
option (google.api.http) = {
get: "/example/{id}"
};
}
}

message GetExampleRequest {
string id = 1;
}

message GetExampleResponse {
string name = 1;
}
```

In addition to the previous way of calling it, this endpoint will be exposed as a REST API:

```http
GET /example/123 HTTP/1.1

HTTP/1.1 200 OK

{
"name": "example"
}
```

Since integration happens on the foundational ScalaPB level, it works with all common GRPC code-generators:

* [ScalaPB](https://scalapb.github.io) services with `Future` monad
* [fs2-grpc](https://github.com/typelevel/fs2-grpc), built on top of `cats-effect` and `fs2`
* [ZIO gRPC](https://scalapb.github.io/zio-grpc/), built on top of `ZIO`

*Note*: at the moment, only unary (non-streaming) methods are supported.

## Usage

Installing with SBT (you also need to install particular `http4s` server implementation):
For SBT (you also need to install particular `http4s` server implementation):

```scala
libraryDependencies ++= Seq(
Expand Down Expand Up @@ -138,7 +165,9 @@ You can read [this](https://zio.dev/guides/interop/with-cats-effect/).

## Development

### Running Connect-RPC conformance tests
### Connect RPC

#### Running Connect-RPC conformance tests

Run the following command to run Connect-RPC conformance tests:

Expand All @@ -149,7 +178,7 @@ docker build . --output "out" --progress=plain
Execution results are output to STDOUT.
Diagnostic data from the server itself is written to the log file `out/out.log`.

### Connect protocol conformance tests status
#### Connect protocol conformance tests status

✅ JSON codec conformance status: __full conformance__.

Expand All @@ -159,9 +188,40 @@ Known issues:

* Errors serialized incorrectly for protobuf codec.

## Future improvements
#### Supported features of the protocol

```yaml
versions: [ HTTP_VERSION_1, HTTP_VERSION_2 ]
protocols: [ PROTOCOL_CONNECT ]
codecs: [ CODEC_JSON, CODEC_PROTO ]
stream_types: [ STREAM_TYPE_UNARY ]
supports_tls: false
supports_trailers: false
supports_connect_get: true
supports_message_receive_limit: false
```

### GRPC Transcoding

#### Support

- [x] GET, POST, PUT, DELETE, PATCH methods
- [x] Path parameters, e.g., `/v1/countries/{name}`
- [x] Query parameters, repeating query parameters (e.g., `?a=1&a=2`) as arrays
- [x] Request body (JSON)
- [ ] Body field mapping, e.g. `body: "request"` (not supported yet), `body: "*"` (supported)
- [ ] Path suffixes, e.g., `/v1/{name=projects/*/locations/*}/datasets` (not supported yet)

### Future improvements

- [x] Support GET-requests ([#10](https://github.com/igor-vovk/connect-rpc-scala/issues/10))
- [ ] Support `google.api.http` annotations (GRPC transcoding) ([#51](https://github.com/igor-vovk/connect-rpc-scala/issues/51))
- [x] Support `google.api.http` annotations (GRPC
transcoding) ([#51](https://github.com/igor-vovk/connect-rpc-scala/issues/51))
- [ ] Support configurable timeouts
- [ ] Support non-unary (streaming) methods

### Thanks

The library is inspired and takes some ideas from the [grpc-json-bridge](https://github.com/avast/grpc-json-bridge).
Which doesn't seem to be supported anymore, + also the library doesn't follow a Connect-RPC standard (while being very
close to it).
Loading