Write Rollups HTTP Server in Go

[gligneul]

📚 Context

The Rollups HTTP server is a Rust program in the machine-emulator-tools repository. This server runs inside the Cartesi Machine allowing the DApp backend to interact with the machine Rollups driver. The server uses the low-level ioctl call to interact with the Kernel driver directly.

With the output unification, this server needs to be rewritten to use a new C API that abstracts the ioctl and the encoding of the Rollups outputs. This new lib is under development by the Machine Unit.

I propose that the Node Unit rewrites this Rollups HTTP Server using this new C API in Go. The implementation of the server should be divided into two parts. The first part should be a library that sets up the HTTP endpoints and delegates the internal behavior to a Rollups Controller interface. The second part should be a binary that uses the endpoints library and implements a controller that calls the C API.

Later, the same Rollups HTTP endpoint library will be used by the DevExecutor in the new node architecture.

[gligneul]

@diegonehab @vfusco @mpolitzer, what do you think?

[mpolitzer]

Sounds good to me. I do have a couple of questions on the details.

Given that:

• The current plan is to have this library live in the tools repository and for it to handle rollup ioctl, EVM encode/decode and the outputs merkle tree.
• We currently provide the linux rollup and yield headers in the toolchain. Files such as linux/cartesi/rollup.h get included by tools, and that goes for the rust code as well (via a C wrapper I believe).

Now to the questions:

• Would the Go service be interested only in the ioctl wrapper or on the EVM codec as well?
  • I ask this because it may have access to libraries that integrate better such as go-ethereum.
• Will this Go service live in this repository or somewhere else?
  • I think cross compiling with a regular Go may be possible if we don't expose/require the kernel headers.
• Does Go handle static libraries (.a) files and headers (.h) files well? would that be enough?

[gligneul]

Would the Go service be interested only in the ioctl wrapper or on the EVM codec as well?
I ask this because it may have access to libraries that integrate better such as go-ethereum.

I see benefits in using the same implementation for the EVM codec. That said, it might be simpler to just use go-ethereum, so we don't need to link to this library when running in dev mode (outside of the Cartesi machine).

Will this Go service live in this repository or somewhere else?

We have a few options.

1. Put go-rollups-lib and go-rollups-server in the emulator tools repo.
2. Put go-rollups-lib in a separate repo, and go-rollups-server in the emulator tools repo.
3. Put go-rollups-lib and go-rollups-server in a separate repo.

Option 1 would be straightforward, but I think it will lead to lack of code ownership. The maintainer of the code would be one unit, but the maintainer of the repository would be another unit.

I don't like much option 2 because these two components will be tightly coupled and should live in the same place.

In option 3, the RISC-V binary would be cross compiled and distributed as a Github artifact. We need to think how you would provide the .h and .a that we need to generate this binary.

I think cross compiling with a regular Go may be possible if we don't expose/require the kernel headers.

I also think so.

Does Go handle static libraries (.a) files and headers (.h) files well? would that be enough?

I have to research that, but I believe so.

[gligneul]

This repo contains an example of linking static and dynamic C libraries with Go.

[gligneul]

We should consider adding HTTP methods that send/receive binary data instead of JSON. For instance, we should have a /finish_bytes that returns the input payload as bytes in the body. The request type should be application/octet-stream.

[gligneul]

We won't maintain the HTTP server in the rollups-node monorepo.