feat: support gRPC endpoints in core

[rach-id]

This is an implementation of a streaming API for blocks in core.

Helps close celestiaorg/celestia-app#3421 but not sure it entirely closes it.

It can easily be used:

package main

import (
	"context"
	"fmt"
	coregrpc "github.com/tendermint/tendermint/rpc/grpc"
)

func main() {
	client := coregrpc.StartBlockAPIGRPCClient("tcp://localhost:9090")

	blockStreamer, err := client.BlockByHeight(context.Background(), &coregrpc.BlockByHeightRequest{Height: 2})
	if err != nil {
		panic(err)
	}
	blockMeta, err := client.BlockMetaByHeight(context.Background(), &coregrpc.BlockMetaByHeightRequest{Height: 2})
	if err != nil {
		panic(err)
	}
	parts := make([]*core.Part, 0)
	for i := 0; i < int(blockMeta.BlockMeta.BlockID.PartSetHeader.Total); i++ {
		resp, err := blockStreamer.Recv()
		if err != nil {
			panic(err)
		}
		parts = append(parts, resp.BlockPart)
		if resp.IsLast && i < int(blockMeta.BlockMeta.BlockID.PartSetHeader.Total)-1 {
			panic("couldn't get all parts")
		} else if resp.IsLast {
			break
		}
	}

	h := types.NewPartSetFromHeader(types.PartSetHeader{
		Total: blockMeta.BlockMeta.BlockID.PartSetHeader.Total,
		Hash:  blockMeta.BlockMeta.BlockID.PartSetHeader.Hash,
	})

	for _, part := range parts {
		ok, err := h.AddPart(&types.Part{
			Index: part.Index,
			Bytes: part.Bytes,
			Proof: merkle.Proof{
				Total:    part.Proof.Total,
				Index:    part.Proof.Index,
				LeafHash: part.Proof.LeafHash,
				Aunts:    part.Proof.Aunts,
			},
		})
		if err != nil {
			panic(err)
		}
		if !ok {
			panic("not okey")
		}
	}
	pbb := new(core.Block)
	bz, err := io.ReadAll(h.GetReader())
	if err != nil {
		panic(err)
	}
	err = proto.Unmarshal(bz, pbb)
	if err != nil {
		panic(err)
	}
	block, err := types.BlockFromProto(pbb)
	if err != nil {
		panic(err)
	}
	fmt.Println(block)

	// get a commit
	commit, err := client.Commit(context.Background(), &coregrpc.CommitRequest{Height: 10})
	if err != nil {
		panic(err)
	}
	fmt.Println(commit)

	// listen for new heights
	streamer, err := client.SubscribeNewHeights(context.Background(), &coregrpc.SubscribeNewHeightsRequest{})
	if err != nil {
		panic(err)
	}
	for {
		resp, err := streamer.Recv()
		if err != nil {
			panic(err)
		}
		fmt.Println(resp)
	}
}

Ps: I didn't add the tests because I didn't find a direct way of mocking the environment without polluting the rest of the repo (exporting some methods, adding new helpers, etc). And I think since the implementation is simple, just querying the block/state stores for results, it's fine to leave it untested.

[rach-id]

Just found out that once we enable the app grpc, these endpoints get overridden, I'll be looking into it

[rach-id]

This branch is now ready to be merged:

• Tests added
• TLS support added to utility as per @vgonkivs request
• latest version tested in node against an app running this version and all seems to be in place

[cmwaters]

Thanks for E2E testing this change @rach-id. Have a few questions before we merge

[cmwaters]

Is it possible to prefix these types as StreamedBlockByHashResponse for example so it's more obvious that this type is streamed. Also could we add a comment that the commit, validator set and block meta is only populated for the first response in that stream? (I guess this is how it works right).

Also why BlockMeta and not just Header?

[rach-id]

Is it possible to prefix these types as StreamedBlockByHashResponse for example so it's more obvious that this type is streamed. Also could we add a comment that the commit, validator set and block meta is only populated for the first response in that stream? (I guess this is how it works right).

ff4f343

Also why BlockMeta and not just Header?

that's what node team asked for

[cmwaters]

I don't see where else we call this. What was the reason for splitting?

[rach-id]

it's called in Celestia-node to have the parts and build the block without verifying the proof

[cmwaters]

why do we want to build the block without verifying the proof

[rach-id]

Since they make the assumption that every bridge node will be connected to a local full node. So the data is trusted and no need to verify inclusion. Also to reduce the size of the parts.

The initial implementation provided the proofs by default, but they said its better to disable it by default

[rootulp]

LGTM. Left a few non-blocking questions / suggestions

[rootulp]

[question] should this log an error and terminate instead of continuing?

Suggested change

	env.Logger.Debug("couldn't cast event data to new block")
	continue
	env.Logger.Error("couldn't cast event data to new block")
	return

[rach-id]

I didn't want to have the gRPC logs polluting the output but it's a good point. I will handle this in a separate PR where I will be doing more testing and adding more logic for failures etc. I didn't want to include everything in the same PR to reduce complexity

[rootulp]

[question] Should we extract a constant for RETRY_ATTEMPTS = 5

Suggested change

	for i := 1; i < 6; i++ {
	for i := 0; i < RETRY_ATTEMPTS; i++ {

[rach-id]

yes, we could. I can extract it if you want