Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Privval protocol incompatibility with namada node #817

Open
Fraccaman opened this issue Dec 5, 2023 · 5 comments
Open

Privval protocol incompatibility with namada node #817

Fraccaman opened this issue Dec 5, 2023 · 5 comments

Comments

@Fraccaman
Copy link

Fraccaman commented Dec 5, 2023

Im using tmkms v0.13.0 with a namada node (which is using cometbft 0.37.0 under the hood).
Running tmkms I get the following error:

2023-12-05T11:18:56.874145Z  INFO tmkms::commands::start: tmkms 0.13.0 starting up...
2023-12-05T11:18:56.876290Z  INFO tmkms::keyring: [keyring:softsign] added consensus Ed25519 key: tnam1zcjduepqe09qfp0stw7ryce4ch8r83frt9qmhseg4t658xrj74mlmx77mspqh20rn7
2023-12-05T11:19:00.890926Z  INFO tmkms::connection::tcp: KMS node ID: fce99ce38f090a9982199c33c88052f154f8a2d3
2023-12-05T11:19:00.894721Z ERROR tmkms::client: [local.e8700a77c10bbc3e43a00-0@tcp://[email protected]:27658] protocol error:
   0: io error
   1: failed to fill whole buffer

And then the namada full node crashes, probably trying to deserialize some protobuf encoded message as I see these kind of message on the full node:

The application panicked (crashed).
Message:  called `Result::unwrap()` on an `Err` value: DecodeError { description: "invalid wire type value: 6", stack: [("Request", "value")] }

or

The application panicked (crashed).
Message:  called `Result::unwrap()` on an `Err` value: DecodeError { description: "unexpected end group tag", stack: [("Request", "value")] }

Do you know if tmkms is compatible with cometbft 0.37.0? if not, is this planned?

@Fraccaman Fraccaman changed the title tendermint v0.37 tendermint/cometbft v0.37 Dec 5, 2023
@tony-iqlusion
Copy link
Member

tony-iqlusion commented Dec 6, 2023

No idea what's happening here.

I would recommend avoiding leaping to conclusions like some outright incompatibility between CometBFT and TMKMS (or rather, the tendermint-p2p crate TMKMS uses), until such time as you can find an actual change to the CometBFT code which would cause this, or demonstrate the incompatibility with more than one CometBFT v0.37 chain. Many of these issues wind up being somewhat chain-specific. See also #729.

These issues are unfortunately painful to reproduce without us setting up an entire testnet node for every chain which experiences them, so I'd appreciate your patience in trying to narrow it down without us having to do that. Ideally if you could provide access to a preconfigured node we could debug which has a full development environment for Rust and the TMKMS source code, that would be helpful.

@tony-iqlusion tony-iqlusion changed the title tendermint/cometbft v0.37 Privval protocol incompatibility with namada node Dec 6, 2023
@tony-iqlusion
Copy link
Member

tony-iqlusion commented Dec 14, 2023

"invalid wire type value: 6"

FWIW, this isn't necessarily due to a schema mismatch. 6 is not a valid wiretype whatsoever (the allowed values are 0-5): https://protobuf.dev/programming-guides/encoding/ (Edit: I guess it could be caused by a mismatch of a prior wiretype misinterpring another field)

I also notice the error looks like a Rust panic, so it's coming from the Namada node, not CometBFT. Any idea where that's happening or what Protobuf it's trying to parse? Perhaps use RUST_BACKTRACE=1?

@tony-iqlusion
Copy link
Member

Also, any idea when Namada will upgrade to CometBFT v0.38?

@tony-iqlusion
Copy link
Member

Please try TMKMS v0.14.0-pre.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants