From 702dc09264192dff0d6577666494069ab2adfe61 Mon Sep 17 00:00:00 2001 From: Elias Tazartes Date: Fri, 12 Jul 2024 11:53:37 +0200 Subject: [PATCH 1/2] fix readme --- README.md | 427 +++++++++++++++++++++++- p2p/diagrams/block_propagation.plantuml | 29 -- p2p/diagrams/chain_sync.plantuml | 22 -- p2p/diagrams/discovery.plantuml | 27 -- p2p/diagrams/state_sync.plantuml | 34 -- p2p/diagrams/txn_pool_sync.plantuml | 19 -- p2p/proto/notes.md | 22 -- p2p/proto/protocols.md | 274 --------------- p2p/starknet-p2p.md | 295 ---------------- 9 files changed, 423 insertions(+), 726 deletions(-) delete mode 100644 p2p/diagrams/block_propagation.plantuml delete mode 100644 p2p/diagrams/chain_sync.plantuml delete mode 100644 p2p/diagrams/discovery.plantuml delete mode 100644 p2p/diagrams/state_sync.plantuml delete mode 100644 p2p/diagrams/txn_pool_sync.plantuml delete mode 100644 p2p/proto/notes.md delete mode 100644 p2p/proto/protocols.md delete mode 100644 p2p/starknet-p2p.md diff --git a/README.md b/README.md index 45e6447..3083b41 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,426 @@ -# StarkNet P2P Specification +# Starknet P2P Protocol -This repo contains and tracks the specification of P2P protocol for StarkNet nodes. +## Content -The main entry point is the [here](./p2p/starknet-p2p.md). +1. [Overview](#overview) +1. [Core Definitions](#core-definitions) +1. [Starknet Protocols](#starknet-protocols) + 1. [Terminology](#terminology) + 1. [Protocols Briefing](#protocols-briefing) + 1. [Headers](#headers) + 1. [Transactions and Receipts](#transactions-and-receipts) + 1. [Events](#events) + 1. [State Diff](#state-diff) + 1. [Classes](#classes) -For a guide on the various protocols of Starknet and how to implement them, see [here](./p2p/proto/protocols.md) +--- + +## Overview + +This Node document aims to provide an overview and necessary details for implementing the P2P protocol for Starknet nodes. + +Familiarity with [Starknet](https://starknet.io/) is assumed. + +### Document Conventions + +Unless otherwise noted, the key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL", NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in This Node document are to be interpreted as described in [RFC 2119](https://datatracker.ietf.org/doc/html/rfc2119). + +## Core Definitions + +As a guiding principle, we leverage [libp2p](https://libp2p.io/) set of [protocols](https://github.com/libp2p/specs) whenever possible. This Node will let us standardize the protocol. It will also allow node implementations to leverage existing implementations of libp2p, or switch between them if necessary. + +As a design choice This Node results in some loss of generality, but seems like a reasonable given that libp2p is becoming a de-facto standard, and we see little value in re-implementing (and re-inventing) lower level network communication protocols. + +Specifically, we identify nodes and address them using [Peer IDs](https://github.com/libp2p/specs/blob/master/peer-ids/peer-ids.md), with key pairs derived using [Ed25519](https://github.com/libp2p/specs/blob/master/peer-ids/peer-ids.md#ed25519) scheme. + +Communication will be done over TCP transport. + +### Identifying Nodes + +An [`Identify` message](https://github.com/libp2p/specs/blob/master/identify/README.md) should include the following fields + +`protocolVersion` := `/starknet/` + version + +Nodes SHOULD use semantic version matching when matching protocols, as described [here](https://docs.libp2p.io/concepts/protocols/#protocol-negotiation). + +`agentVersion` is defined in the form of _agent-name/version_; e.g. `papyrus/0.1.0`. +Agent versions SHOULD follow [semantic versioning](http://semver.org) format. + +### Message Encoding + +Use protobuf (`proto3`) for message encoding/decoding between peers. +Specific message schemas to be defined for each protocol. + +Messages that tie into other messages in one flow, i.e. messages expecting a response and providing a response MUST include a `request_id` (a positive integer) that allows nodes to correlate messages in the same flow. +The request id, together with the sender id, should be enough for identifying the flow. + +--- + +## Starknet Protocols + +### Terminology + +`Request` - The message that is sent in order to request data from another node. + +`Response` - The message that is sent in response to a `Request` message and contains the requested data. + +`Querying peer` - The peer that sent the `Request` message. + +`Replying peer` - The peer that received the `Request` message. + +`Session` - All the messages related to responding to a `Request`. Including both the `Request` +message and all the `Response` messages. This term is used to distinguish between `Response` +messages from one `Request` to another. + +### Protocols Briefing + +The following table describes the different protocols in Starknet, the name that should be used in +negotiation, and the protobuf messages related to the protocol. +| Protocol | Name (for negotiation) | Request Message | Response Message | +| ------------ | -------------- | -------------- | -------------- | +| Headers | /starknet/headers/0.1.0-rc.0 | [BlockHeadersRequest](./header.proto) | [BlockHeadersResponse](./header.proto) | +| StateDiffs | /starknet/state_diffs/0.1.0-rc.0 | [StateDiffsRequest](./state.proto) | [StateDiffsResponse](./state.proto) | +| Classes | /starknet/classes/0.1.0-rc.0 | [ClassesRequest](./class.proto) | [ClassesResponse](./class.proto) | +| Transactions | /starknet/transactions/0.1.0-rc.0 | [TransactionsRequest](./transaction.proto) | [TransactionsResponse](./transaction.proto) | +| Events | /starknet/events/0.1.0-rc.0 | [EventsRequest](./event.proto) | [EventsResponse](./event.proto) | +| Kademlia (for discovery) | /starknet/kad//1.0.0 | + +In addition, nodes should also support the `Identify` protocol, who's name for negotiation is +`/ipfs/id/1.0.0` + +### Properties + +In each Starknet protocol (except for Kademlia and Identify), one peer sends a single request +message and the other peer responds with multiple response messages + +The request is always a range of blocks and the responses are data related to those blocks. + +This allows the querying peer to process data (e.g calculate hashes) before it downloaded all the +data that the replying peer sent in the session. + +The data sent is always a `oneof` of messages containing data and a [Fin](#fin). + +#### Partitioning data into blocks + +Except for the [headers](#headers) protocol, the partition of data into blocks does not appear in +the data itself. + +This partition can be done based on the data in the header. + +The block header contains a data commitment for each protocol. This commitment includes a hash and the number of elements. + +The block hash formula includes all commitments. Meaning that a block's commitments can be validated once we +downloaded the header. + +The length field can be used to determine in each protocol when does the data of one block stops and +the data of the next block after it begins. + +#### Length Prefix + +Each response message is prefixed with the amount of bytes that the encoded message contains. + +The length is written as a [varint](https://protobuf.dev/programming-guides/encoding/#varints) + +The request message is not prefixed with length. The querying peer should close its writer end once +it sends the request message (it's the only message it should send in the session). The replying peer +should read bytes until it reaches EOF and thus it doesn't need to know the length before reading +the message + +#### Limits + +- Each message must be less than 1MB, except for the [Class](./class.proto) message which is allowed + to be up to 4MB. +- TBD limits on the amount of total messages/bytes in a single session +- TBD timeout for a session +- TBD timeout between two messages + +#### Optional fields + +In `proto3`, each field that's a nested message is optional, and the `optional` keyword does nothing +for that field (See this [issue](https://github.com/protocolbuffers/protobuf/issues/249)). + +We want to mark nested fields as required, so every field that is not marked as `optional` is +considered required, even if it's nested. + +If a peer sends a message with a missing required field, they are considered malicious. + +#### Iteration + +The request message in each protocol contains an [Iteration](./common.proto). + +The Iteration message defines an ordered range of blocks. + +The Iteration message has the following fields: + +- **start**: The first block that we request, by hash or by number +- **direction**: Decides whether the blocks we send after the starting block are the blocks after it + or the blocks before it +- **limit**: The amount of blocks requested +- **step**: Given a step `i`, the responder will return every i'th block it sees until it reached `limit` blocks. + +For example. If the request has the values: + +- **start**: 10 +- **direction**: Backward +- **limit**: 3 +- **step**: 3 + +Then the blocks we'll receive will be blocks no. 10, 7, 4 + +#### Fin + +The [Fin](./common.proto) message is an empty message that signals the end of a protocol. +After sending all the data, the replying peer sends a Fin message. +If a replying peer sends any additional messages after sending a fin message, they are considered +malicious and the connection with them should be dropped. + +#### Missing Data + +We currently assume that if a peer has some of the data for a block then it has all the data for +that block. + +If the replying peer doesn't have all the data for all the blocks in the request, it should send +all the data for the blocks it has according to the order of the iteration and stop and send Fin +when it encountered the first missing block (even if it's the first block). + +#### Pre-0.13.2 blocks + +Starting from Starknet v0.13.2, the block hash covers all the data related to the block that a peer +can download from another peer + +The fields that are not in the block hash of blocks before v0.13.2 are: + +- State diff commitment and length +- Receipt commitment +- L1 data availability mode +- Gas and data gas prices +- In event commitment, the emmiting transaction + +This means that a peer cannot download those blocks from an untrusted peer, because +the replying peer can change the value of those fields and the signed block hash won't change. + +In order to solve this issue, we will add a file with all the hashes of blocks from all public +Starknet chains, where every hash is calculated with the v0.13.2 formula. + +### Protocols Breakdown + +#### Headers + +The Headers protocol is used to download block headers and signatures. + +Its name for negotiation is `/starknet/headers/0.1.0-rc.0` + +Each single message is a fin or a [SignedBlockHeader](./header.proto) + +A header is comprised of: + +- block hash +- hash of the parent block +- block number +- metadata for the block (timestamp, sequencer address, protocol version, gas prices) +- Commitments on the data of each of the other protocols. Each commitment is comprised of length and + hash. For more details see the [block hash description](https://docs.google.com/document/d/1EIlHskVJEyztS8eXRyPzd8cZwGuPcIKR5xUAIawzryk/edit) + +There's a single message for each block. That's because we assume that there will never be a header +that's bigger than the [message size limit](#limits), even when including all signatures for this +header. + +```mermaid +sequenceDiagram + participant Querying Peer + participant Replying Peer + Querying Peer->>Replying Peer: BlockHeadersRequest + loop For each block in range + Replying Peer->>Querying Peer: SignedBlockHeader + end + Replying Peer->>Querying Peer: Fin +``` + +#### Transactions and Receipts + +The transactions protocol is used to download the transactions and receipts in a range of blocks. + +Its name for negotiation is `/starknet/transactions/0.1.0-rc.0` + +Each single message is either a fin, or a [TransactionWithReceipt](./transaction.proto) +(A pair of [Transaction](./transaction.proto) and [Receipt](./receipt.proto)) + +Each transaction represents a Starknet transaction. For more detail on the different transaction +types, their content and their hash calculation see [here](https://docs.starknet.io/documentation/architecture_and_concepts/Network_Architecture/transactions/). + +A receipt is comprised of + +- The hash of the transaction this receipt belongs to +- The actual fee paid for this transaction and the price unit used for the payment +- The messages to l1 this transaction generated. +- The execution resources this transaction took. Comprised of Cairo resources and data availability + resources +- The revert reason as a string if this transaction was reverted. If the `revert_reason` field is + missing, it means that this transaction was successful + +The transactions and receipts need to be sent according to the transaction execution order. + +The commitment of the transactions in the header is comprised of the number of transactions and +their merkle root. + +In order to verify the transactions, the node needs to calculate their hash and compare it to the +transactions merkle root in the block header. + +The same is true for receipts. They also have a commitment in the header. + +The number can be used to delimiter between the transactions and receipts of each block. + +In order to avoid potential DDoS, the querying peer should reject the connection and declare the +replying peer malicious once it sent more transactions with receipts than the sum of transaction +amounts in the headers of the requested blocks. + +```mermaid +sequenceDiagram + participant Querying Peer + participant Replying Peer + Querying Peer->>Replying Peer: TransactionsRequest + loop For each transaction in range + Replying Peer->>Querying Peer: TransactionWithReceipt + end + Replying Peer->>Querying Peer: Fin + +``` + +#### Events + +The Events protocol is used to download the events emitted in a range of blocks. + +Its name for negotiation is `/starknet/events/0.1.0-rc.0` + +Each single message is a fin or an [Event](./event.proto) + +Each event contains + +- The hash of the transaction that emitted this event +- The address of the contract that emitted this event. +- The keys and data of this event. + +The commitment of the events in the header is comprised of the number of events and +their merkle root. + +In order to verify the events, the node needs to calculate their hash and compare it to the +events merkle root in the block header. + +The number can be used to delimiter between the events of each block, and the transaction index field +of each event is used to delimiter between the events of each transaction within a block. + +In order to avoid potential DDoS, the querying peer should reject the connection and declare the +replying peer malicious once it sent more events than the sum of event amounts in the headers of the +requested blocks. + +```mermaid + +sequenceDiagram + participant Querying Peer + participant Replying Peer + Querying Peer->>Replying Peer: EventsRequest + loop For each event in range + Replying Peer->>Querying Peer: Event + end + Replying Peer->>Querying Peer: Fin +``` + +#### State Diff + +The state diff protocol is used to download the state diffs created by a range of blocks. +It can be used to avoid running the transactions for computing the state diff. + +Its name for negotiation is `/starknet/state_diffs/0.1.0-rc.0` + +In order to verify the state diff, the node needs to calculate its hash and compare it to the +`state_diff_commitment` field in the block header. The structure of the hash is detailed [here](https://community.starknet.io/t/introducing-p2p-authentication-and-mismatch-resolution-in-v0-12-2/97993). + +```mermaid +sequenceDiagram + participant Querying Peer + participant Replying Peer + Querying Peer->>Replying Peer: StateDiffsRequest + loop For each contract diff or declared class in range + alt ContractDiff + Replying Peer->>Querying Peer: ContractDiff + else DeclaredClass + Replying Peer->>Querying Peer: DeclaredClass + end + end + Replying Peer->>Querying Peer: Fin + +``` + +##### Message structure + +Each single message is either a fin, a [ContractDiff](./state.proto) or a [DeclaredClass](./state.proto). + +A ContractDiff represents all the changes made for a given contract. This includes: + +- Storage diffs - Represented by the `values` field. +- Nonce updates - Represented by the `nonce` field if present. +- Deployed contracts - Represented by the `class_hash` field if present. If it's present it means + that this contract was deployed or replaced in this block. + +A DeclaredClass represents a declared class. If the class is Cairo1 then the `compiled_class_hash` +field will be present + +There's no guarantee on what ordering will the messages have for a single block's state diff. +The only constraint is that each `ContractDiff` can't be empty, i.e one of the following fields have +value: + +- `nonce` +- `class_hash` +- `values` + +##### Length + +As we've seen in other protocols like [Transactions](#transactions and receipts), The node needs to know the +amount of data to expect in order to reject malicious peers trying to DDoS the node. + +In order to do that, the node needs to look at the block header at the field `state_diff_length`. +This field is equal to +`num_storage_diffs + num_nonce_updates + num_deployed_contracts + num_declared_classes` + +Then, whenever the node receives a [StateDiffResponse](./state.proto), it needs to add a number to +a counter and declare the peer malicious if the counter becomes bigger than `state_diff_length`: + +- For DeclaredClass, the counter should increase by 1. +- For ContractDiff, the counter should increase by the length of the field `values`, + and by 1 if the field `class_hash` is present, and by 1 if the field `nonce` is present. + +#### Classes + +The classes protocol is used to download the classes declared in a range of blocks. + +Its name for negotiation is `/starknet/classes/0.1.0-rc.0` + +Each single message is a fin or a [Class](./class.proto). For more information on classes, see +[here](https://docs.starknet.io/documentation/architecture_and_concepts/Smart_Contracts/contract-classes/) + +You should use the [state diff protocol](#state-diff) before using this protocol. +The reason is that the class hashes and amount of declared classes per block are part of the +`state_diff_commitment` in the header, and these values are needed to + +1. Validate the results of the classes protocol. +2. Delimiter between the classes of each block. +3. Reject a replying peer DDoSing us if it sent more classes than the amount of declared classes in + the block range. + +The fact that each class is a single message and each message is up to 4MB gives us a protection +from DDoS the same way other protocols gain protection with a length field. + +```mermaid +sequenceDiagram + participant Querying Peer + participant Replying Peer + Querying Peer->>Replying Peer: ClassesRequest + loop For each class in range + Replying Peer->>Querying Peer: Class + end + Replying Peer->>Querying Peer: Fin +``` + +#### Kademlia + +> TODO diff --git a/p2p/diagrams/block_propagation.plantuml b/p2p/diagrams/block_propagation.plantuml deleted file mode 100644 index 5c08244..0000000 --- a/p2p/diagrams/block_propagation.plantuml +++ /dev/null @@ -1,29 +0,0 @@ -@startuml - -participant "Block Originator" as orig -collections "/blocks subscribers" as block_topic -participant "This Node" as this #lightblue - - -par - orig -> block_topic: NewBlockHeader(header) - block_topic -> this: - this -> this: validate header - alt invalid header - this -> this: update invalid header (originator) - note right - a chance for a node - to mark a peer as problematic - end note - end -else - orig -> block_topic: NewBlockBody(body) - block_topic -> this: - this -> this: validate and\nupdate body -else - orig -> block_topic: NewStateDiff(state_diff) - block_topic -> this: - this -> this: validate and\nupdate state -end - -@enduml diff --git a/p2p/diagrams/chain_sync.plantuml b/p2p/diagrams/chain_sync.plantuml deleted file mode 100644 index 8e2092c..0000000 --- a/p2p/diagrams/chain_sync.plantuml +++ /dev/null @@ -1,22 +0,0 @@ - -@startuml - -participant "This Node" as this #lightblue -participant "Peer" as peer - -this -> peer: GetBlockHeaders(start_block,limit,direction) -note right - start_block: starting block number - limit: how many, at most, headers to retrieve - direction: retrieve direction - forward/backward -end note -peer -> this: BlockHeaders(...) - -par get bodies - this -> peer: GetBlockBodies(start_block,limit,direction) - peer -> this: BlockBodies(...) -else get state - this -> peer: GetStateDiffs(start_block,limit,direction) - peer -> this: StateDiffs(...) -end -@enduml diff --git a/p2p/diagrams/discovery.plantuml b/p2p/diagrams/discovery.plantuml deleted file mode 100644 index e35eb17..0000000 --- a/p2p/diagrams/discovery.plantuml +++ /dev/null @@ -1,27 +0,0 @@ - -@startuml - -participant "NewNode" as nn -collections "`_starknet_nodes/` subscribers" as block_topic -collections "Peers" as peers - - -nn -> block_topic : NewNode(me) -block_topic -> peers: NewNode - -alt valid message & needs to connect - peers -> nn: handshake - nn --> peers: -else valid message - peers -> peers: keep new node -end - -loop every interval - note over block_topic, peers - republish every interval - end note - peers -> block_topic: KnownNodes() - block_topic -> peers: KnownNodes() -end - -@enduml diff --git a/p2p/diagrams/state_sync.plantuml b/p2p/diagrams/state_sync.plantuml deleted file mode 100644 index b3f9d6c..0000000 --- a/p2p/diagrams/state_sync.plantuml +++ /dev/null @@ -1,34 +0,0 @@ - -participant "This Node" as this #lightblue -participant "Peer" as peer - -group sync headers - note over this, peer - as described under "Chain Synchronization" - end note - - this -> peer: - peer --> this: -end - -loop while full state not received - this -> peer: GetSnapshotStateChunk(chunk_number) - peer -> this: StateSnapshotChunk(block_num,chunk_number, state_values, proof) - this -> this: verify proof and apply chunk - - this -> peer: GetClassSnapshotChunk(chunk_number) - note right: can also get class declarations - - peer -> this: ClassSnapshotChunk(block_num, chunk_number, classes, proof) - - this -> this: apply class declarations - - loop for remaining blocks from `block_number` to tip - this -> peer: GetStateDiffs(start_block,limit,direction) - peer -> this: StateDiffs(...) - this -> this: apply state diffs - end - note right - Apply state diffs up to tip for each chunk - end note -end diff --git a/p2p/diagrams/txn_pool_sync.plantuml b/p2p/diagrams/txn_pool_sync.plantuml deleted file mode 100644 index e844540..0000000 --- a/p2p/diagrams/txn_pool_sync.plantuml +++ /dev/null @@ -1,19 +0,0 @@ -@startuml - -participant "This Node" as this #lightblue -participant "Peer" as peer - -note over this,peer: all nodes support transaction propagation - - -this -> peer: h1: KnownPooledTransactions(hashes) -this <- peer: h2: KnownPooledTransactions(hashes) - -this -> peer: GetPooledTransactions(h2 \ h1) -note right of peer: Same flow the other way.\nNot shown for clarity -this <-- peer: PooledTransactions - -this -> this: validate and add transactions to pool - - -@enduml diff --git a/p2p/proto/notes.md b/p2p/proto/notes.md deleted file mode 100644 index f9c87b7..0000000 --- a/p2p/proto/notes.md +++ /dev/null @@ -1,22 +0,0 @@ -* Goals: consensus, scale, validator-fullnode separation, cleanup -* whenever 'bytes' is used to encode large numbers (felts), big endian is used, without storing leading zeros in the number as bytes (so 0x11 is stored as [0x11] not [0x00, 0x00,...,0x11] -* requests are responded to with a stream of messages that are varint message delimited. -* When a stream returnes several logical objects, each in several messages, then each object messages should end with a Fin message -* Responses to events, receipts and transactions also include block hash for cases of reorg. -* request_id is handled at the protocol level, specifically to support multiplexing -* number of messages (especially repeated ones) is capped for ddos -* Getting transactions/events is separate from getting blocks -* Protocols: blocks (headers+bodies or separate?), transactions, events, receipts, mempool. States? fastsync? -* Using Merkle trees so can prove a range of values (in the future). Also, a light node can decommit individual values (e.g. events) -** TBD: can save on the internal merkle nodes by having one where a leaf is the tx data, events and receipts, each hashed separately. But then getting a part of a leaf will require sending the other parts' hashes. -** Events are separate from receipt -* Proof and signatures are required to be returned only for blocks after the last l1 state update -* TBD: consensus messages (tx streaming, proofs separate) -* TBD: reverted transactions -* TBD: stark friendly hashes or not (calculate in os? so light nodes don't trust the consensus) -* TBD: "pubsub" mechanism where a node subscribes to getting push messages instead of GetXXX. -** GetXXX messages will need to be supported for network issues / failures -* Consensus: send (Transactions*, StateDiff*)* Proposal. The state diffs allows paralelization as the transactions following it can be executed from that state in parallel to whatever is already executing -* Assuming L1 state update will have block header hash and number. -* tx calldata limited to 30K - diff --git a/p2p/proto/protocols.md b/p2p/proto/protocols.md deleted file mode 100644 index d8a3758..0000000 --- a/p2p/proto/protocols.md +++ /dev/null @@ -1,274 +0,0 @@ -# Starknet Protocols - -## Terminology -`Request` - The message that is sent in order to request data from another node. - -`Response` - The message that is sent in response to a `Request` message and contains the requested data. - -`Querying peer` - The peer that sent the `Request` message. - -`Replying peer` - The peer that received the `Request` message. - -`Session` - All the messages related to responding to a `Request`. Including both the `Request` -message and all the `Response` messages. This term is used to distinguish between `Response` -messages from one `Request` to another. - -## Protocols Briefing -The following table describes the different protocols in Starknet, the name that should be used in -negotiation, and the protobuf messages related to the protocol. -| Protocol | Name (for negotiation) | Request Message | Response Message | -| ------------ | -------------- | -------------- | -------------- | -| Headers | /starknet/headers/0.1.0-rc.0 | [BlockHeadersRequest](./header.proto) | [BlockHeadersResponse](./header.proto) | -| StateDiffs | /starknet/state_diffs/0.1.0-rc.0 | [StateDiffsRequest](./state.proto) | [StateDiffsResponse](./state.proto) | -| Classes | /starknet/classes/0.1.0-rc.0 | [ClassesRequest](./class.proto) | [ClassesResponse](./class.proto) | -| Transactions | /starknet/transactions/0.1.0-rc.0 | [TransactionsRequest](./transaction.proto) | [TransactionsResponse](./transaction.proto) | -| Events | /starknet/events/0.1.0-rc.0 | [EventsRequest](./event.proto) | [EventsResponse](./event.proto) | -| Kademlia (for discovery) | /starknet/kad//1.0.0 | - -In addition, nodes should also support the `Identify` protocol, who's name for negotiation is -`/ipfs/id/1.0.0` - -## Overview -In each Starknet protocol (except for Kademlia and Identify), one peer sends a single request -message and the other peer responds with multiple response messages - -The request is always a range of blocks and the responses are data related to those blocks. - -This allows the querying peer to process data (e.g calculate hashes) before it downloaded all the -data that the replying peer sent in the session. - -The data sent is always a `oneof` of messages containing data and a [Fin](#fin). - -### Partitioning data into blocks -Except for the [headers](#headers) protocol, The partition of data into blocks does not appear in -the data itself. - -This partition can be done based on the data in the header. - -For each protocol, in the header, there's a commitment on the data given by this protocol. -The commitment is made of hash and number of elements - -The commitment is included inside the block hash. Meaning that it can be validated once we -downloaded the header. - -The length field can be used to determine in each protocol when does the data of one block stops and -the data of the next block after it begins. - -### Length Prefix -Each response message is prefixed with the amount of bytes that the encoded message contains. - -The length is written as a [varint](https://protobuf.dev/programming-guides/encoding/#varints) - -The request message is not prefixed with length. The querying peer should close its writer end once -it sends the request message (it's the only message it should send in the session). The replying peer -should read bytes until it reaches EOF and thus it doesn't need to know the length before reading -the message - -### Limits -- Each message must be less than 1MB, except for the [Class](./class.proto) message which is allowed -to be up to 4MB. -- TBD limits on the amount of total messages/bytes in a single session -- TBD timeout for a session -- TBD timeout between two messages - -### Optional fields -In `proto3`, each field that's a nested message is optional, and the `optional` keyword does nothing -for that field (See this [issue](https://github.com/protocolbuffers/protobuf/issues/249)). - -We want to mark nested fields as required, so every field that is not marked as `optional` is -considered required, even if it's nested. - -If a peer sends a message with a missing required field, they are considered malicious. - -### Iteration -The request message in each protocol contains an [Iteration](./common.proto). - -The Iteration message defines an ordered range of blocks. - -The Iteration message has the following fields: -- **start**: The first block that we request, by hash or by number -- **direction**: Decides whether the blocks we send after the starting block are the blocks after it -or the blocks before it -- **limit**: The amount of blocks requested -- **step**: Given a step `i`, the responder will return every i'th block it sees until it reached `limit` blocks. - -For example. If the request has the values: -- **start**: 10 -- **direction**: Backward -- **limit**: 3 -- **step**: 3 - -Then the blocks we'll receive will be blocks no. 10, 7, 4 - -### Fin -The [Fin](./common.proto) message is an empty message that signals the end of a protocol. -After sending all the data, the replying peer sends a Fin message. -If a replying peer sends any additional messages after sending a fin message, they are considered -malicious and the connection with them should be dropped. - -### Missing Data -We currently assume that if a peer has some of the data for a block then it has all the data for -that block. - -If the replying peer doesn't have all the data for all the blocks in the request, it should send -all the data for the blocks it has according to the order of the iteration and stop and send Fin -when it encountered the first missing block (even if it's the first block). - -### Pre-0.13.2 blocks -Starting from Starknet v0.13.2, the block hash covers all the data related to the block that a peer -can download from another peer - -The fields that are not in the block hash of blocks before v0.13.2 are: -* State diff commitment and length -* Receipt commitment -* L1 data availability mode -* Gas and data gas prices -* In event commitment, the emmiting transaction - -This means that a peer cannot download those blocks from an untrusted peer, because -the replying peer can change the value of those fields and the signed block hash won't change. - -In order to solve this issue, we will add a file with all the hashes of blocks from all public -Starknet chains, where every hash is calculated with the v0.13.2 formula. - -## Protocols Breakdown -### Headers -The Headers protocol is used to download block headers and signatures. - -Its name for negotiation is `/starknet/headers/0.1.0-rc.0` - -Each single message is a fin or a [SignedBlockHeader](./header.proto) - -A header is comprised of: -* block hash -* hash of the parent block -* block number -* metadata for the block (timestamp, sequencer address, protocol version, gas prices) -* Commitments on the data of each of the other protocols. Each commitment is comprised of length and -hash. For more details see the [block hash description](https://docs.google.com/document/d/1EIlHskVJEyztS8eXRyPzd8cZwGuPcIKR5xUAIawzryk/edit) - -There's a single message for each block. That's because we assume that there will never be a header -that's bigger than the [message size limit](#limits), even when including all signatures for this -header. - -### Transactions and Receipts -The transactions protocol is used to download the transactions and receipts in a range of blocks. - -Its name for negotiation is `/starknet/transactions/0.1.0-rc.0` - -Each single message is either a fin, or a [TransactionWithReceipt](./transaction.proto) -(A pair of [Transaction](./transaction.proto) and [Receipt](./receipt.proto)) - -Each transaction represents a Starknet transaction. For more detail on the different transaction -types, their content and their hash calculation see [here](https://docs.starknet.io/documentation/architecture_and_concepts/Network_Architecture/transactions/). - -A receipt is comprised of -* The hash of the transaction this receipt belongs to -* The actual fee paid for this transaction and the price unit used for the payment -* The messages to l1 this transaction generated. -* The execution resources this transaction took. Comprised of Cairo resources and data availability -resources -* The revert reason as a string if this transaction was reverted. If the `revert_reason` field is -missing, it means that this transaction was successful - -The transactions and receipts need to be sent according to the transaction execution order. - -The commitment of the transactions in the header is comprised of the number of transactions and -their merkle root. - -In order to verify the transactions, the node needs to calculate their hash and compare it to the -transactions merkle root in the block header. - -The same is true for receipts. They also have a commitment in the header. - -The number can be used to delimiter between the transactions and receipts of each block. - -In order to avoid potential DDoS, the querying peer should reject the connection and declare the -replying peer malicious once it sent more transactions with receipts than the sum of transaction -amounts in the headers of the requested blocks. - -### Events -The Events protocol is used to download the events emitted in a range of blocks. - -Its name for negotiation is `/starknet/events/0.1.0-rc.0` - -Each single message is a fin or an [Event](./event.proto) - -Each event contains -* The hash of the transaction that emitted this event -* The address of the contract that emitted this event. -* The keys and data of this event. - -The commitment of the events in the header is comprised of the number of events and -their merkle root. - -In order to verify the events, the node needs to calculate their hash and compare it to the -events merkle root in the block header. - -The number can be used to delimiter between the events of each block, and the transaction index field -of each event is used to delimiter between the events of each transaction within a block. - -In order to avoid potential DDoS, the querying peer should reject the connection and declare the -replying peer malicious once it sent more events than the sum of event amounts in the headers of the -requested blocks. - -### State Diff -The state diff protocol is used to download the state diffs created by a range of blocks. -It can be used to avoid running the transactions for computing the state diff. - -Its name for negotiation is `/starknet/state_diffs/0.1.0-rc.0` - -In order to verify the state diff, the node needs to calculate its hash and compare it to the -`state_diff_commitment` field in the block header. The structure of the hash is detailed [here](https://community.starknet.io/t/introducing-p2p-authentication-and-mismatch-resolution-in-v0-12-2/97993) - -#### Message structure -Each single message is either a fin, a [ContractDiff](./state.proto) or a [DeclaredClass](./state.proto). - -A ContractDiff represents all the changes made for a given contract. This includes: -* Storage diffs - Represented by the `values` field. -* Nonce updates - Represented by the `nonce` field if present. -* Deployed contracts - Represented by the `class_hash` field if present. If it's present it means -that this contract was deployed or replaced in this block. - -A DeclaredClass represents a declared class. If the class is Cairo1 then the `compiled_class_hash` -field will be present - -There's no guarantee on what ordering will the messages have for a single block's state diff. -The only constraint is that each `ContractDiff` can't be empty, i.e one of the following fields have -value: -* `nonce` -* `class_hash` -* `values` - -#### Length -As we've seen in other protocols like [Transactions](#transactions and receipts), The node needs to know the -amount of data to expect in order to reject malicious peers trying to DDoS the node. - -In order to do that, the node needs to look at the block header at the field `state_diff_length`. -This field is equal to -`num_storage_diffs + num_nonce_updates + num_deployed_contracts + num_declared_classes` - -Then, whenever the node receives a [StateDiffResponse](./state.proto), it needs to add a number to -a counter and declare the peer malicious if the counter becomes bigger than `state_diff_length`: -* For DeclaredClass, the counter should increase by 1. -* For ContractDiff, the counter should increase by the length of the field `values`, -and by 1 if the field `class_hash` is present, and by 1 if the field `nonce` is present. - -### Classes -The classes protocol is used to download the classes declared in a range of blocks. - -Its name for negotiation is `/starknet/classes/0.1.0-rc.0` - -Each single message is a fin or a [Class](./class.proto). For more information on classes, see -[here](https://docs.starknet.io/documentation/architecture_and_concepts/Smart_Contracts/contract-classes/) - -You should use the [state diff protocol](#state-diff) before using this protocol. -The reason is that the class hashes and amount of declared classes per block are part of the -`state_diff_commitment` in the header, and these values are needed to -1. Validate the results of the classes protocol. -2. Delimiter between the classes of each block. -3. Reject a replying peer DDoSing us if it sent more classes than the amount of declared classes in -the block range. - -The fact that each class is a single message and each message is up to 4MB gives us a protection -from DDoS the same way other protocols gain protection with a length field. diff --git a/p2p/starknet-p2p.md b/p2p/starknet-p2p.md deleted file mode 100644 index 8258375..0000000 --- a/p2p/starknet-p2p.md +++ /dev/null @@ -1,295 +0,0 @@ - - -# Starknet P2P Protocol - -**Content** - -1. [Overview](#overview) -2. [Core Definitions](#core-definitions) -3. [Protocols](#protocols) - 1. [Common Types](#common-types) - 2. [Discovery](#discovery) - 3. [Block Propagation](#block-propagation) - 4. [Synchronization](#synchronization) - 5. [Transaction Propagation](#transaction-propagation) - ----- -## Overview - -This document aims to provide an overview and necessary details for implementing the P2P protocol for Starknet nodes. - -Familiarity with [Starknet](https://starknet.io/) is assumed. - -### Document Conventions - -Unless otherwise noted, the key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL", NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC 2119](https://datatracker.ietf.org/doc/html/rfc2119). - -## Core Definitions - -As a guiding principle, we leverage [libp2p](https://libp2p.io/) set of [protocols](https://github.com/libp2p/specs) whenever possible. This will let us standardize the protocol. It will also allow node implementations to leverage existing implementations of libp2p, or switch between them if necessary. - -As a design choice this results in some loss of generality, but seems like a reasonable given that libp2p is becoming a de-facto standard, and we see little value in re-implementing (and re-inventing) lower level network communication protocols. - -Specifically, we identify nodes and address them using [Peer IDs](https://github.com/libp2p/specs/blob/master/peer-ids/peer-ids.md), with key pairs derived using [Ed25519](https://github.com/libp2p/specs/blob/master/peer-ids/peer-ids.md#ed25519) scheme. - -Communication will be done over TCP transport. - -### Identifying Nodes - -An [`Identify` message](https://github.com/libp2p/specs/blob/master/identify/README.md) should include the following fields - -`protocolVersion` := `/starknet/` + version - -Nodes SHOULD use semantic version matching when matching protocols, as described [here](https://docs.libp2p.io/concepts/protocols/#protocol-negotiation). - -`agentVersion` is defined in the form of _agent-name/version_; e.g. `papyrus/0.1.0`. -Agent versions SHOULD follow [semantic versioning](http://semver.org) format. - -### Message Encoding - -Use protobuf (`proto3`) for message encoding/decoding between peers. -Specific message schemas to be defined for each protocol. - -Messages that tie into other messages in one flow, i.e. messages expecting a response and providing a response MUST include a `request_id` (a positive integer) that allows nodes to correlate messages in the same flow. -The request id, together with the sender id, should be enough for identifying the flow. - ----- - - -## Protocols - -Different flows, namely block propagation, synchronization and transaction propagation, are specified below, on top of lower level network protocols (handshakes, pubsub, etc.). Each section below specifies the flow and the relevant messages passed between different agents. - - -### Common Types - -Common message types, relevant to several protocols are in [common.proto](./proto/common.proto). - -Peer IDs should be displayed and accepted using content identifier (CID) encoding, described [here](https://github.com/libp2p/specs/blob/master/peer-ids/peer-ids.md#peer-ids). - ----- - -### Discovery - -Discovery is the process of joining the network and discovering other relevant peer nodes that are part of the relevant chain. - -Simple protocol for discovery, based on pub sub implementation (similar to [pubsub-based discovery](https://github.com/libp2p/js-libp2p-pubsub-peer-discovery/)): - -![discovery](https://user-images.githubusercontent.com/77265175/189590881-e31ad7d0-be38-49d6-b284-29030853a452.png) - -1. A node connects to at least one node of the network - - Using either [rendezvous protocol](https://github.com/libp2p/specs/tree/master/rendezvous#overview), or bootstrap nodes. - - Rendezvous point: `_starknet_discover/` + configured chain id. - - The bootstrap nodes should allow connecting to the pubsub mechanism (either directly, or providing another nodes that does). - - -2. A node publishes a `NewNode` message to a predefined topic - - The new node message should be signed and identifiable for the node publishing it (based on PeerID) - a node can only publish a `NewNode` message for itself. - - Topic: `_starknet_nodes/` + configured chain id. - - ==Can have subtopics for specific subnets, e.g. discover archive nodes== - -3. Subscribers receive the node, and continue to handshake if necessary. -4. A node re-publishes its known nodes (with ids) every predefined interval. - - - -The chain id is the [starknet chain id](https://docs.starknet.io/documentation/architecture_and_concepts/Blocks/transactions/#chain-id) configured for the node. - -Note that while technically a node can serve both as a full node and a bootstrap node, this isn't recommended, as bootstrap nodes generally need to be available to serve their discovery role. - - -
-OPEN: how to choose peers to prevent eclipse (and other) attacks -
- - - -#### Messages - -Messages are defined [here](./proto/discovery.proto). - -Every node defines a set of capabilities it supports. -Capabilities are represented as strings, with a few capabilities defined as part of the core protocol. -Some capabilities may be parameterized, the exact serialization of the capabilities and their parameters depends on the definition of the capability. - -#### Capabilities - -Capabilities are advertised by nodes, providing information on services these node execute in the network. This is similar in spirit to [discv5 topics](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-theory.md#topic-table) or the [protocols advertised as part of the libp2p identify protocol](https://github.com/libp2p/specs/tree/master/identify#protocols). -These can be used by nodes in their decision to connect (or remain connected) to other peers. - -For the sake of future compatibility and extensions, the Starknet protocol does not enumerate a mandatory list of capabilities. We do define here a scheme for encoding capabilities, and list some core capabilities likely to be used by protocols. - -##### String Encoding - -Capabilities are denoted by identifiers consisting of simple alphanumeric characters (`a-zA-Z0-9`) and underscore (`_`) and hyphen characters (`-`), in ASCII. - -Identifiers can be concatenated using the slashes (`/`), and followed by a version number. This allows for qualifications ("namespaces") and versioning of provided capabilities. - -For example: the capability `core/state/1` provided by a node states that it provides the core state querying capability (its 1st version). -The exact semantics of the capability and its version is to be documented as part of the definition of the capability. The capability usually translates into the protocols it supports and/or API it provides to peers. - - -##### Core Capabilities - -We list here core capabilities that SHOULD be exposed by nodes in the network. - - - -| Capbility | Description | -| -------------- | -------- | -| `core/blocks-sync/1` | The node provides information about past blocks and corresponding transactions | -| `core/state-sync/1` | The node provides state snapshot synchronization capability that can help with state synchronization | -| `core/block-propagate/1` | The node participates in block propagation | -| `core/txn-propagate/1` | The node participates in transaction propagation | - - - ----- -### Block Propagation - -The block propagation flow is the flow where nodes propagate information on newly created blocks. - -The block originator (the one publishing the block) publishes the necessary information to a well-known topic, with other nodes subsribed to that topic, and receiving the information. -The pubsub implementation will take care of publishing the information to available nodes. - -![block_propagation](https://user-images.githubusercontent.com/77265175/189591198-f855f04d-57c4-4faa-87b2-6aa1c43868ee.png) - -Of course, in this case, invalid body or header will invalidate the block (and the node should mark it as problematic.). - -A node SHOULD disregard body and and state diffs for blocks with invalid headers. - -The topic used to publish new blocks will be: `blocks/` + configured chain id. -Where the configured chain id is the [starknet chain id](https://docs.starknet.io/documentation/architecture_and_concepts/Blocks/transactions/#chain-id) configured for the node. -Nodes should subscribe to the topic on initialization. - -#### Validation - -Whenever a new block is received, the block header and body should be validated for consistency. -The validation MUST include: -1. Block number is as expected - consecutive to the parent block. -2. The parent block hash matches the current known tip. -3. Transactions and events in the body must match the commitment in the header. -4. State diff, applied to current state, is consistent with the new state root. - -Note that this assumes blocks received are blocks that are agreed by the consensus, or the node has some way to validate that the block received is agreed by the consensus mechanism. - - -#### Messages - -Messages specification is [here](./proto/propagation.proto) - -Where contract class hash is defined [here](https://docs.starknet.io/documentation/architecture_and_concepts/Contracts/class-hash/) and contract address is as defined [here](https://docs.starknet.io/documentation/architecture_and_concepts/Contracts/contract-address/). - ----- - -### Synchronization - -A full node is expected to synchronize on the current state of the network. -Full state synchronization includes the block headers and bodies (transaction and events) + the current available state at the highest block in the consensus. - - -#### Chain Synchronization - -Chain synchronization is the process of downloading (and verifying) the necessary block information. -The synchronizing node can choose to connect to several peers and ask to get ranges of data. - -
-OPEN: Is it possible that a given node does not support sync? i.e. do we need to verify support of synching in the handshake? -
- -![chain_sync](https://user-images.githubusercontent.com/77265175/189591383-34af1b79-5fcf-440f-8216-e0b420dca130.png) - -The node asks for a _range_ of block headers and bodies. Relevant responses should be of consecutive headers/bodies starting from the request block number and working in the direction requested. -This should allow nodes to ask several peers for different ranges at the same time, as well work however they wish (from the tip backward, or from some point forward) to fill the necessary state. - -In addition, a requesting node can limit responses by size. So responses must be up to the requested size limit. -A peer responding to a message MUST adhere to both limits, i.e. the message MUST NOT violate any of the upper limites. - - -##### Messages - -Messages are defined [here](./proto/sync.proto) - - -
OPEN: allow state diffs to be retrieved in parts?
- - -##### Data Validation - -Validation of the block bodies is the same as defined for [receiving a new block](#Block-Propagation). - - -#### State Synchronization - -State synchronization allows a node to synchronize the up-to-date state of the network, without having to run the transactions or state diffs one at a time for most of the chain. We assume headers are available, as part of the synchronization protocol. Block bodies are not required for state synchronization. - -A node chooses peers to request their latest state snapshot. Peers reply with a subset of the state, and include in the response the block this state is coming from. Part of the response is the block the state is relevant for. Each request is for a _consecutive_ range of storage keys to be retrieved, and the response includes a merkle proof of the retrieved chunk. - -The synchronizing node receives different pieces of the state and "stitches" them together. Note that different peers may respond with state from different blocks. The synchronizing node proceeds from the point of the chain it received (per chunk) forward to the tip, by asking for state diffs, like in the chain synchronization case. - -The size of returned chunks is deterministic and known in advance as a chain-wide parameter. - -![state_sync](https://user-images.githubusercontent.com/77265175/189591568-856067e9-ff66-4555-bb1d-9dd13293fe39.png) - - -The node asks different peers (denoted by `peer` above) for different parts of the snapshot. - -Note that we also have seperate requests for the class declarations, which are also part of the state. Although the request/response is similar, this is defined as a separate set of messages since the proof is a bit different (a different height of merkle tree). It also allows a node to optimize this storate of classes separately from the full contract state. - -Of course, different chunks (of state and class declarations) can be requested from separate peers. - - - -##### Messages - -Messages are specified [here](./proto/state_sync.proto). - -For each diff, the state chunk should provide the merkle path to the state root. - -### Transaction Propagation - -The goal of the transaction propagation protocol is to allow pending transactions to be disseminated -in the network so sequencers can pick them up and add them to blocks. - -A participating node should track the pending transactions it recognizes - a transaction pool. -Transactions are added to this pool when a client submits a new transaction directly to the node, -or when a peer node shares a new transaction (or set of transactions) that it knows of. -Transactions are removed from the pool when some sequencer includes them in a block. - -#### Adding a New Transaction - -When a transaction is submitted to a node, e.g. through [the node API](https://github.com/starkware-libs/starknet-specs/blob/e9415a91dbf08b480fc05bb985d7cf8a93a898bc/api/starknet_write_api.json#L11), -the node should transmit the new transaction to peers. Notification of the new transaction is done using pub-sub mechanism, using a topic known a-priori (similar to block propagation). - -The topic used to notify of new transactions will be `transaction_pool/` + configured chain id. -Where the configured chain id is the [starknet chain id](https://docs.starknet.io/documentation/architecture_and_concepts/Blocks/transactions/#chain-id) configured for the node. -Nodes that maintain a transaction pool should subscribe to the topic on initialization. - -The message transmitted should include the complete new transaction. -A receiving node should validate the transaction, using the same validation used for transactions submitted through the API. Invalid transactions should be discarded. - -The message itself should be the [`Transaction` message](https://github.com/starknet-community-libs/starknet-p2p-specs/blob/d46a92040ba2cc1ceeb3c8f36b10403b345e6636/p2p/proto/common.proto#L68). - -#### Synchronizing Transaction Pools - -![txn_pool_sync](https://user-images.githubusercontent.com/77265175/193442622-7d64bf77-a2fa-4917-97f6-a7791ec99ad4.png) - -The messages described in this section are specified [here](./proto/pool_sync.proto). - -When a new node (re-)joins the network, it should synchronize its transaction pool as well. -The node should connect to peers that support transaction propagation capability, i.e. maintain transaction pools. -At this point it should send and receive the transaction hashes it currently maintains in the pool - the `KnownPooledTransactions` message. This message contains the _hashes_ of the transactions known in the pool. Limit on number of hashes is up to the node. - -A node receiving a set of new transaction hashes can filter out the set of transactions it already knows. -At this point it can ask for the transactions it doesn't recognize from its peers. This is the `GetPooledTransactions` message, containing the hashes of the transaction it wishes to learn about. - -A response to `GetPooledTransactions` is the `PooledTransactions` message, containing the requested `Transaction` information. -The order of the returned `Transaction` objects must be the same as the order of the requested hashes (in `GetPooledTransactions`). -A responding node may impose a lower limit on the returned transactions, i.e. return less transactions than requested. The requesting node can then request these hashes again. - -Transactions that are not in the pool, even if their hash was announced in `KnownPooledTransactions`, should not be returned. -It is ok to return an empty list in `PooledTransactions` if none of the requested hashes is in the pool. - - - - - From 53e85f1bfb14da6314ef8d22d438672d26749c92 Mon Sep 17 00:00:00 2001 From: Elias Tazartes Date: Fri, 12 Jul 2024 12:25:41 +0200 Subject: [PATCH 2/2] fix This Node typo --- README.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 3083b41..12294e1 100644 --- a/README.md +++ b/README.md @@ -17,19 +17,19 @@ ## Overview -This Node document aims to provide an overview and necessary details for implementing the P2P protocol for Starknet nodes. +This document aims to provide an overview and necessary details for implementing the P2P protocol for Starknet nodes. Familiarity with [Starknet](https://starknet.io/) is assumed. ### Document Conventions -Unless otherwise noted, the key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL", NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in This Node document are to be interpreted as described in [RFC 2119](https://datatracker.ietf.org/doc/html/rfc2119). +Unless otherwise noted, the key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL", NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC 2119](https://datatracker.ietf.org/doc/html/rfc2119). ## Core Definitions -As a guiding principle, we leverage [libp2p](https://libp2p.io/) set of [protocols](https://github.com/libp2p/specs) whenever possible. This Node will let us standardize the protocol. It will also allow node implementations to leverage existing implementations of libp2p, or switch between them if necessary. +As a guiding principle, we leverage [libp2p](https://libp2p.io/) set of [protocols](https://github.com/libp2p/specs) whenever possible. This will let us standardize the protocol. It will also allow node implementations to leverage existing implementations of libp2p, or switch between them if necessary. -As a design choice This Node results in some loss of generality, but seems like a reasonable given that libp2p is becoming a de-facto standard, and we see little value in re-implementing (and re-inventing) lower level network communication protocols. +As a design choice this results in some loss of generality, but seems like a reasonable given that libp2p is becoming a de-facto standard, and we see little value in re-implementing (and re-inventing) lower level network communication protocols. Specifically, we identify nodes and address them using [Peer IDs](https://github.com/libp2p/specs/blob/master/peer-ids/peer-ids.md), with key pairs derived using [Ed25519](https://github.com/libp2p/specs/blob/master/peer-ids/peer-ids.md#ed25519) scheme. @@ -49,7 +49,7 @@ Agent versions SHOULD follow [semantic versioning](http://semver.org) format. ### Message Encoding Use protobuf (`proto3`) for message encoding/decoding between peers. -Specific message schemas to be defined for each protocol. +Specific message schemas are to be defined for each protocol. Messages that tie into other messages in one flow, i.e. messages expecting a response and providing a response MUST include a `request_id` (a positive integer) that allows nodes to correlate messages in the same flow. The request id, together with the sender id, should be enough for identifying the flow.