| Name | Number | Description |
@@ -1916,15 +1782,49 @@
- | GNARK |
+ PROCESSING |
1 |
- |
+ PROCESSING means that the blob is currently being processed by the disperser |
- | GOB |
+ CONFIRMED |
2 |
- |
+ CONFIRMED means that the blob has been dispersed to DA Nodes and the dispersed
+batch containing the blob has been confirmed onchain |
+
+
+
+ | FAILED |
+ 3 |
+ FAILED means that the blob has failed permanently (for reasons other than insufficient
+signatures, which is a separate state). This status is somewhat of a catch-all category,
+containg (but not necessarily exclusively as errors can be added in the future):
+ - blob has expired
+ - internal logic error while requesting encoding
+ - blob retry has exceeded its limit while waiting for blob finalization after confirmation.
+ Most likely triggered by a chain reorg: see https://github.com/Layr-Labs/eigenda/blob/master/disperser/batcher/finalizer.go#L179-L189. |
+
+
+
+ | FINALIZED |
+ 4 |
+ FINALIZED means that the block containing the blob's confirmation transaction has been finalized on Ethereum |
+
+
+
+ | INSUFFICIENT_SIGNATURES |
+ 5 |
+ INSUFFICIENT_SIGNATURES means that the confirmation threshold for the blob was not met
+for at least one quorum. |
+
+
+
+ | DISPERSING |
+ 6 |
+ The DISPERSING state is comprised of two separate phases:
+ - Dispersing to DA nodes and collecting signature
+ - Submitting the transaction on chain and waiting for tx receipt |
@@ -1934,8 +1834,8 @@
- Dispersal
-
+ Disperser
+ Disperser defines the public APIs for dispersing blobs.
| Method Name | Request Type | Response Type | Description |
@@ -1943,74 +1843,52 @@ Dispersal
- | StoreChunks |
- StoreChunksRequest |
- StoreChunksReply |
- StoreChunks validates that the chunks match what the Node is supposed to receive (
-different Nodes are responsible for different chunks, as EigenDA is horizontally
-sharded) and is correctly coded (e.g. each chunk must be a valid KZG multiproof)
-according to the EigenDA protocol. It also stores the chunks along with metadata
-for the protocol-defined length of custody. It will return a signature at the
-end to attest to the data in this request it has processed. |
-
-
-
- | StoreBlobs |
- StoreBlobsRequest |
- StoreBlobsReply |
- StoreBlobs is simiar to StoreChunks, but it stores the blobs using a different storage schema
-so that the stored blobs can later be aggregated by AttestBatch method to a bigger batch.
-StoreBlobs + AttestBatch will eventually replace and deprecate StoreChunks method.
-DEPRECATED: StoreBlobs method is not used |
-
-
-
- | AttestBatch |
- AttestBatchRequest |
- AttestBatchReply |
- AttestBatch is used to aggregate the batches stored by StoreBlobs method to a bigger batch.
-It will return a signature at the end to attest to the aggregated batch.
-DEPRECATED: AttestBatch method is not used |
-
-
-
- | NodeInfo |
- NodeInfoRequest |
- NodeInfoReply |
- Retrieve node info metadata |
-
-
-
-
+ DisperseBlob |
+ DisperseBlobRequest |
+ DisperseBlobReply |
+ DisperseBlob accepts a single blob to be dispersed.
+This executes the dispersal async, i.e. it returns once the request
+is accepted. The client should use GetBlobStatus() API to poll the
+processing status of the blob.
-
- Retrieval
-
-
-
- | Method Name | Request Type | Response Type | Description |
-
-
+If DisperseBlob returns the following error codes:
+INVALID_ARGUMENT (400): request is invalid for a reason specified in the error msg.
+RESOURCE_EXHAUSTED (429): request is rate limited for the quorum specified in the error msg.
+ user should retry after the specified duration.
+INTERNAL (500): serious error, user should NOT retry.
+
- | RetrieveChunks |
- RetrieveChunksRequest |
- RetrieveChunksReply |
- RetrieveChunks retrieves the chunks for a blob custodied at the Node. |
+ DisperseBlobAuthenticated |
+ AuthenticatedRequest stream |
+ AuthenticatedReply stream |
+ DisperseBlobAuthenticated is similar to DisperseBlob, except that it requires the
+client to authenticate itself via the AuthenticationData message. The protoco is as follows:
+1. The client sends a DisperseBlobAuthenticated request with the DisperseBlobRequest message
+2. The Disperser sends back a BlobAuthHeader message containing information for the client to
+ verify and sign.
+3. The client verifies the BlobAuthHeader and sends back the signed BlobAuthHeader in an
+ AuthenticationData message.
+4. The Disperser verifies the signature and returns a DisperseBlobReply message. |
- | GetBlobHeader |
- GetBlobHeaderRequest |
- GetBlobHeaderReply |
- GetBlobHeader is similar to RetrieveChunks, this just returns the header of the blob. |
+ GetBlobStatus |
+ BlobStatusRequest |
+ BlobStatusReply |
+ This API is meant to be polled for the blob status. |
- | NodeInfo |
- NodeInfoRequest |
- NodeInfoReply |
- Retrieve node info metadata |
+ RetrieveBlob |
+ RetrieveBlobRequest |
+ RetrieveBlobReply |
+ This retrieves the requested blob from the Disperser's backend.
+This is a more efficient way to retrieve blobs than directly retrieving
+from the DA Nodes (see detail about this approach in
+api/proto/retriever/retriever.proto).
+The blob should have been initially dispersed via this Disperser service
+for this API to work. |
@@ -2020,12 +1898,12 @@ Retrieval
- node/v2/node_v2.protoTop
+ disperser/v2/disperser_v2.protoTop
GetChunksReply
+ Attestation
@@ -2036,10 +1914,38 @@ GetChunksReply
- | chunks |
+ non_signer_pubkeys |
bytes |
repeated |
- All chunks the Node is storing for the requested blob per RetrieveChunksRequest. |
+ Serialized bytes of non signer public keys (G1 points) |
+
+
+
+ | apk_g2 |
+ bytes |
+ |
+ Serialized bytes of G2 point that represents aggregate public key of all signers |
+
+
+
+ | quorum_apks |
+ bytes |
+ repeated |
+ Serialized bytes of aggregate public keys (G1 points) from all nodes for each quorum |
+
+
+
+ | sigma |
+ bytes |
+ |
+ Serialized bytes of aggregate signature |
+
+
+
+ | quorum_numbers |
+ uint32 |
+ repeated |
+ Relevant quorum numbers for the attestation |
@@ -2049,7 +1955,7 @@ GetChunksReply
- GetChunksRequest
+ BlobCommitmentReply
@@ -2060,21 +1966,12 @@ GetChunksRequest
- | blob_key |
- bytes |
+ blob_commitment |
+ common.BlobCommitment |
|
|
-
- | quorum_id |
- uint32 |
- |
- Which quorum of the blob to retrieve for (note: a blob can have multiple
-quorums and the chunks for different quorums at a Node can be different).
-The ID must be in range [0, 254]. |
-
-
@@ -2082,8 +1979,8 @@ GetChunksRequest
- NodeInfoReply
- Node info reply
+ BlobCommitmentRequest
+ Utility method used to generate the commitment of blob given its data. This can be used to construct BlobHeader.commitment
@@ -2093,36 +1990,46 @@ NodeInfoReply
- | semver |
- string |
+ data |
+ bytes |
|
|
-
- | arch |
- string |
- |
- |
-
+
+
+
+
+
+
+
+ BlobStatusReply
+
+
+
+
+
+ | Field | Type | Label | Description |
+
+
- | os |
- string |
+ status |
+ BlobStatus |
|
- |
+ The status of the blob. |
- | num_cpu |
- uint32 |
+ signed_batch |
+ SignedBatch |
|
- |
+ The signed batch |
- | mem_bytes |
- uint64 |
+ blob_verification_info |
+ BlobVerificationInfo |
|
|
@@ -2134,15 +2041,32 @@ NodeInfoReply
- NodeInfoRequest
- Node info request
+ BlobStatusRequest
+ BlobStatusRequest is used to query the status of a blob.
+
+
+ | Field | Type | Label | Description |
+
+
+
+
+ | blob_key |
+ bytes |
+ |
+ |
+
+
+
+
+
+
- StoreChunksReply
-
+ BlobVerificationInfo
+ BlobVerificationInfo is the information needed to verify the inclusion of a blob in a batch.
@@ -2152,12 +2076,26 @@ StoreChunksReply
- | signature |
- bytes |
+ blob_certificate |
+ common.v2.BlobCertificate |
|
|
+
+ | blob_index |
+ uint32 |
+ |
+ blob_index is the index of the blob in the batch |
+
+
+
+ | inclusion_proof |
+ bytes |
+ |
+ inclusion_proof is the inclusion proof of the blob in the batch |
+
+
@@ -2165,7 +2103,7 @@ StoreChunksReply
- StoreChunksRequest
+ DisperseBlobReply
@@ -2176,10 +2114,17 @@ StoreChunksRequest
- | batch |
- common.v2.Batch |
+ result |
+ BlobStatus |
|
- batch of blobs to store |
+ The status of the blob associated with the blob key. |
+
+
+
+ | blob_key |
+ bytes |
+ |
+ |
@@ -2189,40 +2134,130 @@ StoreChunksRequest
+ DisperseBlobRequest
+
-
-
-
-
-
- Dispersal
- WARNING: the following RPCs are experimental and subject to change.
-
-
- | Method Name | Request Type | Response Type | Description |
+
+
+
+ | Field | Type | Label | Description |
+
+
+
+
+ | data |
+ bytes |
+ |
+ The data to be dispersed.
+The size of data must be <= 16MiB. Every 32 bytes of data is interpreted as an integer in big endian format
+where the lower address has more significant bits. The integer must stay in the valid range to be interpreted
+as a field element on the bn254 curve. The valid range is
+0 <= x < 21888242871839275222246405745257275088548364400416034343698204186575808495617
+If any one of the 32 bytes elements is outside the range, the whole request is deemed as invalid, and rejected. |
+
+
+
+ | blob_header |
+ common.v2.BlobHeader |
+ |
+ |
+
+
+
+
+
+
+
+
+
+ SignedBatch
+ SignedBatch is a batch of blobs with a signature.
+
+
+
+
+ | Field | Type | Label | Description |
+
+
+
+
+ | header |
+ common.v2.BatchHeader |
+ |
+ header contains metadata about the batch |
+
+
+
+ | attestation |
+ Attestation |
+ |
+ attestation on the batch |
+
+
+
+
+
+
+
+
+
+
+
+ BlobStatus
+ BlobStatus represents the status of a blob. The status of a blob is updated as the blob is processed by the disperser. The status of a blob can be queried by the client using the GetBlobStatus API. Intermediate states are states that the blob can be in while being processed, and it can be updated to a differet state: - QUEUED - ENCODED Terminal states are states that will not be updated to a different state: - CERTIFIED - FAILED - INSUFFICIENT_SIGNATURES
+
+
+ | Name | Number | Description |
- | StoreChunks |
- StoreChunksRequest |
- StoreChunksReply |
+ UNKNOWN |
+ 0 |
|
- | NodeInfo |
- NodeInfoRequest |
- NodeInfoReply |
- |
+ QUEUED |
+ 1 |
+ QUEUED means that the blob has been queued by the disperser for processing |
+
+
+
+ | ENCODED |
+ 2 |
+ ENCODED means that the blob has been encoded and is ready to be dispersed to DA Nodes |
+
+
+
+ | CERTIFIED |
+ 3 |
+ CERTIFIED means the blob has been dispersed and attested by the DA nodes |
+
+
+
+ | FAILED |
+ 4 |
+ FAILED means that the blob has failed permanently (for reasons other than insufficient
+signatures, which is a separate state) |
+
+
+
+ | INSUFFICIENT_SIGNATURES |
+ 5 |
+ INSUFFICIENT_SIGNATURES means that the confirmation threshold for the blob was not met
+for at least one quorum. |
+
-
- Retrieval
-
+
+
+
+ Disperser
+ Disperser defines the public APIs for dispersing blobs.
| Method Name | Request Type | Response Type | Description |
@@ -2230,17 +2265,27 @@ Retrieval
- | GetChunks |
- GetChunksRequest |
- GetChunksReply |
- GetChunks retrieves the chunks for a blob custodied at the Node. |
+ DisperseBlob |
+ DisperseBlobRequest |
+ DisperseBlobReply |
+ DisperseBlob accepts blob to disperse from clients.
+This executes the dispersal asynchronously, i.e. it returns once the request
+is accepted. The client could use GetBlobStatus() API to poll the the
+processing status of the blob. |
- | NodeInfo |
- NodeInfoRequest |
- NodeInfoReply |
- Retrieve node info metadata |
+ GetBlobStatus |
+ BlobStatusRequest |
+ BlobStatusReply |
+ GetBlobStatus is meant to be polled for the blob status. |
+
+
+
+ | GetBlobCommitment |
+ BlobCommitmentRequest |
+ BlobCommitmentReply |
+ GetBlobCommitment is a utility method that calculates commitment for a blob payload. |
@@ -2250,12 +2295,12 @@ Retrieval
- disperser/v2/disperser_v2.protoTop
+ node/node.protoTop
Attestation
+ AttestBatchReply
@@ -2266,38 +2311,10 @@ Attestation
- | non_signer_pubkeys |
- bytes |
- repeated |
- Serialized bytes of non signer public keys (G1 points) |
-
-
-
- | apk_g2 |
- bytes |
- |
- Serialized bytes of G2 point that represents aggregate public key of all signers |
-
-
-
- | quorum_apks |
- bytes |
- repeated |
- Serialized bytes of aggregate public keys (G1 points) from all nodes for each quorum |
-
-
-
- | sigma |
+ signature |
bytes |
|
- Serialized bytes of aggregate signature |
-
-
-
- | quorum_numbers |
- uint32 |
- repeated |
- Relevant quorum numbers for the attestation |
+ |
@@ -2307,7 +2324,7 @@ Attestation
- BlobCommitmentReply
+ AttestBatchRequest
@@ -2318,10 +2335,17 @@ BlobCommitmentReply
- | blob_commitment |
- common.BlobCommitment |
+ batch_header |
+ BatchHeader |
|
- |
+ header of the batch |
+
+
+
+ | blob_header_hashes |
+ bytes |
+ repeated |
+ the header hashes of all blobs in the batch |
@@ -2331,8 +2355,8 @@ BlobCommitmentReply
- BlobCommitmentRequest
- Utility method used to generate the commitment of blob given its data. This can be used to construct BlobHeader.commitment
+
+ BatchHeader (see core/data.go#BatchHeader)
@@ -2342,10 +2366,17 @@ BlobCommitmentRequest
- | data |
+ batch_root |
bytes |
|
- |
+ The root of the merkle tree with hashes of blob headers as leaves. |
+
+
+
+ | reference_block_number |
+ uint32 |
+ |
+ The Ethereum block number at which the batch is dispersed. |
@@ -2355,8 +2386,8 @@ BlobCommitmentRequest
- BlobStatusReply
-
+ Blob
+ In EigenDA, the original blob to disperse is encoded as a polynomial via taking taking different point evaluations (i.e. erasure coding). These points are split into disjoint subsets which are assigned to different operator nodes in the EigenDA network. The data in this message is a subset of these points that are assigned to a single operator node.
@@ -2366,24 +2397,21 @@ BlobStatusReply
- | status |
- BlobStatus |
- |
- The status of the blob. |
-
-
-
- | signed_batch |
- SignedBatch |
+ header |
+ BlobHeader |
|
- The signed batch |
+ Which (original) blob this is for. |
- | blob_verification_info |
- BlobVerificationInfo |
- |
- |
+ bundles |
+ Bundle |
+ repeated |
+ Each bundle contains all chunks for a single quorum of the blob.
+The number of bundles must be equal to the total number of quorums associated
+with the blob, and the ordering must be the same as BlobHeader.quorum_headers.
+Note: an operator may be in some but not all of the quorums; in that case the
+bundle corresponding to that quorum will be empty. |
@@ -2393,8 +2421,8 @@ BlobStatusReply
- BlobStatusRequest
- BlobStatusRequest is used to query the status of a blob.
+
+
@@ -2404,48 +2432,55 @@ BlobStatusRequest
- | blob_key |
- bytes |
+ commitment |
+ common.G1Commitment |
|
- |
+ The KZG commitment to the polynomial representing the blob. |
-
-
-
-
-
-
-
- BlobVerificationInfo
- BlobVerificationInfo is the information needed to verify the inclusion of a blob in a batch.
-
-
-
-
- | Field | Type | Label | Description |
-
-
+
+ | length_commitment |
+ G2Commitment |
+ |
+ The KZG commitment to the polynomial representing the blob on G2, it is used
+for proving the degree of the polynomial |
+
- | blob_certificate |
- common.v2.BlobCertificate |
+ length_proof |
+ G2Commitment |
|
- |
+ The low degree proof. It's the KZG commitment to the polynomial shifted to
+the largest SRS degree. |
- | blob_index |
+ length |
uint32 |
|
- blob_index is the index of the blob in the batch |
+ The length of the original blob in number of symbols (in the field where
+the polynomial is defined). |
- | inclusion_proof |
- bytes |
+ quorum_headers |
+ BlobQuorumInfo |
+ repeated |
+ The params of the quorums that this blob participates in. |
+
+
+
+ | account_id |
+ string |
|
- inclusion_proof is the inclusion proof of the blob in the batch |
+ The ID of the user who is dispersing this blob to EigenDA. |
+
+
+
+ | reference_block_number |
+ uint32 |
+ |
+ The reference block number whose state is used to encode the blob |
@@ -2455,8 +2490,8 @@ BlobVerificationInfo
- DisperseBlobReply
-
+ BlobQuorumInfo
+ See BlobQuorumParam as defined in api/proto/disperser/disperser.proto
@@ -2466,15 +2501,36 @@ DisperseBlobReply
- | result |
- BlobStatus |
+ quorum_id |
+ uint32 |
|
- The status of the blob associated with the blob key. |
+ |
- | blob_key |
- bytes |
+ adversary_threshold |
+ uint32 |
+ |
+ |
+
+
+
+ | confirmation_threshold |
+ uint32 |
+ |
+ |
+
+
+
+ | chunk_length |
+ uint32 |
+ |
+ |
+
+
+
+ | ratelimit |
+ uint32 |
|
|
@@ -2486,8 +2542,8 @@ DisperseBlobReply
- DisperseBlobRequest
-
+ Bundle
+ A Bundle is the collection of chunks associated with a single blob, for a single operator and a single quorum.
@@ -2497,22 +2553,18 @@ DisperseBlobRequest
- | data |
+ chunks |
bytes |
- |
- The data to be dispersed.
-The size of data must be <= 16MiB. Every 32 bytes of data is interpreted as an integer in big endian format
-where the lower address has more significant bits. The integer must stay in the valid range to be interpreted
-as a field element on the bn254 curve. The valid range is
-0 <= x < 21888242871839275222246405745257275088548364400416034343698204186575808495617
-If any one of the 32 bytes elements is outside the range, the whole request is deemed as invalid, and rejected. |
+ repeated |
+ Each chunk corresponds to a collection of points on the polynomial.
+Each chunk has same number of points. |
- | blob_header |
- common.v2.BlobHeader |
+ bundle |
+ bytes |
|
- |
+ All chunks of the bundle encoded in a byte array. |
@@ -2522,8 +2574,8 @@ DisperseBlobRequest
- SignedBatch
- SignedBatch is a batch of blobs with a signature.
+ G2Commitment
+
@@ -2533,17 +2585,31 @@ SignedBatch
- | header |
- common.v2.BatchHeader |
+ x_a0 |
+ bytes |
|
- header contains metadata about the batch |
+ The A0 element of the X coordinate of G2 point. |
- | attestation |
- Attestation |
+ x_a1 |
+ bytes |
|
- attestation on the batch |
+ The A1 element of the X coordinate of G2 point. |
+
+
+
+ | y_a0 |
+ bytes |
+ |
+ The A0 element of the Y coordinate of G2 point. |
+
+
+
+ | y_a1 |
+ bytes |
+ |
+ The A1 element of the Y coordinate of G2 point. |
@@ -2553,106 +2619,7 @@ SignedBatch
-
-
- BlobStatus
- BlobStatus represents the status of a blob. The status of a blob is updated as the blob is processed by the disperser. The status of a blob can be queried by the client using the GetBlobStatus API. Intermediate states are states that the blob can be in while being processed, and it can be updated to a differet state: - QUEUED - ENCODED Terminal states are states that will not be updated to a different state: - CERTIFIED - FAILED - INSUFFICIENT_SIGNATURES
-
-
- | Name | Number | Description |
-
-
-
-
- | UNKNOWN |
- 0 |
- |
-
-
-
- | QUEUED |
- 1 |
- QUEUED means that the blob has been queued by the disperser for processing |
-
-
-
- | ENCODED |
- 2 |
- ENCODED means that the blob has been encoded and is ready to be dispersed to DA Nodes |
-
-
-
- | CERTIFIED |
- 3 |
- CERTIFIED means the blob has been dispersed and attested by the DA nodes |
-
-
-
- | FAILED |
- 4 |
- FAILED means that the blob has failed permanently (for reasons other than insufficient
-signatures, which is a separate state) |
-
-
-
- | INSUFFICIENT_SIGNATURES |
- 5 |
- INSUFFICIENT_SIGNATURES means that the confirmation threshold for the blob was not met
-for at least one quorum. |
-
-
-
-
-
-
-
-
-
- Disperser
- Disperser defines the public APIs for dispersing blobs.
-
-
- | Method Name | Request Type | Response Type | Description |
-
-
-
-
- | DisperseBlob |
- DisperseBlobRequest |
- DisperseBlobReply |
- DisperseBlob accepts blob to disperse from clients.
-This executes the dispersal asynchronously, i.e. it returns once the request
-is accepted. The client could use GetBlobStatus() API to poll the the
-processing status of the blob. |
-
-
-
- | GetBlobStatus |
- BlobStatusRequest |
- BlobStatusReply |
- GetBlobStatus is meant to be polled for the blob status. |
-
-
-
- | GetBlobCommitment |
- BlobCommitmentRequest |
- BlobCommitmentReply |
- GetBlobCommitment is a utility method that calculates commitment for a blob payload. |
-
-
-
-
-
-
-
-
-
- disperser/disperser.protoTop
- AuthenticatedReply
+
@@ -2663,17 +2630,19 @@ AuthenticatedReply
- | blob_auth_header |
- BlobAuthHeader |
+ blob_header |
+ BlobHeader |
|
- |
+ The header of the blob requested per GetBlobHeaderRequest. |
- | disperse_reply |
- DisperseBlobReply |
+ proof |
+ MerkleProof |
|
- |
+ Merkle proof that returned blob header belongs to the batch and is
+the batch's MerkleProof.index-th blob.
+This can be checked against the batch root on chain. |
@@ -2683,8 +2652,8 @@ AuthenticatedReply
- AuthenticatedRequest
-
+
+ See RetrieveChunksRequest for documentation of each parameter of GetBlobHeaderRequest.
-
-
-
-
-
- AuthenticationData
- AuthenticationData contains the signature of the BlobAuthHeader.
-
-
-
-
- | Field | Type | Label | Description |
-
-
-
- | authentication_data |
- bytes |
+ quorum_id |
+ uint32 |
|
|
@@ -2738,7 +2690,7 @@ AuthenticationData
-
+ MerkleProof
@@ -2749,35 +2701,17 @@
- | batch_root |
- bytes |
- |
- The root of the merkle tree with the hashes of blob headers as leaves. |
-
-
-
- | quorum_numbers |
- bytes |
- |
- All quorums associated with blobs in this batch. Sorted in ascending order.
-Ex. [0, 2, 1] => 0x000102 |
-
-
-
- | quorum_signed_percentages |
+ hashes |
bytes |
- |
- The percentage of stake that has signed for this batch.
-The quorum_signed_percentages[i] is percentage for the quorum_numbers[i]. |
+ repeated |
+ The proof itself. |
- | reference_block_number |
+ index |
uint32 |
|
- The Ethereum block number at which the batch was created.
-The Disperser will encode and disperse the blobs based on the onchain info
-(e.g. operator stakes) at this block number. |
+ Which index (the leaf of the Merkle tree) this proof is for. |
@@ -2787,8 +2721,8 @@
-
-
+ NodeInfoReply
+ Node info reply
@@ -2798,41 +2732,38 @@
- | batch_header |
- BatchHeader |
+ semver |
+ string |
|
|
- | signatory_record_hash |
- bytes |
+ arch |
+ string |
|
- The hash of all public keys of the operators that did not sign the batch. |
+ |
- | fee |
- bytes |
+ os |
+ string |
|
- The fee payment paid by users for dispersing this batch. It's the bytes
-representation of a big.Int value. |
+ |
- | confirmation_block_number |
+ num_cpu |
uint32 |
|
- The Ethereum block number at which the batch is confirmed onchain. |
+ |
- | batch_header_hash |
- bytes |
+ mem_bytes |
+ uint64 |
|
- This is the hash of the ReducedBatchHeader defined onchain, see:
-https://github.com/Layr-Labs/eigenda/blob/master/contracts/src/interfaces/IEigenDAServiceManager.sol#L43
-The is the message that the operators will sign their signatures on. |
+ |
@@ -2842,31 +2773,14 @@
-
- BlobAuthHeader contains information about the blob for the client to verify and sign. - Once payments are enabled, the BlobAuthHeader will contain the KZG commitment to the blob, which the client will verify and sign. Having the client verify the KZG commitment instead of calculating it avoids the need for the client to have the KZG structured reference string (SRS), which can be large. The signed KZG commitment prevents the disperser from sending a different blob to the DA Nodes than the one the client sent. - In the meantime, the BlobAuthHeader contains a simple challenge parameter is used to prevent replay attacks in the event that a signature is leaked.
+ NodeInfoRequest
+ Node info request
-
-
- | Field | Type | Label | Description |
-
-
-
-
- | challenge_parameter |
- uint32 |
- |
- |
-
-
-
-
-
-
-
+ RetrieveChunksReply
@@ -2877,24 +2791,17 @@
- | commitment |
- common.G1Commitment |
- |
- KZG commitment of the blob. |
+ chunks |
+ bytes |
+ repeated |
+ All chunks the Node is storing for the requested blob per RetrieveChunksRequest. |
- | data_length |
- uint32 |
+ chunk_encoding_format |
+ ChunkEncodingFormat |
|
- The length of the blob in symbols (each symbol is 32 bytes). |
-
-
-
- | blob_quorum_params |
- BlobQuorumParam |
- repeated |
- The params of the quorums that this blob participates in. |
+ How the above chunks are encoded. |
@@ -2904,8 +2811,8 @@
- BlobInfo
- BlobInfo contains information needed to confirm the blob against the EigenDA contracts
+ RetrieveChunksRequest
+
@@ -2915,17 +2822,29 @@ BlobInfo
- | blob_header |
- BlobHeader |
+ batch_header_hash |
+ bytes |
|
- |
+ The hash of the ReducedBatchHeader defined onchain, see:
+https://github.com/Layr-Labs/eigenda/blob/master/contracts/src/interfaces/IEigenDAServiceManager.sol#L43
+This identifies which batch to retrieve for. |
- | blob_verification_proof |
- BlobVerificationProof |
+ blob_index |
+ uint32 |
|
- |
+ Which blob in the batch to retrieve for (note: a batch is logically an ordered
+list of blobs). |
+
+
+
+ | quorum_id |
+ uint32 |
+ |
+ Which quorum of the blob to retrieve for (note: a blob can have multiple
+quorums and the chunks for different quorums at a Node can be different).
+The ID must be in range [0, 254]. |
@@ -2935,7 +2854,7 @@ BlobInfo
- BlobQuorumParam
+ StoreBlobsReply
@@ -2946,34 +2865,12 @@ BlobQuorumParam
- | quorum_number |
- uint32 |
- |
- The ID of the quorum. |
-
-
-
- | adversary_threshold_percentage |
- uint32 |
- |
- The max percentage of stake within the quorum that can be held by or delegated
-to adversarial operators. Currently, this and the next parameter are standardized
-across the quorum using values read from the EigenDA contracts. |
-
-
-
- | confirmation_threshold_percentage |
- uint32 |
- |
- The min percentage of stake that must attest in order to consider
-the dispersal is successful. |
-
-
-
- | chunk_length |
- uint32 |
- |
- The length of each chunk. |
+ signatures |
+ google.protobuf.BytesValue |
+ repeated |
+ The operator's BLS sgnature signed on the blob header hashes.
+The ordering of the signatures must match the ordering of the blobs sent
+in the request, with empty signatures in the places for discarded blobs. |
@@ -2983,7 +2880,7 @@ BlobQuorumParam
- BlobStatusReply
+ StoreBlobsRequest
@@ -2994,17 +2891,17 @@ BlobStatusReply
- | status |
- BlobStatus |
- |
- The status of the blob. |
+ blobs |
+ Blob |
+ repeated |
+ Blobs to store |
- | info |
- BlobInfo |
+ reference_block_number |
+ uint32 |
|
- The blob info needed for clients to confirm the blob against the EigenDA contracts. |
+ The reference block number whose state is used to encode the blobs |
@@ -3014,8 +2911,8 @@ BlobStatusReply
- BlobStatusRequest
- BlobStatusRequest is used to query the status of a blob.
+ StoreChunksReply
+
@@ -3025,10 +2922,10 @@ BlobStatusRequest
- | request_id |
+ signature |
bytes |
|
- |
+ The operator's BLS signature signed on the batch header hash. |
@@ -3038,7 +2935,7 @@ BlobStatusRequest
- BlobVerificationProof
+ StoreChunksRequest
@@ -3049,54 +2946,17 @@ BlobVerificationProof
- | batch_id |
- uint32 |
- |
- batch_id is an incremental ID assigned to a batch by EigenDAServiceManager |
-
-
-
- | blob_index |
- uint32 |
- |
- The index of the blob in the batch (which is logically an ordered list of blobs). |
-
-
-
- | batch_metadata |
- BatchMetadata |
- |
- |
-
-
-
- | inclusion_proof |
- bytes |
+ batch_header |
+ BatchHeader |
|
- inclusion_proof is a merkle proof for a blob header's inclusion in a batch |
+ Which batch this request is for. |
- | quorum_indexes |
- bytes |
- |
- indexes of quorums in BatchHeader.quorum_numbers that match the quorums in BlobHeader.blob_quorum_params
-Ex. BlobHeader.blob_quorum_params = [
- {
- quorum_number = 0,
- ...
- },
- {
- quorum_number = 3,
- ...
- },
- {
- quorum_number = 5,
- ...
- },
-]
-BatchHeader.quorum_numbers = [0, 5, 3] => 0x000503
-Then, quorum_indexes = [0, 2, 1] => 0x000201 |
+ blobs |
+ Blob |
+ repeated |
+ The chunks for each blob in the batch to be stored in an EigenDA Node. |
@@ -3106,7 +2966,133 @@ BlobVerificationProof
- DisperseBlobReply
+
+
+
+ This describes how the chunks returned in RetrieveChunksReply are encoded. Used to facilitate the decoding of chunks.
+
+
+ | Name | Number | Description |
+
+
+
+
+ | UNKNOWN |
+ 0 |
+ |
+
+
+
+ | GNARK |
+ 1 |
+ |
+
+
+
+ | GOB |
+ 2 |
+ |
+
+
+
+
+
+
+
+
+
+ Dispersal
+
+
+
+ | Method Name | Request Type | Response Type | Description |
+
+
+
+
+ | StoreChunks |
+ StoreChunksRequest |
+ StoreChunksReply |
+ StoreChunks validates that the chunks match what the Node is supposed to receive (
+different Nodes are responsible for different chunks, as EigenDA is horizontally
+sharded) and is correctly coded (e.g. each chunk must be a valid KZG multiproof)
+according to the EigenDA protocol. It also stores the chunks along with metadata
+for the protocol-defined length of custody. It will return a signature at the
+end to attest to the data in this request it has processed. |
+
+
+
+ | StoreBlobs |
+ StoreBlobsRequest |
+ StoreBlobsReply |
+ StoreBlobs is simiar to StoreChunks, but it stores the blobs using a different storage schema
+so that the stored blobs can later be aggregated by AttestBatch method to a bigger batch.
+StoreBlobs + AttestBatch will eventually replace and deprecate StoreChunks method.
+DEPRECATED: StoreBlobs method is not used |
+
+
+
+ | AttestBatch |
+ AttestBatchRequest |
+ AttestBatchReply |
+ AttestBatch is used to aggregate the batches stored by StoreBlobs method to a bigger batch.
+It will return a signature at the end to attest to the aggregated batch.
+DEPRECATED: AttestBatch method is not used |
+
+
+
+ | NodeInfo |
+ NodeInfoRequest |
+ NodeInfoReply |
+ Retrieve node info metadata |
+
+
+
+
+
+
+ Retrieval
+
+
+
+
+
+
+
+ node/v2/node_v2.protoTop
+ GetChunksReply
@@ -3117,24 +3103,10 @@ DisperseBlobReply
- | result |
- BlobStatus |
- |
- The status of the blob associated with the request_id. Will always be PROCESSING. |
-
-
-
- | request_id |
+ chunks |
bytes |
- |
- The request ID generated by the disperser.
-Once a request is accepted (although not processed), a unique request ID will be
-generated.
-Two different DisperseBlobRequests (determined by the hash of the DisperseBlobRequest)
-will have different IDs, and the same DisperseBlobRequest sent repeatedly at different
-times will also have different IDs.
-The client should use this ID to query the processing status of the request (via
-the GetBlobStatus API). |
+ repeated |
+ All chunks the Node is storing for the requested blob per RetrieveChunksRequest. |
@@ -3144,7 +3116,7 @@ DisperseBlobReply
- DisperseBlobRequest
+ GetChunksRequest
@@ -3155,33 +3127,19 @@ DisperseBlobRequest
- | data |
+ blob_key |
bytes |
|
- The data to be dispersed.
-The size of data must be <= 16MiB. Every 32 bytes of data is interpreted as an integer in big endian format
-where the lower address has more significant bits. The integer must stay in the valid range to be interpreted
-as a field element on the bn254 curve. The valid range is
-0 <= x < 21888242871839275222246405745257275088548364400416034343698204186575808495617
-If any one of the 32 bytes elements is outside the range, the whole request is deemed as invalid, and rejected. |
+ |
- | custom_quorum_numbers |
+ quorum_id |
uint32 |
- repeated |
- The quorums to which the blob will be sent, in addition to the required quorums which are configured
-on the EigenDA smart contract. If required quorums are included here, an error will be returned.
-The disperser will ensure that the encoded blobs for each quorum are all processed
-within the same batch. |
-
-
-
- | account_id |
- string |
|
- The account ID of the client. This should be a hex-encoded string of the ECSDA public key
-corresponding to the key used by the client to sign the BlobAuthHeader. |
+ Which quorum of the blob to retrieve for (note: a blob can have multiple
+quorums and the chunks for different quorums at a Node can be different).
+The ID must be in range [0, 254]. |
@@ -3191,8 +3149,8 @@ DisperseBlobRequest
- DispersePaidBlobRequest
-
+ NodeInfoReply
+ Node info reply
@@ -3202,42 +3160,56 @@ DispersePaidBlobRequest
- | data |
- bytes |
+ semver |
+ string |
|
- The data to be dispersed. Same requirements as DisperseBlobRequest. |
+ |
- | quorum_numbers |
- uint32 |
- repeated |
- The quorums to which the blob to be sent |
+ arch |
+ string |
+ |
+ |
- | payment_header |
- common.PaymentHeader |
+ os |
+ string |
|
- Payment header contains AccountID, BinIndex, and CumulativePayment |
+ |
- | payment_signature |
- bytes |
+ num_cpu |
+ uint32 |
|
- signature of payment_header |
+ |
+
+
+
+ | mem_bytes |
+ uint64 |
+ |
+ |
-
+
+
+
+
+ NodeInfoRequest
+ Node info request
+
+
- RetrieveBlobReply
- RetrieveBlobReply contains the retrieved blob data
+ StoreChunksReply
+
@@ -3247,7 +3219,7 @@ RetrieveBlobReply
- | data |
+ signature |
bytes |
|
|
@@ -3260,8 +3232,8 @@ RetrieveBlobReply
- RetrieveBlobRequest
- RetrieveBlobRequest contains parameters to retrieve the blob.
+ StoreChunksRequest
+
@@ -3271,17 +3243,10 @@ RetrieveBlobRequest
- | batch_header_hash |
- bytes |
- |
- |
-
-
-
- | blob_index |
- uint32 |
+ batch |
+ common.v2.Batch |
|
- |
+ batch of blobs to store |
@@ -3293,75 +3258,38 @@ RetrieveBlobRequest
- BlobStatus
- BlobStatus represents the status of a blob. The status of a blob is updated as the blob is processed by the disperser. The status of a blob can be queried by the client using the GetBlobStatus API. Intermediate states are states that the blob can be in while being processed, and it can be updated to a differet state: - PROCESSING - DISPERSING - CONFIRMED Terminal states are states that will not be updated to a different state: - FAILED - FINALIZED - INSUFFICIENT_SIGNATURES
+
+
+
+
+ Dispersal
+ WARNING: the following RPCs are experimental and subject to change.
- | Name | Number | Description |
+ | Method Name | Request Type | Response Type | Description |
- | UNKNOWN |
- 0 |
+ StoreChunks |
+ StoreChunksRequest |
+ StoreChunksReply |
|
- | PROCESSING |
- 1 |
- PROCESSING means that the blob is currently being processed by the disperser |
-
-
-
- | CONFIRMED |
- 2 |
- CONFIRMED means that the blob has been dispersed to DA Nodes and the dispersed
-batch containing the blob has been confirmed onchain |
-
-
-
- | FAILED |
- 3 |
- FAILED means that the blob has failed permanently (for reasons other than insufficient
-signatures, which is a separate state). This status is somewhat of a catch-all category,
-containg (but not necessarily exclusively as errors can be added in the future):
- - blob has expired
- - internal logic error while requesting encoding
- - blob retry has exceeded its limit while waiting for blob finalization after confirmation.
- Most likely triggered by a chain reorg: see https://github.com/Layr-Labs/eigenda/blob/master/disperser/batcher/finalizer.go#L179-L189. |
-
-
-
- | FINALIZED |
- 4 |
- FINALIZED means that the block containing the blob's confirmation transaction has been finalized on Ethereum |
-
-
-
- | INSUFFICIENT_SIGNATURES |
- 5 |
- INSUFFICIENT_SIGNATURES means that the confirmation threshold for the blob was not met
-for at least one quorum. |
-
-
-
- | DISPERSING |
- 6 |
- The DISPERSING state is comprised of two separate phases:
- - Dispersing to DA nodes and collecting signature
- - Submitting the transaction on chain and waiting for tx receipt |
+ NodeInfo |
+ NodeInfoRequest |
+ NodeInfoReply |
+ |
-
-
-
-
- Disperser
- Disperser defines the public APIs for dispersing blobs.
+
+ Retrieval
+
| Method Name | Request Type | Response Type | Description |
@@ -3369,52 +3297,17 @@ Disperser
- | DisperseBlob |
- DisperseBlobRequest |
- DisperseBlobReply |
- DisperseBlob accepts a single blob to be dispersed.
-This executes the dispersal async, i.e. it returns once the request
-is accepted. The client should use GetBlobStatus() API to poll the
-processing status of the blob.
-
-If DisperseBlob returns the following error codes:
-INVALID_ARGUMENT (400): request is invalid for a reason specified in the error msg.
-RESOURCE_EXHAUSTED (429): request is rate limited for the quorum specified in the error msg.
- user should retry after the specified duration.
-INTERNAL (500): serious error, user should NOT retry. |
-
-
-
- | DisperseBlobAuthenticated |
- AuthenticatedRequest stream |
- AuthenticatedReply stream |
- DisperseBlobAuthenticated is similar to DisperseBlob, except that it requires the
-client to authenticate itself via the AuthenticationData message. The protoco is as follows:
-1. The client sends a DisperseBlobAuthenticated request with the DisperseBlobRequest message
-2. The Disperser sends back a BlobAuthHeader message containing information for the client to
- verify and sign.
-3. The client verifies the BlobAuthHeader and sends back the signed BlobAuthHeader in an
- AuthenticationData message.
-4. The Disperser verifies the signature and returns a DisperseBlobReply message. |
-
-
-
- | GetBlobStatus |
- BlobStatusRequest |
- BlobStatusReply |
- This API is meant to be polled for the blob status. |
+ GetChunks |
+ GetChunksRequest |
+ GetChunksReply |
+ GetChunks retrieves the chunks for a blob custodied at the Node. |
- | RetrieveBlob |
- RetrieveBlobRequest |
- RetrieveBlobReply |
- This retrieves the requested blob from the Disperser's backend.
-This is a more efficient way to retrieve blobs than directly retrieving
-from the DA Nodes (see detail about this approach in
-api/proto/retriever/retriever.proto).
-The blob should have been initially dispersed via this Disperser service
-for this API to work. |
+ NodeInfo |
+ NodeInfoRequest |
+ NodeInfoReply |
+ Retrieve node info metadata |
@@ -3694,6 +3587,113 @@ Relay
+
+
+ retriever/retriever.protoTop
+ BlobReply
+
+
+
+
+
+ | Field | Type | Label | Description |
+
+
+
+
+ | data |
+ bytes |
+ |
+ The blob retrieved and reconstructed from the EigenDA Nodes per BlobRequest. |
+
+
+
+
+
+
+
+
+
+ BlobRequest
+
+
+
+
+
+ | Field | Type | Label | Description |
+
+
+
+
+ | batch_header_hash |
+ bytes |
+ |
+ The hash of the ReducedBatchHeader defined onchain, see:
+https://github.com/Layr-Labs/eigenda/blob/master/contracts/src/interfaces/IEigenDAServiceManager.sol#L43
+This identifies the batch that this blob belongs to. |
+
+
+
+ | blob_index |
+ uint32 |
+ |
+ Which blob in the batch this is requesting for (note: a batch is logically an
+ordered list of blobs). |
+
+
+
+ | reference_block_number |
+ uint32 |
+ |
+ The Ethereum block number at which the batch for this blob was constructed. |
+
+
+
+ | quorum_id |
+ uint32 |
+ |
+ Which quorum of the blob this is requesting for (note a blob can participate in
+multiple quorums). |
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Retriever
+ The Retriever is a service for retrieving chunks corresponding to a blob from the EigenDA operator nodes and reconstructing the original blob from the chunks. This is a client-side library that the users are supposed to operationalize. Note: Users generally have two ways to retrieve a blob from EigenDA: 1) Retrieve from the Disperser that the user initially used for dispersal: the API is Disperser.RetrieveBlob() as defined in api/proto/disperser/disperser.proto 2) Retrieve directly from the EigenDA Nodes, which is supported by this Retriever. The Disperser.RetrieveBlob() (the 1st approach) is generally faster and cheaper as the Disperser manages the blobs that it has processed, whereas the Retriever.RetrieveBlob() (the 2nd approach here) removes the need to trust the Disperser, with the downside of worse cost and performance.
+
+
+ | Method Name | Request Type | Response Type | Description |
+
+
+
+
+ | RetrieveBlob |
+ BlobRequest |
+ BlobReply |
+ This fans out request to EigenDA Nodes to retrieve the chunks and returns the
+reconstructed original blob in response. |
+
+
+
+
+
+
+
Scalar Value Types
diff --git a/api/docs/eigenda-protos.md b/api/docs/eigenda-protos.md
index a5a01cac1..ad012427c 100644
--- a/api/docs/eigenda-protos.md
+++ b/api/docs/eigenda-protos.md
@@ -11,11 +11,10 @@
- [Churner](#churner-Churner)
-- [retriever/retriever.proto](#retriever_retriever-proto)
- - [BlobReply](#retriever-BlobReply)
- - [BlobRequest](#retriever-BlobRequest)
-
- - [Retriever](#retriever-Retriever)
+- [common/common.proto](#common_common-proto)
+ - [BlobCommitment](#common-BlobCommitment)
+ - [G1Commitment](#common-G1Commitment)
+ - [PaymentHeader](#common-PaymentHeader)
- [common/v2/common.proto](#common_v2_common-proto)
- [Batch](#common-v2-Batch)
@@ -23,10 +22,43 @@
- [BlobCertificate](#common-v2-BlobCertificate)
- [BlobHeader](#common-v2-BlobHeader)
-- [common/common.proto](#common_common-proto)
- - [BlobCommitment](#common-BlobCommitment)
- - [G1Commitment](#common-G1Commitment)
- - [PaymentHeader](#common-PaymentHeader)
+- [disperser/disperser.proto](#disperser_disperser-proto)
+ - [AuthenticatedReply](#disperser-AuthenticatedReply)
+ - [AuthenticatedRequest](#disperser-AuthenticatedRequest)
+ - [AuthenticationData](#disperser-AuthenticationData)
+ - [BatchHeader](#disperser-BatchHeader)
+ - [BatchMetadata](#disperser-BatchMetadata)
+ - [BlobAuthHeader](#disperser-BlobAuthHeader)
+ - [BlobHeader](#disperser-BlobHeader)
+ - [BlobInfo](#disperser-BlobInfo)
+ - [BlobQuorumParam](#disperser-BlobQuorumParam)
+ - [BlobStatusReply](#disperser-BlobStatusReply)
+ - [BlobStatusRequest](#disperser-BlobStatusRequest)
+ - [BlobVerificationProof](#disperser-BlobVerificationProof)
+ - [DisperseBlobReply](#disperser-DisperseBlobReply)
+ - [DisperseBlobRequest](#disperser-DisperseBlobRequest)
+ - [DispersePaidBlobRequest](#disperser-DispersePaidBlobRequest)
+ - [RetrieveBlobReply](#disperser-RetrieveBlobReply)
+ - [RetrieveBlobRequest](#disperser-RetrieveBlobRequest)
+
+ - [BlobStatus](#disperser-BlobStatus)
+
+ - [Disperser](#disperser-Disperser)
+
+- [disperser/v2/disperser_v2.proto](#disperser_v2_disperser_v2-proto)
+ - [Attestation](#disperser-v2-Attestation)
+ - [BlobCommitmentReply](#disperser-v2-BlobCommitmentReply)
+ - [BlobCommitmentRequest](#disperser-v2-BlobCommitmentRequest)
+ - [BlobStatusReply](#disperser-v2-BlobStatusReply)
+ - [BlobStatusRequest](#disperser-v2-BlobStatusRequest)
+ - [BlobVerificationInfo](#disperser-v2-BlobVerificationInfo)
+ - [DisperseBlobReply](#disperser-v2-DisperseBlobReply)
+ - [DisperseBlobRequest](#disperser-v2-DisperseBlobRequest)
+ - [SignedBatch](#disperser-v2-SignedBatch)
+
+ - [BlobStatus](#disperser-v2-BlobStatus)
+
+ - [Disperser](#disperser-v2-Disperser)
- [node/node.proto](#node_node-proto)
- [AttestBatchReply](#node-AttestBatchReply)
@@ -65,44 +97,6 @@
- [Dispersal](#node-v2-Dispersal)
- [Retrieval](#node-v2-Retrieval)
-- [disperser/v2/disperser_v2.proto](#disperser_v2_disperser_v2-proto)
- - [Attestation](#disperser-v2-Attestation)
- - [BlobCommitmentReply](#disperser-v2-BlobCommitmentReply)
- - [BlobCommitmentRequest](#disperser-v2-BlobCommitmentRequest)
- - [BlobStatusReply](#disperser-v2-BlobStatusReply)
- - [BlobStatusRequest](#disperser-v2-BlobStatusRequest)
- - [BlobVerificationInfo](#disperser-v2-BlobVerificationInfo)
- - [DisperseBlobReply](#disperser-v2-DisperseBlobReply)
- - [DisperseBlobRequest](#disperser-v2-DisperseBlobRequest)
- - [SignedBatch](#disperser-v2-SignedBatch)
-
- - [BlobStatus](#disperser-v2-BlobStatus)
-
- - [Disperser](#disperser-v2-Disperser)
-
-- [disperser/disperser.proto](#disperser_disperser-proto)
- - [AuthenticatedReply](#disperser-AuthenticatedReply)
- - [AuthenticatedRequest](#disperser-AuthenticatedRequest)
- - [AuthenticationData](#disperser-AuthenticationData)
- - [BatchHeader](#disperser-BatchHeader)
- - [BatchMetadata](#disperser-BatchMetadata)
- - [BlobAuthHeader](#disperser-BlobAuthHeader)
- - [BlobHeader](#disperser-BlobHeader)
- - [BlobInfo](#disperser-BlobInfo)
- - [BlobQuorumParam](#disperser-BlobQuorumParam)
- - [BlobStatusReply](#disperser-BlobStatusReply)
- - [BlobStatusRequest](#disperser-BlobStatusRequest)
- - [BlobVerificationProof](#disperser-BlobVerificationProof)
- - [DisperseBlobReply](#disperser-DisperseBlobReply)
- - [DisperseBlobRequest](#disperser-DisperseBlobRequest)
- - [DispersePaidBlobRequest](#disperser-DispersePaidBlobRequest)
- - [RetrieveBlobReply](#disperser-RetrieveBlobReply)
- - [RetrieveBlobRequest](#disperser-RetrieveBlobRequest)
-
- - [BlobStatus](#disperser-BlobStatus)
-
- - [Disperser](#disperser-Disperser)
-
- [relay/relay.proto](#relay_relay-proto)
- [ChunkRequest](#node-ChunkRequest)
- [ChunkRequestByIndex](#node-ChunkRequestByIndex)
@@ -114,6 +108,12 @@
- [Relay](#node-Relay)
+- [retriever/retriever.proto](#retriever_retriever-proto)
+ - [BlobReply](#retriever-BlobReply)
+ - [BlobRequest](#retriever-BlobRequest)
+
+ - [Retriever](#retriever-Retriever)
+
- [Scalar Value Types](#scalar-value-types)
@@ -225,72 +225,69 @@ https://github.com/Layr-Labs/eigenlayer-middleware/blob/master/src/interfaces/IB
-
+
Top
-## retriever/retriever.proto
-
+## common/common.proto
-
-### BlobReply
+
+### BlobCommitment
+BlobCommitment represents commitment of a specific blob, containing its
+KZG commitment, degree proof, the actual degree, and data length in number of symbols.
| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
-| data | [bytes](#bytes) | | The blob retrieved and reconstructed from the EigenDA Nodes per BlobRequest. |
+| commitment | [bytes](#bytes) | | |
+| length_commitment | [bytes](#bytes) | | |
+| length_proof | [bytes](#bytes) | | |
+| length | [uint32](#uint32) | | |
-
+
-### BlobRequest
+### G1Commitment
| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
-| batch_header_hash | [bytes](#bytes) | | The hash of the ReducedBatchHeader defined onchain, see: https://github.com/Layr-Labs/eigenda/blob/master/contracts/src/interfaces/IEigenDAServiceManager.sol#L43 This identifies the batch that this blob belongs to. |
-| blob_index | [uint32](#uint32) | | Which blob in the batch this is requesting for (note: a batch is logically an ordered list of blobs). |
-| reference_block_number | [uint32](#uint32) | | The Ethereum block number at which the batch for this blob was constructed. |
-| quorum_id | [uint32](#uint32) | | Which quorum of the blob this is requesting for (note a blob can participate in multiple quorums). |
+| x | [bytes](#bytes) | | The X coordinate of the KZG commitment. This is the raw byte representation of the field element. |
+| y | [bytes](#bytes) | | The Y coordinate of the KZG commitment. This is the raw byte representation of the field element. |
-
-
+
-
+### PaymentHeader
-
-### Retriever
-The Retriever is a service for retrieving chunks corresponding to a blob from
-the EigenDA operator nodes and reconstructing the original blob from the chunks.
-This is a client-side library that the users are supposed to operationalize.
+| Field | Type | Label | Description |
+| ----- | ---- | ----- | ----------- |
+| account_id | [string](#string) | | |
+| bin_index | [uint32](#uint32) | | |
+| cumulative_payment | [bytes](#bytes) | | |
-Note: Users generally have two ways to retrieve a blob from EigenDA:
- 1) Retrieve from the Disperser that the user initially used for dispersal: the API
- is Disperser.RetrieveBlob() as defined in api/proto/disperser/disperser.proto
- 2) Retrieve directly from the EigenDA Nodes, which is supported by this Retriever.
-The Disperser.RetrieveBlob() (the 1st approach) is generally faster and cheaper as the
-Disperser manages the blobs that it has processed, whereas the Retriever.RetrieveBlob()
-(the 2nd approach here) removes the need to trust the Disperser, with the downside of
-worse cost and performance.
-| Method Name | Request Type | Response Type | Description |
-| ----------- | ------------ | ------------- | ------------|
-| RetrieveBlob | [BlobRequest](#retriever-BlobRequest) | [BlobReply](#retriever-BlobReply) | This fans out request to EigenDA Nodes to retrieve the chunks and returns the reconstructed original blob in response. |
+
+
+
+
+
+
+
@@ -379,392 +376,296 @@ BlobCertificate is what gets attested by the network
-
+
Top
-## common/common.proto
+## disperser/disperser.proto
-
+
+
+### AuthenticatedReply
-### BlobCommitment
-BlobCommitment represents commitment of a specific blob, containing its
-KZG commitment, degree proof, the actual degree, and data length in number of symbols.
| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
-| commitment | [bytes](#bytes) | | |
-| length_commitment | [bytes](#bytes) | | |
-| length_proof | [bytes](#bytes) | | |
-| length | [uint32](#uint32) | | |
+| blob_auth_header | [BlobAuthHeader](#disperser-BlobAuthHeader) | | |
+| disperse_reply | [DisperseBlobReply](#disperser-DisperseBlobReply) | | |
-
+
-### G1Commitment
+### AuthenticatedRequest
| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
-| x | [bytes](#bytes) | | The X coordinate of the KZG commitment. This is the raw byte representation of the field element. |
-| y | [bytes](#bytes) | | The Y coordinate of the KZG commitment. This is the raw byte representation of the field element. |
-
+| disperse_request | [DisperseBlobRequest](#disperser-DisperseBlobRequest) | | |
+| authentication_data | [AuthenticationData](#disperser-AuthenticationData) | | |
-
-### PaymentHeader
+
+### AuthenticationData
+AuthenticationData contains the signature of the BlobAuthHeader.
| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
-| account_id | [string](#string) | | |
-| bin_index | [uint32](#uint32) | | |
-| cumulative_payment | [bytes](#bytes) | | |
+| authentication_data | [bytes](#bytes) | | |
-
-
+
-
+### BatchHeader
-
+| Field | Type | Label | Description |
+| ----- | ---- | ----- | ----------- |
+| batch_root | [bytes](#bytes) | | The root of the merkle tree with the hashes of blob headers as leaves. |
+| quorum_numbers | [bytes](#bytes) | | All quorums associated with blobs in this batch. Sorted in ascending order. Ex. [0, 2, 1] => 0x000102 |
+| quorum_signed_percentages | [bytes](#bytes) | | The percentage of stake that has signed for this batch. The quorum_signed_percentages[i] is percentage for the quorum_numbers[i]. |
+| reference_block_number | [uint32](#uint32) | | The Ethereum block number at which the batch was created. The Disperser will encode and disperse the blobs based on the onchain info (e.g. operator stakes) at this block number. |
+
-
-Top
-## node/node.proto
-
+
-### AttestBatchReply
+### BatchMetadata
| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
-| signature | [bytes](#bytes) | | |
-
+| batch_header | [BatchHeader](#disperser-BatchHeader) | | |
+| signatory_record_hash | [bytes](#bytes) | | The hash of all public keys of the operators that did not sign the batch. |
+| fee | [bytes](#bytes) | | The fee payment paid by users for dispersing this batch. It's the bytes representation of a big.Int value. |
+| confirmation_block_number | [uint32](#uint32) | | The Ethereum block number at which the batch is confirmed onchain. |
+| batch_header_hash | [bytes](#bytes) | | This is the hash of the ReducedBatchHeader defined onchain, see: https://github.com/Layr-Labs/eigenda/blob/master/contracts/src/interfaces/IEigenDAServiceManager.sol#L43 The is the message that the operators will sign their signatures on. |
-
-### AttestBatchRequest
+
+### BlobAuthHeader
+BlobAuthHeader contains information about the blob for the client to verify and sign.
+- Once payments are enabled, the BlobAuthHeader will contain the KZG commitment to the blob, which the client
+will verify and sign. Having the client verify the KZG commitment instead of calculating it avoids
+the need for the client to have the KZG structured reference string (SRS), which can be large.
+The signed KZG commitment prevents the disperser from sending a different blob to the DA Nodes
+than the one the client sent.
+- In the meantime, the BlobAuthHeader contains a simple challenge parameter is used to prevent
+replay attacks in the event that a signature is leaked.
| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
-| batch_header | [BatchHeader](#node-BatchHeader) | | header of the batch |
-| blob_header_hashes | [bytes](#bytes) | repeated | the header hashes of all blobs in the batch |
+| challenge_parameter | [uint32](#uint32) | | |
-
+
+
+### BlobHeader
-### BatchHeader
-BatchHeader (see core/data.go#BatchHeader)
| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
-| batch_root | [bytes](#bytes) | | The root of the merkle tree with hashes of blob headers as leaves. |
-| reference_block_number | [uint32](#uint32) | | The Ethereum block number at which the batch is dispersed. |
+| commitment | [common.G1Commitment](#common-G1Commitment) | | KZG commitment of the blob. |
+| data_length | [uint32](#uint32) | | The length of the blob in symbols (each symbol is 32 bytes). |
+| blob_quorum_params | [BlobQuorumParam](#disperser-BlobQuorumParam) | repeated | The params of the quorums that this blob participates in. |
-
+
-### Blob
-In EigenDA, the original blob to disperse is encoded as a polynomial via taking
-taking different point evaluations (i.e. erasure coding). These points are split
-into disjoint subsets which are assigned to different operator nodes in the EigenDA
-network.
-The data in this message is a subset of these points that are assigned to a
-single operator node.
+### BlobInfo
+BlobInfo contains information needed to confirm the blob against the EigenDA contracts
| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
-| header | [BlobHeader](#node-BlobHeader) | | Which (original) blob this is for. |
-| bundles | [Bundle](#node-Bundle) | repeated | Each bundle contains all chunks for a single quorum of the blob. The number of bundles must be equal to the total number of quorums associated with the blob, and the ordering must be the same as BlobHeader.quorum_headers. Note: an operator may be in some but not all of the quorums; in that case the bundle corresponding to that quorum will be empty. |
+| blob_header | [BlobHeader](#disperser-BlobHeader) | | |
+| blob_verification_proof | [BlobVerificationProof](#disperser-BlobVerificationProof) | | |
-
-
-### BlobHeader
-
-
-
-| Field | Type | Label | Description |
-| ----- | ---- | ----- | ----------- |
-| commitment | [common.G1Commitment](#common-G1Commitment) | | The KZG commitment to the polynomial representing the blob. |
-| length_commitment | [G2Commitment](#node-G2Commitment) | | The KZG commitment to the polynomial representing the blob on G2, it is used for proving the degree of the polynomial |
-| length_proof | [G2Commitment](#node-G2Commitment) | | The low degree proof. It's the KZG commitment to the polynomial shifted to the largest SRS degree. |
-| length | [uint32](#uint32) | | The length of the original blob in number of symbols (in the field where the polynomial is defined). |
-| quorum_headers | [BlobQuorumInfo](#node-BlobQuorumInfo) | repeated | The params of the quorums that this blob participates in. |
-| account_id | [string](#string) | | The ID of the user who is dispersing this blob to EigenDA. |
-| reference_block_number | [uint32](#uint32) | | The reference block number whose state is used to encode the blob |
-
-
-
-
-
-
-
-
-### BlobQuorumInfo
-See BlobQuorumParam as defined in
-api/proto/disperser/disperser.proto
-
-
-| Field | Type | Label | Description |
-| ----- | ---- | ----- | ----------- |
-| quorum_id | [uint32](#uint32) | | |
-| adversary_threshold | [uint32](#uint32) | | |
-| confirmation_threshold | [uint32](#uint32) | | |
-| chunk_length | [uint32](#uint32) | | |
-| ratelimit | [uint32](#uint32) | | |
-
-
-
-
-
-
-
-
-### Bundle
-A Bundle is the collection of chunks associated with a single blob, for a single
-operator and a single quorum.
-
-
-| Field | Type | Label | Description |
-| ----- | ---- | ----- | ----------- |
-| chunks | [bytes](#bytes) | repeated | Each chunk corresponds to a collection of points on the polynomial. Each chunk has same number of points. |
-| bundle | [bytes](#bytes) | | All chunks of the bundle encoded in a byte array. |
-
-
-
-
-
-
-
-
-### G2Commitment
-
-
-
-| Field | Type | Label | Description |
-| ----- | ---- | ----- | ----------- |
-| x_a0 | [bytes](#bytes) | | The A0 element of the X coordinate of G2 point. |
-| x_a1 | [bytes](#bytes) | | The A1 element of the X coordinate of G2 point. |
-| y_a0 | [bytes](#bytes) | | The A0 element of the Y coordinate of G2 point. |
-| y_a1 | [bytes](#bytes) | | The A1 element of the Y coordinate of G2 point. |
-
-
-
-
-
-
-
-
-### GetBlobHeaderReply
-
-
-
-| Field | Type | Label | Description |
-| ----- | ---- | ----- | ----------- |
-| blob_header | [BlobHeader](#node-BlobHeader) | | The header of the blob requested per GetBlobHeaderRequest. |
-| proof | [MerkleProof](#node-MerkleProof) | | Merkle proof that returned blob header belongs to the batch and is the batch's MerkleProof.index-th blob. This can be checked against the batch root on chain. |
-
-
-
-
-
+
-
+### BlobQuorumParam
-### GetBlobHeaderRequest
-See RetrieveChunksRequest for documentation of each parameter of GetBlobHeaderRequest.
| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
-| batch_header_hash | [bytes](#bytes) | | |
-| blob_index | [uint32](#uint32) | | |
-| quorum_id | [uint32](#uint32) | | |
+| quorum_number | [uint32](#uint32) | | The ID of the quorum. |
+| adversary_threshold_percentage | [uint32](#uint32) | | The max percentage of stake within the quorum that can be held by or delegated to adversarial operators. Currently, this and the next parameter are standardized across the quorum using values read from the EigenDA contracts. |
+| confirmation_threshold_percentage | [uint32](#uint32) | | The min percentage of stake that must attest in order to consider the dispersal is successful. |
+| chunk_length | [uint32](#uint32) | | The length of each chunk. |
-
+
-### MerkleProof
+### BlobStatusReply
| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
-| hashes | [bytes](#bytes) | repeated | The proof itself. |
-| index | [uint32](#uint32) | | Which index (the leaf of the Merkle tree) this proof is for. |
+| status | [BlobStatus](#disperser-BlobStatus) | | The status of the blob. |
+| info | [BlobInfo](#disperser-BlobInfo) | | The blob info needed for clients to confirm the blob against the EigenDA contracts. |
-
+
-### NodeInfoReply
-Node info reply
+### BlobStatusRequest
+BlobStatusRequest is used to query the status of a blob.
| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
-| semver | [string](#string) | | |
-| arch | [string](#string) | | |
-| os | [string](#string) | | |
-| num_cpu | [uint32](#uint32) | | |
-| mem_bytes | [uint64](#uint64) | | |
-
-
-
-
-
-
-
-
-### NodeInfoRequest
-Node info request
+| request_id | [bytes](#bytes) | | |
-
+
-### RetrieveChunksReply
+### BlobVerificationProof
| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
-| chunks | [bytes](#bytes) | repeated | All chunks the Node is storing for the requested blob per RetrieveChunksRequest. |
-| chunk_encoding_format | [ChunkEncodingFormat](#node-ChunkEncodingFormat) | | How the above chunks are encoded. |
+| batch_id | [uint32](#uint32) | | batch_id is an incremental ID assigned to a batch by EigenDAServiceManager |
+| blob_index | [uint32](#uint32) | | The index of the blob in the batch (which is logically an ordered list of blobs). |
+| batch_metadata | [BatchMetadata](#disperser-BatchMetadata) | | |
+| inclusion_proof | [bytes](#bytes) | | inclusion_proof is a merkle proof for a blob header's inclusion in a batch |
+| quorum_indexes | [bytes](#bytes) | | indexes of quorums in BatchHeader.quorum_numbers that match the quorums in BlobHeader.blob_quorum_params Ex. BlobHeader.blob_quorum_params = [ { quorum_number = 0, ... }, { quorum_number = 3, ... }, { quorum_number = 5, ... }, ] BatchHeader.quorum_numbers = [0, 5, 3] => 0x000503 Then, quorum_indexes = [0, 2, 1] => 0x000201 |
-
+
-### RetrieveChunksRequest
+### DisperseBlobReply
| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
-| batch_header_hash | [bytes](#bytes) | | The hash of the ReducedBatchHeader defined onchain, see: https://github.com/Layr-Labs/eigenda/blob/master/contracts/src/interfaces/IEigenDAServiceManager.sol#L43 This identifies which batch to retrieve for. |
-| blob_index | [uint32](#uint32) | | Which blob in the batch to retrieve for (note: a batch is logically an ordered list of blobs). |
-| quorum_id | [uint32](#uint32) | | Which quorum of the blob to retrieve for (note: a blob can have multiple quorums and the chunks for different quorums at a Node can be different). The ID must be in range [0, 254]. |
+| result | [BlobStatus](#disperser-BlobStatus) | | The status of the blob associated with the request_id. Will always be PROCESSING. |
+| request_id | [bytes](#bytes) | | The request ID generated by the disperser. Once a request is accepted (although not processed), a unique request ID will be generated. Two different DisperseBlobRequests (determined by the hash of the DisperseBlobRequest) will have different IDs, and the same DisperseBlobRequest sent repeatedly at different times will also have different IDs. The client should use this ID to query the processing status of the request (via the GetBlobStatus API). |
-
+
-### StoreBlobsReply
+### DisperseBlobRequest
| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
-| signatures | [google.protobuf.BytesValue](#google-protobuf-BytesValue) | repeated | The operator's BLS sgnature signed on the blob header hashes. The ordering of the signatures must match the ordering of the blobs sent in the request, with empty signatures in the places for discarded blobs. |
+| data | [bytes](#bytes) | | The data to be dispersed. The size of data must be <= 16MiB. Every 32 bytes of data is interpreted as an integer in big endian format where the lower address has more significant bits. The integer must stay in the valid range to be interpreted as a field element on the bn254 curve. The valid range is 0 <= x < 21888242871839275222246405745257275088548364400416034343698204186575808495617 If any one of the 32 bytes elements is outside the range, the whole request is deemed as invalid, and rejected. |
+| custom_quorum_numbers | [uint32](#uint32) | repeated | The quorums to which the blob will be sent, in addition to the required quorums which are configured on the EigenDA smart contract. If required quorums are included here, an error will be returned. The disperser will ensure that the encoded blobs for each quorum are all processed within the same batch. |
+| account_id | [string](#string) | | The account ID of the client. This should be a hex-encoded string of the ECSDA public key corresponding to the key used by the client to sign the BlobAuthHeader. |
-
+
-### StoreBlobsRequest
+### DispersePaidBlobRequest
| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
-| blobs | [Blob](#node-Blob) | repeated | Blobs to store |
-| reference_block_number | [uint32](#uint32) | | The reference block number whose state is used to encode the blobs |
-
+| data | [bytes](#bytes) | | The data to be dispersed. Same requirements as DisperseBlobRequest. |
+| quorum_numbers | [uint32](#uint32) | repeated | The quorums to which the blob to be sent |
+| payment_header | [common.PaymentHeader](#common-PaymentHeader) | | Payment header contains AccountID, BinIndex, and CumulativePayment |
+| payment_signature | [bytes](#bytes) | | signature of payment_header |
-
-### StoreChunksReply
+
+### RetrieveBlobReply
+RetrieveBlobReply contains the retrieved blob data
| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
-| signature | [bytes](#bytes) | | The operator's BLS signature signed on the batch header hash. |
-
+| data | [bytes](#bytes) | | |
-
-### StoreChunksRequest
+
+### RetrieveBlobRequest
+RetrieveBlobRequest contains parameters to retrieve the blob.
| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
-| batch_header | [BatchHeader](#node-BatchHeader) | | Which batch this request is for. |
-| blobs | [Blob](#node-Blob) | repeated | The chunks for each blob in the batch to be stored in an EigenDA Node. |
+| batch_header_hash | [bytes](#bytes) | | |
+| blob_index | [uint32](#uint32) | | |
@@ -773,175 +674,50 @@ Node info request
-
+
-### ChunkEncodingFormat
-This describes how the chunks returned in RetrieveChunksReply are encoded.
-Used to facilitate the decoding of chunks.
+### BlobStatus
+BlobStatus represents the status of a blob.
+The status of a blob is updated as the blob is processed by the disperser.
+The status of a blob can be queried by the client using the GetBlobStatus API.
+Intermediate states are states that the blob can be in while being processed, and it can be updated to a differet state:
+- PROCESSING
+- DISPERSING
+- CONFIRMED
+Terminal states are states that will not be updated to a different state:
+- FAILED
+- FINALIZED
+- INSUFFICIENT_SIGNATURES
| Name | Number | Description |
| ---- | ------ | ----------- |
| UNKNOWN | 0 | |
-| GNARK | 1 | |
-| GOB | 2 | |
-
-
-
-
-
-
-
-
-
-### Dispersal
-
-
-| Method Name | Request Type | Response Type | Description |
-| ----------- | ------------ | ------------- | ------------|
-| StoreChunks | [StoreChunksRequest](#node-StoreChunksRequest) | [StoreChunksReply](#node-StoreChunksReply) | StoreChunks validates that the chunks match what the Node is supposed to receive ( different Nodes are responsible for different chunks, as EigenDA is horizontally sharded) and is correctly coded (e.g. each chunk must be a valid KZG multiproof) according to the EigenDA protocol. It also stores the chunks along with metadata for the protocol-defined length of custody. It will return a signature at the end to attest to the data in this request it has processed. |
-| StoreBlobs | [StoreBlobsRequest](#node-StoreBlobsRequest) | [StoreBlobsReply](#node-StoreBlobsReply) | StoreBlobs is simiar to StoreChunks, but it stores the blobs using a different storage schema so that the stored blobs can later be aggregated by AttestBatch method to a bigger batch. StoreBlobs + AttestBatch will eventually replace and deprecate StoreChunks method. DEPRECATED: StoreBlobs method is not used |
-| AttestBatch | [AttestBatchRequest](#node-AttestBatchRequest) | [AttestBatchReply](#node-AttestBatchReply) | AttestBatch is used to aggregate the batches stored by StoreBlobs method to a bigger batch. It will return a signature at the end to attest to the aggregated batch. DEPRECATED: AttestBatch method is not used |
-| NodeInfo | [NodeInfoRequest](#node-NodeInfoRequest) | [NodeInfoReply](#node-NodeInfoReply) | Retrieve node info metadata |
-
-
-
-
-### Retrieval
-
-
-| Method Name | Request Type | Response Type | Description |
-| ----------- | ------------ | ------------- | ------------|
-| RetrieveChunks | [RetrieveChunksRequest](#node-RetrieveChunksRequest) | [RetrieveChunksReply](#node-RetrieveChunksReply) | RetrieveChunks retrieves the chunks for a blob custodied at the Node. |
-| GetBlobHeader | [GetBlobHeaderRequest](#node-GetBlobHeaderRequest) | [GetBlobHeaderReply](#node-GetBlobHeaderReply) | GetBlobHeader is similar to RetrieveChunks, this just returns the header of the blob. |
-| NodeInfo | [NodeInfoRequest](#node-NodeInfoRequest) | [NodeInfoReply](#node-NodeInfoReply) | Retrieve node info metadata |
-
-
-
-
-
-
-Top
-
-## node/v2/node_v2.proto
-
-
-
-
-
-### GetChunksReply
-
-
-
-| Field | Type | Label | Description |
-| ----- | ---- | ----- | ----------- |
-| chunks | [bytes](#bytes) | repeated | All chunks the Node is storing for the requested blob per RetrieveChunksRequest. |
-
-
-
-
-
-
-
-
-### GetChunksRequest
-
-
-
-| Field | Type | Label | Description |
-| ----- | ---- | ----- | ----------- |
-| blob_key | [bytes](#bytes) | | |
-| quorum_id | [uint32](#uint32) | | Which quorum of the blob to retrieve for (note: a blob can have multiple quorums and the chunks for different quorums at a Node can be different). The ID must be in range [0, 254]. |
-
-
-
-
-
-
-
-
-### NodeInfoReply
-Node info reply
-
-
-| Field | Type | Label | Description |
-| ----- | ---- | ----- | ----------- |
-| semver | [string](#string) | | |
-| arch | [string](#string) | | |
-| os | [string](#string) | | |
-| num_cpu | [uint32](#uint32) | | |
-| mem_bytes | [uint64](#uint64) | | |
-
-
-
-
-
-
-
-
-### NodeInfoRequest
-Node info request
-
-
-
-
-
-
-
-
-### StoreChunksReply
-
-
-
-| Field | Type | Label | Description |
-| ----- | ---- | ----- | ----------- |
-| signature | [bytes](#bytes) | | |
-
-
-
-
-
-
-
-
-### StoreChunksRequest
-
-
-
-| Field | Type | Label | Description |
-| ----- | ---- | ----- | ----------- |
-| batch | [common.v2.Batch](#common-v2-Batch) | | batch of blobs to store |
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-### Dispersal
-WARNING: the following RPCs are experimental and subject to change.
-
-| Method Name | Request Type | Response Type | Description |
-| ----------- | ------------ | ------------- | ------------|
-| StoreChunks | [StoreChunksRequest](#node-v2-StoreChunksRequest) | [StoreChunksReply](#node-v2-StoreChunksReply) | |
-| NodeInfo | [NodeInfoRequest](#node-v2-NodeInfoRequest) | [NodeInfoReply](#node-v2-NodeInfoReply) | |
+| PROCESSING | 1 | PROCESSING means that the blob is currently being processed by the disperser |
+| CONFIRMED | 2 | CONFIRMED means that the blob has been dispersed to DA Nodes and the dispersed batch containing the blob has been confirmed onchain |
+| FAILED | 3 | FAILED means that the blob has failed permanently (for reasons other than insufficient signatures, which is a separate state). This status is somewhat of a catch-all category, containg (but not necessarily exclusively as errors can be added in the future): - blob has expired - internal logic error while requesting encoding - blob retry has exceeded its limit while waiting for blob finalization after confirmation. Most likely triggered by a chain reorg: see https://github.com/Layr-Labs/eigenda/blob/master/disperser/batcher/finalizer.go#L179-L189. |
+| FINALIZED | 4 | FINALIZED means that the block containing the blob's confirmation transaction has been finalized on Ethereum |
+| INSUFFICIENT_SIGNATURES | 5 | INSUFFICIENT_SIGNATURES means that the confirmation threshold for the blob was not met for at least one quorum. |
+| DISPERSING | 6 | The DISPERSING state is comprised of two separate phases: - Dispersing to DA nodes and collecting signature - Submitting the transaction on chain and waiting for tx receipt |
-
+
-### Retrieval
+
+
+
+
+### Disperser
+Disperser defines the public APIs for dispersing blobs.
| Method Name | Request Type | Response Type | Description |
| ----------- | ------------ | ------------- | ------------|
-| GetChunks | [GetChunksRequest](#node-v2-GetChunksRequest) | [GetChunksReply](#node-v2-GetChunksReply) | GetChunks retrieves the chunks for a blob custodied at the Node. |
-| NodeInfo | [NodeInfoRequest](#node-v2-NodeInfoRequest) | [NodeInfoReply](#node-v2-NodeInfoReply) | Retrieve node info metadata |
+| DisperseBlob | [DisperseBlobRequest](#disperser-DisperseBlobRequest) | [DisperseBlobReply](#disperser-DisperseBlobReply) | DisperseBlob accepts a single blob to be dispersed. This executes the dispersal async, i.e. it returns once the request is accepted. The client should use GetBlobStatus() API to poll the processing status of the blob.
+
+If DisperseBlob returns the following error codes: INVALID_ARGUMENT (400): request is invalid for a reason specified in the error msg. RESOURCE_EXHAUSTED (429): request is rate limited for the quorum specified in the error msg. user should retry after the specified duration. INTERNAL (500): serious error, user should NOT retry. |
+| DisperseBlobAuthenticated | [AuthenticatedRequest](#disperser-AuthenticatedRequest) stream | [AuthenticatedReply](#disperser-AuthenticatedReply) stream | DisperseBlobAuthenticated is similar to DisperseBlob, except that it requires the client to authenticate itself via the AuthenticationData message. The protoco is as follows: 1. The client sends a DisperseBlobAuthenticated request with the DisperseBlobRequest message 2. The Disperser sends back a BlobAuthHeader message containing information for the client to verify and sign. 3. The client verifies the BlobAuthHeader and sends back the signed BlobAuthHeader in an AuthenticationData message. 4. The Disperser verifies the signature and returns a DisperseBlobReply message. |
+| GetBlobStatus | [BlobStatusRequest](#disperser-BlobStatusRequest) | [BlobStatusReply](#disperser-BlobStatusReply) | This API is meant to be polled for the blob status. |
+| RetrieveBlob | [RetrieveBlobRequest](#disperser-RetrieveBlobRequest) | [RetrieveBlobReply](#disperser-RetrieveBlobReply) | This retrieves the requested blob from the Disperser's backend. This is a more efficient way to retrieve blobs than directly retrieving from the DA Nodes (see detail about this approach in api/proto/retriever/retriever.proto). The blob should have been initially dispersed via this Disperser service for this API to work. |
@@ -1147,296 +923,469 @@ Disperser defines the public APIs for dispersing blobs.
-
+
Top
-## disperser/disperser.proto
+## node/node.proto
-
+
-### AuthenticatedReply
+### AttestBatchReply
| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
-| blob_auth_header | [BlobAuthHeader](#disperser-BlobAuthHeader) | | |
-| disperse_reply | [DisperseBlobReply](#disperser-DisperseBlobReply) | | |
+| signature | [bytes](#bytes) | | |
-
+
-### AuthenticatedRequest
+### AttestBatchRequest
| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
-| disperse_request | [DisperseBlobRequest](#disperser-DisperseBlobRequest) | | |
-| authentication_data | [AuthenticationData](#disperser-AuthenticationData) | | |
+| batch_header | [BatchHeader](#node-BatchHeader) | | header of the batch |
+| blob_header_hashes | [bytes](#bytes) | repeated | the header hashes of all blobs in the batch |
-
+
-### AuthenticationData
-AuthenticationData contains the signature of the BlobAuthHeader.
+### BatchHeader
+BatchHeader (see core/data.go#BatchHeader)
| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
-| authentication_data | [bytes](#bytes) | | |
+| batch_root | [bytes](#bytes) | | The root of the merkle tree with hashes of blob headers as leaves. |
+| reference_block_number | [uint32](#uint32) | | The Ethereum block number at which the batch is dispersed. |
-
+
+
+### Blob
+In EigenDA, the original blob to disperse is encoded as a polynomial via taking
+taking different point evaluations (i.e. erasure coding). These points are split
+into disjoint subsets which are assigned to different operator nodes in the EigenDA
+network.
+The data in this message is a subset of these points that are assigned to a
+single operator node.
+
+
+| Field | Type | Label | Description |
+| ----- | ---- | ----- | ----------- |
+| header | [BlobHeader](#node-BlobHeader) | | Which (original) blob this is for. |
+| bundles | [Bundle](#node-Bundle) | repeated | Each bundle contains all chunks for a single quorum of the blob. The number of bundles must be equal to the total number of quorums associated with the blob, and the ordering must be the same as BlobHeader.quorum_headers. Note: an operator may be in some but not all of the quorums; in that case the bundle corresponding to that quorum will be empty. |
+
-### BatchHeader
+
+
+
+
+
+
+### BlobHeader
| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
-| batch_root | [bytes](#bytes) | | The root of the merkle tree with the hashes of blob headers as leaves. |
-| quorum_numbers | [bytes](#bytes) | | All quorums associated with blobs in this batch. Sorted in ascending order. Ex. [0, 2, 1] => 0x000102 |
-| quorum_signed_percentages | [bytes](#bytes) | | The percentage of stake that has signed for this batch. The quorum_signed_percentages[i] is percentage for the quorum_numbers[i]. |
-| reference_block_number | [uint32](#uint32) | | The Ethereum block number at which the batch was created. The Disperser will encode and disperse the blobs based on the onchain info (e.g. operator stakes) at this block number. |
+| commitment | [common.G1Commitment](#common-G1Commitment) | | The KZG commitment to the polynomial representing the blob. |
+| length_commitment | [G2Commitment](#node-G2Commitment) | | The KZG commitment to the polynomial representing the blob on G2, it is used for proving the degree of the polynomial |
+| length_proof | [G2Commitment](#node-G2Commitment) | | The low degree proof. It's the KZG commitment to the polynomial shifted to the largest SRS degree. |
+| length | [uint32](#uint32) | | The length of the original blob in number of symbols (in the field where the polynomial is defined). |
+| quorum_headers | [BlobQuorumInfo](#node-BlobQuorumInfo) | repeated | The params of the quorums that this blob participates in. |
+| account_id | [string](#string) | | The ID of the user who is dispersing this blob to EigenDA. |
+| reference_block_number | [uint32](#uint32) | | The reference block number whose state is used to encode the blob |
-
+
-### BatchMetadata
+### BlobQuorumInfo
+See BlobQuorumParam as defined in
+api/proto/disperser/disperser.proto
+
+
+| Field | Type | Label | Description |
+| ----- | ---- | ----- | ----------- |
+| quorum_id | [uint32](#uint32) | | |
+| adversary_threshold | [uint32](#uint32) | | |
+| confirmation_threshold | [uint32](#uint32) | | |
+| chunk_length | [uint32](#uint32) | | |
+| ratelimit | [uint32](#uint32) | | |
+
+
+
+
+
+
+
+### Bundle
+A Bundle is the collection of chunks associated with a single blob, for a single
+operator and a single quorum.
| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
-| batch_header | [BatchHeader](#disperser-BatchHeader) | | |
-| signatory_record_hash | [bytes](#bytes) | | The hash of all public keys of the operators that did not sign the batch. |
-| fee | [bytes](#bytes) | | The fee payment paid by users for dispersing this batch. It's the bytes representation of a big.Int value. |
-| confirmation_block_number | [uint32](#uint32) | | The Ethereum block number at which the batch is confirmed onchain. |
-| batch_header_hash | [bytes](#bytes) | | This is the hash of the ReducedBatchHeader defined onchain, see: https://github.com/Layr-Labs/eigenda/blob/master/contracts/src/interfaces/IEigenDAServiceManager.sol#L43 The is the message that the operators will sign their signatures on. |
+| chunks | [bytes](#bytes) | repeated | Each chunk corresponds to a collection of points on the polynomial. Each chunk has same number of points. |
+| bundle | [bytes](#bytes) | | All chunks of the bundle encoded in a byte array. |
-
+
+
+### G2Commitment
+
+
+
+| Field | Type | Label | Description |
+| ----- | ---- | ----- | ----------- |
+| x_a0 | [bytes](#bytes) | | The A0 element of the X coordinate of G2 point. |
+| x_a1 | [bytes](#bytes) | | The A1 element of the X coordinate of G2 point. |
+| y_a0 | [bytes](#bytes) | | The A0 element of the Y coordinate of G2 point. |
+| y_a1 | [bytes](#bytes) | | The A1 element of the Y coordinate of G2 point. |
+
+
+
+
+
+
+
+
+### GetBlobHeaderReply
+
+
+
+| Field | Type | Label | Description |
+| ----- | ---- | ----- | ----------- |
+| blob_header | [BlobHeader](#node-BlobHeader) | | The header of the blob requested per GetBlobHeaderRequest. |
+| proof | [MerkleProof](#node-MerkleProof) | | Merkle proof that returned blob header belongs to the batch and is the batch's MerkleProof.index-th blob. This can be checked against the batch root on chain. |
+
+
+
+
+
+
+
+
+### GetBlobHeaderRequest
+See RetrieveChunksRequest for documentation of each parameter of GetBlobHeaderRequest.
+
+
+| Field | Type | Label | Description |
+| ----- | ---- | ----- | ----------- |
+| batch_header_hash | [bytes](#bytes) | | |
+| blob_index | [uint32](#uint32) | | |
+| quorum_id | [uint32](#uint32) | | |
+
+
+
+
+
+
+
+
+### MerkleProof
+
+
+
+| Field | Type | Label | Description |
+| ----- | ---- | ----- | ----------- |
+| hashes | [bytes](#bytes) | repeated | The proof itself. |
+| index | [uint32](#uint32) | | Which index (the leaf of the Merkle tree) this proof is for. |
+
+
+
+
+
+
+
+
+### NodeInfoReply
+Node info reply
+
+
+| Field | Type | Label | Description |
+| ----- | ---- | ----- | ----------- |
+| semver | [string](#string) | | |
+| arch | [string](#string) | | |
+| os | [string](#string) | | |
+| num_cpu | [uint32](#uint32) | | |
+| mem_bytes | [uint64](#uint64) | | |
+
+
+
+
+
+
+
+
+### NodeInfoRequest
+Node info request
+
+
+
+
+
+
+
+
+### RetrieveChunksReply
+
+
+
+| Field | Type | Label | Description |
+| ----- | ---- | ----- | ----------- |
+| chunks | [bytes](#bytes) | repeated | All chunks the Node is storing for the requested blob per RetrieveChunksRequest. |
+| chunk_encoding_format | [ChunkEncodingFormat](#node-ChunkEncodingFormat) | | How the above chunks are encoded. |
+
+
+
+
+
+
+
+
+### RetrieveChunksRequest
+
+
+
+| Field | Type | Label | Description |
+| ----- | ---- | ----- | ----------- |
+| batch_header_hash | [bytes](#bytes) | | The hash of the ReducedBatchHeader defined onchain, see: https://github.com/Layr-Labs/eigenda/blob/master/contracts/src/interfaces/IEigenDAServiceManager.sol#L43 This identifies which batch to retrieve for. |
+| blob_index | [uint32](#uint32) | | Which blob in the batch to retrieve for (note: a batch is logically an ordered list of blobs). |
+| quorum_id | [uint32](#uint32) | | Which quorum of the blob to retrieve for (note: a blob can have multiple quorums and the chunks for different quorums at a Node can be different). The ID must be in range [0, 254]. |
+
+
+
+
+
+
+
+
+### StoreBlobsReply
-### BlobAuthHeader
-BlobAuthHeader contains information about the blob for the client to verify and sign.
-- Once payments are enabled, the BlobAuthHeader will contain the KZG commitment to the blob, which the client
-will verify and sign. Having the client verify the KZG commitment instead of calculating it avoids
-the need for the client to have the KZG structured reference string (SRS), which can be large.
-The signed KZG commitment prevents the disperser from sending a different blob to the DA Nodes
-than the one the client sent.
-- In the meantime, the BlobAuthHeader contains a simple challenge parameter is used to prevent
-replay attacks in the event that a signature is leaked.
| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
-| challenge_parameter | [uint32](#uint32) | | |
+| signatures | [google.protobuf.BytesValue](#google-protobuf-BytesValue) | repeated | The operator's BLS sgnature signed on the blob header hashes. The ordering of the signatures must match the ordering of the blobs sent in the request, with empty signatures in the places for discarded blobs. |
-
+
-### BlobHeader
+### StoreBlobsRequest
| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
-| commitment | [common.G1Commitment](#common-G1Commitment) | | KZG commitment of the blob. |
-| data_length | [uint32](#uint32) | | The length of the blob in symbols (each symbol is 32 bytes). |
-| blob_quorum_params | [BlobQuorumParam](#disperser-BlobQuorumParam) | repeated | The params of the quorums that this blob participates in. |
+| blobs | [Blob](#node-Blob) | repeated | Blobs to store |
+| reference_block_number | [uint32](#uint32) | | The reference block number whose state is used to encode the blobs |
-
+
+
+### StoreChunksReply
-### BlobInfo
-BlobInfo contains information needed to confirm the blob against the EigenDA contracts
| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
-| blob_header | [BlobHeader](#disperser-BlobHeader) | | |
-| blob_verification_proof | [BlobVerificationProof](#disperser-BlobVerificationProof) | | |
+| signature | [bytes](#bytes) | | The operator's BLS signature signed on the batch header hash. |
-
+
-### BlobQuorumParam
+### StoreChunksRequest
| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
-| quorum_number | [uint32](#uint32) | | The ID of the quorum. |
-| adversary_threshold_percentage | [uint32](#uint32) | | The max percentage of stake within the quorum that can be held by or delegated to adversarial operators. Currently, this and the next parameter are standardized across the quorum using values read from the EigenDA contracts. |
-| confirmation_threshold_percentage | [uint32](#uint32) | | The min percentage of stake that must attest in order to consider the dispersal is successful. |
-| chunk_length | [uint32](#uint32) | | The length of each chunk. |
+| batch_header | [BatchHeader](#node-BatchHeader) | | Which batch this request is for. |
+| blobs | [Blob](#node-Blob) | repeated | The chunks for each blob in the batch to be stored in an EigenDA Node. |
+
-
-### BlobStatusReply
+
+
+### ChunkEncodingFormat
+This describes how the chunks returned in RetrieveChunksReply are encoded.
+Used to facilitate the decoding of chunks.
+| Name | Number | Description |
+| ---- | ------ | ----------- |
+| UNKNOWN | 0 | |
+| GNARK | 1 | |
+| GOB | 2 | |
-| Field | Type | Label | Description |
-| ----- | ---- | ----- | ----------- |
-| status | [BlobStatus](#disperser-BlobStatus) | | The status of the blob. |
-| info | [BlobInfo](#disperser-BlobInfo) | | The blob info needed for clients to confirm the blob against the EigenDA contracts. |
+
+
+
+### Dispersal
-
+| Method Name | Request Type | Response Type | Description |
+| ----------- | ------------ | ------------- | ------------|
+| StoreChunks | [StoreChunksRequest](#node-StoreChunksRequest) | [StoreChunksReply](#node-StoreChunksReply) | StoreChunks validates that the chunks match what the Node is supposed to receive ( different Nodes are responsible for different chunks, as EigenDA is horizontally sharded) and is correctly coded (e.g. each chunk must be a valid KZG multiproof) according to the EigenDA protocol. It also stores the chunks along with metadata for the protocol-defined length of custody. It will return a signature at the end to attest to the data in this request it has processed. |
+| StoreBlobs | [StoreBlobsRequest](#node-StoreBlobsRequest) | [StoreBlobsReply](#node-StoreBlobsReply) | StoreBlobs is simiar to StoreChunks, but it stores the blobs using a different storage schema so that the stored blobs can later be aggregated by AttestBatch method to a bigger batch. StoreBlobs + AttestBatch will eventually replace and deprecate StoreChunks method. DEPRECATED: StoreBlobs method is not used |
+| AttestBatch | [AttestBatchRequest](#node-AttestBatchRequest) | [AttestBatchReply](#node-AttestBatchReply) | AttestBatch is used to aggregate the batches stored by StoreBlobs method to a bigger batch. It will return a signature at the end to attest to the aggregated batch. DEPRECATED: AttestBatch method is not used |
+| NodeInfo | [NodeInfoRequest](#node-NodeInfoRequest) | [NodeInfoReply](#node-NodeInfoReply) | Retrieve node info metadata |
-### BlobStatusRequest
-BlobStatusRequest is used to query the status of a blob.
+
-| Field | Type | Label | Description |
-| ----- | ---- | ----- | ----------- |
-| request_id | [bytes](#bytes) | | |
+### Retrieval
+| Method Name | Request Type | Response Type | Description |
+| ----------- | ------------ | ------------- | ------------|
+| RetrieveChunks | [RetrieveChunksRequest](#node-RetrieveChunksRequest) | [RetrieveChunksReply](#node-RetrieveChunksReply) | RetrieveChunks retrieves the chunks for a blob custodied at the Node. |
+| GetBlobHeader | [GetBlobHeaderRequest](#node-GetBlobHeaderRequest) | [GetBlobHeaderReply](#node-GetBlobHeaderReply) | GetBlobHeader is similar to RetrieveChunks, this just returns the header of the blob. |
+| NodeInfo | [NodeInfoRequest](#node-NodeInfoRequest) | [NodeInfoReply](#node-NodeInfoReply) | Retrieve node info metadata |
+
-
+
+Top
-### BlobVerificationProof
+## node/v2/node_v2.proto
+
+
+
+
+
+### GetChunksReply
| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
-| batch_id | [uint32](#uint32) | | batch_id is an incremental ID assigned to a batch by EigenDAServiceManager |
-| blob_index | [uint32](#uint32) | | The index of the blob in the batch (which is logically an ordered list of blobs). |
-| batch_metadata | [BatchMetadata](#disperser-BatchMetadata) | | |
-| inclusion_proof | [bytes](#bytes) | | inclusion_proof is a merkle proof for a blob header's inclusion in a batch |
-| quorum_indexes | [bytes](#bytes) | | indexes of quorums in BatchHeader.quorum_numbers that match the quorums in BlobHeader.blob_quorum_params Ex. BlobHeader.blob_quorum_params = [ { quorum_number = 0, ... }, { quorum_number = 3, ... }, { quorum_number = 5, ... }, ] BatchHeader.quorum_numbers = [0, 5, 3] => 0x000503 Then, quorum_indexes = [0, 2, 1] => 0x000201 |
+| chunks | [bytes](#bytes) | repeated | All chunks the Node is storing for the requested blob per RetrieveChunksRequest. |
-
+
-### DisperseBlobReply
+### GetChunksRequest
| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
-| result | [BlobStatus](#disperser-BlobStatus) | | The status of the blob associated with the request_id. Will always be PROCESSING. |
-| request_id | [bytes](#bytes) | | The request ID generated by the disperser. Once a request is accepted (although not processed), a unique request ID will be generated. Two different DisperseBlobRequests (determined by the hash of the DisperseBlobRequest) will have different IDs, and the same DisperseBlobRequest sent repeatedly at different times will also have different IDs. The client should use this ID to query the processing status of the request (via the GetBlobStatus API). |
-
+| blob_key | [bytes](#bytes) | | |
+| quorum_id | [uint32](#uint32) | | Which quorum of the blob to retrieve for (note: a blob can have multiple quorums and the chunks for different quorums at a Node can be different). The ID must be in range [0, 254]. |
-
-### DisperseBlobRequest
+
+### NodeInfoReply
+Node info reply
| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
-| data | [bytes](#bytes) | | The data to be dispersed. The size of data must be <= 16MiB. Every 32 bytes of data is interpreted as an integer in big endian format where the lower address has more significant bits. The integer must stay in the valid range to be interpreted as a field element on the bn254 curve. The valid range is 0 <= x < 21888242871839275222246405745257275088548364400416034343698204186575808495617 If any one of the 32 bytes elements is outside the range, the whole request is deemed as invalid, and rejected. |
-| custom_quorum_numbers | [uint32](#uint32) | repeated | The quorums to which the blob will be sent, in addition to the required quorums which are configured on the EigenDA smart contract. If required quorums are included here, an error will be returned. The disperser will ensure that the encoded blobs for each quorum are all processed within the same batch. |
-| account_id | [string](#string) | | The account ID of the client. This should be a hex-encoded string of the ECSDA public key corresponding to the key used by the client to sign the BlobAuthHeader. |
-
-
+| semver | [string](#string) | | |
+| arch | [string](#string) | | |
+| os | [string](#string) | | |
+| num_cpu | [uint32](#uint32) | | |
+| mem_bytes | [uint64](#uint64) | | |
-
-### DispersePaidBlobRequest
+
+### NodeInfoRequest
+Node info request
-| Field | Type | Label | Description |
-| ----- | ---- | ----- | ----------- |
-| data | [bytes](#bytes) | | The data to be dispersed. Same requirements as DisperseBlobRequest. |
-| quorum_numbers | [uint32](#uint32) | repeated | The quorums to which the blob to be sent |
-| payment_header | [common.PaymentHeader](#common-PaymentHeader) | | Payment header contains AccountID, BinIndex, and CumulativePayment |
-| payment_signature | [bytes](#bytes) | | signature of payment_header |
+
-
+### StoreChunksReply
-### RetrieveBlobReply
-RetrieveBlobReply contains the retrieved blob data
| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
-| data | [bytes](#bytes) | | |
+| signature | [bytes](#bytes) | | |
-
+
+
+### StoreChunksRequest
-### RetrieveBlobRequest
-RetrieveBlobRequest contains parameters to retrieve the blob.
| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
-| batch_header_hash | [bytes](#bytes) | | |
-| blob_index | [uint32](#uint32) | | |
+| batch | [common.v2.Batch](#common-v2-Batch) | | batch of blobs to store |
@@ -1444,51 +1393,31 @@ RetrieveBlobRequest contains parameters to retrieve the blob.
+
-
+
-### BlobStatus
-BlobStatus represents the status of a blob.
-The status of a blob is updated as the blob is processed by the disperser.
-The status of a blob can be queried by the client using the GetBlobStatus API.
-Intermediate states are states that the blob can be in while being processed, and it can be updated to a differet state:
-- PROCESSING
-- DISPERSING
-- CONFIRMED
-Terminal states are states that will not be updated to a different state:
-- FAILED
-- FINALIZED
-- INSUFFICIENT_SIGNATURES
-| Name | Number | Description |
-| ---- | ------ | ----------- |
-| UNKNOWN | 0 | |
-| PROCESSING | 1 | PROCESSING means that the blob is currently being processed by the disperser |
-| CONFIRMED | 2 | CONFIRMED means that the blob has been dispersed to DA Nodes and the dispersed batch containing the blob has been confirmed onchain |
-| FAILED | 3 | FAILED means that the blob has failed permanently (for reasons other than insufficient signatures, which is a separate state). This status is somewhat of a catch-all category, containg (but not necessarily exclusively as errors can be added in the future): - blob has expired - internal logic error while requesting encoding - blob retry has exceeded its limit while waiting for blob finalization after confirmation. Most likely triggered by a chain reorg: see https://github.com/Layr-Labs/eigenda/blob/master/disperser/batcher/finalizer.go#L179-L189. |
-| FINALIZED | 4 | FINALIZED means that the block containing the blob's confirmation transaction has been finalized on Ethereum |
-| INSUFFICIENT_SIGNATURES | 5 | INSUFFICIENT_SIGNATURES means that the confirmation threshold for the blob was not met for at least one quorum. |
-| DISPERSING | 6 | The DISPERSING state is comprised of two separate phases: - Dispersing to DA nodes and collecting signature - Submitting the transaction on chain and waiting for tx receipt |
+
+### Dispersal
+WARNING: the following RPCs are experimental and subject to change.
-
+| Method Name | Request Type | Response Type | Description |
+| ----------- | ------------ | ------------- | ------------|
+| StoreChunks | [StoreChunksRequest](#node-v2-StoreChunksRequest) | [StoreChunksReply](#node-v2-StoreChunksReply) | |
+| NodeInfo | [NodeInfoRequest](#node-v2-NodeInfoRequest) | [NodeInfoReply](#node-v2-NodeInfoReply) | |
-
+
-
+### Retrieval
-### Disperser
-Disperser defines the public APIs for dispersing blobs.
| Method Name | Request Type | Response Type | Description |
| ----------- | ------------ | ------------- | ------------|
-| DisperseBlob | [DisperseBlobRequest](#disperser-DisperseBlobRequest) | [DisperseBlobReply](#disperser-DisperseBlobReply) | DisperseBlob accepts a single blob to be dispersed. This executes the dispersal async, i.e. it returns once the request is accepted. The client should use GetBlobStatus() API to poll the processing status of the blob.
-
-If DisperseBlob returns the following error codes: INVALID_ARGUMENT (400): request is invalid for a reason specified in the error msg. RESOURCE_EXHAUSTED (429): request is rate limited for the quorum specified in the error msg. user should retry after the specified duration. INTERNAL (500): serious error, user should NOT retry. |
-| DisperseBlobAuthenticated | [AuthenticatedRequest](#disperser-AuthenticatedRequest) stream | [AuthenticatedReply](#disperser-AuthenticatedReply) stream | DisperseBlobAuthenticated is similar to DisperseBlob, except that it requires the client to authenticate itself via the AuthenticationData message. The protoco is as follows: 1. The client sends a DisperseBlobAuthenticated request with the DisperseBlobRequest message 2. The Disperser sends back a BlobAuthHeader message containing information for the client to verify and sign. 3. The client verifies the BlobAuthHeader and sends back the signed BlobAuthHeader in an AuthenticationData message. 4. The Disperser verifies the signature and returns a DisperseBlobReply message. |
-| GetBlobStatus | [BlobStatusRequest](#disperser-BlobStatusRequest) | [BlobStatusReply](#disperser-BlobStatusReply) | This API is meant to be polled for the blob status. |
-| RetrieveBlob | [RetrieveBlobRequest](#disperser-RetrieveBlobRequest) | [RetrieveBlobReply](#disperser-RetrieveBlobReply) | This retrieves the requested blob from the Disperser's backend. This is a more efficient way to retrieve blobs than directly retrieving from the DA Nodes (see detail about this approach in api/proto/retriever/retriever.proto). The blob should have been initially dispersed via this Disperser service for this API to work. |
+| GetChunks | [GetChunksRequest](#node-v2-GetChunksRequest) | [GetChunksReply](#node-v2-GetChunksReply) | GetChunks retrieves the chunks for a blob custodied at the Node. |
+| NodeInfo | [NodeInfoRequest](#node-v2-NodeInfoRequest) | [NodeInfoReply](#node-v2-NodeInfoReply) | Retrieve node info metadata |
@@ -1639,6 +1568,77 @@ Relay is a service that provides access to public relay functionality.
+
+Top
+
+## retriever/retriever.proto
+
+
+
+
+
+### BlobReply
+
+
+
+| Field | Type | Label | Description |
+| ----- | ---- | ----- | ----------- |
+| data | [bytes](#bytes) | | The blob retrieved and reconstructed from the EigenDA Nodes per BlobRequest. |
+
+
+
+
+
+
+
+
+### BlobRequest
+
+
+
+| Field | Type | Label | Description |
+| ----- | ---- | ----- | ----------- |
+| batch_header_hash | [bytes](#bytes) | | The hash of the ReducedBatchHeader defined onchain, see: https://github.com/Layr-Labs/eigenda/blob/master/contracts/src/interfaces/IEigenDAServiceManager.sol#L43 This identifies the batch that this blob belongs to. |
+| blob_index | [uint32](#uint32) | | Which blob in the batch this is requesting for (note: a batch is logically an ordered list of blobs). |
+| reference_block_number | [uint32](#uint32) | | The Ethereum block number at which the batch for this blob was constructed. |
+| quorum_id | [uint32](#uint32) | | Which quorum of the blob this is requesting for (note a blob can participate in multiple quorums). |
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+### Retriever
+The Retriever is a service for retrieving chunks corresponding to a blob from
+the EigenDA operator nodes and reconstructing the original blob from the chunks.
+This is a client-side library that the users are supposed to operationalize.
+
+Note: Users generally have two ways to retrieve a blob from EigenDA:
+ 1) Retrieve from the Disperser that the user initially used for dispersal: the API
+ is Disperser.RetrieveBlob() as defined in api/proto/disperser/disperser.proto
+ 2) Retrieve directly from the EigenDA Nodes, which is supported by this Retriever.
+
+The Disperser.RetrieveBlob() (the 1st approach) is generally faster and cheaper as the
+Disperser manages the blobs that it has processed, whereas the Retriever.RetrieveBlob()
+(the 2nd approach here) removes the need to trust the Disperser, with the downside of
+worse cost and performance.
+
+| Method Name | Request Type | Response Type | Description |
+| ----------- | ------------ | ------------- | ------------|
+| RetrieveBlob | [BlobRequest](#retriever-BlobRequest) | [BlobReply](#retriever-BlobReply) | This fans out request to EigenDA Nodes to retrieve the chunks and returns the reconstructed original blob in response. |
+
+
+
+
+
## Scalar Value Types
| .proto Type | Notes | C++ | Java | Python | Go | C# | PHP | Ruby |
|