feat(conductor)!: rate limit sequencer cometbft requests #1068
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Limits the number of requests conductor sends to the Sequencer CometBFT endpoint to 100 per minute.
Background
During sync conductor can DOS Sequencer's CometBFT node by sending too many requests for commits and validator sets. With the batching logic introduced in #1049 there can be dozens (or more) blocks stored in each Celestia blob, each of which needs to be checked separately. With several blobs being fetched at once during, this can quickly spiral into hundreds (if not thousands) requests per minute.
Note that only calls to
/commit
and/validators
are rate limited, because there is currently no way to enforce this at the transport layer, see this issue: informalsystems/tendermint-rs#1420However, the only other calls are to
/genesis
(once at startup), and/abci_info
(every block-time period, usually every 2 seconds), which is rare enough to not need a rate limit.Changes
RateLimit
middleware around a tendermint-rsHttpClient
only send up to 100 requests per minute.Breaking changes
ASTRIA_CONDUCTOR_SEQUENCER_REQUESTS_PER_SECOND
to configure rate-limiting of requests sent to the Sequencer CometBFT node for verification of Sequencer block data fetched from Celestia blobsTesting
This needs to be observed end-to-end, potentially letting conductor run for a very long time with only soft commits, and then turning firm commits on.
Related Issues
closes #1064