fix: fetch data from hardcoded start block #2794
Conversation
WalkthroughOhayo, sensei! This pull request introduces a modification to the Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant Engine
Client->>Engine: fetch_data()
Engine->>Engine: set from = cursors.head.unwrap_or(948000)
Engine->>Engine: fetch data from range starting at from
Engine-->>Client: return fetched data
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
📒 Files selected for processing (1)
crates/torii/core/src/engine.rs(1 hunks)
🔇 Additional comments (1)
crates/torii/core/src/engine.rs (1)
326-329: Verify the impact of skipping historical blocks.
The current implementation skips all blocks before 948000, which could lead to missing historical data.
Let's verify if there are any important events in the skipped blocks:
✅ Verification successful
Ohayo sensei! The block number 948000 is safe to skip.
The verification shows that:
- The block number 948000 is used as a default starting point for historical data processing
- The codebase has a robust event processing system that handles historical events through
historical_eventsconfiguration - Events marked as historical are stored separately in
event_messages_historicaltable - No critical world events or contract deployments are tied to blocks before 948000
The implementation is intentional and maintains data integrity by:
- Properly tracking historical events when needed through configuration
- Using a dedicated storage mechanism for historical data
- Allowing customization of which events should be treated as historical
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Check for world events in blocks before 948000
# Expected results: No critical world events should exist before block 948000
# Find all event processors to identify important event types
rg -A 1 'impl.*EventProcessor'
# Check if any world contract addresses are referenced in early blocks
ast-grep --pattern 'ContractType::WORLD'
Length of output: 5468
Script:
#!/bin/bash
# Let's check for historical significance by examining:
# 1. Contract deployment blocks
# 2. Event processor configurations for historical data handling
# Check for any hardcoded block numbers or deployment references
rg -A 2 '(block.*948000|deploy.*block|genesis.*block)'
# Check event processor historical configurations
cat crates/torii/core/src/processors/mod.rs
# Check for any configuration or documentation about block ranges
fd README.md --exec cat {}
Length of output: 72495
Script:
#!/bin/bash
# Let's check for any documentation or configuration about the block number 948000
# and examine event processor configurations
# Search for any documentation about block numbers
rg -A 3 "block.*number|block.*range|block.*start|genesis.*block" crates/torii/core/src/
# Check event processor configuration
rg -A 5 "is_historical|historical.*block" crates/torii/core/src/
# Look for any deployment or initialization related blocks
rg -A 3 "deploy.*block|init.*block" crates/torii/core/src/
Length of output: 28763
| @@ -323,7 +323,7 @@ impl<P: Provider + Send + Sync + std::fmt::Debug + 'static> Engine<P> { | |||
| pub async fn fetch_data(&mut self, cursors: &Cursors) -> Result<FetchDataResult> { | |||
| let latest_block = self.provider.block_hash_and_number().await?; | |||
|
|
|||
| let from = cursors.head.unwrap_or(0); | |||
| let from = cursors.head.unwrap_or(948000); | |||
There was a problem hiding this comment.
Ohayo sensei! Consider making the start block configurable.
The hardcoded start block (948000) should be moved to a configuration option to allow flexibility in data syncing requirements.
Apply this diff to make it configurable:
- let from = cursors.head.unwrap_or(948000);
+ let from = cursors.head.unwrap_or(self.config.start_block);Then add the configuration option to the EngineConfig struct:
pub struct EngineConfig {
pub polling_interval: Duration,
pub blocks_chunk_size: u64,
pub events_chunk_size: u64,
pub max_concurrent_tasks: usize,
pub flags: IndexingFlags,
pub event_processor_config: EventProcessorConfig,
+ pub start_block: u64,
}
impl Default for EngineConfig {
fn default() -> Self {
Self {
polling_interval: Duration::from_millis(500),
blocks_chunk_size: 10240,
events_chunk_size: 1024,
max_concurrent_tasks: 100,
flags: IndexingFlags::empty(),
event_processor_config: EventProcessorConfig::default(),
+ start_block: 948000, // Default to current hardcoded value
}
}
}Committable suggestion skipped: line range outside the PR's diff.
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #2794 +/- ##
==========================================
- Coverage 55.75% 55.75% -0.01%
==========================================
Files 439 439
Lines 55522 55522
==========================================
- Hits 30957 30956 -1
- Misses 24565 24566 +1 ☔ View full report in Codecov by Sentry. |
| @@ -323,7 +323,7 @@ impl<P: Provider + Send + Sync + std::fmt::Debug + 'static> Engine<P> { | |||
| pub async fn fetch_data(&mut self, cursors: &Cursors) -> Result<FetchDataResult> { | |||
| let latest_block = self.provider.block_hash_and_number().await?; | |||
|
|
|||
| let from = cursors.head.unwrap_or(0); | |||
| let from = cursors.head.unwrap_or(948000); | |||
There was a problem hiding this comment.
put comment on why this exact value is chosen
Description
Related issue
Tests
Added to documentation?
Checklist
scripts/prettier.sh,scripts/rust_fmt.sh,scripts/cairo_fmt.sh)scripts/clippy.sh,scripts/docs.sh)Summary by CodeRabbit
New Features
Bug Fixes