feat(katana): forked provider fetch remote non-state data

[fishseabowl]

This PR adds the following functions to fetch non-state data from the fork network.

pub fn do_get_events(
        &self,
        filter: EventFilter,
        continuation_token: Option<String>,
        chunks_size: ChunkSize,
    ) -> Result<EventsPage, ForkedBackendError>

pub fn do_get_block_with_tx_hashes(
        &self,
        block_id: BlockIdOrTag,
    ) -> Result<starknet::core::types::MaybePendingBlockWithTxHashes, ForkedBackendError>

pub fn do_get_block_with_txs(
        &self,
        block_id: BlockIdOrTag,
    ) -> Result<starknet::core::types::MaybePendingBlockWithTxs, ForkedBackendError>

pub fn do_get_transaction_by_block_id_and_index(
        &self,
        block_id: BlockIdOrTag,
        index: TxNumber,
    ) -> Result<Transaction, ForkedBackendError>

pub fn do_get_transaction_by_hash(
        &self,
        transaction_hash: TxHash
    ) -> Result<Transaction, ForkedBackendError>

pub fn do_get_transaction_receipt(
        &self,
        transaction_hash: TxHash
    ) -> Result<starknet::core::types::MaybePendingTransactionReceipt, ForkedBackendError>

Closes #1466
Closes DOJ-128

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 147 lines in your changes missing coverage. Please review.

Project coverage is 67.93%. Comparing base (328004d) to head (2ff6312).
Report is 38 commits behind head on main.

❗ Current head 2ff6312 differs from pull request most recent head 0423f89

Please upload reports for the commit 0423f89 to get more accurate results.

Files	Patch %	Lines
...ana/storage/provider/src/providers/fork/backend.rs	0.00%	147 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1700      +/-   ##
==========================================
- Coverage   68.83%   67.93%   -0.91%     
==========================================
  Files         318      274      -44     
  Lines       38068    29863    -8205     
==========================================
- Hits        26205    20286    -5919     
+ Misses      11863     9577    -2286     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[glihm]

@fishseabowl thanks for the work done here!
On my end I was to suggest using a constant for the logger Target. Beyond than that, @kariy could you have a look to it? :)

[glihm]

Isn't pending that you want to use here?

[fishseabowl]

@glihm Thank you for your comments. Could you share more details or relevant documentation?

[glihm]

Currently you're reading BlockTag::Latest, and I was wondering if instead we want to use BlockTag::Pending.
But re-reading your test, I think it's better what you've did with the latest. @kariy could you confirm?

[kariy]

sorry for the late review @fishseabowl i forgot about this 🙏 .

i realized that this is a bit more complex than what i had anticipated. due to how katana's storage provider works, we cant exactly map the RPC calls to one of the forked backend requests. it needs to call the same provider traits methods just like the other provider types.

for example, when the getBlockWithTxs is called on the RPC server,

dojo/crates/katana/rpc/rpc/src/starknet.rs

Line 290 in e9fab2c

async fn block_with_txs(&self, block_id: BlockIdOrTag) -> RpcResult<MaybePendingBlockWithTxs> {

it needs to go through all these provider methods (ie BlockHashProvider::block_hash_by_id, BlockProvider::block, and BlockStatusProvider::block_status) to build the right return type that the RPC expect.

dojo/crates/katana/rpc/rpc-types-builder/src/block.rs

Lines 22 to 33 in e9fab2c

	pub fn build(self) -> ProviderResult<Option<BlockWithTxs>> {
	let Some(hash) = BlockHashProvider::block_hash_by_id(&self.provider, self.block_id)? else {
	return Ok(None);
	};
	let block = BlockProvider::block(&self.provider, self.block_id)?
	.expect("should exist if hash exists");
	let finality_status = BlockStatusProvider::block_status(&self.provider, self.block_id)?
	.expect("should exist if block exists");
	Ok(Some(BlockWithTxs::new(hash, block, finality_status)))
	}

this works well for non-forked provider types but for forked provider, ideally we want to just call the appropriate RPC methods to the forked network and just return the result without any modification as the type is already in the RPC format.

i still think this is possible, we just have to do some redundant storage operations in order to conform to the provider abstraction that we have. though, it may requires storing the fetched remote data, which involves refactoring the ForkedProvider storage layout.

does that make sense ?

[fishseabowl]

sorry for the late review @fishseabowl i forgot about this 🙏 .

i realized that this is a bit more complex than what i had anticipated. due to how katana's storage provider works, we cant exactly map the RPC calls to one of the forked backend requests. it needs to call the same provider traits methods just like the other provider types.

for example, when the getBlockWithTxs is called on the RPC server,

dojo/crates/katana/rpc/rpc/src/starknet.rs

Line 290 in e9fab2c

async fn block_with_txs(&self, block_id: BlockIdOrTag) -> RpcResult<MaybePendingBlockWithTxs> {

it needs to go through all these provider methods (ie BlockHashProvider::block_hash_by_id, BlockProvider::block, and BlockStatusProvider::block_status) to build the right return type that the RPC expect.

dojo/crates/katana/rpc/rpc-types-builder/src/block.rs

Lines 22 to 33 in e9fab2c

	pub fn build(self) -> ProviderResult<Option<BlockWithTxs>> {
	let Some(hash) = BlockHashProvider::block_hash_by_id(&self.provider, self.block_id)? else {
	return Ok(None);
	};
	let block = BlockProvider::block(&self.provider, self.block_id)?
	.expect("should exist if hash exists");
	let finality_status = BlockStatusProvider::block_status(&self.provider, self.block_id)?
	.expect("should exist if block exists");
	Ok(Some(BlockWithTxs::new(hash, block, finality_status)))
	}

this works well for non-forked provider types but for forked provider, ideally we want to just call the appropriate RPC methods to the forked network and just return the result without any modification as the type is already in the RPC format.

i still think this is possible, we just have to do some redundant storage operations in order to conform to the provider abstraction that we have. though, it may requires storing the fetched remote data, which involves refactoring the ForkedProvider storage layout.

does that make sense ?

@kariy Thank you for your comments. Let me try this.

[glihm]

@kariy how do you feel about this? Do you think we definitively need more rework to support that? Or @fishseabowl suggestion can be a good candidate?

[fishseabowl]

@kariy how do you feel about this? Do you think we definitively need more rework to support that? Or @fishseabowl suggestion can be a good candidate?

@glihm @kariy I have updated the relevant code, can you help check what is missing? Thanks

[glihm]

Merged in #2592.
@fishseabowl thank you again for the time and effort here, please ping me on the dojo discord or telegram @glihm when you have a chance. 👍