IF: deep-mind logging updates for 6.0 instant finality

[heifner]

Currently deep-mind logs block_state renamed to block_state_legacy in main.

   void deep_mind_handler::on_accepted_block(const std::shared_ptr<block_state_legacy>& bsp) {
      auto packed_blk = fc::raw::pack(*bsp);
      fc_dlog(_logger, "ACCEPTED_BLOCK ${num} ${blk}", ("num", bsp->block_num())(blk", fc::to_hex(packed_blk)));
   }

block_state_legacy will not be used once instant finality is activated and enabled on-chain.

See https://github.com/pinax-network/firehose-antelope/blob/deff673da16baf0441bdf316c9e546ef7f34fb09/codec/consolereader.go#L360
See https://github.com/pinax-network/firehose-antelope/blob/develop/codec/antelope/v3.1/hydrator.go#L22

It is unclear if all these values are actively used downstream. Some are no longer meaningful after instant finality is enabled. For example: DPoSIrreversibleBlockNum, DPoSProposedIrreversibleBlockNum, ConfirmCount, ProducerToLastProduced , ProducerToLastImpliedIrb, PendingSchedule. There are new values in instant finality which may be useful such as ActiveProposerPolicy, ActiveFinalizerPolicy.

Proposal:
Add an ACCEPTED_BLOCK_V2 which will output ${num} ${blk2}. Where blk2 is struct deepmind::block_state_v2;

The definition of deepmind::block_state_v2 needs to be defined.

@YaroShkvorets @fschoell @matthewdarwin Can you provide information on what values are used by firehose/substreams?

[fschoell]

DPoSIrreversibleBlockNum is used to set the LIB number on the generic blocks (see here), which is required for the fork handling logic, so this is rather important.

Are there any resources how the instant finality is going to work? I assume this is only impacting single blocks, while the LIB logic stays for all the others?

Add an ACCEPTED_BLOCK_V2 which will output ${num} ${blk2}. Where blk2 is struct deepmind::block_state_v2;

I guess that makes sense. So each instant final block will still have ACCEPTED_BLOCK with the DPoSIrreversibleBlockNum field which is then going to be used to do the fork handling downstreams as before? And then we can discuss with Streamingfast on how to include the new instant finality into Firehose and Substreams based on ACCEPTED_BLOCK_V2

[heifner]

DPoSIrreversibleBlockNum is used to set the LIB number on the generic blocks (see here), which is required for the fork handling logic, so this is rather important.

This exact field will not be available after instant finality is enabled. We can either create a struct deepmind::block_state_v2 with analogous information, or export signed_block instead along with LIB. The exact details of internal representation of block_state is still be worked out. However, we don't want to make the mistake of exporting the new block_state as that should not be an external ABI. I think a tuple of [block_num, irreversible_block_num, to_hex(signed_block)] would be best. Are there any other parts of the current block_state other than signed_block that are used?

Are there any resources how the instant finality is going to work? I assume this is only impacting single blocks, while the LIB logic stays for all the others?

We will be creating more information as we move closer to release. I'm not sure I understand your question about LIB. Instant finality will have a different mechanism for determining LIB.

Add an ACCEPTED_BLOCK_V2 which will output ${num} ${blk2}. Where blk2 is struct deepmind::block_state_v2;

I guess that makes sense. So each instant final block will still have ACCEPTED_BLOCK with the DPoSIrreversibleBlockNum field which is then going to be used to do the fork handling downstreams as before? And then we can discuss with Streamingfast on how to include the new instant finality into Firehose and Substreams based on ACCEPTED_BLOCK_V2

I am proposing that each block will be either ACCEPTED_BLOCK or ACCEPTED_BLOCK_V2. ACCEPTED_BLOCK_V2 will be exported for blocks after instant finality is enabled. ACCEPTED_BLOCK will be exported until instant finality is enabled.

[fschoell]

This exact field will not be available after instant finality is enabled. We can either create a struct deepmind::block_state_v2 with analogous information, or export signed_block instead along with LIB. The exact details of internal representation of block_state is still be worked out. However, we don't want to make the mistake of exporting the new block_state as that should not be an external ABI. I think a tuple of [block_num, irreversible_block_num, to_hex(signed_block)] would be best. Are there any other parts of the current block_state other than signed_block that are used?

For internal Firehose/Substreams logic it's important we are able to implement those two interfaces: https://github.com/streamingfast/firehose-core/blob/df7ce843a541ecdb8cac20dfdc04b4049d9bdce1/types.go#L16-L94

I don't see any other issues here besides the DPoSIrreversibleBlockNum (I assume everything else will be available in the block header as before). If we get the LIB number from another struct that's also fine.

I am proposing that each block will be either ACCEPTED_BLOCK or ACCEPTED_BLOCK_V2. ACCEPTED_BLOCK_V2 will be exported for blocks after instant finality is enabled. ACCEPTED_BLOCK will be exported until instant finality is enabled.

We can handle it that way.

We will be creating more information as we move closer to release.

Looking forward to this.

[arhag]

This was resolved by AntelopeIO/spring#16.