Support for arbitrary zKEVM types pre 2-to-1 aggregation

[Nashtare]

Sharing some thoughts I had with @muursh to the greater audience.

We currently have all the circuits living in AllRecursiveCircuits struct, which are flavor specific (by flavor I mean here the specific type of zkEVM, not just Type-1 vs Type-2 but also the intrinsic differences of chains aiming at using a given type, for instance ImmutableX chain vs Polygon PoS chain).

We made the choice of supporting conditional compilation in the KERNEL to improve developer experience and reduce the proving overhead, by not having to perform conditional branching over all existing variants we aim at supporting on several places of the ASM code.

However because of how all our circuits are arranged, this makes the two final layers (block wrapper, chain specific, and 2-to-1 aggreg, chain agnostic) pegged to a given zkEVM flavor version.

Instead, I'm proposing to decouple these from the current, chain-specific, AllRecursiveCircuits object, and turn the block-wrapper layer into a $N$-conditional selective verification circuit (while preserving its current opacification of remaining chain-private data), where `$N$ is the number of distinct flavor circuits that would be supported.

If circuit size is a concern (for the 2-to-1 aggregation step specifically), we can internally make the wrapping stage a 2-step phase to shrink the proof to be emitted to the AggLayer for aggregation with others.

[Nashtare]

Multi-Provers Aggregation design

Summarizing the description above, we'd obtain something as is:

flowchart TD
    subgraph "Prover State (Type 1)"
        A1["Segment circuit"] --> A2["Segment aggregation circuit"]
        A2 --> A3["Batch aggregation circuit"]
        A3 --> A4["Block circuit"]
        A4 --> A5["Block wrapper circuit"]
    end

    subgraph "Prover State (Type 1')"
        B1["Segment circuit"] --> B2["Segment aggregation circuit"]
        B2 --> B3["Batch aggregation circuit"]
        B3 --> B4["Block circuit"]
        B4 --> B5["Block wrapper circuit"]
    end

    subgraph "Prover State (Type 2)"
        C1["Segment circuit"] --> C2["Segment aggregation circuit"]
        C2 --> C3["Batch aggregation circuit"]
        C3 --> C4["Block circuit"]
        C4 --> C5["Block wrapper circuit"]
    end

    subgraph "Aggregator State"
        D1["Block aggregation circuit"]
    end

    A5 --finalized proof<br>b1 thru b100--> D1
    B5 --finalized proof<br>b60 thru b70--> D1
    C5 --finalized proof<br>b1000 thru b2000--> D1

    D1 --> E1{{"Aggregated proof"}}

Loading

where:

• the leader binary is in charge of generating individual chains block proofs (as currently done)
• the aggregator binary is in charge of gathering (possibly) distinct chains block proofs and aggregate them all together

Aggregator initialization

Given a set of known prover states (for each flavor of zkEVM we need to support), we would then initialize a common aggregator based of each individual prover states' final circuits (i.e. block wrapper¹).

In a production setting, we would probably host a verified version of all officially supported prover states and their combined aggregator state, but for the time being, we'd need some local way of generating them.
Given that prover states are feature gated, we'd most likely want to initialize the Aggregator State through a specification of all the existing (i.e. serialized) disk states on file.

Open Questions

• do we want to specify manually all the identifiers (i.e. kernel_hash prefixes for those) or load by default all the ones in the cache?
• quid of identical identifiers / different ranges? (won't happen in production, can happen during dev)

Proof files identifiers

We currently write to disk block proofs as b${n}.zkproof where n is the latest block included in the proof. This needs to be changed to at least include the chain_id, to identify distinct proofs at same height from different networks. Same approach can be used for wrapped block proofs (currently b${n}_wrapper.zkproof). #656 suggests to use the PI hash for aggregated block proofs, which could be kept here.

Verifier binary

The verifier binary currently (as of #656) supports verifying all 3 highest level proofs (block / wrapper / aggreg). As it is generated from its prover counterpart, following the separation of ProverStates and AggregatorState, we'd most likely need 2 associated verifier binaries.

Aggregator input types

#656 currently only supports stdio mode for the AggregatorState. It also expects a compact file serializing all block proofs in an array fashion, while they are being generated and stored independently in distinct files, which requires additional preprocessing / manual handling.

Open Questions

• How should the aggregator stdio mode work? Should it expect a dir_path and load all proofs found? (requiring to store wrapped block proofs in a different folder by default), or should it not do anything and keep expecting a single array file?
• Do we want to support RPC / HTTP?

Footnotes

1. The block wrapper circuit is currently used by the aggregator, this would need to change and be placed under the leader responsability either as additional command to be called once the final block proof for a given window has been generated successfully, or automated once a range is being proven, which change the expectation with respect to proof inputs (i.e. the specified range should not be incomplete, as we cannot aggregate a wrapped block proof with incoming new block proofs, these behave as new checkpoint heights). ↩