New event driven workflow for chain events

[kurtisassad]

Description

The way we handle contest events is not abstract enough to handle all events. Also it adds coupling with the eventMappers. A new proposal for the event handling is as follows:

1. The main idea is to move the EvmEventSources from the db to static code in order to get build time typing. This will serve as the source of truth for everything chain related. We have a shared folder with a mapping of const eventRegistry = Map<contractAddress, eventStuff>. When adding a new contractAddress we need to add the entry. When adding new events we need to update the entry to the existing contractAddress.

const eventRegistry = {
    `${contestsContractAddress}`: {
        abi: ...
        events: [
            ContestCreated: z.object(...)
            ContestFinished: z.object(...)
        ],
       chain_node_ids: [...],
    },
    `${launchpadContractAddress}`: {
        abi: ...,
        events: [
          TokenLaunched: z.object(...)
        ],
       chain_node_ids: [...],
    }
};

( Note that these contractAddresses will just reference the hardcoded contractAddresses variables we have, to ensure a single source of truth)

2. From here the producer (evm-ce) will go through this registry and parse the corresponding events from each specified chain. It will then emit these raw events into the outbox. There will be no need for validation here as the web3 contract parsing will fail if the data does not match the abi.
3. The outbox will be free from any business logic. This will improve the mental model as we will no longer need to think about it or make any changes when adding events.
4. The policy (consumer) will listen to some queue. It will use the eventRegistry to obtain the zod schema for the event it is listening to in order to parse this data from the queue. A new Policy handler will need to be added every time we add an event we want business logic to operate on.

With this proposal, we will accomplish the following:

1. Decouple any logic from touching the producer, instead we have the source of truth be the eventRegistry. All producer related logic is downstream from this.
2. No more outbox in the mental model. Since this just contains event related data. There will be no preprocessing step required to emit events to the outbox.

[timolegros]

I agree that the event mapping is overkill and the entire process of adding events could be largely simplified if we emit ABI-parsed events and not do any additional mapping to custom internal events before emitting to the Outbox. Mapping to some internal structure reduces flexibility - each time you need a new data point you need to modify the schema and the mappers or in more extreme cases you need to modify the entire mapping pattern (e.g. adding chain_node_id for @kurtisassad). Additionally, it seems that mappers mainly rename properties and convert values from BigNumber to strings. There is no need for such a complex/robust mapping system when converting values can be done in the consumer when that value is needed.

The event system is designed to route according to the abstractions we have. That is, event names === routing keys and policy names === queues. If we want to remove mapping and simplify adding new chain event processing we need to go back to using ChainEventCreated. To support the existing routing functionality with ChainEventCreated we have 2 challenges:

1. Modify routing to use payload or metadata (anything other than event_name) so that we can still have separate queues for different sets of correlated events.
   • This may affect our ability to automate routing in the future.
2. Coerce types according to the event signature or somehow infer the specific ChainEventCreated (essentially the parsedArgs type) schema from the event signature (this would be ideal).

A downside here is that chain event policies now only hold a single processChainEventCreated handler which itself has an event_signature based switch to map events to respective handlers.

• We may want to update the Policy framework (or have a special case for chain event policies) to accept event signatures as input keys (rather than event_names). Paired with (2) I think this would provide the ideal development experience.

So to recap there are a couple separate streams of work we can ticket here:

1. Switch from DB-defined event sources to static in-memory eventRegistry for internal protocol events -> this would essentially involve modifying getEventSources.ts
2. Switch from mappers to ChainEventCreated schemas
   • Route ChainEventCreated events according to metadata
   • Modify existing policies to have a single event handler with a switch over event signatures
   • Modify existing handlers to use raw Ethers result rather than mapped values
3. Create ChainEvent policies where input keys are event signatures

[timolegros]

Another note from my previous comment: we will soon have policies that include handlers for chain events and "regular" events (ThreadCreated). For example, we will need to add chain-events (contents, referrals, etc) to the Notifications policy. This may cause issues if we want types/handlers derived from event signatures.

[timolegros]

We will create a generic chain-event zod schema which we extend with the event names from the event registry (dynamically?). Consumers will use the https://abitype.dev/api/zod package/utils to parse events.

[timolegros]

The work that arose from this discussion is detailed in #9895.