There are many cases where it is valuable to be able to observe events on the Convex network.
Examples are:
- Peer operators wish to monitor performance of their peer(s) and the service levels they are offering to their clients
- dApp developers looking to observe events in the smart contracts they have deployed
- Traders looking to monitor price movements to inform their trading decisions
- Centralised exchanges monitoring for fraud and other suspicious activity
Different observers can be plugged in to support different use cases. Examples might be:
- Sending events to a Kafka topic where they can subsequently be filtered, processed and analysed by various monitoring tools
- Visualising events in a realtime monitoring tool
The observability model is opt-in from the perspective of peer operators. Peer operators may choose which events they wish to observe, and how they want to consume these on a per-peer basis.
The following topics will be available as observability hooks:
- Client Transactions
Client transactions are transactions that are handled on behalf of a client from a specific peer.
The following events related to client transactions will be available
- Request (when the peer accepts a transaction from a client)
- Response (when the peer retuens the result to the client)
The primary key for transactions should be regarded as (Peer Key, Transaction ID, Event type)
Ordering events are any time a peer recieves an updated ordering, directly or indirectly, from another Peer.
Important fields:
peer= Peer public keyhash= Hash of ordering (this can deduplicate identical orderings)cps= Consensus points (including ordering length)ts= Timestamp (at which Ordering validated by peer)
Events related to consenus update
Peer status changes are any time a Peer has an update in the global state:
- Peer added
- Peer removed
- Peer stake change
- Peer metadata change
CVM log events are events emitted via the CVM log instruction.
As this is an application specific log capability, it is likely that observers will want to filter this event stream for events of particular interest. Examples might be:
- DAO votes
- Token trades
- Auction bids
Peer performance stats detail live performance metrics from a peer. Fields include:
- Thread-based CPU utilisation (time not blocked)
- Internal queue sizes (and total capacity)
- Rates of events per second
Default is to produce one performance record every second during peer operations
Message events are low level peer protocol messages.
Key fields:
- Event type "SND" or "RCV"
- Timestamp
- Remote address / port
- Message type
- Length
The Kafka consumer provides a general purpose observability framework
This Consumer works on the following basis:
- Events are serialised as JSON
- Events are posted to a Kafka queue via Strimzi (over https:)
- There is some buffering at the client side
- Different events are normally routed to different Kafka topics (configurable)
The NOP consumer ignores events, and produces no output.
It has the advantage of near-zero overhead for Peer operators.
The NOP Consumer is the default, unless come other Consumer is specifcied.
It is assumed that the observability service will not usually be directly accessible to end users, e.g. clients using a Convex dApp.
Peer operators may of course choose to make data from their observability service publicly visible, e.g. performance graphs.