diff --git a/docs/overview/features/_category_.json b/docs/overview/features/_category_.json new file mode 100644 index 000000000..1990c26df --- /dev/null +++ b/docs/overview/features/_category_.json @@ -0,0 +1,13 @@ +{ + "position": 6, + "label": "Features", + "collapsible": true, + "collapsed": true, + "className": "red", + "link": { + "title": "Features" + }, + "customProps": { + "description": "Listing different endpoints available for different sources for Osmosis" + } +} \ No newline at end of file diff --git a/docs/overview/integrate/Concentrated Liquidity/1-overview.md b/docs/overview/features/concentrated-liquidity.md similarity index 96% rename from docs/overview/integrate/Concentrated Liquidity/1-overview.md rename to docs/overview/features/concentrated-liquidity.md index 2c468b143..ba67e7c51 100644 --- a/docs/overview/integrate/Concentrated Liquidity/1-overview.md +++ b/docs/overview/features/concentrated-liquidity.md @@ -1,21 +1,8 @@ --- -sidebar_position: 3 +sidebar_position: 2 --- -# Overview - -## Index -- [Overview](#Overview) - - [Background](#background) - - [Notable Features](#notable-features) -- [Our Implementation](#our-implementation) - - [Incentives](#Incentive-Creation-and-Querying) - - [Reward Splitting Between Classic and CL pools](#Reward-Splitting-Between-Classic-and-CL-pools) -- [TWAP Integration](#twap-integration) -- [Precision Issues With Price](#precision-issues-with-price) - - [Solution](#solution) -- [Terminology](#terminology) -- [External Sources](#external-sources) +# Concentrated Liquidity ## Background Concentrated liquidity is a novel Automated Market Maker (AMM) design introduced @@ -272,3 +259,6 @@ Note, that the current tick of 155 is defined inside the bucket over a range of - [Uniswap V3 Whitepaper](https://uniswap.org/whitepaper-v3.pdf) - [Technical Note on Liquidity Math](https://atiselsts.github.io/pdfs/uniswap-v3-liquidity-math.pdf) + +## Repository +For more an in-depth docs and implementation, please visit: https://github.com/osmosis-labs/osmosis/tree/main/x/concentrated-liquidity \ No newline at end of file diff --git a/docs/overview/features/eip-1559.md b/docs/overview/features/eip-1559.md new file mode 100644 index 000000000..50b8e9fef --- /dev/null +++ b/docs/overview/features/eip-1559.md @@ -0,0 +1,42 @@ +--- +sidebar_position: 3 +--- + +# EIP-1559 + +## Introduction +The following documentation explains the logic behind the Osmosis EIP-1559 fee market implementation. This implementation is designed to prevent spam attacks on the network by dynamically adjusting transaction fees based on network congestion. + +## Base Fee Adjustment + +The heart of the EIP-1559 fee market is the adjustment of the base fee. The base fee represents the minimum fee required to include a transaction in a block. + +The base fee adjustment is based on the concept of "block space competition." If the network is congested and more users want to send transactions, the base fee increases to allocate block space to the highest bidders. Conversely, if the network is underutilized, the base fee decreases to reduce transaction costs. + +The adjustment is calculated using the formula: +``` +baseFeeMultiplier = 1 + (gasUsed - targetGas) / targetGas * maxChangeRate +newBaseFee = baseFee * baseFeeMultiplier +``` +- gasUsed: The total gas used in a block. +- targetGas: The target gas wanted per block, defined as 75,000,000. +- maxChangeRate: The maximum allowable change rate for the base fee, defined as 1/10. +The calculated baseFeeMultiplier is then used to update the CurBaseFee. + +The base fee is constrained by minimum and maximum values defined as MinBaseFee (0.0025) and MaxBaseFee (5), respectively. + +## Integration +The current chain base fee can be retrieved from the following end point: + `${LCD_ENDPOINT}/osmosis/txfees/v1beta1/cur_eip_base_fee` + +User txs will get priced at `multiplier` * B_t, where B_t is base fee at time of submission, and multiplier is an arbitrary constant that can be set in wallets / dapps. + +An example fee calculation can be as the following: +``` +low = base_fee * 1.05 +average = base_fee * 2 +high = base_fee * 3 +``` + +Note that setting a conservative, high multiplier can ensure getting user transactions even throughout high transaction volumes, whilst it can lead to users paying higher transaction fees than needed. + diff --git a/docs/overview/features/fee-abstraction.md b/docs/overview/features/fee-abstraction.md new file mode 100644 index 000000000..a4dac27d3 --- /dev/null +++ b/docs/overview/features/fee-abstraction.md @@ -0,0 +1,89 @@ +--- +sidebar_position: 4 +--- + +# Fee Abstraction + +## Context + +The concrete use cases which motivated this module include: + +- The desire to use IBC token as transaction fees on any chain instead of having to use native token as fee. +- To fully take advantage of the newly represented Osmosis [``swap router``](https://github.com/osmosis-labs/osmosis/tree/main/cosmwasm/contracts) with the [``ibc-hooks``](https://github.com/osmosis-labs/osmosis/tree/main/x/ibc-hooks) module and the [``async-icq``](https://github.com/strangelove-ventures/async-icq) module. + + +## Description + +Fee abstraction modules enable users on any Cosmos chain with IBC connections to pay fee using ibc token. + +Fee-abs implementation: + +- Fee-abs module imported to the customer chain. + +The implememtation also uses Osmosis swap router and async-icq module which are already deployed on Osmosis testnet. + +## Prototype + +Fee-abs mechanism in a nutshell: + + 1. Pulling `twap data` and update exchange rate: + +- Periodically pulling `twap data` from osmosis by ibc-ing to `async-icq` module on Osmosis, this `twap data` will update the exchange rate of osmosis to customer chain's native token. + + 2. Handling txs with ibc-token fee: + +- The exchange rate is used to calculate the amount of ibc-token needed for tx fee allowing users to pay ibc-token for tx fee instead of chain's native token. + + 3. Swap accumulated ibc-token fee: + +- The collected ibc-token users use for tx fee is periodically swaped back to customer chain's native token using osmosis. + +We'll goes into all the details now: + +#### Pulling `twap data` and update exchange rate + +For this to work, we first has to set up an ibc channel from `feeabs` to `async-icq`. This channel set-up process can be done by anyone, just like setting up an ibc transfer channel. Once that ibc channel is there, we'll use that channel to ibc-query Twap data. Let's call this the querying channel. + +The process of pulling Twap data and update exchange rate : + +![Diagram of the process of pulling Twap data and updating exchange rate](https://i.imgur.com/HJ9a26H.png "Diagram of the process of pulling Twap data and updating exchange rate") + +Description : + For every `update exchange rate period`, at fee-abs `BeginBlocker()` we submit a `InterchainQueryPacketData` which wrapped `QueryArithmeticTwapToNowRequest` to the querying channel on the customer chain's end. Then relayers will submit `MsgReceivePacket` so that our `QueryTwapPacket` which will be routed to `async-icq` module to be processed. `async-icq` module then unpack `InterchainQueryPacketData` and send query to TWAP module. The correspone response will be wrapped in the ibc acknowledgement. Relayers then submit `MsgAcknowledgement` to the customer chain so that the ibc acknowledgement is routed to fee-abs to be processed. Fee-abs then update exchange rate according to the Twap wrapped in the ibc acknowledgement. + +#### Handling txs with ibc-token fee + +We modified `MempoolFeeDecorator` so that it can handle ibc-token as fee. If the tx has ibc-token fee, the AnteHandler will first check if that token is allowed (which is setup by Gov) we basically replace the amount of ibc-token with the equivalent native-token amount which is calculated by `exchange rate` * `ibc-token amount`. + +We have an account to manage the ibc-token user used to pay for tx fee. The collected ibc-token fee is sent to that account instead of community pool account. + +#### Swap accumulated ibc-tokens fee + +Fee-abstraction will use osmosis's Cross chain Swap (XCS) feature to do this. We basically ibc transfer to the osmosis crosschain swap contract with custom memo to swap the osmosis fee back to customer chain's native-token and ibc transfer back to the customer chain. + +##### How XCS work + +###### Reverse With Path-unwinding to get Ibc-token on Osmosis + +- Create a ibc transfer message with a specific MEMO to work with ibc [``packet-forward-middleware``](https://github.com/strangelove-ventures/packet-forward-middleware) which is path-unwinding (an ibc feature that allow to automatic define the path and ibc transfer multiple hop follow the defined path) +- Ibc transfer the created packet to get the fee Ibc-token on Osmosis + +Ex: When you sent STARS on Hub to Osmosis, you will get Osmosis(Hub(STARS)) which is different with STARS on Osmosis Osmosis(STARS). It will reverse back Osmosis(Hub(STARS)) to Osmosis(STARS): + +![Diagram of the process of swapping accumulated ibc-tokens fee](https://i.imgur.com/D1wSrMm.png "Diagram of the process of swapping accumulated ibc-tokens fee") + +###### Swap Ibc-token + +###### Swap Ibc-token + +After reverse the ibc-token, XCS will : + +- Swap with the specific pool (which is defined in the transfer packet from Feeabs-chain) to get Feeabs-chain native-token +- Transfer back Feeabs-chain native-token to Feeabs module account (will use to pay fee for other transaction) + +![Diagram of the process of swapping accumulated ibc-tokens fee](https://i.imgur.com/YKOK8mr.png "Diagram of the process of swapping accumulated ibc-tokens fee") + +Current version of fee-abstraction working with XCSv2 + +## Repository +For more an in-depth docs and implementation, please visit: https://github.com/osmosis-labs/fee-abstraction \ No newline at end of file diff --git a/docs/overview/features/ibc-hooks.md b/docs/overview/features/ibc-hooks.md new file mode 100644 index 000000000..3bd75da00 --- /dev/null +++ b/docs/overview/features/ibc-hooks.md @@ -0,0 +1,308 @@ +--- +sidebar_position: 5 +--- + +# IBC Hooks + +## Wasm Hooks + +The wasm hook is an IBC middleware which is used to allow ICS-20 token transfers to initiate contract calls. +This allows cross-chain contract calls, that involve token movement. +This is useful for a variety of usecases. +One of primary importance is cross-chain swaps, which is an extremely powerful primitive. + +The mechanism enabling this is a `memo` field on every ICS20 transfer packet as of [IBC v3.4.0](https://medium.com/the-interchain-foundation/moving-beyond-simple-token-transfers-d42b2b1dc29b). +Wasm hooks is an IBC middleware that parses an ICS20 transfer, and if the `memo` field is of a particular form, executes a wasm contract call. We now detail the `memo` format for `wasm` contract calls, and the execution guarantees provided. + +### Cosmwasm Contract Execution Format + +Before we dive into the IBC metadata format, we show the cosmwasm execute message format, so the reader has a sense of what are the fields we need to be setting in. +The cosmwasm `MsgExecuteContract` is defined [here](https://github.com/CosmWasm/wasmd/blob/4fe2fbc8f322efdaf187e2e5c99ce32fd1df06f0/x/wasm/types/tx.pb.go#L340-L349 +) as the following type: + +```go +type MsgExecuteContract struct { + // Sender is the actor that committed the message in the sender chain + Sender string + // Contract is the address of the smart contract + Contract string + // Msg json encoded message to be passed to the contract + Msg RawContractMessage + // Funds coins that are transferred to the contract on execution + Funds sdk.Coins +} +``` + +So we detail where we want to get each of these fields from: + +* Sender: We cannot trust the sender of an IBC packet, the counterparty chain has full ability to lie about it. +We cannot risk this sender being confused for a particular user or module address on Osmosis. +So we replace the sender with an account to represent the sender prefixed by the channel and a wasm module prefix. +This is done by setting the sender to `Bech32(Hash("ibc-wasm-hook-intermediary" || channelID || sender))`, where the channelId is the channel id on the local chain. +* Contract: This field should be directly obtained from the ICS-20 packet metadata +* Msg: This field should be directly obtained from the ICS-20 packet metadata. +* Funds: This field is set to the amount of funds being sent over in the ICS 20 packet. One detail is that the denom in the packet is the counterparty chains representation of the denom, so we have to translate it to Osmosis' representation. + +> **_WARNING:_** Due to a [bug](https://twitter.com/SCVSecurity/status/1682329758020022272) in the packet forward middleware, we cannot trust the sender from chains that use PFM. Until that is fixed, we recommend chains to not trust the sender on contracts executed via IBC hooks. + +So our constructed cosmwasm message that we execute will look like: + +```go +msg := MsgExecuteContract{ + // Sender is the that actor that signed the messages + Sender: "osmo1-hash-of-channel-and-sender", + // Contract is the address of the smart contract + Contract: packet.data.memo["wasm"]["ContractAddress"], + // Msg json encoded message to be passed to the contract + Msg: packet.data.memo["wasm"]["Msg"], + // Funds coins that are transferred to the contract on execution + Funds: sdk.NewCoin{Denom: ibc.ConvertSenderDenomToLocalDenom(packet.data.Denom), Amount: packet.data.Amount} +``` + +### ICS20 packet structure + +So given the details above, we propagate the implied ICS20 packet data structure. +ICS20 is JSON native, so we use JSON for the memo format. + +```json +{ + //... other ibc fields that we don't care about + "data":{ + "denom": "denom on counterparty chain (e.g. uatom)", // will be transformed to the local denom (ibc/...) + "amount": "1000", + "sender": "addr on counterparty chain", // will be transformed + "receiver": "contract addr or blank", + "memo": { + "wasm": { + "contract": "osmo1contractAddr", + "msg": { + "raw_message_fields": "raw_message_data", + } + } + } + } +} +``` + +An ICS20 packet is formatted correctly for wasmhooks iff the following all hold: + +* `memo` is not blank +* `memo` is valid JSON +* `memo` has at least one key, with value `"wasm"` +* `memo["wasm"]` has exactly two entries, `"contract"` and `"msg"` +* `memo["wasm"]["msg"]` is a valid JSON object +* `receiver == "" || receiver == memo["wasm"]["contract"]` + +We consider an ICS20 packet as directed towards wasmhooks iff all of the following hold: + +* `memo` is not blank +* `memo` is valid JSON +* `memo` has at least one key, with name `"wasm"` + +If an ICS20 packet is not directed towards wasmhooks, wasmhooks doesn't do anything. +If an ICS20 packet is directed towards wasmhooks, and is formatted incorrectly, then wasmhooks returns an error. + +### Execution flow + +Pre wasm hooks: + +* Ensure the incoming IBC packet is cryptogaphically valid +* Ensure the incoming IBC packet is not timed out. + +In Wasm hooks, pre packet execution: + +* Ensure the packet is correctly formatted (as defined above) +* Edit the receiver to be the hardcoded IBC module account + +In wasm hooks, post packet execution: + +* Construct wasm message as defined before +* Execute wasm message +* if wasm message has error, return ErrAck +* otherwise continue through middleware + +## Ack callbacks + +A contract that sends an IBC transfer, may need to listen for the ACK from that packet. To allow +contracts to listen on the ack of specific packets, we provide Ack callbacks. + +### Design + +The sender of an IBC transfer packet may specify a callback for when the ack of that packet is received in the memo +field of the transfer packet. + +Crucially, _only_ the IBC packet sender can set the callback. + +### Use case + +The crosschain swaps implementation sends an IBC transfer. If the transfer were to fail, we want to allow the sender +to be able to retrieve their funds (which would otherwise be stuck in the contract). To do this, we allow users to +retrieve the funds after the timeout has passed, but without the ack information, we cannot guarantee that the send +hasn't failed (i.e.: returned an error ack notifying that the receiving change didn't accept it) + +### Implementation + +#### Callback information in memo + +For the callback to be processed, the transfer packet's memo should contain the following in its JSON: + +`{"ibc_callback": "osmo1contractAddr"}` + +The wasm hooks will keep the mapping from the packet's channel and sequence to the contract in storage. When an ack is +received, it will notify the specified contract via a sudo message. + +#### Interface for receiving the Acks and Timeouts + +The contract that awaits the callback should implement the following interface for a sudo message: + +```rust +#[cw_serde] +pub enum IBCLifecycleComplete { + #[serde(rename = "ibc_ack")] + IBCAck { + /// The source channel (osmosis side) of the IBC packet + channel: String, + /// The sequence number that the packet was sent with + sequence: u64, + /// String encoded version of the ack as seen by OnAcknowledgementPacket(..) + ack: String, + /// Weather an ack is a success of failure according to the transfer spec + success: bool, + }, + #[serde(rename = "ibc_timeout")] + IBCTimeout { + /// The source channel (osmosis side) of the IBC packet + channel: String, + /// The sequence number that the packet was sent with + sequence: u64, + }, +} + +/// Message type for `sudo` entry_point +#[cw_serde] +pub enum SudoMsg { + #[serde(rename = "ibc_lifecycle_complete")] + IBCLifecycleComplete(IBCLifecycleComplete), +} +``` + +### Async Acks + +IBC supports the ability to send an ack back to the sender of the packet asynchronously. This is useful for +cases where the packet is received, but the ack is not immediately known. For example, if the packet is being +forwarded to another chain, the ack may not be known until the packet is received on the other chain. + +Note this ACK does not imply full revertability. It is possible that unrevertable actions have occurred +even if there is an Ack Error. (This is distinct from the behavior of ICS-20 transfers). If you want to ensure +revertability, your contract should be implemented in a way that actions are not finalized until a success ack +is received. + +#### Use case + +Async acks are useful in cases where the contract needs to wait for a response from another chain before +returning a result to the caller. + +For example, if you want to send tokens to another chain after the contract is executed you need to +add a new ibc packet and wait for its ack. + +In the synchronous acks case, the caller will receive an ack from the contract before the second packet +has been processed. This means that the caller will have to wait (and potentially track) if the second +packet has been processed successfully or not. + +With async acks, you contract can take this responsibility and only send an ack to the caller once the +second packet has been processed + +#### Making contract Acks async + +To support this, we allow contracts to return an `IBCAsync` response from the function being executed when the +packet is received. That response specifies that the ack should be handled asynchronously. + +Concretely the contract should return: + +```rust +#[cw_serde] +pub struct OnRecvPacketAsyncResponse { + pub is_async_ack: bool, +} +``` + +if `is_async_ack` is set to true, `OnRecvPacket` will return `nil` and the ack will not be written. Instead, the +contract will be stored as the "ack actor" for the packet so that only that contract is allowed to send an ack +for it. + +It is up to the contract developers to decide which conditions will trigger the ack to be sent. + +#### Sending an async ack + +To send the async ack, the contract needs to send the MsgEmitIBCAck message to the chain. This message will +then make a sudo call to the contract requesting the ack and write the ack to state. + +That message can be specified in the contract as: + +```rust +#[derive( + Clone, + PartialEq, + Eq, + ::prost::Message, + serde::Serialize, + serde::Deserialize, + schemars::JsonSchema, + CosmwasmExt, +)] +#[proto_message(type_url = "/osmosis.ibchooks.MsgEmitIBCAck")] +pub struct MsgEmitIBCAck { + #[prost(string, tag = "1")] + pub sender: ::prost::alloc::string::String, + #[prost(uint64, tag = "2")] + pub packet_sequence: u64, + #[prost(string, tag = "3")] + pub channel: ::prost::alloc::string::String, +} +``` + +The contract is expected to implement the following sudo message handler: + +```rust +#[cw_serde] +pub enum IBCAsyncOptions { + #[serde(rename = "request_ack")] + RequestAck { + /// The source channel (osmosis side) of the IBC packet + source_channel: String, + /// The sequence number that the packet was sent with + packet_sequence: u64, + }, +} + +#[cw_serde] +pub enum SudoMsg { + #[serde(rename = "ibc_async")] + IBCAsync(IBCAsyncOptions), +} +``` + +and that sudo call should return an `IBCAckResponse`: + +```rust +#[cw_serde] +#[serde(tag = "type", content = "content")] +pub enum IBCAck { + AckResponse{ + packet: Packet, + contract_ack: ContractAck, + }, + AckError { + packet: Packet, + error_description: String, + error_response: String, + } +} +``` + +Note: the sudo call is required to potentially allow anyone to send the MsgEmitIBCAck message. For now, however, +this is artificially limited so that the message can only be send by the same contract. This could be expanded in +the future if needed. + +# Repository +For more an in-depth docs and implementation, please visit: https://github.com/osmosis-labs/osmosis/tree/main/x/ibc-hooks \ No newline at end of file diff --git a/docs/overview/features/ibc-rate-limit.md b/docs/overview/features/ibc-rate-limit.md new file mode 100644 index 000000000..79cb55e58 --- /dev/null +++ b/docs/overview/features/ibc-rate-limit.md @@ -0,0 +1,314 @@ +--- +sidebar_position: 6 +--- + +# IBC Rate Limit + +The IBC Rate Limit module is responsible for adding a governance-configurable rate limit to IBC transfers. +This is a safety control, intended to protect assets on osmosis in event of: + +* a bug/hack on osmosis +* a bug/hack on the counter-party chain +* a bug/hack in IBC itself + +This is done in exchange for a potential (one-way) bridge liveness tradeoff, in periods of high deposits or withdrawals. + +The architecture of this package is a minimal go package which implements an [IBC Middleware](https://github.com/cosmos/ibc-go/blob/f57170b1d4dd202a3c6c1c61dcf302b6a9546405/docs/ibc/middleware/develop.md) that wraps the [ICS20 transfer](https://ibc.cosmos.network/main/apps/transfer/overview.html) app, and calls into a cosmwasm contract. +The cosmwasm contract then has all of the actual IBC rate limiting logic. +The Cosmwasm code can be found in the [`contracts`](./contracts/) package, with bytecode findable in the [`bytecode`](./bytecode/) folder. The cosmwasm VM usage allows Osmosis chain governance to choose to change this safety control with no hard forks, via a parameter change proposal, a great mitigation for faster threat adaptavity. + +The status of the module is being in a state suitable for some initial governance settable rate limits for high value bridged assets. +Its not in its long term / end state for all channels by any means, but does act as a strong protection we +can instantiate today for high value IBC connections. + +## Motivation + +The motivation of IBC-rate-limit comes from the empirical observations of blockchain bridge hacks that a rate limit would have massively reduced the stolen amount of assets in: + +- [Polynetwork Bridge Hack ($611 million)](https://rekt.news/polynetwork-rekt/) +- [BNB Bridge Hack ($586 million)](https://rekt.news/bnb-bridge-rekt/) +- [Wormhole Bridge Hack ($326 million)](https://rekt.news/wormhole-rekt/) +- [Nomad Bridge Hack ($190 million)](https://rekt.news/nomad-rekt/) +- [Harmony Bridge Hack ($100 million)](https://rekt.news/harmony-rekt/) - (Would require rate limit + monitoring) +- [Dragonberry IBC bug](https://forum.cosmos.network/t/ibc-security-advisory-dragonberry/7702) (can't yet disclose amount at risk, but was saved due to being found first by altruistic Osmosis core developers) + +In the presence of a software bug on Osmosis, IBC itself, or on a counterparty chain, we would like to prevent the bridge from being fully depegged. +This stems from the idea that a 30% asset depeg is ~infinitely better than a 100% depeg. +Its _crazy_ that today these complex bridged assets can instantly go to 0 in event of bug. +The goal of a rate limit is to raise an alert that something has potentially gone wrong, allowing validators and developers to have time to analyze, react, and protect larger portions of user funds. + +The thesis of this is that, it is worthwhile to sacrifice liveness in the case of legitimate demand to send extreme amounts of funds, to prevent the terrible long-tail full fund risks. +Rate limits aren't the end-all of safety controls, they're merely the simplest automated one. More should be explored and added onto IBC! + +## Rate limit types + +We express rate limits in time-based periods. +This means, we set rate limits for (say) 6-hour, daily, and weekly intervals. +The rate limit for a given time period stores the relevant amount of assets at the start of the rate limit. +Rate limits are then defined on percentage terms of the asset. +The time windows for rate limits are currently _not_ rolling, they have discrete start/end times. + +We allow setting separate rate limits for the inflow and outflow of assets. +We do all of our rate limits based on the _net flow_ of assets on a channel pair. This prevents DOS issues, of someone repeatedly sending assets back and forth, to trigger rate limits and break liveness. + +We currently envision creating two kinds of rate limits: + +* Per denomination rate limits + - allows safety statements like "Only 30% of Stars on Osmosis can flow out in one day" or "The amount of Atom on Osmosis can at most double per day". +* Per channel rate limits + - Limit the total inflow and outflow on a given IBC channel, based on "USDC" equivalent, using Osmosis as the price oracle. + +We currently only implement per denomination rate limits for non-native assets. We do not yet implement channel based rate limits. + +Currently these rate limits automatically "expire" at the end of the quota duration. TODO: Think of better designs here. E.g. can we have a constant number of subsequent quotas start filled? Or perhaps harmonically decreasing amounts of next few quotas pre-filled? Halted until DAO override seems not-great. + +## Instantiating rate limits + +Today all rate limit quotas must be set manually by governance. +In the future, we should design towards some conservative rate limit to add as a safety-backstop automatically for channels. +Ideas for how this could look: + +* One month after a channel has been created, automatically add in some USDC-based rate limit +* One month after governance incentivizes an asset, add on a per-denomination rate limit. + +Definitely needs far more ideation and iteration! + +## Parameterizing the rate limit + +One element is we don't want any rate limit timespan that's too short, e.g. not enough time for humans to react to. So we wouldn't want a 1 hour rate limit, unless we think that if its hit, it could be assessed within an hour. + +### Handling rate limit boundaries + +We want to be safe against the case where say we have a daily rate limit ending at a given time, and an adversary attempts to attack near the boundary window. +We would not like them to be able to "double extract funds" by timing their extraction near a window boundary. + +Admittedly, not a lot of thought has been put into how to deal with this well. +Right now we envision simply handling this by saying if you want a quota of duration D, instead include two quotas of duration D, but offset by `D/2` from each other. + +Ideally we can change windows to be more 'rolling' in the future, to avoid this overhead and more cleanly handle the problem. (Perhaps rolling ~1 hour at a time) + +### Inflow parameterization + +The "Inflow" side of a rate limit is essentially protection against unforeseen bug on a counterparty chain. +This can be quite conservative (e.g. bridged amount doubling in one week). This covers a few cases: + +* Counter-party chain B having a token theft attack + - TODO: description of how this looks +* Counter-party chain B runaway mint + - TODO: description of how this looks +* IBC theft + - TODO: description of how this looks + +It does get more complex when the counterparty chain is itself a DEX, but this is still much more protection than nothing. + +### Outflow parameterization + +The "Outflow" side of a rate limit is protection against a bug on Osmosis OR IBC. +This has potential for much more user-frustrating issues, if set too low. +E.g. if there's some event that causes many people to suddenly withdraw many STARS or many USDC. + +So this parameterization has to contend with being a tradeoff of withdrawal liveness in high volatility periods vs being a crucial safety rail, in event of on-Osmosis bug. + +TODO: Better fill out + +### Example suggested parameterization + +## Code structure + +As mentioned at the beginning of the README, the go code is a relatively minimal ICS 20 wrapper, that dispatches relevant calls to a cosmwasm contract that implements the rate limiting functionality. + +### Go Middleware + +To achieve this, the middleware needs to implement the `porttypes.Middleware` interface and the +`porttypes.ICS4Wrapper` interface. This allows the middleware to send and receive IBC messages by wrapping +any IBC module, and be used as an ICS4 wrapper by a transfer module (for sending packets or writing acknowledgements). + +Of those interfaces, just the following methods have custom logic: + +* `ICS4Wrapper.SendPacket` forwards to contract, with intent of tracking of value sent via an ibc channel +* `Middleware.OnRecvPacket` forwards to contract, with intent of tracking of value received via an ibc channel +* `Middleware.OnAcknowledgementPacket` forwards to contract, with intent of undoing the tracking of a sent packet if the acknowledgment is not a success +* `OnTimeoutPacket` forwards to contract, with intent of undoing the tracking of a sent packet if the packet times out (is not relayed) + +All other methods from those interfaces are passthroughs to the underlying implementations. + +#### Parameters + +The middleware uses the following parameters: + +| Key | Type | +|-----------------|--------| +| ContractAddress | string | + +1. **ContractAddress** - + The contract address is the address of an instantiated version of the contract provided under `./contracts/` + +### Cosmwasm Contract Concepts + +Something to keep in mind with all of the code, is that we have to reason separately about every item in the following matrix: + +| Native Token | Non-Native Token | +|----------------------|--------------------------| +| Send Native Token | Send Non-Native Token | +| Receive Native Token | Receive Non-Native Token | +| Timeout Native Send | Timeout Non-native Send | + +(Error ACK can reuse the same code as timeout) + +TODO: Spend more time on sudo messages in the following description. We need to better describe how we map the quota concepts onto the code. +Need to describe how we get the quota beginning balance, and that its different for sends and receives. +Explain intracacies of tracking that a timeout and/or ErrorAck must appear from the same quota, else we ignore its update to the quotas. + + +The tracking contract uses the following concepts + +1. **RateLimit** - tracks the value flow transferred and the quota for a path. +2. **Path** - is a (denom, channel) pair. +3. **Flow** - tracks the value that has moved through a path during the current time window. +4. **Quota** - is the percentage of the denom's total value that can be transferred through the path in a given period of time (duration) + +#### Messages + +The contract specifies the following messages: + +##### Query + +* GetQuotas - Returns the quotas for a path + +##### Exec + +* AddPath - Adds a list of quotas for a path +* RemovePath - Removes a path +* ResetPathQuota - If a rate limit has been reached, the contract's governance address can reset the quota so that transfers are allowed again + +##### Sudo + +Sudo messages can only be executed by the chain. + +* SendPacket - Increments the amount used out of the send quota and checks that the send is allowed. If it isn't, it will return a RateLimitExceeded error +* RecvPacket - Increments the amount used out of the receive quota and checks that the receive is allowed. If it isn't, it will return a RateLimitExceeded error +* UndoSend - If a send has failed, the undo message is used to remove its cost from the send quota + +All of these messages receive the packet from the chain and extract the necessary information to process the packet and determine if it should be the rate limited. + +### Necessary information + +To determine if a packet should be rate limited, we need: + +* Channel: The channel on the Osmosis side: `packet.SourceChannel` for sends, and `packet.DestinationChannel` for receives. +* Denom: The denom of the token being transferred as known on the Osmosis side (more on that below) +* Channel Value: The total value of the channel denominated in `Denom` (i.e.: channel-17 is worth 10k osmo). +* Funds: the amount being transferred + +#### Notes on Channel +The contract also supports quotas on a custom channel called "any" that is checked on every transfer. If either the +transfer channel or the "any" channel have a quota that has been filled, the transaction will be rate limited. + +#### Notes on Denom +We always use the the denom as represented on Osmosis. For native assets that is the local denom, and for non-native +assets it's the "ibc" prefix and the sha256 hash of the denom trace (`ibc/...`). + +##### Sends + +For native denoms, we can just use the denom in the packet. If the denom is invalid, it will fail somewhere else along the chain. Example result: `uosmo` + +For non-native denoms, the contract needs to hash the denom trace and append it to the `ibc/` prefix. The +contract always receives the parsed denom (i.e.: `transfer/channel-32/uatom` instead of +`ibc/27394FB092D2ECCD56123C74F36E4C1F926001CEADA9CA97EA622B25F41E5EB2`). This is because of the order in which +the middleware is called. When sending a non-native denom, the packet contains `transfer/source-channel/denom` as it +is built on the `relay.SendTransfer()` in the transfer module and then passed to the middleware. Example result: `ibc/` + +##### Receives + +This behaves slightly different if the asset is an osmosis asset that was sent to the counterparty and is being +returned to the chain, or if the asset is being received by the chain and originates on the counterparty. In ibc this +is called being a "source" or a "sink" respectively. + +If the chain is a sink for the denom, we build the local denom by prefixing the port and the channel +(`transfer/local-channel`) and hashing that denom. Example result: `ibc/` + +If the chain is the source for the denom, there are two possibilities: + +* The token is a native token, in which case we just remove the prefix added by the counterparty. Example result: `uosmo` +* The token is a non-native token, in which case we remove the extra prefix and hash it. Example result `ibc/` + +#### Notes on Channel Value +We have iterated on different strategies for calculating the channel value. Our preferred strategy is the following: +* For non-native tokens (`ibc/...`), the channel value should be the supply of those tokens in Osmosis +* For native tokens, the channel value should be the total amount of tokens in escrow across all ibc channels + +The later ensures the limits are lower and represent the amount of native tokens that exist outside Osmosis. This is +beneficial as we assume the majority of native tokens exist on the native chain and the amount "normal" ibc transfers is +proportional to the tokens that have left the chain. + +This strategy cannot be implemented at the moment because IBC does not track the amount of tokens in escrow across +all channels ([github issue](https://github.com/cosmos/ibc-go/issues/2664)). Instead, we use the current supply on +Osmosis for all denoms (i.e.: treat native and non-native tokens the same way). Once that ticket is fixed, we will +update this strategy. + +##### Caching + +The channel value varies constantly. To have better predictability, and avoid issues of the value growing if there is +a potential infinite mint bug, we cache the channel value at the beginning of the period for every quota. + +This means that if we have a daily quota of 1% of the osmo supply, and the channel value is 1M osmo at the beginning of +the quota, no more than 100k osmo can transferred during that day. If 10M osmo were to be minted or IBC'd in during that +period, the quota will not increase until the period expired. Then it will be 1% of the new channel value (~11M) + +### Integration + +The rate limit middleware wraps the `transferIBCModule` and is added as the entry route for IBC transfers. + +The module is also provided to the underlying `transferIBCModule` as its `ICS4Wrapper`; previously, this would have +pointed to a channel, which also implements the `ICS4Wrapper` interface. + +This integration can be seen in [osmosis/app/keepers/keepers.go](https://github.com/osmosis-labs/osmosis/blob/main/app/keepers/keepers.go) + +## Testing strategy + + +A general testing strategy is as follows: + +* Setup two chains. +* Send some tokens from A->B and some from B->A (so that there are IBC tokens to play with in both sides) +* Add the rate limiter on A with low limits (i.e. 1% of supply) +* Test Function for chains A' and B' and denom d + * Send some d tokens from A' to B' and get close to the limit. + * Do the same transfer making sure the amount is above the quota and verify it fails with the rate limit error + * Wait until the reset time has passed, and send again. The transfer should now succeed +* Repeat the above test for the following combination of chains and tokens: `(A,B,a)`, `(B,A,a)`, `(A,B,b)`, `(B,A,b)`, + where `a` and `b` are native tokens to chains A and B respectively. + +For more comprehensive tests we can also: +* Add a third chain C and make sure everything works properly for C tokens that have been transferred to A and to B +* Test that the contracts gov address can reset rate limits if the quota has been hit +* Test the queries for getting information about the state of the quotas +* Test that rate limit symmetries hold (i.e.: sending the a token through a rate-limited channel and then sending back + reduces the rate limits by the same amount that it was increased during the first send) +* Ensure that the channels between the test chains have different names (A->B="channel-0", B->A="channel-1", for example) + +## Known Future work + +Items that have been highlighted above: + +* Making automated rate limits get added for channels, instead of manual configuration only +* Improving parameterization strategies / data analysis +* Adding the USDC based rate limits +* We need better strategies for how rate limits "expire". + +Not yet highlighted + +* Making monitoring tooling to know when approaching rate limiting and when they're hit +* Making tooling to easily give us summaries we can use, to reason about "bug or not bug" in event of rate limit being hit +* Enabling ways to pre-declare large transfers so as to not hit rate limits. + * Perhaps you can on-chain declare intent to send these assets with a large delay, that raises monitoring but bypasses rate limits? + * Maybe contract-based tooling to split up the transfer suffices? +* Strategies to account for high volatility periods without hitting rate limits + * Can imagine "Hop network" style markets emerging + * Could imagine tieng it into looking at AMM volatility, or off-chain oracles + * but these are both things we should be wary of security bugs in. + * Maybe [constraint based programming with tracking of provenance](https://youtu.be/HB5TrK7A4pI?t=2852) as a solution +* Analyze changing denom-based rate limits, to just overall withdrawal amount for Osmosis + +## Repository +For more an in-depth docs and implementation, please visit: https://github.com/osmosis-labs/osmosis/tree/main/x/ibc-rate-limit \ No newline at end of file diff --git a/docs/overview/features/index.mdx b/docs/overview/features/index.mdx new file mode 100644 index 000000000..de065dc4c --- /dev/null +++ b/docs/overview/features/index.mdx @@ -0,0 +1,21 @@ +--- +title: Overview +sidebar_position: 1 +--- +import DocCardList from '@theme/DocCardList'; + +# Features + + +This page serves as an informative gateway to explore the innovative and distinctive features that Osmosis brings to the decentralized finance (DeFi) landscape. Leveraging its pioneering App Chain architecture, Osmosis offers a unique DeFi experience that sets it apart from traditional platforms. + +## Table of Contents + - [Concentrated Liquidity](/overview/features/concentrated-liquidity) + - [EIP-1559](/overview/features/eip-1559) + - [Fee Abstraction](/overview/features/fee-abstraction) + - [IBC Hooks](/overview/features/ibc-hooks) + - [IBC Rate Limit](/overview/features/ibc-rate-limit) + - [Superfluid Staking](/overview/features/superfluid) + - [Token Factory](/overview/features/tokenfactory) + - [Protorev](/overview/features/protorev) + diff --git a/docs/overview/features/protorev.md b/docs/overview/features/protorev.md new file mode 100644 index 000000000..2a7f1c18f --- /dev/null +++ b/docs/overview/features/protorev.md @@ -0,0 +1,744 @@ +--- +sidebar_position: 9 +--- + +# Protorev + +# Abstract + +ProtoRev is a module that: + +1. Runs during the Posthandler (core trading execution) and Epoch Hook (keeper store updating) +2. In the posthandler of a tx, checks if that tx swaps (has SwapExactAmountIn or SwapExactAmountOut as Msgs) +3. If a tx swaps, generates routes related to the pool swapped against that may contain cyclic arbitrage opportunities after the user’s swap +4. For each route, determines the optimal amount of the asset to swap in that results in maximum amount of the same asset out (profit) +5. Compares profits and selects the route that generates the most profit and is greater than 0 +6. Mints the optimal amount of asset to swap in from the Bank module (as determined previously) +7. Executes the MultiHopSwapExactAmountIn with the optimal input amount for the route +8. Burns the same amount of asset previously minted to execute the swap +9. Redistributes the profit captured back to the Osmosis ecosystem based on Governance. + +For ecosystem context about the purpose of the module, please see the ProtoRev governance proposal discussion: [https://gov.osmosis.zone/discussion/7078-skip-x-osmosis-proposal-to-capture-mev-as-protocol-revenue-on-chain](https://gov.osmosis.zone/discussion/7078-skip-x-osmosis-proposal-to-capture-mev-as-protocol-revenue-on-chain) + +# Concepts + +## Cyclic Arbitrage + +Cyclic arbitrage is a series of swaps that results in more of the same asset that was initially swapped in. An example of this is as follows: + +Assume there exist three pools with the following asset pairs: + +```bash +1. A/B +2. B/C +3. C/A +``` + +A user executes a multi-hop swap that swaps between pools 1, 2, and 3 with the following outcome (user inputs 10A into pool 1, and receives 15A from pool 3): + +```bash +User -> 10A -> Pool 1 -> 5B -> Pool 2 -> 20C -> Pool 3 -> 15A -> User +``` + +This series of swaps is known as a cyclic swap because it starts and ends in the same asset. A cyclic swap is known as a cyclic arbitrage swap when the output amount is greater than the input amount, for the same asset. + +## Cyclic Arbitrage Route + +A Cyclic Arbitrage Route describes an ordered set of pools that need to be swapped through in consecutive order to capture a cyclic arbitrage opportunity. A Cyclic Route can be determined without knowing current reserve ratios of pools by assessing if one can swap in an asset into the series of pools and receive the same asset out. + +So for the same pools as the example above, an exhaustive list of Cyclic Routes are as follows: + +```python +1. A/B +2. B/C +3. C/A + +(1,2,3) # Asset A in, Asset A Out +(3,2,1) # Asset A in, Asset A Out +(3,1,2) # Asset C in, Asset C Out +(2,1,3) # Asset C in, Asset C Out +(2,3,1) # Asset B in, Asset B Out +(1,3,2) # Asset B in, Asset B Out +``` + +What determines if a Cyclic Route is a Cyclic Arbitrage Route at any given state of the chain (state of pool reserves) is if there exists an amount of an asset to be swapped into the route that results in more of the same asset out (10A in, 10A+ Out). + +## Optimal Amount In to Swap + +When given an ordered route against a specific chain state (state of pool reserves) where a cyclic arbitrage opportunity exists, one must then determine how much to swap in to capture maximum profits (where profits is defined as Asset Out Amount - Asset In Amount). + +ProtoRev uses a binary search algorithm to determine the optimal amount in to swap, using functions from the PoolManager module for calculations and swap execution. + +# State + +## State Object + +The `x/protorev` module keeps the following objects in state: + +| State Object | Description | Key | Values | Store | +| --- | --- | --- | --- | --- | +| TokenPairArbRoutes | TokenPairRoutes tracks cyclic arb routes that can be used to create a MultiHopSwap given two denoms | []byte{1} + []byte{inputDenom} +[]byte{outputDenom} | []byte{TokenPairArbRoutes} | KV | +| DenomPairToPool | Tracks the pool ids of the highest liquidity pools matched with a given denom[]byte{2} | []byte{2} + []byte{baseDenom} + []byte{denomToMatch} | []byte{poolID} | KV | +| BaseDenoms | Tracks all of the base denominations that will be used to construct arbitrage routes | []byte{3} | []byte{[]BaseDenoms{}} | KV | +| NumberOfTrades | Tracks the number of trades protorev has executed | []byte{4} | []byte{numberOfTrades} | KV | +| ProfitsByDenom | Tracks the profits protorev has made | []byte{5} + []byte{tokenDenom} | []byte{sdk.Coin} | KV | +| TradesByRoute | Tracks the number of trades the module has executed on a given route | []byte{6} + []byte{route} | []byte{numberOfTrades} | KV | +| ProfitsByRoute | Tracks the profits the module has accumulated after trading on a given route | []byte{7} + []byte{route} | []byte{sdk.Coin} | KV | +| DeveloperAccount | Tracks the developer account for protorev | []byte{8} | []byte{sdk.AccAddress} | KV | +| DaysSinceModuleGenesis | Tracks the number of days since the module was initialized. Used to track profits that can be withdrawn by the developer account | []byte{9} | []byte{uint} | KV | +| DeveloperFees | Tracks the profits that the developer account can withdraw | []byte{10} + []byte{tokenDenom} | []byte{sdk.Coin} | KV | +| MaxPoolPointsPerTx | Tracks the maximum number of pool points that can be consumed per tx | []byte{11} | []byte{uint64} | KV | +| MaxPoolPointsPerBlock | Tracks the maximum number of pool points that can be consumed per block | []byte{12} | []byte{uint64} | KV | +| PoolPointCountForBlock | Tracks the number of pool points that have been consumed in this block | []byte{13} | []byte{uint64} | KV | +| LatestBlockHeight | Tracks the latest recorded block height | []byte{14} | []byte{uint64} | KV | +| PoolWeights | Tracks the weights (pool points) of the different pool types | []byte{15} | []byte{PoolWeights} | KV | + +### TokenPairArbRoutes + +TokenPairArbRoutes are cyclic arbitrage routes that are not going to be captured by the highest liquidity method (described in state transitions below). If there is a cyclic arbitrage route that is frequently being utilized by searchers, `x/protorev` can manually enter this route - through the admin account - and allow it to be used for trading. Each TokenPairArbRoutes object tracks a directional swap of two assets, and associates the swap with cyclic routes. When the module sees a swap of (`token_in`, `token_out`), it will extract the `arb_routes` that should be used and will simulate trades and execute them if profitable. + +```go +// TokenPairArbRoutes tracks all of the hot routes for a given pair of tokens +message TokenPairArbRoutes { + option (gogoproto.equal) = true; + + // Stores all of the possible hot paths for a given pair of tokens + repeated Route arb_routes = 1; + // Token denomination of the first asset + string token_in = 2; + // Token denomination of the second asset + string token_out = 3; +} + +// Route is a hot route for a given pair of tokens +message Route { + option (gogoproto.equal) = true; + + // The pool IDs that are traversed in the directed cyclic graph (traversed left + // -> right) + repeated Trade trades = 1; + // The step size that will be used to find the optimal swap amount in the + // binary search + string step_size = 2 [ + (gogoproto.customtype) = "github.com/cosmos/cosmos-sdk/types.Int", + (gogoproto.nullable) = true + ]; +} + +// Trade is a single trade in a route +message Trade { + option (gogoproto.equal) = true; + + // The pool IDs that are traversed in the directed cyclic graph (traversed left + // -> right) + uint64 pool = 1; + // The denom of token A that is traded + string token_in = 2; + // The denom of token B that is traded + string token_out = 3; +} +``` + +### DenomPairToPool + +DenomPairToPool takes in a base denomination (read below) – denom that is used to build routes (ex. osmo, atom, usdc) – and a denom to match (akash, juno) and returns the highest liquidity pool id between the pair of denominations. For example, an input might look like (osmo, juno) —> poolID: 5. This store is directly tied to the highest liquidity method (described in state transitions below). Each base denomination is going to have its own set of denominations it maps to. + +### BaseDenoms + +BaseDenoms are the denominations that are used to build the highest liquidity routes. This will be configurable by the admin account, but will always maintain at least `uosmo` as a base denom. A base denom just means the denomination that will be used to start and end a cyclic arbitrage route. Base denoms can be added on as needed basis. + +***NOTE***: BaseDenoms do have a priority that is directly tied down to the order in the list of base denoms that are used i.e. BaseDenoms that are closer to the front of the list will likely be simulated and executed more often than those later in the list. This is done by design so that we can prioritize certain denoms over others in order to simulate and execute the most profitable trades. + +### NumberOfTrades + +This will store the total number of arbitrage trades that `x/protorev` has executed since genesis. This gets incremented every time the module executes a trade. + +### ProfitsByDenom + +This will store the profits `x/protorev` has accumulated for a given denom. + +### TradesByRoute & ProfitsByRoute + +These stores allow users and researchers to query the number of cyclic arbitrage trades that have been executed by `x/protorev` on an cyclic arbitrage route as well as all of the profits captured on that same route. Routes are denoted by the pool ids in the route i.e. []uint64{1,2,3}. + +### ProtoRevEnabled + +`x/protorev` can be enabled or disabled through governance. As a proposal is a stateful change, we store whether the module is currently enabled or disabled in the module. + +### AdminAccount + +The admin account is set through governance and has permissions to set hot routes, the maximum number of pool points per transaction, maximum number of pool points per block, pool type weights, base denoms and the developer account. On genesis, the admin account is set to a trusted address that is stored on a ledger - currently configured to be the Skip dev team's address. Note that governance has full ability to change this live on-chain, and this admin can at most prevent `x/protorev` from working. All the admin account's controls have limits, so it can't lead to a chain halt, excess processing time or prevention of swaps. + +### DeveloperAccount + +The developer account is set through a MsgSetDeveloperAccount tx. This is the account that will be able to withdraw a portion of the profits from `x/protorev` as specified by the Osmosis ↔ Skip proposal. Only the admin account has permission to make this message. + +### DaysSinceModuleGenesis + +`x/protorev` will distribute 20% of profits to the developer account in year 1, 10% of profits in year 2, and 5% thereafter. To track how much profit can be distributed to the developer account at any given moment, we store the amount of days since module genesis. + +### DeveloperFees (DEPRECATED IN v16) + +DeveloperFees tracks the total amount of profit that can be withdrawn by the developer account. These fees are sent to the developer account, if set, every week through the `epoch` hook. If unset, the funds are held in the module account. All `x/protorev` profits are going to be stored on the module account. + +### MaxPoolPointsPerTx + +A pool point roughly corresponds to a millisecond of trading simulation and execution time. In order to bound the compute time of `x/protorev` , we set a maximum number of pool points (execution time) per transaction and per block. MaxPoolPointsPerTx tracks the maximum number of pool points that can be consumed in a given transaction. This is configurable (but bounded) by the admin account. We limit the number of pool points per transaction so that all `x/protorev` execution is not limited to the top of the block. + +### MaxPoolPointsPerBlock + +MaxPoolPointsPerBlock tracks the maximum number of pool points that can be consumed in a given block. This is configurable (but bounded) by the admin account. We limit the number of pool points per block so that the execution time of the `x/protorev` posthandler is reasonably bounded to ensure that block time remains as is. + +### PoolPointCountForBlock + +PoolPointCountForBlock tracks the number of pool points that have been consumed in the current block. Used to ensure that the module is not slowing down block speed. + +### LatestBlockHeight + +LatestBlockHeight tracks the latest recorded block height. This is used to update and reset the pool point count within a block and after new blocks are proposed. + +### PoolWeights + +PoolWeights assigns each pool type to a number of pool points it will approximately consume. This tracks the pool points or weight of each pool type that can be traversed. This distinction is necessary because different pool types have different simulation and execution times. + +```go +// PoolWeights contains the weights of all of the different pool types. This +// distinction is made and necessary because the execution time ranges +// significantly between the different pool types. Each weight roughly +// corresponds to the amount of time (in ms) it takes to execute a swap on that +// pool type. +type PoolWeights struct { + // The weight of a stableswap pool + StableWeight uint64 `protobuf:"varint,1,opt,name=stable_weight,json=stableWeight,proto3" json:"stable_weight,omitempty"` + // The weight of a balancer pool + BalancerWeight uint64 `protobuf:"varint,2,opt,name=balancer_weight,json=balancerWeight,proto3" json:"balancer_weight,omitempty"` + // The weight of a concentrated pool + ConcentratedWeight uint64 `protobuf:"varint,3,opt,name=concentrated_weight,json=concentratedWeight,proto3" json:"concentrated_weight,omitempty"` +} +``` + +### GenesisState + +There is only one configurable parameter for the genesis state —> whether protorev is enabled or not. + +```go +// GenesisState defines the protorev module's genesis state. +type GenesisState struct { + // Module Parameters + Params Params `protobuf:"bytes,1,opt,name=params,proto3" json:"params"` +} +``` + +# State Transitions + +The `protorev` module triggers state transitions in the `postHandler` , governance proposals, and admin account transactions. After each `sdk.Tx`, the `postHandler` will determine whether there were any `MsgSwapExactAmountIn` or `MsgSwapExactAmountOut` in the transaction. If so, the module gets all of the pools that were used in the swap(s), temporarily stores the pool ids accessed along with their respective tokenIn/tokenOut denoms, and then builds cyclic arbitrage routes for each pool swapped against. + +## Route Generation + +There are two primary methods for route generation: **Highest Liquidity Pools** and **Hot Routes**. + +### Highest Liquidity Pool Method + +The highest liquidity pool method will always create cyclic arbitrage routes that have three pools. The routes that are created will always start and end with one of the denominations that are stored in BaseDenoms. The pool swapped against that the `postHandler` processes will always be the 2nd pool in the three-pool cyclic arbitrage route. + +**Highest Liquidity Pools:** Updated via the daily epoch, the module iterates through all the pools and stores the highest liquidity pool for every asset that pairs with any of the base denominations the module stores (for example, the osmo/juno key will have a single pool id stored, that pool id having the most liquidity out of all the osmo/juno pools). New base denominations can be added or removed on an as needed basis by the admin account. A base denomination is just another way of describing the denomination we want to use for cyclic arbitrage. This store is then used to create routes at runtime after analyzing a swap. This store is updated through the `epoch` hook and when the admin account submits a `MsgSetBaseDenoms` tx. + +The simplest way to conceptualize how the route is generated is by the following example. Assume we have two base denominations that `x/protorev` is currently tracking. + +BaseDenoms + +- Osmosis +- Atom + +Lets say the `postHandler` receives a transaction that contains a swap of **Juno** —> **Akash** on pool **4**. In this case, the module will attempt to create three-pool route where a base denomination is on either side of the route. For example, a route that it might create is + +- Osmosis —> Akash (on pool 1), Akash —> Juno (on pool 4), Juno —> Osmosis (on pool 2) + +It does so by finding the highest liquidity pool between (Osmosis, Akash) —> pool 1 and the highest liquidity pool between (Osmosis, Juno) —> pool 2. If there is no highest liquidity pool pair between (Osmosis, Juno) or (Osmosis, Akash), no route will be generated. + +**NOTE: Cyclic arbitrage routes will always go in the opposite direction of the original swap i.e. in this case we see Juno —> Akash so we know that the route must include a swap of Akash —> Juno.** + +The same line of reasoning exists for Atom. `x/protorev` will attempt to find the highest liquidity pool between (Atom, Akash) and (Atom, Juno). If these pools exist, they will be added to the list of routes that can be simulated later in the pipeline. If not, the route is discarded. + +In both cases, the route that is built will always surround the pool of the original swap that was made. However, we allow for more flexibility in route generation as the highest liquidity method may not be optimal, hence the additional of hot routes. + +### Hot Route Method + +Populated through the admin account, the module’s keeper holds a KV store that associates token pairs (for example, osmo/juno) to the routes that result in a high percentage of arbitrage profit on Osmosis (as determined by external analysis). + +The purpose of storing Hot Routes is a recognition that the Highest Liquidity Pool method may not present the best arbitrage routes. As such, hot routes can be configured by the admin account to store additional routes that may be more effective at capturing arbitrage opportunities. Each hot route will store a placeholder for where the current swapped pool will fit into the trade. + +### Pool Rebalancing + +Now that we have a list of cyclic routes for each pool swapped by the user’s tx, we then determine if any of the routes are profitable. We determine this using a binary search algorithm that finds the amount of the asset to swap in that results in the most of that same asset out. We then calculate profits by taking the difference between the amount of the asset out and amount of the asset in. By iterating through the routes and storing the route, optimal input amount, and profit of the route with the highest profit > 0, we are left with the route and amount to execute the MultiHopSwap against. + +Each swap will generate its own set of routes and `x/protorev` will execute only the most profitable route. + +The module mints the optimal input amount of the coin to swap in from the `bankkeeper` to the `x/protorev` module account, executes the MultiHopSwap by interacting with the `x/poolmanager` module, burns the optimal input amount of the coin minted to execute the MultiHopSwap, and sends subsequent profits to the module account. + +## Governance Proposals + +`x/protorev` implements two different governance proposals. + +**SetProtoRevAdminAccountProposal** + +As the landscape of pools on Osmosis evolves, an admin account will be able to add and remove routes for `x/protorev` to check for cyclic arbitrage opportunities along with several other optimization txs. Largely, the purpose of maintaining hot routes is to reduce the amount of computation that would otherwise be required to determine optimal paths at runtime. + +This proposal is put in place in case the admin account needs to be transferred over. However, as mentioned above, it will be initialized to a trusted address on genesis. + +**SetProtoRevEnabledProposal** + +This proposal type allows the chain to turn the module on or off. This is meant to be used as a fail safe in the case stakers and the chain decide to turn the module off. This might be used to halt the execution of trades in the case that the `x/gamm` module has significant upgrades that might produce unexpected behavior from the module. + +## PostHandler + +The `postHandler` extracts pools that were swapped in a transaction and determines if there is a cyclic arbitrage opportunity. If so, the handler will find an optimal route and execute it - rebalancing the pool and returning arbitrage profits to the module account. + +1. Check if the module is enabled. + 1. If the module is disabled, nothing happens. +2. Extract all pools that were traded on in the transaction (`ExtractSwappedPools`) as well as the direction of the trade. +3. Create cyclic arbitrage routes for each of the swaps above (`BuildRoutes`) +4. For each feasible route, determine if there is a cyclic arbitrage opportunity (`IterateRoutes`) + 1. Determine the optimal amount to swap in and its respective profits via binary search over range of potential input amounts (`FindMaxProfitForRoute`) + 2. Compare profits of each route, keep the best route and input amount with the highest profit +5. If the best route and input amount has a profit > 0, execute the trade (`ExecuteTrade`) and rebalance the pools on-behalf of the chain through the `poolmanagerkeeper` (`MultiHopSwapExactAmountIn`) +6. Keep the profits in the module’s account for subsequent distribution. + +### ExtractSwappedPools + +Checks if there were any swaps made on pools in a transaction, returning the pool ids and input/output denoms for each pool that was traded on. + +### BuildRoutes + +BuildRoutes takes a token pair (input and output denom) as well as the pool id and returns a list of routes for that token pair that potentially contain a cyclic arbitrage opportunity, populated via the Hot Route and Highest Liquidity Pools method as described above. + +### IterateRoutes + +IterateRoutes iterates through a list of routes, determining the route and input amount that results in the highest cyclic arbitrage profits.. + +### FindMaxProfitForRoute + +This will take in a route and determine the optimal amount to swap in to maximize profits, given the reserves of all of the pools that are swapped against in the route. The bounds of the binary search are dynamic and update per route (see `UpdateSearchRangeIfNeeded`) based on how computationally expensive (in terms of gas) swapping can be on that route. For instance, moving across several ticks on a concentrated pool is relatively expensive, so the bounds of the binary search with a route that includes that pool type may be smaller than a route that does not include that pool type. + +### ExecuteTrade + +Execute trade takes the route and optimal input amount as params, mints the optimal amount of input coin, executes the swaps via `poolmanagerKeeper`’s `MultiHopSwapExactAmountIn`, and then burns the amount of coins originally minted, storing the profits in it’s own module account. + +This will also update various trading statistics in the module’s store. It will update the total number of trades the module has executed, total profits captured, profits made on this specific route, share of profits the developer account can withdraw, and more. + +## Execution Guardrails + +`x/protorev` is bounded and limited in the number of trades the module can execute per block. The purpose of doing so is to ensure that the current block time does not substantially change and that the module does not introduce a new attack vector. + +Execution is currently limited in the following ways + +1. The binary search method for finding input amounts is bounded by some number of iterations. +2. The number of routes that can be traversed in a given transaction is bounded by some number. +3. The number of routes that can be traversed in a given block is bounded by some number. + +# Hooks + +The `x/protorev` module implements epoch hooks in order to trigger the recalculation of the highest liquidity pools paired with any of the base denominations, manages the distribution of developer profits over time, and updates pool point information. + +## Epoch Hook + +The Epoch hook allows the module to update the information listed above using the epoch identifier `day`. + +### Highest Liquidity Pools + +As described above, one method of determining cyclic arbitrage opportunities is to use the highest liquidity pools paired with any base denomination. While this calculation is done on genesis (with only Osmo configured), the pools may restructure over time and new tokens may end up being traded heavily with the base denominations. As such, it is necessary to update this over time so that the module’s logic in determining cyclic arbitrage opportunities is most optimal and updated. Using the `AfterEpochEnd` hook in combination with the `day` epoch identifier, we are able to successfully update the pool information every day. At runtime, `UpdatePools` will be executed and all of the internal pool info will be updated. + +### Profit Distribution + +Profits accumulated by the module will be partially distributed to the developers that built the module in accordance with the governance proposal that was passed: year 1 is 20% of profits, year 2 is 10%, and subsequent years is 5%. + +In order to track how much profit the developers can withdraw at any given moment, the module tracks the number of days since module genesis. This gets incremented in the epoch hook after every day. When a trade gets executed by the module, the module will determine how much of the profit from the trade the developers can receive by using `daysSinceModuleGenesis` in a simple calculation. + +If the developer account is not set (which it is not on genesis), all funds are held in the module account. Once the developer address is set by the admin account, the developer address will start to automatically receive a share of profits after every trade. The distribution of funds from the module account is done through `SendDeveloperFees`. + +# Governance Proposals + +This section defines the governance proposals that result in the state transitions defined on the previous section. + +## **`SetProtoRevAdminAccountProposal`** + +A gov `content` type to set the admin account which will be overseeing the selection of hot routes, developer account, and more. Governance users vote on this proposal and it automatically executes the custom handler for `SetProtoRevAdminAccountProposal` when the vote passes. + +```go +// SetProtoRevAdminAccountProposal is a gov Content type to set the admin +// account that will receive permissions to alter hot routes +type SetProtoRevAdminAccountProposal struct { + Title string `protobuf:"bytes,1,opt,name=title,proto3" json:"title,omitempty"` + Description string `protobuf:"bytes,2,opt,name=description,proto3" json:"description,omitempty"` + Account string `protobuf:"bytes,3,opt,name=account,proto3" json:"account,omitempty"` +} +``` + +The proposal content stateless validation fails if: + +- The account entered is not a valid bech32 address. + +## **`SetProtoRevEnabledProposal`** + +A gov `content` type to enable or disable the `x/protorev` module. Governance users vote on this proposal and it automatically executes the custom handler for `SetProtoRevEnabledProposal` when the vote passes. + +```go +// SetProtoRevEnabledProposal is a gov Content type to update the proto rev +// enabled field in the params +type SetProtoRevEnabledProposal struct { + Title string `protobuf:"bytes,1,opt,name=title,proto3" json:"title,omitempty"` + Description string `protobuf:"bytes,2,opt,name=description,proto3" json:"description,omitempty"` + Enabled bool `protobuf:"varint,3,opt,name=enabled,proto3" json:"enabled,omitempty"` +} +``` + +The proposal content stateless validation fails if: + +- The entered field to `enabled` is not a boolean. + +# Transactions + +This section defines the `sdk.Msg` concrete types that result in the state transitions defined on the previous section. + +## `MsgSetDeveloperAccount` + +The admin account broadcasts a `MsgSetDeveloperAccount` to set the developer account. + +```go +// MsgSetDeveloperAccount defines the Msg/SetDeveloperAccount request type. +type MsgSetDeveloperAccount struct { + // admin is the account that is authorized to set the developer account. + Admin string `protobuf:"bytes,1,opt,name=admin,proto3" json:"admin,omitempty"` + // developer_account is the account that will receive a portion of the profit + // from the protorev module. + DeveloperAccount string `protobuf:"bytes,2,opt,name=developer_account,json=developerAccount,proto3" json:"developer_account,omitempty"` +} +``` + +Message stateless validation fails if: + +- The admin is not a valid bech32 address +- The signature of the user does not match the admin account’s +- The developer account is not a valid bech32 address + +Message stateful validation fails if: + +- The admin is not set in state +- The admin entered in the message does not match the admin on chain +- The admin’s signatures are not the same + +## `MsgSetHotRoutes` + +The admin account broadcasts a `MsgSetHotRoutes` to set the hot routes. + +```go +// MsgSetHotRoutes defines the Msg/SetHotRoutes request type. +type MsgSetHotRoutes struct { + // admin is the account that is authorized to set the hot routes. + Admin string `protobuf:"bytes,1,opt,name=admin,proto3" json:"admin,omitempty"` + // hot_routes is the list of hot routes to set. + HotRoutes []*TokenPairArbRoutes `protobuf:"bytes,2,rep,name=hot_routes,json=hotRoutes,proto3" json:"hot_routes,omitempty"` +} +``` + +Message statless validation fails if: + +- The admin is not a valid bech32 address +- The signature of the user does not match the admin account’s +- The hot routes are not valid + - The starting and ending denominations for each route must be the same + - The routes in between must have valid swaps i.e. a → b, b → c, c → a and not a → b, c → b, b → a + - None of the routes can be nil + - The step size must be set for each route - the step size is used in the binary search method + - There must be at least two hops in each route + - There are duplicate token pairs in the msg + +Message stateful validation fails if: + +- The admin is not set in state +- The admin entered in the message does not match the admin on chain +- The admin’s signatures are not the same +- `NewMsgSetHotRoutes` + +## **`MsgSetMaxPoolPointsPerTx`** + +The admin account broadcasts a **`MsgSetMaxPoolPointsPerTx`** to set the maximum number of pool points that can consumed per transaction. + +```go +// MsgSetMaxPoolPointsPerTx defines the Msg/SetMaxPoolPointsPerTx request type. +type MsgSetMaxPoolPointsPerTx struct { + // admin is the account that is authorized to set the max pool points per tx. + Admin string `protobuf:"bytes,1,opt,name=admin,proto3" json:"admin,omitempty"` + // max_pool_points_per_tx is the maximum number of pool points that can be + // consumed per transaction. + MaxPoolPointsPerTx uint64 `protobuf:"varint,2,opt,name=max_pool_points_per_tx,json=maxPoolPointsPerTx,proto3" json:"max_pool_points_per_tx,omitempty"` +} +``` + +Message statless validation fails if: + +- The admin is not a valid bech32 address +- The signature of the user does not match the admin account’s +- The MaxPoolPointsPerTx is out of range of the limits we hardcode + +Message stateful validation fails if: + +- The admin is not set in state +- The admin entered in the message does not match the admin on chain +- The admin’s signatures are not the same + +## `MsgSetMaxPoolPointsPerBlock` + +The admin account broadcasts a `MsgSetMaxPoolPointsPerBlock` to set the maximum number of pool points that can consumed per block. + +```go +// MsgSetMaxPoolPointsPerBlock defines the Msg/SetMaxPoolPointsPerBlock request +// type. +type MsgSetMaxPoolPointsPerBlock struct { + // admin is the account that is authorized to set the max pool points per + // block. + Admin string `protobuf:"bytes,1,opt,name=admin,proto3" json:"admin,omitempty"` + // max_pool_points_per_block is the maximum number of pool points that can be + // consumed per block. + MaxPoolPointsPerBlock uint64 `protobuf:"varint,2,opt,name=max_pool_points_per_block,json=maxPoolPointsPerBlock,proto3" json:"max_pool_points_per_block,omitempty"` +} +``` + +Message statless validation fails if: + +- The admin is not a valid bech32 address +- The signature of the user does not match the admin account’s +- The MaxRoutesPerBlock is out of range of the limits we hardcode + +Message stateful validation fails if: + +- The admin is not set in state +- The admin entered in the message does not match the admin on chain +- The admin’s signatures are not the same + +## **`MsgSetPoolWeights`** + +The admin account broadcasts a **`MsgSetPoolWeights`** to set the pool weights. The pool weights roughly correspond to the execution time of a swap on that pool type (stable, balancer, concentrated). + +```go +// MsgSetPoolWeights defines the Msg/SetPoolWeights request type. +type MsgSetPoolWeights struct { + // admin is the account that is authorized to set the pool weights. + Admin string `protobuf:"bytes,1,opt,name=admin,proto3" json:"admin,omitempty"` + // pool_weights is the list of pool weights to set. + PoolWeights *PoolWeights `protobuf:"bytes,2,opt,name=pool_weights,json=poolWeights,proto3" json:"pool_weights,omitempty"` +} + +// PoolWeights contains the weights of all of the different pool types. This +// distinction is made and necessary because the execution time ranges +// significantly between the different pool types. Each weight roughly +// corresponds to the amount of time (in ms) it takes to execute a swap on that +// pool type. +type PoolWeights struct { + // The weight of a stableswap pool + StableWeight uint64 `protobuf:"varint,1,opt,name=stable_weight,json=stableWeight,proto3" json:"stable_weight,omitempty"` + // The weight of a balancer pool + BalancerWeight uint64 `protobuf:"varint,2,opt,name=balancer_weight,json=balancerWeight,proto3" json:"balancer_weight,omitempty"` + // The weight of a concentrated pool + ConcentratedWeight uint64 `protobuf:"varint,3,opt,name=concentrated_weight,json=concentratedWeight,proto3" json:"concentrated_weight,omitempty"` +} +``` + +Message statless validation fails if: + +- The admin is not a valid bech32 address +- The signature of the user does not match the admin account’s +- Any of the pool weights is not set or is less than or equal to 0. + +Message stateful validation fails if: + +- The admin is not set in state +- The admin entered in the message does not match the admin on chain +- The admin’s signatures are not the same + +## **`MsgSetBaseDenoms`** + +The admin account broadcasts a **`MsgSetBaseDenoms`** to set the base denominations the module will use to create cyclic arbitrage routes. + +```go +// MsgSetBaseDenoms defines the Msg/SetBaseDenoms request type. +type MsgSetBaseDenoms struct { + // admin is the account that is authorized to set the base denoms. + Admin string `protobuf:"bytes,1,opt,name=admin,proto3" json:"admin,omitempty"` + // base_denoms is the list of base denoms to set. + BaseDenoms []*BaseDenom `protobuf:"bytes,2,rep,name=base_denoms,json=baseDenoms,proto3" json:"base_denoms,omitempty"` +} + +// BaseDenom represents a single base denom that the module uses for its +// arbitrage trades. It contains the denom name alongside the step size of the +// binary search that is used to find the optimal swap amount +type BaseDenom struct { + // The denom i.e. name of the base denom (ex. uosmo) + Denom string `protobuf:"bytes,1,opt,name=denom,proto3" json:"denom,omitempty"` + // The step size of the binary search that is used to find the optimal swap + // amount + StepSize github_com_cosmos_cosmos_sdk_types.Int `protobuf:"bytes,2,opt,name=step_size,json=stepSize,proto3,customtype=github.com/cosmos/cosmos-sdk/types.Int" json:"step_size"` +} +``` + +Message statless validation fails if: + +- The admin is not a valid bech32 address +- The signature of the user does not match the admin account’s +- Osmosis is not the first base denom in the list +- The step size for any of the base denoms is not set +- There are duplicate base denoms + +Message stateful validation fails if: + +- The admin is not set in state +- The admin entered in the message does not match the admin on chain +- The admin’s signatures are not the same + +# Parameters + +Tracks whether the module is enabled on genesis. + +```go +// Params defines the parameters for the module. +type Params struct { + // Boolean whether the module is going to be enabled + Enabled bool `protobuf:"varint,1,opt,name=enabled,proto3" json:"enabled,omitempty"` +} +``` + +## Enabled + +The `Enabled` parameters toggles all state transitions in the module. When the parameter is disabled, it will prevent all module functionality. + +# Clients + +## CLI + +Find below a lost of `osmosisd` commands added with the `x/protorev` module. A CLI command can look like this: + +```bash +osmosisd query protorev params +``` + +### Queries + +| Command | Subcommand | Description | +| --- | --- | --- | +| query protorev | params | Queries the parameters of the module | +| query protorev | number-of-trades | Queries the number of cyclic arbitrage trades ProtoRev has executed | +| query protorev | profits-by-denom [denom] | Queries ProtoRev profits by denom | +| query protorev | all-profits | Queries all ProtoRev profits | +| query protorev | statistics-by-route [route] where route is the list of pool ids i.e. [1,2,3] | Queries ProtoRev statistics by route | +| query protorev | all-statistics | Queries all ProtoRev statistics | +| query protorev | hot-routes | Queries the ProtoRev token pair arb routes | +| query protorev | admin-account | Queries the ProtoRev admin account | +| query protorev | developer-account | Queries the ProtoRev developer account | +| query protorev | max-pool-points-per-tx | Queries the ProtoRev max pool points per transaction | +| query protorev | max-pool-points-per-block | Queries the ProtoRev max pool points per block | +| query protorev | base-denoms | Queries the ProtoRev base denoms used to create cyclic arbitrage routes | +| query protorev | enabled | Queries whether the ProtoRev module is currently enabled | +| query protorev | pool-weights | Queries the pool weights used to determine how computationally expensive a route is | +| query protorev | pool | Queries the pool id for a given denom pair stored in ProtoRev | + +### Proposals + +| Command | Subcommand | Description | +| --- | --- | --- | +| tx protorev | set-pool-weights [path/to/file.json] | Submit a tx to set the pool weights for ProtoRev | +| tx protorev | set-hot-routes [path/to/file.json] | Submit a tx to set the hot routes for ProtoRev | +| tx protorev | set-base-denoms [path/to/file.json] | Submit a tx to set the base denoms for ProtoRev | +| tx protorev | set-max-pool-points-per-block [uint64] | Submit a tx to set the max pool points per block for ProtoRev | +| tx protorev | set-max-pool-points-per-tx [uint64] | Submit a tx to set the max pool points per transaction for ProtoRev | +| tx protorev | set-developer-account [sdk.AccAddress] | Submit a tx to set the developer account for ProtoRev | +| tx protorev | set-admin-account-proposal [sdk.AccAddress] | Submit a proposal to set the admin account for ProtoRev | +| tx protorev | set-enabled-proposal [boolean] | Submit a proposal to disable/enable the ProtoRev module | + +## gRPC & REST + +### Queries + +| Verb | Method | Description | +| --- | --- | --- | +| gRPC | osmosis.protorev.Query/Params | Queries the parameters of the module | +| gRPC | osmosis.protorev.Query/GetProtoRevNumberOfTrades | Queries the number of arbitrage trades the module has executed | +| gRPC | osmosis.protorev.Query/GetProtoRevProfitsByDenom | Queries the profits of the module by denom | +| gRPC | osmosis.protorev.Query/GetProtoRevAllProfits | Queries all of the profits from the module | +| gRPC | osmosis.protorev.Query/GetProtoRevStatisticsByRoute | Queries the number of arbitrages and profits that have been executed for a given route | +| gRPC | osmosis.protorev.Query/GetProtoRevAllStatistics | Queries all of routes that the module has arbitrage against and the number of trades and profits that have been executed for each route | +| gRPC | osmosis.protorev.Query/GetProtoRevTokenPairArbRoutes | Queries all of the hot routes that the module is currently arbitraging | +| gRPC | osmosis.protorev.Query/GetProtoRevMaxPoolPointsPerTx | Queries the ProtoRev max pool points per transaction | +| gRPC | osmosis.protorev.Query/GetProtoRevMaxPoolPointsPerBlock | Queries the ProtoRev max pool points per block | +| gRPC | osmosis.protorev.Query/GetProtoRevAdminAccount | Queries the admin account of the ProtoRev | +| gRPC | osmosis.protorev.Query/GetProtoRevDeveloperAccount | Queries the developer account of the ProtoRev | +| gRPC | osmosis.protorev.Query/GetProtoRevBaseDenoms | Queries the ProtoRev base denoms used to create cyclic arbitrage routes | +| gRPC | osmosis.protorev.Query/GetProtoRevEnabled | Queries whether the ProtoRev module is currently enabled | +| gRPC | osmosis.protorev.Query/GetProtoRevPoolWeights | Queries the number of pool points each pool type will consume when executing and simulating trades | +| gRPC | osmosis.protorev.Query/GetProtoRevPool | Queries the pool id for a given denom pair stored in ProtoRev | +| GET | /osmosis/protorev/params | Queries the parameters of the module | +| GET | /osmosis/protorev/number_of_trades | Queries the number of arbitrage trades the module has executed | +| GET | /osmosis/protorev/profits_by_denom | Queries the profits of the module by denom | +| GET | /osmosis/protorev/all_profits | Queries all of the profits from the module | +| GET | /osmosis/protorev/statistics_by_route | Queries the number of arbitrages and profits that have happened for a given route | +| GET | /osmosis/protorev/all_route_statistics | Queries all of routes that the module has arbitrage against and the number of trades and profits that have happened for each route | +| GET | /osmosis/protorev/token_pair_arb_routes | Queries all of the hot routes that the module is currently arbitraging | +| GET | /osmosis/protorev/max_pool_points_per_tx | Queries the maximum number of pool points that can be consumed per transaction | +| GET | /osmosis/protorev/max_pool_points_per_block | Queries the maximum number of pool points that can be consumed per block | +| GET | /osmosis/protorev/admin_account | Queries the admin account of the ProtoRev | +| GET | /osmosis/protorev/developer_account | Queries the developer account of the ProtoRev | +| GET | /osmosis/protorev/base_denoms | Queries the base denominations ProtoRev is currently using to create cyclic arbitrage routes | +| GET | /osmosis/protorev/enabled | Queries whether the ProtoRev module is currently enabled | +| GET | /osmosis/protorev/pool_weights | Queries the number of pool points each pool type will consume when executing and simulating trades | +| GET | /osmosis/protorev/pool | Queries the pool id for a given denom pair stored in ProtoRev | + +### Transactions + +| Verb | Method | Description | +| --- | --- | --- | +| gRPC | osmosis.protorev.Msg/SetHotRoutes | Sets the hot routes that will be explored when creating cyclic arbitrage routes. Can only be called by the admin account | +| gRPC | osmosis.protorev.Msg/SetDeveloperAccount | Sets the account that can withdraw a portion of the profit from the ProtoRev module. Can only be called by the admin account | +| gRPC | osmosis.protorev.Msg/SetMaxPoolPointsPerTx | Sets the maximum number of pool points that can be consumed per transaction | +| gRPC | osmosis.protorev.Msg/SetMaxPoolPointsPerBlock | Sets the maximum number of routes that can be iterated per block | +| gRPC | osmosis.protorev.Msg/SetBaseDenoms | Sets the base denominations the ProtoRev module will use to create cyclic arbitrage routes | +| gRPC | osmosis.protorev.Msg/SetPoolWeights | Sets the amount of pool points each pool type will consume when executing and simulating trades | +| POST | /osmosis/protorev/set_hot_routes | Sets the hot routes that will be explored when creating cyclic arbitrage routes. Can only be called by the admin account | +| POST | /osmosis/protorev/set_developer_account | Sets the account that can withdraw a portion of the profit from the ProtoRev module. Can only be called by the admin account | +| POST | /osmosis/protorev/set_max_pool_points_per_tx | Sets the maximum number of pool points that can be consumed per transaction | +| POST | /osmosis/protorev/set_max_pool_points_per_block | Sets the maximum number of pool points that can be consumed per block | +| POST | /osmosis/protorev/set_pool_weights | Sets the amount of pool points each pool type will consume when executing and simulating trades | +| POST | /osmosis/protorev/set_base_denoms | Sets the base denominations that will be used by ProtoRev to construct cyclic arbitrage routes | + +## Events + +There is 1 type of event that exists in ProtoRev: + +* `types.TypeEvtBackrun` - "protorev_backrun" + +### `types.TypeEvtBackrun` + +This event is emitted after ProtoRev successfully backruns a transaction. + +It consists of the following attributes: + +* `types.AttributeValueCategory` - "ModuleName" + * The value is the module's name - "protorev". +* `types.AttributeKeyUserPoolId` + * The value is the pool id that the user swapped on that ProtoRev backran. +* `types.AttributeKeyTxHash` + * The value is the transaction hash that ProtoRev backran. +* `types.AttributeKeyUserDenomIn` + * The value is the user denom in for the swap ProtoRev backran. +* `types.AttributeKeyUserDenomOut` + * The value is the user denom out for the swap ProtoRev backran. +* `types.AttributeKeyBlockPoolPointsRemaining` + * The value is the remaining block pool points ProtoRev can still use after the backrun. +* `types.AttributeKeyTxPoolPointsRemaining` + * The value is the remaining tx pool points ProtoRev can still use after the backrun. +* `types.AttributeKeyProtorevProfit` + * The value is the profit ProtoRev captured through the backrun. +* `types.AttributeKeyProtorevAmountIn` + * The value is the amount Protorev swapped in to execute the backrun. +* `types.AttributeKeyProtorevAmountOut` + * The value is the amount Protorev got out of the backrun swap. +* `types.AttributeKeyProtorevArbDenom` + * The value is the denom that ProtoRev swapped in/out to execute the backrun. diff --git a/docs/overview/features/superfluid.md b/docs/overview/features/superfluid.md new file mode 100644 index 000000000..6b0ad4a50 --- /dev/null +++ b/docs/overview/features/superfluid.md @@ -0,0 +1,217 @@ +--- +sidebar_position: 7 +--- + +# Superfluid Staking + +## Abstract + +Superfluid Staking provides the consensus layer more security with a +sort of "Proof of Useful Stake". Each person gets an amount of Osmo +representative of the value of their share of liquidity pool tokens +staked and delegated to validators, resulting in the security guarantee +of the consensus layer to also be based on GAMM LP shares. The OSMO +token is minted and burned in the context of Superfluid Staking. +Throughout all of this, OSMO's supply is preserved in queries to the +bank module. + +### The process + +All of the below methods are found under the [Superfluid +modules](https://github.com/osmosis-labs/osmosis/tree/main/x/superfluid). + +- The `SuperfluidDelegate` method stores your share of bonded + liquidity pool tokens, with `validateLock` as a verifier for lockup + time. +- `GetSuperfluidOsmo` mints OSMO tokens each day for delegation as a + representative of the value of your pool share. This amount is + minted because the staking module at the moment requires staked + tokens to be in OSMO. This amount is burned each day and re-minted + to keep the representative amount of the value of your pool share + accurate. The lockup duration is guaranteed from the underlying + lockup module. +- `GetExpectedDelegationAmount` iterates over each (denom, delegate) + pair and checks for how much OSMO we have delegated. The difference + from the current balance to what is expected is burned / minted to + match with the expected. +- A `messageServer` method executes the Superfluid delegate message. +- `syntheticLockup` is used to index bond holders and tracking their + addresses for reward distribution or potentially slashing purposes. + These track whether if your Superfluid stake is currently bonding or + unbonding. +- An `IntermediaryAccount` is mostly used for the actual reward + distribution or slashing events, and are responsible for + establishing the connection between each superfluid staked lock and + their delegation to the validator. These work by transferring the + superfluid OSMO to their respective delegators. Rewards are linearly + scaled based on how much you have locked for a given (validator, + denom) pair. Rewards are first moved to the incentive gauges, then + distributed from the gauges. In this way, we're using the existing + gauge reward system for paying out superfluid staking rewards and + tracking the amount you have superfluidly staked using the lockup + module. +- Rewards are distributed per epoch, which is currently a day. + `abci.go` checks whether or not the current block is at the + beginning of the epoch using `BeginBlock`. +- Superfluid staking will continue to expand to other Osmosis pools + based on governance proposals and vote turnouts. + +### Example + +If Alice has 500 GAMM tokens bonded to the ATOM \<\> OSMO, she will have +the equivalent value of OSMO minted, delegated to her chosen staker, and +burned for her each day with Superfluid staking. On the user side, all +she has to know is who she wants to delegate her tokens to. In order to +switch delegation, she has to unbond her tokens from the pool first and +then redeposit. Bob, who has a share of the same liquidity pool before +Superfluid Staking went live, also has to re-deposit into the pool for +the above process to kickstart. + +### Why mint Osmo? How is this method safe and accurate? + +Superfluid staking requires the minting of OSMO because in order to +stake on the Osmosis chain, OSMO tokens are required as the chosen +collateral. Synthetic Osmo is minted here as a representative of the +value of each superfluid staker's liquidity pool tokens. + +The pool tokens are acquired by the user from normally staking in a +liquidity pool. They get minted an amount of OSMO equivalent to the +value of their GAMM pool tokens. This method is accurate because +querying the value OSMO every day allows for burning and minting +according to the difference in value of OSMO relative to the expected +delegation amount (as seen with +[GetExpectedDelegationAmount](https://github.com/osmosis-labs/osmosis/blob/main/x/superfluid/keeper/stake.go)). +It's like having a price oracle for fairly calculating the amount the +user has superfluidly staked. + +On epoch (start of every day), we read from the lockup module how much +GAMM tokens we have locked which acts as an oracle for the +representative price of the GAMM token shares. The superfluid module has +"hooks" messages to refresh delegation amounts +(`RefreshIntermediaryDelegationAmounts`) and to increase delegation on +lockup (`IncreaseSuperfluidDelegation`). Then, we see whether or not the +superfluid OSMO currently delegated is worth more or less than this +expected delegation amount amount. If the OSMO is worth more, we do +instant undelegations and immediately burn the OSMO. If less, we mint +OSMO and update the amount delegated. + +This minting is safe because we strict constrain the permissions of Bank +(the module that burns and mints OSMO) to do what it's designed to do. +The authority is mediated through `mintOsmoTokensAndDelegate` and +`forceUndelegateAndBurnOsmoTokens` keeper methods called by the +`SuperfluidDelegate` and `SuperfluidUndelegate` message handlers for the +tokens. The hooks above that increase delegation and refresh delegation +amounts also call this keeper method. + +The delegation is then verified to not already be associated with an +intermediary account (to prevent double-staking), and is always +delegated or withdrawn taking into account various multipliers for +synthetic OSMO value (its worth with respect to the liquidity pool, and +a risk modifier) to prevent mint inaccuracies. Before minting, we also +check that the message sender is the owner of the locked funds; that the +lock is not unlocking; is locked for at least the unbonding period, and +is bonded to a single asset. We also check to see if the lock isn't +already in superfluid and that the same lock isn't currently being +unbonded. + +On the end of each epoch, we iterate through all intermediary accounts +to withdraw delegation rewards they may have received and put it all +into the perpetual gauges corresponding to each account for reward +delegation. + +### Bonding, unbonding, slashing + +Here, we describe how token bonding and unbonding works, and what +happens to your superfluid tokens in the case of a slashing event. + +### Bonding + +When bonding, your input tokens are locked up and you are given GAMM +pool tokens in exchange. These GAMM pool tokens represent a share of the +total liquidity pool, and allows you to get transaction fees or +participate in external incentive gauge token distributions. When +bonding, on top of the regular bonding transaction there will also be a +selection of validators. As stated above, OSMO is also minted and burned +each day and superfluidly staked to whoever you have chosen to be your +validator. You gain additional APR as a reward for bolstering the +Osmosis chain's consensus integrity by delegating. + +### Unbonding + +When unbonding, superfluid tokens get un-delegated. After making sure +that the unbond message sender is the owner of their corresponding +locked funds, the existing synthetic lockup is deleted and replaced with +a new synthetic lockup for unbonding purposes. The undelegated OSMO is +then instantly withdrawn from the intermediate account and validator +using the InstantUndelegate function. The OSMO that was originally used +for representing your LP shares are burnt. Moves the tracker for +unbonding, allows the underlying lock to start unlocking if desired + +## Concepts + +### SyntheticLockups + +SyntheticLockups are synthetica of PeriodLocks, but different in the +sense that they store suffix, which is a combination of +bonding/unbonding status + validator address. This is mainly used to +track whether an individual lock that has been superfluid staked has an +bonding status or a unbonding status from the staking delegations. + +### Intermediary Account + +Intermediary Accounts establishes the connections between the superfluid +staked locks and delegations to the validator. Intermediary accounts +exists for every denom + validator combination, so that it would group +locks with the same denom + validator selection. Superfluid staking a +lock would mint equivalent amount of OSMO of the lock and send it to the +intermediary account and the intermediarry accounts would be delegating +to the specified validator. + +### Intermediary Account Connection + +Intermediary Accounts Connection serves the role of tracking the locks +that an Intermediary Account is dedicated to. + +## State + +### Superfluid Asset + +A superfluid asset is a alternative asset (non-OSMO) that is allowed by +governance to be used for staking. + +It can only be updated by governance proposals. We validate at proposal +creation time that the denom + pool exists. (Are we going to ignore edge +cases around a reference pool getting deleted it) + +### Intermediary Accounts + +Lots of questions to be answered here + +### Dedicated Gauges + +Each intermediary account has has dedicated gauge where it sends the +delegation rewards to. Gauges are distributing the rewards to end users +at the end of the epoch. + +### Synthetic Lockups created + +At the moment, one lock can only be fully bonded to one validator. + +### Osmo Equivalent Multipliers + +The Osmo Equivalent Multiplier for an asset is the multiplier it has for +its value relative to OSMO. + +Different types of assets can have different functions for calculating +their multiplier. We currently support two asset types. + +1. Native Token + +The multiplier for OSMO is always 1. + +2. Gamm LP Shares + +Currently we use the spot price for an asset based on a designated +osmo-basepair pool of an asset. The multiplier is set once per epoch, at +the beginning of the epoch. In the future, we will switch this out to +use a TWAP instead. diff --git a/docs/overview/features/tokenfactory.md b/docs/overview/features/tokenfactory.md new file mode 100644 index 000000000..c85357300 --- /dev/null +++ b/docs/overview/features/tokenfactory.md @@ -0,0 +1,158 @@ +--- +sidebar_position: 8 +--- + +# Token Factory + +The tokenfactory module allows any account to create a new token with +the name `factory/{creator address}/{subdenom}`. Because tokens are +namespaced by creator address, this allows token minting to be +permissionless, due to not needing to resolve name collisions. A single +account can create multiple denoms, by providing a unique subdenom for each +created denom. Once a denom is created, the original creator is given +"admin" privileges over the asset. This allows them to: + +- Mint their denom to any account +- Burn their denom from any account +- Create a transfer of their denom between any two accounts +- Change the admin. In the future, more admin capabilities may be added. Admins + can choose to share admin privileges with other accounts using the authz + module. The `ChangeAdmin` functionality, allows changing the master admin + account, or even setting it to `""`, meaning no account has admin privileges + of the asset. + +## Bank hooks (`TrackBeforeSend`, `BlockBeforeSend`) +In our fork of [cosmos-sdk](https://github.com/osmosis-labs/cosmos-sdk), we have added two hooks: TrackBeforeSend and BlockBeforeSend. + +The APIs for TrackBeforeSend and BlockBeforeSend are as follows: + +```go +TrackBeforeSend(ctx sdk.Context, from, to sdk.AccAddress, amount sdk.Coins) +BlockBeforeSend(ctx sdk.Context, from, to sdk.AccAddress, amount sdk.Coins) error +``` + +Note that both hooks take the same arguments, but BlockBeforeSend returns and triggers an error, while TrackBeforeSend does not. That is, any error triggered by the BlockBeforeSend hook implementation would cancel the state transition and, consequently, the send itself, while any error omitted from TrackBeforeSend would be gracefully silenced. + +TrackBeforeSend and BlockBeforeSend are both triggered before any send action occurs, specifically before we call sendCoins, the internal API for transferring coins. + +```go +func (k BaseSendKeeper) SendCoins(ctx sdk.Context, fromAddr sdk.AccAddress, toAddr sdk.AccAddress, amt sdk.Coins) error { + // BlockBeforeSend hook should always be called before the TrackBeforeSend hook. + err := k.BlockBeforeSend(ctx, fromAddr, toAddr, amt) + if err != nil { + return err + } + + return k.sendCoins(ctx, fromAddr, toAddr, amt) +} + +``` +Note that for Module to Module send, the BlockBeforeSend hooks are not triggered, as we do not want to block module-to-module sends in any case. + +Please see [PR421](https://github.com/osmosis-labs/cosmos-sdk/pull/421) for more implementation details. + + +### Token factory integration with Bank Hooks +Due to the difference two hooks mentioned above, `TrackBeforeSend` is useful for cases when a contract needs to track specific send actions of the token factory denom, whilst `BlockBeforeSend` would be more useful for situations when we want to block specific sends using contracts. + +Each Token Factory denom allows the registration of one contract address. This contract is sudo-called every time the aforementioned bank hooks are activated. + +Contracts are able to integrate with these hooks by implementing `BlockBeforeSend` and `TrackBeforeSend` message as the following example: + +```rust +#[entry_point] +pub fn sudo(deps: DepsMut, env: Env, msg: SudoMsg) -> StdResult { + match &msg{ + SudoMsg::BlockBeforeSend { from, to, amount} => { + Ok(Response::new().add_attributes(vec![ + ("hook", "block"), + ("from", from), + ("to", to), + ("amount", &amount.to_string()) + ])) + }, + SudoMsg::TrackBeforeSend { from, to, amount} => { + Ok(Response::new().add_attributes(vec![ + ("hook", "track"), + ("from", from), + ("to", to), + ("amount", &amount.to_string()) + ])) + } + } +} +``` + + +Note that since `TrackBeforeSend` hook can also be triggered upon module to module send (which is not gas metered), we internally gas meter `TrackBeforeSend` with a gas limit of 100_000. + + +## Expectations from the chain + +The chain's bech32 prefix for addresses can be at most 16 characters long. + +This comes from denoms having a 128 byte maximum length, enforced from the SDK, +and us setting longest_subdenom to be 44 bytes. + +A token factory token's denom is: `factory/{creator address}/{subdenom}` + +Splitting up into sub-components, this has: + +- `len(factory) = 7` +- `2 * len("/") = 2` +- `len(longest_subdenom)` +- `len(creator_address) = len(bech32(longest_addr_length, chain_addr_prefix))`. + +Longest addr length at the moment is `32 bytes`. Due to SDK error correction +settings, this means `len(bech32(32, chain_addr_prefix)) = len(chain_addr_prefix) + 1 + 58`. +Adding this all, we have a total length constraint of `128 = 7 + 2 + len(longest_subdenom) + len(longest_chain_addr_prefix) + 1 + 58`. +Therefore `len(longest_subdenom) + len(longest_chain_addr_prefix) = 128 - (7 + 2 + 1 + 58) = 60`. + +The choice between how we standardized the split these 60 bytes between maxes +from longest_subdenom and longest_chain_addr_prefix is somewhat arbitrary. +Considerations going into this: + +- Per [BIP-0173](https://github.com/bitcoin/bips/blob/master/bip-0173.mediawiki#bech32) + the technically longest HRP for a 32 byte address ('data field') is 31 bytes. + (Comes from encode(data) = 59 bytes, and max length = 90 bytes) +- subdenom should be at least 32 bytes so hashes can go into it +- longer subdenoms are very helpful for creating human readable denoms +- chain addresses should prefer being smaller. The longest HRP in cosmos to date is 11 bytes. (`persistence`) + +For explicitness, its currently set to `len(longest_subdenom) = 44` and `len(longest_chain_addr_prefix) = 16`. + +Please note, if the SDK increases the maximum length of a denom from 128 bytes, +these caps should increase. + +So please don't make code rely on these max lengths for parsing. + +# Examples +To create a new token, use the create-denom command from the tokenfactory module. The following example uses the address osmo1c584m4lq25h83yp6ag8hh4htjr92d954vklzja from mylocalwallet as the default admin for the new token. + +## Creating a token +To create a new token we can use the create-denom command. + +```sh +osmosisd tx tokenfactory create-denom ufoo --keyring-backend=test --from mylocalwallet +``` + +## Mint a new token +Once a new token is created, it can be minted using the mint command in the tokenfactory module. Note that the complete tokenfactory address, in the format of factory/{creator address}/{subdenom}, must be used to mint the token. + +```sh +osmosisd tx tokenfactory mint 100000000000factory/osmo1c584m4lq25h83yp6ag8hh4htjr92d954vklzja/ufoo --keyring-backend=test --from mylocalwallet +``` + +## Checking Token metadata +To view a token's metadata, use the denom-metadata command in the bank module. The following example queries the metadata for the token factory/osmo1c584m4lq25h83yp6ag8hh4htjr92d954vklzja/ufoo: + +```sh +osmosisd query bank denom-metadata --denom factory/osmo1c584m4lq25h83yp6ag8hh4htjr92d954vklzja/ufoo +``` + +## Check the tokens created by an account +To see a list of tokens created by a specific account, use the denoms-from-creator command in the tokenfactory module. The following example shows tokens created by the account osmo1c584m4lq25h83yp6ag8hh4htjr92d954vklzja: + +```sh +osmosisd query tokenfactory denoms-from-creator osmo1c584m4lq25h83yp6ag8hh4htjr92d954vklzja +``` diff --git a/docs/overview/integrate/Concentrated Liquidity/README.md b/docs/overview/integrate/Concentrated Liquidity/README.md deleted file mode 100644 index 1dbebc962..000000000 --- a/docs/overview/integrate/Concentrated Liquidity/README.md +++ /dev/null @@ -1,275 +0,0 @@ ---- -sidebar_position: 3 ---- - -# Concentrated Liquidity - -## Index -- [Concentrated Liquidity](#concentrated-liquidity) - - [Background](#background) - - [Notable Features](#notable-features) -- [Our Implementation](#our-implementation) - - [Incentives](#Incentive-Creation-and-Querying) - - [Reward Splitting Between Classic and CL pools](#Reward-Splitting-Between-Classic-and-CL-pools) -- [TWAP Integration](#twap-integration) -- [Precision Issues With Price](#precision-issues-with-price) - - [Solution](#solution) -- [Terminology](#terminology) -- [External Sources](#external-sources) - -## Background - -Concentrated liquidity is a novel Automated Market Maker (AMM) design introduced -by Uniswap that allows for more efficient use of capital. The improvement is -achieved by providing liquidity in specific price ranges chosen by the user. - -For instance, a pool with stablecoin pairs like USDC/USDT has a spot price that -should always be trading near 1. As a result, Liquidity Providers (LPs) can -focus their capital in a small range around 1, rather than the full range from 0 -to infinity. This approach leads to an average of 200-300x higher capital -efficiency. Moreover, traders benefit from lower price impact because the pool -incentivizes greater depth around the current price. - -Concentrated liquidity also opens up new opportunities for providing liquidity -rewards to desired strategies. For example, it's possible to incentivize LPs -based on their position's proximity to the current price and the time spent -within that position. This design also allows for a new "range order" type, -similar to a limit order with order-books. - -For comprehensive technical information, we highly recommend reading the complete README of concentrated liquidity, available at https://github.com/osmosis-labs/osmosis/blob/main/x/concentrated-liquidity/README.md - - -## Our Implementation -At launch, Osmosis's CL incentives will primarily be in the format described [here](https://github.com/osmosis-labs/osmosis/blob/main/x/concentrated-liquidity/README.md) while we iron out a mechanism that achieves the remaining two properties predictably and effectively. As a piece of foreshadowing, the primary problem space we will be tackling is the following: status quo incentives advantage LPs who keep their liquidity off the books until a trade happens, ultimately pushing liquidity off of the DEX and creating ambiguity around the "real" liquidity depth. This forces traders to make uninformed decisions about where to trade their assets (or worse, accept worse execution on an inferior venue). - -In other words, instead of having incentives go towards bootstrapping healthy liquidity pools, they risk going towards adversely pushing volume to other exchanges at the cost of the DEX, active LPs, and ultimately traders. - -Note on supported and authorized uptimes -If you dig through our incentives logic, you might find code dealing with notions of Supported Uptimes and Authorized Uptimes. These are for an uptime incentivization mechanism we are keeping off at launch while we refine a more sophisticated version. We leave the state-related parts in core logic to ensure that if we do decide to turn the feature on (even if just to experiment), it could be done by a simple governance proposal (to add more supported uptimes to the list of authorized uptimes) and not require a state migration for pools. At launch, only the 1ns uptime will be authorized, which is roughly equivalent to status quo CL incentives with the small difference that positions that are created and closed in the same block are not eligible for any incentives. - -For the sake of clarity, this mechanism functions very similarly to status quo incentives, but it has a separate accumulator for each supported uptime and ensures that only liquidity that has been in the pool for the required amount of time qualifies for claiming incentives. - - -### Incentive Creation and Querying - -While it is technically possible for Osmosis to enable the creation of incentive records directly in the CL module, incentive creation is currently funneled through existing gauge infrastructure in the `x/incentives` module. This simplifies UX drastically for frontends, external incentive creators, and governance, while making CL incentives fully backwards-compatible with incentive creation and querying flows that everyone is already used to. As of the initial version of Osmosis's CL, all incentive creation and querying logic will be handled by respective gauge functions (e.g. the `IncentivizedPools` query in the `x/incentives` module will include CL pools that have internal incentives on them). - -To create a gauge dedicated to the concentrated liquidity pool, run a `MsgCreateGauge` message in the `x/incentives` module with the following parameter constraints: -- `PoolId`: The ID of the CL pool to create a gauge for. -- `DistrTo.LockQueryType` must be set to `locktypes.LockQueryType.NoLock` -- `DistrTo.Denom` must be an empty string. - -The rest of the parameters can be set according to the desired configuration of the gauge. Please read the `x/incentives` module documentation for more information on how to configure gauges. - -Note, that the created gauge will start emitting at the first epoch after the given `StartTime`. During the epoch, a `x/concentrated-liquidity` -module `IncentiveRecord` will be created for every denom in the gauge. This incentive record will be configured to emit all given incentives -over the period of an epoch. If the gauge is non-perpetual (emits over several epochs), the distribution will be split evenly between the epochs. -and a new `IncentiveRecord` will be created for each denom every epoch with the emission rate and token set to finish emitting at the end of the epoch. - -### Reward Splitting Between Classic and CL pools - -While we want to nudge Classic pool LPs to transition to CL pools, we also want to ensure that we do not have a hard cutoff for incentives where past a certain point it is no longer worth it to provide liquidity to Classic pools. This is because we want to ensure that we have a healthy transition period where liquidity is not split between Classic and CL pools, but rather that liquidity is added to CL pools while Classic pools are slowly drained of liquidity. - -To achieve this in a way that is difficult to game and efficient for the chain to process, we will be using a **reward-splitting** mechanism that treats _bonded_ liquidity in a Classic pool that is paired by governance to a CL pool (e.g. for the purpose of migration) as a single full-range position on the CL pool for the purpose of calculating incentives. Note that this _does not affect spread reward distribution_ and only applies to the flow of incentives through a CL pool. - -One implication of this mechanism is that it moves the incentivization process to a higher level of abstraction (incentivizing _pairs_ instead of _pools_). For internal incentives (which are governance managed), this is in line with the goal of continuing to push governance to require less frequent actions, which this change ultimately does. - -To keep a small but meaningful incentive for LPs to still migrate their positions, we have added a **discount rate** to incentives that are redirected to Classic pools. This is initialized to 5% by default but is a governance-upgradable parameter that can be increased in the future. A discount rate of 100% is functionally equivalent to all the incentives staying in the CL pool. - -## TWAP Integration - -In the context of twap, concentrated liquidity pools function differently from -CFMM pools. - -There are 2 major differences that stem from how the liquidity is added and -removed in concentrated-liquidity. - -The first one is given by the fact that a user does not provide liquidity at -pool creation time. Instead, they have to issue a separate message post-pool -creation. As a result, there can be a time where there is no valid spot price -initialized for a concentrated liquidity pool. When a concentrated liquidity pool -is created, the `x/twap` module still initializes the twap records. However, these -records are invalidated by setting the "last error time" field to the block time -at pool creation. Only adding liquidity to the pool will initialize the spot price -and twap records correctly. One technical detail to note is that adding liquidity -in the same block as pool creation will still set the "last error time" field to -the block time despite spot price already being initialized. Although we fix an -error within that block, it still occurs. As a result, this is deemed acceptable. -However, this is a technical trade-off for implementation simplicity and not an -intentional design decision. - -The second difference from balancer pools is focused around the fact that -liquidity can be completely removed from a concentrated liquidity pool, -making its spot price be invalid. - -To recap the basic LP functionality in concentrated liquidity, a user adds -liqudity by creating a position. To remove liquidity, they withdraw their -position. Contrary to CFMM pools, adding or removing liquidity does not affect -the price in 99% of the cases in concentrated liquidity. The only two exceptions -to this rule are: - -**Creating the first position in the pool.** - -In this case, we transition from invalid state where there is no liqudity, and -the spot price is uninitialized to the state where there is some liqudity, and -as a result a valid spot price. - -Note, that if there is a pool where liqudiity is completely drained and re-added, -the TWAP's last error time will be pointing at the time when the liquidity was drained. -This is different from how twap functions in CFMM pool where liquidity cannot -be removed in-full. - -**Removing the last position in the pool.** - -In this case, we transition from a valid state with liquidity and spot price to -an invalid state where there is no liquidity and, as a result, no valid spot -price anymore. The last spot price error will be set to the block time of when -the last position was removed. - -To reiterate, the above two exceptions are the only cases where twap is updated -due to adding or removing liquidity. - -The major source of updates with respect to twap is the swap logic. It functions -similarly to CFMM pools where upon the completion of a swap, a listener `AfterConcentratedPoolSwap` -propagates the execution to the twap module for the purposes of tracking state updates -necessary to retrieve the spot price and update the twap accumulators -(more details in x/twap module). - -Lastly, see the "Listeners" section for more details on how twap is enabled by -the use of these hooks. - -## Precision Issues With Price - -There are precision issues that we must be considerate of in our design. - -Consider the balancer pool between `arb` base unit and `uosmo`: - -```bash -osmosisd q gamm pool 1011 -pool: - '@type': /osmosis.gamm.v1beta1.Pool - address: osmo1pv6ffw8whyle2nyxhh8re44k4mu4smqd7fd66cu2y8gftw3473csxft8y5 - future_pool_governor: 24h - id: "1011" - pool_assets: - - token: - amount: "101170077995723619690981" - denom: ibc/10E5E5B06D78FFBB61FD9F89209DEE5FD4446ED0550CBB8E3747DA79E10D9DC6 - weight: "536870912000000" - - token: - amount: "218023341414" - denom: uosmo - weight: "536870912000000" - pool_params: - exit_fee: "0.000000000000000000" - smooth_weight_change_params: null - swap_fee: "0.002000000000000000" - total_shares: - amount: "18282469846754434906194" - denom: gamm/pool/1011 - total_weight: "1073741824000000" -``` - -Let's say we want to migrate this into a CL pool where `uosmo` is the quote -asset and `arb` base unit is the base asset. - -Note that quote asset is denom1 and base asset is denom0. -We want quote asset to be `uosmo` so that limit orders on ticks -have tick spacing in terms of `uosmo` as the quote. - -Note: -- OSMO has precision of 6. 1 OSMO = 10**6 `uosmo` -- ARB has precision of 18. 1 ARB = 10**18 `arb` base unit - -Therefore, the true price of the pool is: -```python ->>> (218023341414 / 10**6) / (101170077995723619690981 / 10**18) -2.1550180224553714 -``` - -However, in our core logic it is represented as: - -```python -218023341414 / 101170077995723619690981 -2.1550180224553714e-12 -``` - -or - -```python -osmosisd q gamm spot-price 1011 uosmo ibc/10E5E5B06D78FFBB61FD9F89209DEE5FD4446ED0550CBB8E3747DA79E10D9DC6 -spot_price: "0.000000000002155018" -``` - -As a protocol, we need to accomodate prices that are very far apart. -In the example above, the difference between `10**6 and 10**18` - -Most of the native precision is 10**6. However, most of the ETH -precision is 10**18. - -This starts to matter for assets such as `upepe`. That have -a precision of 18 and a very low price level relative to -the quote asset that has precision of 6 (e.g `uosmo` or `uusdc`). - -The true price of PEPE in USDC terms is `0.0000009749`. - -In the "on-chain representation", this would be: -`0.0000009749 * 10**6 / 10**18 = 9.749e-19` - -Note that this is below the minimum precision of `sdk.Dec`. - -Additionally, there is a problem with tick to sqrt price conversions -where at small price levels, two sqrt prices can map to the same -tick. - -As a workaround, we have decided to limit min spot price to 10^-12 -and min tick to `-108000000`. It has been shown at at price levels -below 10^-12, this issue is most apparent. See this issue for details: - - -Now, we have a problem that we cannot handle pairs where -the quote asset has a precision of 6 and the base asset has a -precision of 18. - -Note that this is not a problem for pairs where the quote asset -has a precision of 18 and the base asset has a precision of 6. -E.g. OSMO/DAI. - -### Solution - -At launch, pool creation is permissioned. Therefore, we can -ensure correctness for the initial set of pools. - -Long term, we will implement a wrapper contract around concentrated liquidity -that will handle the precision issues and scale the prices to all have a precision of at most 12. - -The contract will have to handle truncation and rounding to determine -how to handle dust during this process. The truncated amount can be significant. -That being said, this problem is out of scope for this document. - -## Terminology - -We will use the following terms throughout the document and our codebase: - -- `Tick` - a unit that has a 1:1 mapping with price - -- `Bucket` - an area between two initialized ticks. - -- `Tick Range` - a general term to describe a concept with lower and upper bound. - * Position is defined on a tick range. - * Bucket is defined on a tick range. - * A trader performs a swap over a tick range. - -- `Tick Spacing` - the distance between two ticks that can be initialized. This is -what defines the minimum bucket size. - -Note that ticks are defined inside buckets. Assume tick spacing is 100. A liquidity provider -creates a position with amounts such that the current tick is 155 between ticks 100 and 200. - -Note, that the current tick of 155 is defined inside the bucket over a range of 100 to 200. - - -## External Sources - -- [Uniswap V3 Whitepaper](https://uniswap.org/whitepaper-v3.pdf) -- [Technical Note on Liquidity Math](https://atiselsts.github.io/pdfs/uniswap-v3-liquidity-math.pdf) diff --git a/docs/overview/integrate/Concentrated Liquidity/on-chain.md b/docs/overview/integrate/on-chain.md similarity index 99% rename from docs/overview/integrate/Concentrated Liquidity/on-chain.md rename to docs/overview/integrate/on-chain.md index 4a6fdd1d2..7d8a3d782 100644 --- a/docs/overview/integrate/Concentrated Liquidity/on-chain.md +++ b/docs/overview/integrate/on-chain.md @@ -1,4 +1,8 @@ -# On-chain Integration +--- +sidebar_position: 3 +--- + +# Concentraed Liquidity Integration Concentrated liquidity is a novel Automated Market Maker (AMM) design introduced by Uniswap that allows for more efficient use of capital. The improvement is achieved by providing liquidity in specific price ranges chosen by the user. diff --git a/docusaurus.config.js b/docusaurus.config.js index ec9ab2a4b..503c115f7 100644 --- a/docusaurus.config.js +++ b/docusaurus.config.js @@ -160,6 +160,11 @@ const config = { to: 'overview/endpoints', position: 'left', }, + { + label: 'Features', + to: 'overview/features', + position: 'left', + }, { href: 'https://github.com/osmosis-labs', className: 'pseudo-icon github-icon',