feat: SVM priority fee oracles and transaction submitters, more consistent tx submission #4959

tkporter · 2024-12-06T01:31:54Z

Description

The fruition of https://www.notion.so/hyperlanexyz/Landing-Solana-Transactions-15a6d35200d680cbaf5eea5f9dbec1b1

Major highlights:

Instead of using the max CUs, estimate CU usage and apply a 10% buffer
Allow for priority fees to be configured. If not specified, defaults to a constant priority fee of 0. Supported options are:
- a constant fee
- the Helius priority fee API (https://docs.helius.dev/solana-apis/priority-fee-api)
Allow for transaction submission to be configured. If not specified, defaults to the configured RPC url for the chain. Supported options are:
- An arbitrary RPC URL, which will use priority fee auctions for fees
- Jito, which will use a tip for fees
In our own infra, we have logic to plumb through the Helius URL found at https://console.cloud.google.com/security/secret-manager/secret/mainnet3-rpc-endpoint-helius-solanamainnet/versions?project=abacus-labs-dev
Signs txs with a finalized blockhash, as suggested by How to Land Transactions on Solana
Partially implements process_estimate_costs to test that messages are processable. Does not yet actually enforce gas limits, because this will take some rollout effort. This is part of https://www.notion.so/hyperlanexyz/Alerting-on-critical-processing-issues-for-any-and-all-messages-15b6d35200d680a79d40fabc2e4e49da

Drive-by changes

Related issues

Backward compatibility

Testing

… trevor/helius-priority-fee-oracle

codecov · 2024-12-06T01:39:35Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 77.53%. Comparing base (fd20bb1) to head (356c107).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #4959   +/-   ##
=======================================
  Coverage   77.53%   77.53%           
=======================================
  Files         103      103           
  Lines        2110     2110           
  Branches      190      190           
=======================================
  Hits         1636     1636           
  Misses        453      453           
  Partials       21       21

Components	Coverage Δ
core	`87.80% <ø> (ø)`
hooks	`79.39% <ø> (ø)`
isms	`83.68% <ø> (ø)`
token	`91.27% <ø> (ø)`
middlewares	`79.80% <ø> (ø)`

… trevor/helius-priority-fee-oracle

changeset-bot · 2024-12-06T18:33:40Z

⚠️ No Changeset found

Latest commit: 570c52b

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes changesets to release 9 packages

Name	Type
@hyperlane-xyz/cli	Patch
@hyperlane-xyz/ccip-server	Patch
@hyperlane-xyz/github-proxy	Patch
@hyperlane-xyz/helloworld	Patch
@hyperlane-xyz/infra	Patch
@hyperlane-xyz/sdk	Patch
@hyperlane-xyz/utils	Patch
@hyperlane-xyz/widgets	Patch
@hyperlane-xyz/core	Patch

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

… trevor/helius-priority-fee-oracle

tkporter · 2024-12-13T04:41:16Z

rust/main/config/mainnet_config.json

@@ -785,7 +785,7 @@
      "index": {
        "from": 1,
        "mode": "sequence",
-        "chunk": 100
+        "chunk": 20


these are some drive-bys configs that were applied for operational reasons to the statefulsets but hadn't yet made it into the mainnet config

could you also update the metadata in registry pls so future config generation respects the values 🙏

tkporter · 2024-12-13T05:16:31Z

rust/main/chains/hyperlane-sealevel/src/rpc/client.rs

+    }
+
+    /// Gets the estimated costs for a given instruction.
+    pub async fn get_estimated_costs_for_instruction(


honestly not sure if these fns should live here, in provider.rs, or somewhere else - curious if anyone has strong takes

tkporter · 2024-12-13T05:17:09Z

rust/main/helm/hyperlane-agent/templates/relayer-statefulset.yaml

@@ -17,7 +17,9 @@ spec:
    metadata:
      annotations:
        checksum/configmap: {{ include (print $.Template.BasePath "/configmap.yaml") . | sha256sum }}
+        checksum/external-secret: {{ include (print $.Template.BasePath "/external-secret.yaml") . | sha256sum }}


drive-bys to make sure pods get updated if doing a release and there's a checksum change of the external secrets

tkporter · 2024-12-13T05:18:00Z

typescript/infra/config/environments/mainnet3/agent.ts

@@ -501,7 +550,7 @@ const releaseCandidate: RootAgentConfig = {
    rpcConsensusType: RpcConsensusType.Fallback,
    docker: {
      repo,
-      tag: '4cb2c9a-20241205-142854',
+      tag: 'b98678c-20241213-043725',


you can see this running in RC atm https://solscan.io/account/G6fWKEBSrDtR8Uzx4kdVRRhrsHZ6zmGX1G2TYMmMgK6u - though might run out of funds. I'd expect some reverts as it's competing with the hyperlane relayer

check out the fees!

tkporter · 2024-12-13T05:20:04Z

typescript/infra/src/config/agent/agent.ts

+}
+
+// An ugly way to mark a URL as a the secret Helius URL when Helm templating
+export const HELIUS_SECRET_URL_MARKER = 'helius';


intent here is to let us easily say in code "I want the helius RPC URL here" and have external secrets replace it

tkporter · 2024-12-13T05:22:17Z

rust/main/chains/hyperlane-sealevel/src/mailbox.rs


+        // TODO: not sure if this actually checks if the transaction was executed / reverted?


was poking into this and I think we may need to change this - iiuc TxOutcome.executed is supposed to be false if the tx reverted, but I think executed will be true just if the tx lands

paulbalaji · 2024-12-13T11:31:33Z

typescript/sdk/src/metadata/agentConfig.ts

+    // If the protocol type is Sealevel, require everything in AgentSealevelChainMetadataSchema
+    if (metadata.protocol === ProtocolType.Sealevel) {
+      if (!AgentSealevelChainMetadataSchema.safeParse(metadata).success) {
+        return false;
+      }
+    }
+


i can't see where these new params are in the agent config?

paulbalaji · 2024-12-13T11:31:59Z

.changeset/long-llamas-fly.md

+'@hyperlane-xyz/cli': patch
 ---

 Suppress help on CLI failures


need a new changeset for the SDK changes

ameten · 2024-12-13T11:53:34Z

rust/main/chains/hyperlane-sealevel/src/priority_fee.rs

@@ -0,0 +1,225 @@
+#![allow(dead_code)]


We can remove it if we add annotations directly to fields which are not used:

#[derive(Debug, Deserialize)] #[serde(rename_all = "camelCase")] struct JsonRpcResult<T> { #[allow(dead_code)] jsonrpc: String, #[allow(dead_code)] id: String, result: T, }

ameten · 2024-12-13T11:55:18Z

rust/main/chains/hyperlane-sealevel/src/tx_submitter.rs

@@ -0,0 +1,121 @@
+#![allow(dead_code)]


We can remove it if we remove this unused function:

pub fn new_with_rpc_client(rpc_client: SealevelRpcClient) -> Self { Self { rpc_client } }

ameten · 2024-12-13T11:57:21Z

rust/main/chains/hyperlane-sealevel/src/mailbox.rs

+        // The retuend costs are unused at the moment - we simply want to perform a simulation to
+        // determine if the message will revert or not.


Suggested change

// The retuend costs are unused at the moment - we simply want to perform a simulation to

// determine if the message will revert or not.

// The retuned costs are unused at the moment - we simply want to perform a simulation to

// determine if the message will revert or not.

jmrossy · 2024-12-13T15:44:10Z

typescript/sdk/src/metadata/agentConfig.ts

+      if (!AgentSealevelChainMetadataSchema.safeParse(metadata).success) {
+        return false;


nit: return AgentSealevelChainMetadataSchema.safeParse(metadata).success

tkporter added 7 commits December 5, 2024 17:47

everything innit

ed035c3

fee level parser

b527bb8

yeeter skeeter

0e8e2c9

Merge branch 'main' of github.com:abacus-network/abacus-monorepo into…

2c5a227

… trevor/helius-priority-fee-oracle

only sign when needed

106472f

nit

55c0b9f

update schema, nit

8deaf03

tkporter added 2 commits December 6, 2024 11:33

some tweaks

d01cbf3

Merge branch 'main' of github.com:abacus-network/abacus-monorepo into…

cb5b893

… trevor/helius-priority-fee-oracle

tkporter added 19 commits December 6, 2024 12:39

external secrinnit

7b7856c

oops

c68b314

config stuff

8e6427c

drivebys

e556629

cmon

b12bb5c

test the anti-jito hunch

2c881f7

pretty

43733b5

Merge branch 'main' of github.com:abacus-network/abacus-monorepo into…

3be422c

… trevor/helius-priority-fee-oracle

nit

9df2c05

micro lamports, oopsie

fd48da8

better

172da22

update image

2a021dc

try with jito again

ba3efa1

add tx submitter

e0d8da2

probably better config parsing

0b61be6

update agent config

74afde8

Merge branch 'main' of github.com:abacus-network/abacus-monorepo into…

80971fe

… trevor/helius-priority-fee-oracle

Merge branch 'main' of github.com:abacus-network/abacus-monorepo into…

dc1d953

… trevor/helius-priority-fee-oracle

Move to recommended, reshuffle some stuff

2365692

tkporter added 6 commits December 11, 2024 21:56

some cleanup

4124a15

Merge branch 'main' of github.com:abacus-network/abacus-monorepo into…

20221b3

… trevor/helius-priority-fee-oracle

Estimate message delivery success on SVM

1fe1914

prettier

26e1ed8

documenting

669c85f

move some fns into the RPC client

b98678c

tkporter changed the title ~~[wip] feat: Helius and constant priority fee oracles~~ feat: SVM priority fee oracles and transaction submitters, more consistent tx submission Dec 13, 2024

final touches

356c107

tkporter commented Dec 13, 2024

View reviewed changes

tkporter marked this pull request as ready for review December 13, 2024 05:23

tkporter requested review from paulbalaji, Mo-Hussain, yorhodes, jmrossy, ltyu and daniel-savu as code owners December 13, 2024 05:23

paulbalaji reviewed Dec 13, 2024

View reviewed changes

ameten reviewed Dec 13, 2024

View reviewed changes

ameten approved these changes Dec 13, 2024

View reviewed changes

jmrossy reviewed Dec 13, 2024

View reviewed changes

add recommended

570c52b

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: SVM priority fee oracles and transaction submitters, more consistent tx submission #4959

feat: SVM priority fee oracles and transaction submitters, more consistent tx submission #4959

tkporter commented Dec 6, 2024 •

edited

Loading

codecov bot commented Dec 6, 2024 •

edited

Loading

changeset-bot bot commented Dec 6, 2024 •

edited

Loading

tkporter Dec 13, 2024

paulbalaji Dec 13, 2024

tkporter Dec 13, 2024

tkporter Dec 13, 2024

tkporter Dec 13, 2024

tkporter Dec 13, 2024

tkporter Dec 13, 2024

paulbalaji Dec 13, 2024

paulbalaji Dec 13, 2024

ameten Dec 13, 2024

ameten Dec 13, 2024

ameten Dec 13, 2024

jmrossy Dec 13, 2024


		// TODO: not sure if this actually checks if the transaction was executed / reverted?

		// The retuend costs are unused at the moment - we simply want to perform a simulation to
		// determine if the message will revert or not.

		if (!AgentSealevelChainMetadataSchema.safeParse(metadata).success) {
		return false;

feat: SVM priority fee oracles and transaction submitters, more consistent tx submission #4959

Are you sure you want to change the base?

feat: SVM priority fee oracles and transaction submitters, more consistent tx submission #4959

Conversation

tkporter commented Dec 6, 2024 • edited Loading

Description

Drive-by changes

Related issues

Backward compatibility

Testing

codecov bot commented Dec 6, 2024 • edited Loading

Codecov Report

changeset-bot bot commented Dec 6, 2024 • edited Loading

⚠️ No Changeset found

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tkporter commented Dec 6, 2024 •

edited

Loading

codecov bot commented Dec 6, 2024 •

edited

Loading

changeset-bot bot commented Dec 6, 2024 •

edited

Loading