Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/675 on chain ouis devaddrs #431

Open
wants to merge 51 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 36 commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
6060787
Move SubDao::dc_key() -> Dao::dc_key()
michaeldjeffrey Aug 1, 2024
8cae4db
Add `dc::burn_delegated` function
michaeldjeffrey Aug 1, 2024
58cf64b
add helper for going from solana keypair to wrapped helium-lib keypair
michaeldjeffrey Aug 2, 2024
d2bf60c
Add boosting module, function for starting hex boost
michaeldjeffrey Aug 2, 2024
b1734cc
merge imports
michaeldjeffrey Oct 22, 2024
c755093
add Io error to encode error when attempting to write a keypair
michaeldjeffrey Oct 22, 2024
bd87fdd
Move ReadWrite trait to helium-lib
michaeldjeffrey Oct 22, 2024
db7f998
remove after bad merge
michaeldjeffrey Oct 22, 2024
2031f39
update to use new dc_key location
michaeldjeffrey Oct 22, 2024
b2f4780
remove useless ? and Ok
michaeldjeffrey Oct 22, 2024
7a67eed
move ReadWrite back to wallet. avoid file io in helium-lib
michaeldjeffrey Oct 22, 2024
e827eeb
upgrade spl-account-compression to version compatible with anchor-lan…
michaeldjeffrey Oct 23, 2024
dc47049
move back to anchor-lang 0.29
michaeldjeffrey Oct 23, 2024
040fc93
update helium-anchor-gen with looser dep around anchor-client
michaeldjeffrey Oct 23, 2024
0e750c9
go back to master helium-anchor-gen that has updated anchor-lang >=0.28
michaeldjeffrey Oct 23, 2024
9ee2bb8
wip moving iot_routing_manager sdk here
bryzettler Oct 30, 2024
3b38d76
Pdas and instructions creators for iot_routing_manager plus a refactor
bryzettler Nov 1, 2024
74c9d34
added update org
bryzettler Nov 4, 2024
381423a
move config key to hem
bryzettler Nov 4, 2024
1e6ee06
Us as_ref for solana client
bryzettler Nov 4, 2024
3f245d4
added send instructions and restructured sol transaction utils
bryzettler Nov 5, 2024
3b7f815
structure client better
bryzettler Nov 6, 2024
1ea9ba3
add wrapper fn for getting the SolanaRpcClient
bryzettler Nov 6, 2024
02d5791
.
bryzettler Nov 6, 2024
3abf8a9
added wallet/payer logic to solana_client
bryzettler Nov 6, 2024
e14d61a
Pass tpu helium-lib
bryzettler Nov 6, 2024
174b779
add AsRef for SolanaClient
bryzettler Nov 7, 2024
3d10078
Building
bryzettler Nov 7, 2024
3b2249d
bump bubblegum
bryzettler Nov 7, 2024
0a36e98
wip
bryzettler Nov 11, 2024
b5ababb
Fix logic for existing account
bryzettler Nov 20, 2024
7d558b4
Merge branch 'master' into feat/675-on-chain-ouis-devaddrs
bryzettler Dec 5, 2024
0623a99
Fix errors after merging master
bryzettler Dec 5, 2024
c8a587f
testing
bryzettler Dec 6, 2024
d4ef2c9
remove cargo patches
bryzettler Dec 6, 2024
ca0308c
self review
bryzettler Dec 6, 2024
c9431ea
remove unused priority_fee
bryzettler Dec 10, 2024
e79b802
pr feedback
bryzettler Dec 10, 2024
7557c35
fold helium_sub_daos back into dao
bryzettler Dec 18, 2024
9c907a8
fold helium_entity_manger back into dao/asset
bryzettler Dec 18, 2024
4642b8f
Use dao/asset in iot
bryzettler Dec 18, 2024
8fde91e
update refrences of helium_entity_manger
bryzettler Dec 19, 2024
fbdf4d3
PR feedback
bryzettler Dec 19, 2024
25147f3
fix
bryzettler Dec 20, 2024
d46f7e9
Merge branch 'master' into feat/675-on-chain-ouis-devaddrs
bryzettler Dec 20, 2024
2527773
fold metaplex into asset
bryzettler Jan 14, 2025
0dd0e45
pr feedback
bryzettler Jan 14, 2025
16239b5
remove refrences to tpu
bryzettler Jan 15, 2025
fb7c2c4
pr feedback tweaks
bryzettler Jan 15, 2025
4d8ea0f
fix compute budget estimate
bryzettler Jan 17, 2025
a62acdb
unpatch crates
bryzettler Jan 17, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -23,3 +23,4 @@ helium-proto = { git = "https://github.com/helium/proto", branch = "master", fea
"services",
] }
clap = { version = "4", features = ["derive"] }
tokio = { version = "1.0", features = ["full"] }
bryzettler marked this conversation as resolved.
Show resolved Hide resolved
2 changes: 1 addition & 1 deletion helium-lib/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -48,4 +48,4 @@ clap = {workspace = true, optional = true}
helium-mnemonic = { path = "../helium-mnemonic", optional = true }

[dev-dependencies]
rand = "0.8"
rand = "0.8"
265 changes: 217 additions & 48 deletions helium-lib/src/client.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,32 @@ use crate::{
error::{DecodeError, Error},
is_zero,
keypair::{self, Pubkey},
solana_client,
solana_client::{
self,
nonblocking::tpu_client::TpuClient,
send_and_confirm_transactions_in_parallel::{
send_and_confirm_transactions_in_parallel, SendAndConfirmConfig,
},
tpu_client::TpuClientConfig,
},
solana_sdk::{
commitment_config::CommitmentConfig, instruction::Instruction, message::Message,
signature::Keypair, signer::Signer,
},
solana_transaction_utils::{
pack::pack_instructions_into_transactions, priority_fee::auto_compute_limit_and_price,
},
};

use futures::{stream, StreamExt, TryStreamExt};
use itertools::Itertools;
use jsonrpc_client::{JsonRpcError, SendRequest};
use std::{marker::Send, sync::Arc};
use solana_sdk::signer::EncodableKey;
use std::{marker::Send, path::PathBuf, sync::Arc};
use tracing::instrument;

pub use solana_client::nonblocking::rpc_client::RpcClient as SolanaRpcClient;

pub static ONBOARDING_URL_MAINNET: &str = "https://onboarding.dewi.org/api/v3";
pub static ONBOARDING_URL_DEVNET: &str = "https://onboarding.web.test-helium.com/api/v3";

Expand All @@ -23,17 +41,200 @@ pub static SOLANA_URL_DEVNET: &str = "https://solana-rpc.web.test-helium.com?ses
pub static SOLANA_URL_MAINNET_ENV: &str = "SOLANA_MAINNET_URL";
pub static SOLANA_URL_DEVNET_ENV: &str = "SOLANA_DEVNET_URL";

pub use solana_client::nonblocking::rpc_client::RpcClient as SolanaRpcClient;
static USER_AGENT: &str = concat!(env!("CARGO_PKG_NAME"), "/", env!("CARGO_PKG_VERSION"),);

#[derive(Clone)]
pub struct SolanaClient {
pub inner: Arc<SolanaRpcClient>,
pub base_url: String,
pub wallet: Option<Arc<Keypair>>,
bryzettler marked this conversation as resolved.
Show resolved Hide resolved
}

impl Default for SolanaClient {
fn default() -> Self {
Self::new(SOLANA_URL_MAINNET, None).unwrap()
}
}

impl SolanaClient {
pub fn new(url: &str, wallet_path: Option<PathBuf>) -> Result<Self, Error> {
let client = Arc::new(
solana_client::nonblocking::rpc_client::RpcClient::new_with_commitment(
url.to_string(),
CommitmentConfig::finalized(),
),
);

let wallet = if let Some(path) = wallet_path {
Some(Arc::new(
Keypair::read_from_file(&path).map_err(Error::from)?,
bryzettler marked this conversation as resolved.
Show resolved Hide resolved
))
} else {
None
};

Ok(Self {
inner: client,
base_url: url.to_string(),
wallet,
})
}

pub fn solana_rpc_client(&self) -> Arc<SolanaRpcClient> {
self.inner.clone()
bryzettler marked this conversation as resolved.
Show resolved Hide resolved
}

pub fn ws_url(&self) -> String {
self.base_url
.replace("https", "wss")
.replace("http", "ws")
.replace("127.0.0.1:8899", "127.0.0.1:8900")
}

pub fn wallet(&self) -> Result<Pubkey, Error> {
self.wallet
bryzettler marked this conversation as resolved.
Show resolved Hide resolved
.as_ref()
.map(|wallet| wallet.pubkey())
.ok_or_else(|| Error::Other("Wallet not configured".to_string()))
}

pub async fn send_instructions(
bryzettler marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curiously, where is this function actuallly used?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

used in multiple places on this pr helium/helium-config-service-cli#67

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

right which really doesn't need to use instructions but can use the transaction form constructor functions, since they're all just passing in a single or fixed number of ixs

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Im not following? Is it bad to just pass the instructions to send_instructions and have it do the transaction forming and sending? You're saying I should just call pack_instructions in the config-service followed by a subsequent call of sending those transactions?

&self,
ixs: Vec<Instruction>,
extra_signers: &[Keypair],
sequentially: bool,
) -> Result<(), Error> {
let wallet = self
.wallet
.as_ref()
.ok_or_else(|| Error::Other("Wallet not configured".to_string()))?;

let (blockhash, _) = self
.inner
.as_ref()
.get_latest_blockhash_with_commitment(CommitmentConfig::finalized())
.await
.expect("Failed to get latest blockhash");

let txs = pack_instructions_into_transactions(vec![ixs], &wallet);
let mut with_auto_compute: Vec<Message> = Vec::new();
let keys: Vec<&dyn Signer> = std::iter::once(&wallet as &dyn Signer)
.chain(extra_signers.iter().map(|k| k as &dyn Signer))
.collect();

for (tx, _) in &txs {
// This is just a tx with compute ixs. Skip it
if tx.len() == 2 {
continue;
}

let computed = auto_compute_limit_and_price(
&self.solana_rpc_client(),
tx.clone(),
&keys,
1.2,
Some(&wallet.pubkey()),
Some(blockhash),
)
.await
.unwrap();

with_auto_compute.push(Message::new(&computed, Some(&wallet.pubkey())));
}

if with_auto_compute.is_empty() {
return Ok(());
}

let results;
let tpu_client = TpuClient::new(
"helium-lib",
self.solana_rpc_client(),
&self.ws_url(),
TpuClientConfig::default(),
)
.await?;

match sequentially {
true => {
results = tpu_client
.send_and_confirm_messages_with_spinner(&with_auto_compute, &keys)
.await?;
}
false => {
results = send_and_confirm_transactions_in_parallel(
self.solana_rpc_client(),
Some(tpu_client),
&with_auto_compute,
&keys,
SendAndConfirmConfig {
with_spinner: true,
resign_txs_count: Some(5),
},
)
.await?;
}
}

if let Some(err) = results.into_iter().flatten().next() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is just called into_iter().try_collect()?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

getting unstable library feature 'iterator_try_collect'

return Err(Error::from(err));
}

Ok(())
}
}

#[derive(Clone)]
pub struct Client {
pub solana_client: Arc<SolanaRpcClient>,
pub solana_client: Arc<SolanaClient>,
pub das_client: Arc<DasClient>,
}

impl TryFrom<&str> for Client {
type Error = Error;
fn try_from(value: &str) -> Result<Self, Self::Error> {
fn env_or(key: &str, default: &str) -> String {
std::env::var(key).unwrap_or_else(|_| default.to_string())
}
let url = match value {
"m" | "mainnet-beta" => &env_or(SOLANA_URL_MAINNET_ENV, SOLANA_URL_MAINNET),
"d" | "devnet" => &env_or(SOLANA_URL_DEVNET_ENV, SOLANA_URL_DEVNET),
url => url,
};
let das_client = Arc::new(DasClient::with_base_url(url)?);
let solana_client = Arc::new(SolanaClient::new(url, None)?);

Ok(Self {
solana_client,
das_client,
})
}
}

impl AsRef<SolanaRpcClient> for SolanaClient {
fn as_ref(&self) -> &SolanaRpcClient {
&self.inner
}
}

impl AsRef<SolanaRpcClient> for Client {
fn as_ref(&self) -> &SolanaRpcClient {
&self.solana_client.inner
}
}

impl AsRef<DasClient> for Client {
fn as_ref(&self) -> &DasClient {
&self.das_client
}
}

#[async_trait::async_trait]
pub trait GetAnchorAccount {
async fn anchor_account<T: AccountDeserialize>(&self, pubkey: &Pubkey) -> Result<T, Error>;
async fn anchor_account<T: AccountDeserialize>(
&self,
pubkey: &Pubkey,
) -> Result<Option<T>, Error>;
async fn anchor_accounts<T: AccountDeserialize + Send>(
&self,
pubkeys: &[Pubkey],
Expand All @@ -42,18 +243,21 @@ pub trait GetAnchorAccount {

#[async_trait::async_trait]
impl GetAnchorAccount for SolanaRpcClient {
async fn anchor_account<T: AccountDeserialize>(&self, pubkey: &Pubkey) -> Result<T, Error> {
async fn anchor_account<T: AccountDeserialize>(
&self,
pubkey: &Pubkey,
) -> Result<Option<T>, Error> {
let account = self.get_account(pubkey).await?;
let decoded = T::try_deserialize(&mut account.data.as_ref())?;
Ok(decoded)
Ok(Some(decoded))
}

async fn anchor_accounts<T: AccountDeserialize + Send>(
&self,
pubkeys: &[Pubkey],
) -> Result<Vec<Option<T>>, Error> {
async fn get_accounts<A: AccountDeserialize + Send>(
client: &SolanaRpcClient,
client: &solana_client::nonblocking::rpc_client::RpcClient,
pubkeys: &[Pubkey],
) -> Result<Vec<Option<A>>, Error> {
let accounts = client.get_multiple_accounts(pubkeys).await?;
Expand Down Expand Up @@ -86,46 +290,14 @@ impl GetAnchorAccount for Client {
async fn anchor_account<T: AccountDeserialize>(
&self,
pubkey: &keypair::Pubkey,
) -> Result<T, Error> {
self.solana_client.anchor_account(pubkey).await
) -> Result<Option<T>, Error> {
self.solana_client.inner.anchor_account(pubkey).await
}
async fn anchor_accounts<T: AccountDeserialize + Send>(
&self,
pubkeys: &[Pubkey],
) -> Result<Vec<Option<T>>, Error> {
self.solana_client.anchor_accounts(pubkeys).await
}
}

impl TryFrom<&str> for Client {
type Error = Error;
fn try_from(value: &str) -> Result<Self, Self::Error> {
fn env_or(key: &str, default: &str) -> String {
std::env::var(key).unwrap_or_else(|_| default.to_string())
}
let url = match value {
"m" | "mainnet-beta" => &env_or(SOLANA_URL_MAINNET_ENV, SOLANA_URL_MAINNET),
"d" | "devnet" => &env_or(SOLANA_URL_DEVNET_ENV, SOLANA_URL_DEVNET),
url => url,
};
let das_client = Arc::new(DasClient::with_base_url(url)?);
let solana_client = Arc::new(SolanaRpcClient::new(url.to_string()));
Ok(Self {
solana_client,
das_client,
})
}
}

impl AsRef<SolanaRpcClient> for Client {
fn as_ref(&self) -> &SolanaRpcClient {
&self.solana_client
}
}

impl AsRef<DasClient> for Client {
fn as_ref(&self) -> &DasClient {
&self.das_client
self.solana_client.inner.anchor_accounts(pubkeys).await
}
}

Expand Down Expand Up @@ -199,8 +371,6 @@ impl DasClientError {
#[jsonrpc_client::api]
pub trait DAS {}

static USER_AGENT: &str = concat!(env!("CARGO_PKG_NAME"), "/", env!("CARGO_PKG_VERSION"),);

#[jsonrpc_client::implement(DAS)]
#[derive(Debug, Clone)]
pub struct DasClient {
Expand All @@ -210,7 +380,6 @@ pub struct DasClient {

impl Default for DasClient {
fn default() -> Self {
// safe to unwrap
bryzettler marked this conversation as resolved.
Show resolved Hide resolved
Self::with_base_url(SOLANA_URL_MAINNET).unwrap()
}
}
Expand Down Expand Up @@ -369,12 +538,12 @@ pub mod config {
}

#[derive(Clone)]
pub enum Client {
pub enum ClientType {
bryzettler marked this conversation as resolved.
Show resolved Hide resolved
Iot(iot::Client),
Mobile(mobile::Client),
}

impl Client {
impl ClientType {
pub fn for_subdao(
subdao: SubDao,
config: &str,
Expand Down
Loading
Loading