Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NDEV-3112. Optimize getting of deactivated features #566

Open
wants to merge 4 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions evm_loader/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions evm_loader/lib/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ lazy_static = "1.5.0"
elsa = "1.10.0"
arrayref = "0.3.8"
futures = "0.3.30"
once_cell = "1.19.0"

[dev-dependencies]
hex-literal = "0.4.1"
Expand Down
3 changes: 2 additions & 1 deletion evm_loader/lib/src/rpc/db_call_client.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
use super::{e, Rpc, SliceConfig};
use crate::types::deactivated_features::get_deactivated_features;
use crate::types::{TracerDb, TracerDbTrait};
use crate::NeonError;
use crate::NeonError::RocksDb;
Expand Down Expand Up @@ -88,6 +89,6 @@ impl Rpc for CallDbClient {
}

async fn get_deactivated_solana_features(&self) -> ClientResult<Vec<Pubkey>> {
Ok(vec![]) // TODO
get_deactivated_features(self, Some(self.slot)).await

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You have to use CloneRpcClient to get_deactivated_features or the logic of it will be broken.

}
}
50 changes: 2 additions & 48 deletions evm_loader/lib/src/rpc/validator_client.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
use crate::{config::APIOptions, Config};
use crate::{config::APIOptions, types::deactivated_features::get_deactivated_features, Config};

use super::{Rpc, SliceConfig};
use async_trait::async_trait;
Expand Down Expand Up @@ -179,52 +179,6 @@ impl Rpc for CloneRpcClient {
}

async fn get_deactivated_solana_features(&self) -> ClientResult<Vec<Pubkey>> {
use std::time::{Duration, Instant};
use tokio::sync::Mutex;

struct Cache {
data: Vec<Pubkey>,
timestamp: Instant,
}

static CACHE: Mutex<Option<Cache>> = Mutex::const_new(None);
let mut cache = CACHE.lock().await;

if let Some(cache) = cache.as_ref() {
if cache.timestamp.elapsed() < Duration::from_secs(24 * 60 * 60) {
return Ok(cache.data.clone());
}
}

let feature_keys: Vec<Pubkey> = solana_sdk::feature_set::FEATURE_NAMES
.keys()
.copied()
.collect();

let features = Rpc::get_multiple_accounts(self, &feature_keys).await?;

let mut result = Vec::with_capacity(feature_keys.len());
for (pubkey, feature) in feature_keys.iter().zip(features) {
let is_activated = feature
.and_then(|a| solana_sdk::feature::from_account(&a))
.and_then(|f| f.activated_at)
.is_some();

if !is_activated {
result.push(*pubkey);
}
}

for feature in &result {
debug!("Deactivated feature: {}", feature);
}

cache.replace(Cache {
data: result.clone(),
timestamp: Instant::now(),
});
drop(cache);

Ok(result)
get_deactivated_features(self, None).await
}
}
82 changes: 82 additions & 0 deletions evm_loader/lib/src/types/deactivated_features.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
// use crate::tracing::tracers::state_diff::Account;
use crate::rpc::Rpc;
use once_cell::sync::Lazy;
use solana_client::client_error::Result as ClientResult;
use solana_sdk::pubkey::Pubkey;
use std::collections::HashMap;
use std::sync::Arc;
use tokio::sync::Mutex;

static DEACTIVATED_FEATURES_UPDATE_SLOTS: u64 = 24 * 60 * 60; // approximately each 12h

#[derive(Debug, Eq, PartialEq, Clone, Default)]
struct DeactivatedFeaturesCache {
// feature to slot when activated, if not then None
pub data: HashMap<Pubkey, Option<u64>>,
pub last_slot: u64,
}

impl DeactivatedFeaturesCache {
fn should_update(&self, slot: u64) -> bool {
self.last_slot >= DEACTIVATED_FEATURES_UPDATE_SLOTS + slot
}
}

static DEACTIVATED_FEATURES: Lazy<Arc<Mutex<DeactivatedFeaturesCache>>> =

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why don't you use RwLock instead of Mutex?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because we cannot call await of async function under lock of RwLock. In our case we need to call async Rpc::get_multiple_accounts

Lazy::new(|| Arc::new(Mutex::new(DeactivatedFeaturesCache::default())));

async fn get_multiple_features(rpc: &dyn Rpc) -> ClientResult<HashMap<Pubkey, Option<u64>>> {
let feature_keys: Vec<Pubkey> = solana_sdk::feature_set::FEATURE_NAMES
.keys()
.copied()
.collect();

let features = Rpc::get_multiple_accounts(rpc, &feature_keys).await?;

let mut result = HashMap::<Pubkey, Option<u64>>::new();
for (pubkey, feature) in feature_keys.iter().zip(features) {
let slot = feature
.and_then(|a| solana_sdk::feature::from_account(&a))
.and_then(|f| f.activated_at);

result.insert(*pubkey, slot);
}

Ok(result)
}

async fn deactivated_features_get_instance(
rpc: &dyn Rpc,
) -> ClientResult<DeactivatedFeaturesCache> {
let mut cache = DEACTIVATED_FEATURES.lock().await;

let rpc_slot = rpc.get_slot().await?;
if cache.should_update(rpc_slot) {
cache.last_slot = rpc_slot;
cache.data = get_multiple_features(rpc).await?;
}

Ok((*cache).clone())
}

pub async fn get_deactivated_features(
rpc: &dyn Rpc,
slot: Option<u64>,
) -> ClientResult<Vec<Pubkey>> {
let features = deactivated_features_get_instance(rpc).await?;

Ok(features
.data
.into_iter()
.filter_map(|(pubkey, activated_at)| match (slot, activated_at) {
(_, None) => Some(pubkey),
(Some(slot), Some(activated_at)) => {
if activated_at > slot {
return Some(pubkey);
}
None
}
(None, Some(_)) => None,
})
.collect())
}
4 changes: 2 additions & 2 deletions evm_loader/lib/src/types/mod.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
pub mod tracer_ch_common;

pub mod deactivated_features;
pub mod programs_cache;
pub mod tracer_ch_common;
pub(crate) mod tracer_ch_db;
pub mod tracer_rocks_db;

Expand Down
Loading