Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

plugin API rework, plugin status support #583

Merged
merged 123 commits into from
Jan 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
123 commits
Select commit Hold shift + click to select a range
0164dd8
made runtime state fields private
milyin Oct 9, 2023
e0dbcf5
RuntimeState private
milyin Oct 10, 2023
47daf4c
removing ValidationFunction unfinished
milyin Oct 10, 2023
b920073
make validator function
milyin Oct 11, 2023
735ae2b
removed validation function
milyin Oct 11, 2023
c4e95b9
removed cyclic reference
milyin Oct 11, 2023
56f43be
format fix
milyin Oct 11, 2023
8d69198
clippy fix
milyin Oct 11, 2023
def94fb
Merge branch 'master' into runtime_blob
milyin Oct 12, 2023
27ebcd5
unfinished
milyin Oct 12, 2023
1dc7662
zenoh-plugin-trait corrected
milyin Oct 12, 2023
c059cc0
version control unfinished
milyin Oct 12, 2023
ee44f0e
loader compatibility version
milyin Oct 13, 2023
b1780e1
string version with features
milyin Oct 13, 2023
63346f0
concat_enabled_features
milyin Oct 16, 2023
a91e0cf
example storage plugin
milyin Oct 17, 2023
5fdbe66
config corrected for example storage plugin
milyin Oct 18, 2023
be46e36
example storage plugin works as memory one
milyin Oct 19, 2023
89e67dd
support load api from example stroage plugin
milyin Oct 19, 2023
84442f5
unifying plugn loading unfinished
milyin Oct 22, 2023
684c279
replaced volumes map to plugin_manager
milyin Oct 23, 2023
3f23cbc
static memory unfinihed
milyin Oct 23, 2023
41f5216
plugin manager refactor
milyin Oct 24, 2023
84498a4
plugin manager api update
milyin Oct 25, 2023
ef7bfaa
running plugin index added
milyin Oct 26, 2023
7d4856b
safer plugin load api, unfinished
milyin Oct 27, 2023
6a84494
compiles
milyin Oct 27, 2023
21f03a8
static memory plugin fix
milyin Oct 27, 2023
fafb5cd
compare version improved
milyin Oct 28, 2023
f080df6
moved feature list to crate level
milyin Oct 29, 2023
7a2ddb1
Merge branch 'master' into simplified_load_plugin
milyin Oct 30, 2023
0eaa517
linter fix
milyin Oct 30, 2023
be3382d
unfinished
milyin Oct 31, 2023
13381dd
unfinished
milyin Nov 2, 2023
2c90716
send + sync problem
milyin Nov 4, 2023
9e6a2db
unfinished
milyin Nov 6, 2023
57bec10
unfinished
milyin Nov 7, 2023
4e63949
compilation errors fixed
milyin Nov 7, 2023
2ccb9dd
compiles
milyin Nov 8, 2023
ef81a46
memory plugin loading fix
milyin Nov 8, 2023
1d136dc
startargs and instance traits
milyin Nov 8, 2023
9ee0e8b
Merge branch 'master'
milyin Nov 9, 2023
515d63e
plruings method cow
milyin Nov 10, 2023
85193fb
plugin trait sources reorganised
milyin Nov 10, 2023
9957d44
get plugin names list recursively
milyin Nov 10, 2023
7e586cb
pligins method unifinshed
milyin Nov 13, 2023
0cf6ca4
strip_prefix
milyin Nov 13, 2023
19a3c68
adminspace
milyin Nov 13, 2023
1317544
renamed loading to manager
milyin Nov 14, 2023
d850216
log prints, recurse info
milyin Nov 14, 2023
50564bb
adminspace works
milyin Nov 14, 2023
7a49a71
fixed compilation of example-plugin
milyin Nov 14, 2023
11e573c
debug error removed
milyin Nov 16, 2023
27c4841
unfinished
milyin Nov 17, 2023
52e0ea5
plugin version in compatibility api
milyin Nov 18, 2023
e1b5ed5
version in plugin status unfinished
milyin Nov 18, 2023
d1b4af8
version works
milyin Nov 19, 2023
d1e4b29
status method removed from trait
milyin Nov 19, 2023
ce3f585
removed tuple
milyin Nov 19, 2023
8ce56fc
Merge branch 'master' into simplified_load_plugin
milyin Nov 20, 2023
0d403ab
format fix
milyin Nov 20, 2023
2e5b259
commented out plugins section in config sample
milyin Nov 20, 2023
a40e7e4
Merge branch 'master' into simplified_load_plugin
milyin Nov 20, 2023
c7e51bf
doc test fixed
milyin Nov 20, 2023
785a76f
no-mangled feature added to test
milyin Nov 23, 2023
d5c95f9
test for default zenohd features
milyin Nov 27, 2023
e77acb8
tests added, leaking through zenoh-ext fixed
milyin Nov 27, 2023
d7b0ea8
no feature test added
milyin Nov 28, 2023
6a4b366
cargo fmt
milyin Nov 28, 2023
a0fd931
removed global Arc import as it's under features
milyin Nov 28, 2023
bd051fa
cargo fmt
milyin Nov 28, 2023
b358102
removed incorrect test, right one in zenohd
milyin Nov 28, 2023
1aae796
Merge branch 'master' into feature_cleanup2
milyin Nov 29, 2023
19f2a9f
Merge branch 'feature_cleanup2' into simplified_load_plugin
milyin Nov 29, 2023
eaa7b96
duplicated import fix
milyin Nov 29, 2023
3b7a188
compilation fix
milyin Nov 29, 2023
ef192c7
Merge branch 'master' into simplified_load_plugin
milyin Nov 29, 2023
8c8e502
compilation fix
milyin Nov 29, 2023
12a8187
Merge branch 'master' into simplified_load_plugin
milyin Nov 30, 2023
297b202
compilation fix
milyin Nov 30, 2023
fc53c38
compilation errors fixed
milyin Nov 30, 2023
5df923c
Merge branch 'master' into simplified_load_plugin
milyin Dec 1, 2023
4439ce3
nom_mangle in lib removed
milyin Dec 4, 2023
51d5eb3
no full failure due to one storage
milyin Dec 5, 2023
9e375e9
Merge branch 'master' into simplified_load_plugin
milyin Dec 5, 2023
c976e1c
config for testing plugins, error s logging
milyin Dec 5, 2023
03d426d
no_mangle in example
milyin Dec 5, 2023
3d7c614
compile fixes
milyin Dec 5, 2023
536d808
dds plugin tested
milyin Dec 5, 2023
296142b
cargo fmt
milyin Dec 5, 2023
0135797
Merge branch 'master' into simplified_load_plugin
milyin Dec 6, 2023
bc0d0d2
Merge branch 'master' into simplified_load_plugin
milyin Jan 8, 2024
6af0ee7
Merge branch 'simplified_load_plugin' of github.com:eclipse-zenoh/zen…
milyin Jan 8, 2024
486ed78
removed reexport of ZResult and Runtime types from plugin mod
milyin Jan 9, 2024
c960dc4
doc comments updated
milyin Jan 9, 2024
db01409
renaming, removed unused type
milyin Jan 10, 2024
7650185
constant version string
milyin Jan 10, 2024
456d48b
macro simplified
milyin Jan 10, 2024
8c88341
doc updated, version macro added
milyin Jan 10, 2024
5d1cd5d
version simplified, cargo fmt
milyin Jan 10, 2024
0db65a9
unused use removed
milyin Jan 10, 2024
39f49d8
long_version field added
milyin Jan 12, 2024
522a10d
Merge branch 'main' into simplified_load_plugin
milyin Jan 15, 2024
428bae7
default impls for RunningPluginTrait, doc comments
milyin Jan 15, 2024
0d3393e
todos added
milyin Jan 15, 2024
18e6212
cargo fmt
milyin Jan 16, 2024
8ccf6da
diagnostic improved, test config partially made, path renamings
milyin Jan 17, 2024
328371e
diagnostic improved, test config partially made, path renamings
milyin Jan 17, 2024
4e1b13a
incorrect warn replaced to debug
milyin Jan 17, 2024
912f5ae
no_mangle into plugin's code
milyin Jan 17, 2024
48060b9
all plugins covered in test config
milyin Jan 17, 2024
31cacc1
comments update
milyin Jan 18, 2024
a0d2811
Merge branch 'main' into simplified_load_plugin
milyin Jan 18, 2024
95234a4
cargo fmt
milyin Jan 18, 2024
9f36a02
restored cargo.lock
milyin Jan 18, 2024
be24a42
undo changes in DEFAULT_CONFIG
milyin Jan 18, 2024
3b10fd7
compatibility code in separate .rs, minor changes
milyin Jan 18, 2024
279b3a8
comments to declare_plugin
milyin Jan 18, 2024
0b72c53
Merge branch 'main' into simplified_load_plugin
milyin Jan 23, 2024
ff0ab93
renamings, removed unnecessary sample configs
milyin Jan 23, 2024
98277b0
cargo fmt
milyin Jan 23, 2024
581f765
debug code removed
milyin Jan 23, 2024
0cc130a
log::info changed to log::debug
milyin Jan 23, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 31 additions & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ members = [
"io/zenoh-links/zenoh-link-ws/",
"io/zenoh-links/zenoh-link-unixpipe/",
"io/zenoh-transport",
"plugins/zenoh-backend-example",
"plugins/zenoh-plugin-example",
"plugins/zenoh-backend-traits",
"plugins/zenoh-plugin-rest",
Expand Down
51 changes: 28 additions & 23 deletions commons/zenoh-config/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -26,13 +26,13 @@ use serde_json::Value;
use std::convert::TryFrom; // This is a false positive from the rust analyser
use std::{
any::Any,
collections::{HashMap, HashSet},
collections::HashSet,
fmt,
io::Read,
marker::PhantomData,
net::SocketAddr,
path::Path,
sync::{Arc, Mutex, MutexGuard},
sync::{Arc, Mutex, MutexGuard, Weak},
};
use validated_struct::ValidatedMapAssociatedTypes;
pub use validated_struct::{GetError, ValidatedMap};
Expand Down Expand Up @@ -70,15 +70,21 @@ impl Zeroize for SecretString {

pub type SecretValue = Secret<SecretString>;

pub type ValidationFunction = std::sync::Arc<
dyn Fn(
&str,
&serde_json::Map<String, serde_json::Value>,
&serde_json::Map<String, serde_json::Value>,
) -> ZResult<Option<serde_json::Map<String, serde_json::Value>>>
+ Send
+ Sync,
>;
pub trait ConfigValidator: Send + Sync {
fn check_config(
&self,
_plugin_name: &str,
_path: &str,
_current: &serde_json::Map<String, serde_json::Value>,
_new: &serde_json::Map<String, serde_json::Value>,
) -> ZResult<Option<serde_json::Map<String, serde_json::Value>>> {
Ok(None)
}
}

// Necessary to allow to set default emplty weak referece value to plugin.validator field
// because empty weak value is not allowed for Arc<dyn Trait>
impl ConfigValidator for () {}

/// Creates an empty zenoh net Session configuration.
pub fn empty() -> Config {
Expand Down Expand Up @@ -520,8 +526,8 @@ fn config_deser() {
}

impl Config {
pub fn add_plugin_validator(&mut self, name: impl Into<String>, validator: ValidationFunction) {
self.plugins.validators.insert(name.into(), validator);
pub fn set_plugin_validator<T: ConfigValidator + 'static>(&mut self, validator: Weak<T>) {
self.plugins.validator = validator;
}

pub fn plugin(&self, name: &str) -> Option<&Value> {
Expand Down Expand Up @@ -882,7 +888,7 @@ fn user_conf_validator(u: &UsrPwdConf) -> bool {
#[derive(Clone)]
pub struct PluginsConfig {
values: Value,
validators: HashMap<String, ValidationFunction>,
validator: std::sync::Weak<dyn ConfigValidator>,
}
fn sift_privates(value: &mut serde_json::Value) {
match value {
Expand Down Expand Up @@ -948,11 +954,9 @@ impl PluginsConfig {
Some(first_in_plugin) => first_in_plugin,
None => {
self.values.as_object_mut().unwrap().remove(plugin);
self.validators.remove(plugin);
return Ok(());
}
};
let validator = self.validators.get(plugin);
let (old_conf, mut new_conf) = match self.values.get_mut(plugin) {
Some(plugin) => {
let clone = plugin.clone();
Expand Down Expand Up @@ -993,8 +997,9 @@ impl PluginsConfig {
}
other => bail!("{} cannot be indexed", other),
}
let new_conf = if let Some(validator) = validator {
match validator(
let new_conf = if let Some(validator) = self.validator.upgrade() {
match validator.check_config(
plugin,
&key[("plugins/".len() + plugin.len())..],
old_conf.as_object().unwrap(),
new_conf.as_object().unwrap(),
Expand Down Expand Up @@ -1023,7 +1028,7 @@ impl Default for PluginsConfig {
fn default() -> Self {
Self {
values: Value::Object(Default::default()),
validators: Default::default(),
validator: std::sync::Weak::<()>::new(),
}
}
}
Expand All @@ -1033,8 +1038,8 @@ impl<'a> serde::Deserialize<'a> for PluginsConfig {
D: serde::Deserializer<'a>,
{
Ok(PluginsConfig {
validators: Default::default(),
values: serde::Deserialize::deserialize(deserializer)?,
validator: std::sync::Weak::<()>::new(),
})
}
}
Expand Down Expand Up @@ -1122,7 +1127,6 @@ impl validated_struct::ValidatedMap for PluginsConfig {
validated_struct::InsertionError: From<D::Error>,
{
let (plugin, key) = validated_struct::split_once(key, '/');
let validator = self.validators.get(plugin);
let new_value: Value = serde::Deserialize::deserialize(deserializer)?;
let value = self
.values
Expand All @@ -1131,8 +1135,9 @@ impl validated_struct::ValidatedMap for PluginsConfig {
.entry(plugin)
.or_insert(Value::Null);
let mut new_value = value.clone().merge(key, new_value)?;
if let Some(validator) = validator {
match validator(
if let Some(validator) = self.validator.upgrade() {
match validator.check_config(
plugin,
key,
value.as_object().unwrap(),
new_value.as_object().unwrap(),
Expand Down
45 changes: 45 additions & 0 deletions plugins/zenoh-backend-example/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
#
# Copyright (c) 2023 ZettaScale Technology
#
# This program and the accompanying materials are made available under the
# terms of the Eclipse Public License 2.0 which is available at
# http://www.eclipse.org/legal/epl-2.0, or the Apache License, Version 2.0
# which is available at https://www.apache.org/licenses/LICENSE-2.0.
#
# SPDX-License-Identifier: EPL-2.0 OR Apache-2.0
#
# Contributors:
# ZettaScale Zenoh Team, <[email protected]>
#
[package]
rust-version = { workspace = true }
name = "zenoh-backend-example"
version = { workspace = true }
authors = { workspace = true }
edition = { workspace = true }

[features]
default = ["no_mangle"]
no_mangle = []

[lib]
name = "zenoh_backend_example"
# This crate type will make `cargo` output a dynamic library instead of a rust static library
crate-type = ["cdylib"]

[dependencies]
async-std = { workspace = true, features = ["default"] }
clap = { workspace = true }
const_format = { workspace = true }
env_logger = { workspace = true }
futures = { workspace = true }
git-version = { workspace = true }
log = { workspace = true }
serde_json = { workspace = true }
zenoh = { workspace = true }
zenoh-core = { workspace = true }
zenoh-plugin-trait = { workspace = true }
zenoh-result = { workspace = true }
zenoh-util = { workspace = true }
async-trait = { workspace = true }
zenoh_backend_traits = { workspace = true }
Loading