Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: start network docker container with cli #1107

Merged
merged 60 commits into from
Mar 4, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
5ab4070
Add a subcommand to network to start a network docker container
elizabethengelman Nov 27, 2023
dc68f6c
Add a network stop subcommand
elizabethengelman Nov 30, 2023
eae93e4
Refactor
elizabethengelman Dec 4, 2023
685d993
Allow for a --detached flag
elizabethengelman Dec 4, 2023
e942860
Refactor and address clippy suggestions
elizabethengelman Dec 4, 2023
b3c1392
Check in docs update
elizabethengelman Dec 4, 2023
ce941dd
Address PR feedback
elizabethengelman Dec 6, 2023
bd348bc
A few tweaks to doc comments
elizabethengelman Dec 7, 2023
e396839
Specify image by network option
elizabethengelman Jan 9, 2024
12a2261
Handle docker command flags with a slop arg
elizabethengelman Jan 9, 2024
a3da80f
Remove docker image arg and always use quickstart image
elizabethengelman Jan 9, 2024
a5d3d9d
Allow for overriding the docker image tag
elizabethengelman Jan 9, 2024
2bfad41
Update doc strings
elizabethengelman Jan 9, 2024
12b9602
Some fixes for clippy
elizabethengelman Jan 9, 2024
323c3aa
Allow for disabling soroban rpc
elizabethengelman Jan 9, 2024
383759c
Add optional args for protocol-version and limits
elizabethengelman Jan 10, 2024
53eda85
Cleanup
elizabethengelman Jan 10, 2024
b7c343f
Add generated docs
elizabethengelman Jan 12, 2024
49f8b27
Clippy
elizabethengelman Jan 12, 2024
2115fe3
Apply suggestions from code review
elizabethengelman Jan 31, 2024
cf62ae9
Make network start an async fn
elizabethengelman Jan 31, 2024
cfcfe1e
Add bollard crate
elizabethengelman Jan 31, 2024
1fa251f
Use bollard crate for docker interaction
elizabethengelman Feb 6, 2024
a3c54a7
Cleanup / refactor
elizabethengelman Feb 7, 2024
a615966
Allow for passing in a custom docker socket path
elizabethengelman Feb 7, 2024
9482311
Address clippy suggestions & other cleanup
elizabethengelman Feb 7, 2024
db30946
Use bollard for soroban network stop
elizabethengelman Feb 7, 2024
c04bc85
Remove slop arg
elizabethengelman Feb 7, 2024
9244987
Move doc comments to be displayed as help for network start
elizabethengelman Feb 7, 2024
316689d
Make network into an enum
elizabethengelman Feb 7, 2024
cbedeb1
Use network name as the container name
elizabethengelman Feb 7, 2024
d6c60f2
Factor out shared code from start and stop
elizabethengelman Feb 8, 2024
f09e103
Refine error handling
elizabethengelman Feb 9, 2024
9bd7c45
Move bollard options inline
elizabethengelman Feb 12, 2024
c68670a
Improve messaging for start and stop
elizabethengelman Feb 13, 2024
6c71396
Check in Cargo.lock
elizabethengelman Feb 13, 2024
a05bdef
Generated docs
elizabethengelman Feb 13, 2024
ed39e51
Update futurenet tag to future (from soroban-dev)
elizabethengelman Feb 13, 2024
8256f50
Merge branch 'main' into feat/start-network-with-cli
elizabethengelman Feb 21, 2024
1f6f0ae
Apply suggestions from code review on network doc strings
elizabethengelman Feb 21, 2024
2114d1d
Change --docker-socket-path to --docker-host and allow for this to be…
elizabethengelman Feb 21, 2024
2e706d8
Allow for DOCKER_HOST to be used for any connection type
elizabethengelman Feb 21, 2024
4d246d7
Update limit arg to limits for consistency
elizabethengelman Feb 21, 2024
7aec389
Remove --disable-soroban-rpc flag
elizabethengelman Feb 22, 2024
1878fe9
Merge branch 'main' into feat/start-network-with-cli
elizabethengelman Feb 22, 2024
f1ac455
Try to connect to DOCKER_HOST and if that fails try toconnect to dock…
elizabethengelman Feb 22, 2024
6637872
Update auto generated docs
elizabethengelman Feb 22, 2024
589137f
Update Cargo.lock file
elizabethengelman Feb 22, 2024
2057341
Merge branch 'main' into feat/start-network-with-cli
elizabethengelman Feb 26, 2024
8ec1c0a
Merge branch 'main' into feat/start-network-with-cli
elizabethengelman Feb 28, 2024
7a77e57
Default to connecting to docker using bollard::connect_with_socket_de…
elizabethengelman Feb 28, 2024
d7370d8
Set the default DOCKER_HOST for unix and windows
elizabethengelman Feb 28, 2024
9717201
Return a custom error from connect_to_docker
elizabethengelman Feb 28, 2024
68dbef7
Refactor retrying with docker desktop socket for unix only
elizabethengelman Feb 28, 2024
b039029
Update generated docs
elizabethengelman Feb 28, 2024
3a036a5
Updates for windows clippy checks
elizabethengelman Feb 28, 2024
9367847
Make builds and tests consistent (#1222)
leighmcculloch Feb 28, 2024
ba67e52
Merge branch 'main' into feat/start-network-with-cli
elizabethengelman Feb 28, 2024
270bce5
Merge branch 'main' into feat/start-network-with-cli
elizabethengelman Mar 4, 2024
70dccbd
Checkin updated Cargo.lock
elizabethengelman Mar 4, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
79 changes: 79 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions cmd/soroban-cli/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,9 @@ ureq = { version = "2.9.1", features = ["json"] }
tempfile = "3.8.1"
toml_edit = "0.21.0"
rust-embed = { version = "8.2.0", features = ["debug-embed"] }
bollard = "0.15.0"
futures-util = "0.3.30"
home = "0.5.9"
# For hyper-tls
[target.'cfg(unix)'.dependencies]
openssl = { version = "=0.10.55", features = ["vendored"] }
Expand Down
2 changes: 1 addition & 1 deletion cmd/soroban-cli/src/commands/config/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ impl Cmd {
pub async fn run(&self) -> Result<(), Error> {
match &self {
Cmd::Identity(identity) => identity.run().await?,
Cmd::Network(network) => network.run()?,
Cmd::Network(network) => network.run().await?,
}
Ok(())
}
Expand Down
2 changes: 1 addition & 1 deletion cmd/soroban-cli/src/commands/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ impl Root {
Cmd::Contract(contract) => contract.run(&self.global_args).await?,
Cmd::Events(events) => events.run().await?,
Cmd::Lab(lab) => lab.run().await?,
Cmd::Network(network) => network.run()?,
Cmd::Network(network) => network.run().await?,
Cmd::Version(version) => version.run(),
Cmd::Keys(id) => id.run().await?,
Cmd::Config(c) => c.run().await?,
Expand Down
24 changes: 23 additions & 1 deletion cmd/soroban-cli/src/commands/network/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,9 @@ pub const LOCAL_NETWORK_PASSPHRASE: &str = "Standalone Network ; February 2017";
pub mod add;
pub mod ls;
pub mod rm;
pub mod shared;
pub mod start;
pub mod stop;

#[derive(Debug, Parser)]
pub enum Cmd {
Expand All @@ -26,6 +29,17 @@ pub enum Cmd {
Rm(rm::Cmd),
/// List networks
Ls(ls::Cmd),
/// Start network
///
/// Start a container running a Stellar node, RPC, API, and friendbot (faucet).
///
/// soroban network start <NETWORK> [OPTIONS]
///
/// By default, when starting a testnet container, without any optional arguments, it will run the equivalent of the following docker command:
/// docker run --rm -p 8000:8000 --name stellar stellar/quickstart:testing --testnet --enable-soroban-rpc
Start(start::Cmd),
/// Stop a network started with `network start`. For example, if you ran `soroban network start local`, you can use `soroban network stop local` to stop it.
Stop(stop::Cmd),
}

#[derive(thiserror::Error, Debug)]
Expand All @@ -39,6 +53,12 @@ pub enum Error {
#[error(transparent)]
Ls(#[from] ls::Error),

#[error(transparent)]
Start(#[from] start::Error),

#[error(transparent)]
Stop(#[from] stop::Error),

#[error(transparent)]
Config(#[from] locator::Error),

Expand All @@ -61,11 +81,13 @@ pub enum Error {
}

impl Cmd {
pub fn run(&self) -> Result<(), Error> {
pub async fn run(&self) -> Result<(), Error> {
match self {
Cmd::Add(cmd) => cmd.run()?,
Cmd::Rm(new) => new.run()?,
Cmd::Ls(cmd) => cmd.run()?,
Cmd::Start(cmd) => cmd.run().await?,
Cmd::Stop(cmd) => cmd.run().await?,
};
Ok(())
}
Expand Down
150 changes: 150 additions & 0 deletions cmd/soroban-cli/src/commands/network/shared.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,150 @@
use core::fmt;

use bollard::{ClientVersion, Docker};
use clap::ValueEnum;
#[allow(unused_imports)]
// Need to add this for windows, since we are only using this crate for the unix fn try_docker_desktop_socket
use home::home_dir;

pub const DOCKER_HOST_HELP: &str = "Optional argument to override the default docker host. This is useful when you are using a non-standard docker host path for your Docker-compatible container runtime, e.g. Docker Desktop defaults to $HOME/.docker/run/docker.sock instead of /var/run/docker.sock";

// DEFAULT_DOCKER_HOST is from the bollard crate on the main branch, which has not been released yet: https://github.com/fussybeaver/bollard/blob/0972b1aac0ad5c08798e100319ddd0d2ee010365/src/docker.rs#L64
#[cfg(unix)]
pub const DEFAULT_DOCKER_HOST: &str = "unix:///var/run/docker.sock";

#[cfg(windows)]
pub const DEFAULT_DOCKER_HOST: &str = "npipe:////./pipe/docker_engine";

// DEFAULT_TIMEOUT and API_DEFAULT_VERSION are from the bollard crate
const DEFAULT_TIMEOUT: u64 = 120;
const API_DEFAULT_VERSION: &ClientVersion = &ClientVersion {
major_version: 1,
minor_version: 40,
};

#[derive(thiserror::Error, Debug)]
pub enum Error {
#[error("⛔ ️Failed to start container: {0}")]
BollardErr(#[from] bollard::errors::Error),

#[error("URI scheme is not supported: {uri}")]
UnsupportedURISchemeError { uri: String },
}

#[derive(ValueEnum, Debug, Clone, PartialEq)]
pub enum Network {
Local,
Testnet,
Futurenet,
Pubnet,
}

impl fmt::Display for Network {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
let variant_str = match self {
Network::Local => "local",
Network::Testnet => "testnet",
Network::Futurenet => "futurenet",
Network::Pubnet => "pubnet",
};

write!(f, "{variant_str}")
}
}

pub async fn connect_to_docker(docker_host: &Option<String>) -> Result<Docker, Error> {
// if no docker_host is provided, use the default docker host:
// "unix:///var/run/docker.sock" on unix machines
// "npipe:////./pipe/docker_engine" on windows machines

let host = docker_host
.clone()
.unwrap_or(DEFAULT_DOCKER_HOST.to_string());

// this is based on the `connect_with_defaults` method which has not yet been released in the bollard crate
// https://github.com/fussybeaver/bollard/blob/0972b1aac0ad5c08798e100319ddd0d2ee010365/src/docker.rs#L660
let connection = match host.clone() {
// if tcp or http, use connect_with_http_defaults
// if unix and host starts with "unix://" use connect_with_unix
// if windows and host starts with "npipe://", use connect_with_named_pipe
// else default to connect_with_unix
h if h.starts_with("tcp://") || h.starts_with("http://") => {
Docker::connect_with_http_defaults()
}
#[cfg(unix)]
h if h.starts_with("unix://") => {
Docker::connect_with_unix(&h, DEFAULT_TIMEOUT, API_DEFAULT_VERSION)
}
#[cfg(windows)]
h if h.starts_with("npipe://") => {
Docker::connect_with_named_pipe(&h, DEFAULT_TIMEOUT, API_DEFAULT_VERSION)
}
_ => {
return Err(Error::UnsupportedURISchemeError {
uri: host.to_string(),
});
}
}?;

match check_docker_connection(&connection).await {
Ok(()) => Ok(connection),
// If we aren't able to connect with the defaults, or with the provided docker_host
// try to connect with the default docker desktop socket since that is a common use case for devs
#[allow(unused_variables)]
Err(e) => {
// if on unix, try to connect to the default docker desktop socket
#[cfg(unix)]
{
let docker_desktop_connection = try_docker_desktop_socket(&host)?;
match check_docker_connection(&docker_desktop_connection).await {
Ok(()) => Ok(docker_desktop_connection),
Err(err) => Err(err)?,
}
}

#[cfg(windows)]
{
Err(e)?
}
}
}
}

#[cfg(unix)]
fn try_docker_desktop_socket(host: &str) -> Result<Docker, bollard::errors::Error> {
let default_docker_desktop_host =
format!("{}/.docker/run/docker.sock", home_dir().unwrap().display());
println!("Failed to connect to DOCKER_HOST: {host}.\nTrying to connect to the default Docker Desktop socket at {default_docker_desktop_host}.");

Docker::connect_with_unix(
&default_docker_desktop_host,
DEFAULT_TIMEOUT,
API_DEFAULT_VERSION,
)
}

// When bollard is not able to connect to the docker daemon, it returns a generic ConnectionRefused error
// This method attempts to connect to the docker daemon and returns a more specific error message
async fn check_docker_connection(docker: &Docker) -> Result<(), bollard::errors::Error> {
// This is a bit hacky, but the `client_addr` field is not directly accessible from the `Docker` struct, but we can access it from the debug string representation of the `Docker` struct
let docker_debug_string = format!("{docker:#?}");
let start_of_client_addr = docker_debug_string.find("client_addr: ").unwrap();
let end_of_client_addr = docker_debug_string[start_of_client_addr..]
.find(',')
.unwrap();
// Extract the substring containing the value of client_addr
let client_addr = &docker_debug_string
[start_of_client_addr + "client_addr: ".len()..start_of_client_addr + end_of_client_addr]
.trim()
.trim_matches('"');

match docker.version().await {
Ok(_version) => Ok(()),
Err(err) => {
println!(
"⛔️ Failed to connect to the Docker daemon at {client_addr:?}. Is the docker daemon running?\nℹ️ Running a local Stellar network requires a Docker-compatible container runtime.\nℹ️ Please note that if you are using Docker Desktop, you may need to utilize the `--docker-host` flag to pass in the location of the docker socket on your machine.\n"
);
Err(err)
}
}
}
Loading
Loading