Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add prometheus env #56

Merged
merged 3 commits into from
Apr 29, 2024
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,8 @@ services:
- hdrop
- backend
environment:
- PORT=80
- HDROP_PORT=80
- PROMETHEUS_PORT=3001
- STORAGE_PROVIDER=s3
- S3_ACCESS_KEY_ID=dev
- S3_SECRET_ACCESS_KEY=dev
Expand Down
5 changes: 3 additions & 2 deletions hdrop-server/.env.example
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
PORT=8080
HDROP_PORT=8080
PROMETHEUS_PORT=3001

S3_REGION=example-region-1
S3_ENDPOINT=https://foo.s3.example.org
Expand All @@ -7,4 +8,4 @@ S3_SECRET_ACCESS_KEY=
S3_BUCKET_NAME=hdrop
S3_PUBLIC_URL=https://storage.example.org

CORS_ORIGIN=*
CORS_ORIGIN=*
2 changes: 1 addition & 1 deletion hdrop-server/src/server/hdrop_server.rs
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,7 @@ impl Server {
);

// Server configuration
let addr = SocketAddr::from(([0; 4], env::port().unwrap_or(8080)));
let addr = SocketAddr::from(([0; 4], env::hdrop_port().unwrap_or(8080)));

tracing::info!("Starting server on {addr}");

Expand Down
4 changes: 2 additions & 2 deletions hdrop-server/src/server/prometheus_metrics_server.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
use std::{future::ready, net::SocketAddr};

use axum::{routing::get, Router};
use hdrop_shared::metrics::names;
use hdrop_shared::{env, metrics::names};
use metrics_exporter_prometheus::{Matcher, PrometheusBuilder, PrometheusHandle};
use tokio::net::TcpListener;

Expand Down Expand Up @@ -52,7 +52,7 @@ impl PrometheusMetricsServer {
/// Run the metrics server.
pub async fn run(self) {
let app = self.metrics_router();
let addr = SocketAddr::from(([0; 4], 3001));
let addr = SocketAddr::from(([0; 4], env::prometheus_port().unwrap_or(3001)));

// Bind the listener to the address
let listener = match TcpListener::bind(&addr).await {
Expand Down
59 changes: 38 additions & 21 deletions hdrop-shared/src/env.rs
Original file line number Diff line number Diff line change
Expand Up @@ -43,29 +43,46 @@ macro_rules! env_get {
};
}

// Server
env_get!(port => u16);
env_get!(cors_origin);
env_get!(single_file_limit_mb => usize);
env_get!(storage_provider);
macro_rules! env_and_const {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer a name like map_env because it describes an action whereas env_and_const doesn't, but it's really not that important. Up to you if you wanna change it or not

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is a good suggestion. I will reconsider it later today. The term "generate" or "gen" could also be an appropriate choice.

($($name:ident => $type:ty),* $(,)?) => {
const ENV_VARS_BACKEND: &[&str] = &[$(stringify!($name),)*];
$(
env_get!($name => $type);
)*
};
}

// Generate the environment variables metadata and fetch functions
env_and_const!(
// Server
hdrop_port => u16,
prometheus_port => u16,
cors_origin => String,
single_file_limit_mb => usize,
storage_provider => String,
// Database
env_get!(database_url);

database_url => String,
// Cache
env_get!(cache_strategy);
env_get!(cache_memory_limit_mb => usize);
env_get!(cache_disk_limit_mb => usize);
env_get!(cache_dir => PathBuf);

cache_strategy => String,
cache_memory_limit_mb => usize,
cache_disk_limit_mb => usize,
cache_dir => PathBuf,
// S3 Provider
env_get!(s3_region);
env_get!(s3_endpoint);
env_get!(s3_access_key_id);
env_get!(s3_secret_access_key);
env_get!(s3_bucket_name);
env_get!(s3_public_url);

s3_region => String,
s3_endpoint => String,
s3_access_key_id => String,
s3_secret_access_key => String,
s3_bucket_name => String,
s3_public_url => String,
// Local Provider
env_get!(local_storage_dir => PathBuf);
env_get!(local_storage_limit_mb => usize);
local_storage_dir => PathBuf,
local_storage_limit_mb => usize,
);

/// Get a list of all environment variables used in the hdrop backend.
pub fn get_env_vars() -> Vec<String> {
ENV_VARS_BACKEND
.iter()
.map(|&var| var.to_uppercase())
.collect()
}
Comment on lines +82 to +88
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see this being used for anything, what's the purpose?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that hdrop-shared contains all formats for the API responses and requests, as well as the names of the metrics and the environment getters, it was a logical choice to include environment variables data in this crate. The objective is to minimize the maintenance required to update the environment variables across multiple tools. A metadata function has therefore been implemented that displays all available environment variables. This allows other programs to fetch these environment variables when using hdrop-shared as a dependency, thereby establishing it as the single source of truth

3 changes: 2 additions & 1 deletion start_api.sh
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,8 @@ popd

cargo build --release -p hdrop-server
export RUST_BACKTRACE=1
export PORT=8080
export HDROP_PORT=8080
export PROMETHEUS_PORT=3001
export CORS_ORIGIN="*"
export S3_ACCESS_KEY_ID="dev"
export S3_BUCKET_NAME="hdrop"
Expand Down
Loading