Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add SessionStateBuilder::with_object_store method #12578

Merged
merged 1 commit into from
Sep 23, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions datafusion/core/src/execution/session_state.rs
Original file line number Diff line number Diff line change
Expand Up @@ -68,13 +68,15 @@ use datafusion_sql::parser::{DFParser, Statement};
use datafusion_sql::planner::{ContextProvider, ParserOptions, PlannerContext, SqlToRel};
use itertools::Itertools;
use log::{debug, info};
use object_store::ObjectStore;
use sqlparser::ast::Expr as SQLExpr;
use sqlparser::dialect::dialect_from_str;
use std::any::Any;
use std::collections::hash_map::Entry;
use std::collections::{HashMap, HashSet};
use std::fmt::Debug;
use std::sync::Arc;
use url::Url;
use uuid::Uuid;

/// `SessionState` contains all the necessary state to plan and execute queries,
Expand Down Expand Up @@ -1229,6 +1231,41 @@ impl SessionStateBuilder {
self
}

/// Register an `ObjectStore` to the [`RuntimeEnv`]. See [`RuntimeEnv::register_object_store`]
/// for more details.
///
/// Note that this creates a default [`RuntimeEnv`] if there isn't one passed in already.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

///
/// ```
/// # use datafusion::prelude::*;
/// # use datafusion::execution::session_state::SessionStateBuilder;
/// # use datafusion_execution::runtime_env::RuntimeEnv;
/// # use url::Url;
/// # use std::sync::Arc;
/// # let http_store = object_store::local::LocalFileSystem::new();
/// let url = Url::try_from("file://").unwrap();
/// let object_store = object_store::local::LocalFileSystem::new();
/// let state = SessionStateBuilder::new()
/// .with_config(SessionConfig::new())
/// .with_object_store(&url, Arc::new(object_store))
/// .with_default_features()
/// .build();
/// ```
pub fn with_object_store(
mut self,
url: &Url,
object_store: Arc<dyn ObjectStore>,
) -> Self {
if self.runtime_env.is_none() {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue #12553 proposes creating a method runtime_env to set the default and return a reference as the author does here. However, there already is a method runtime_env and its signature doesn't mesh well with that proposal. Currently I set a default runtime in with_object_store instead.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

self.runtime_env = Some(Arc::new(RuntimeEnv::default()));
}
self.runtime_env
.as_ref()
.unwrap()
.register_object_store(url, object_store);
self
}

/// Builds a [`SessionState`] with the current configuration.
///
/// Note that there is an explicit option for enabling catalog and schema defaults
Expand Down
Loading