-
Notifications
You must be signed in to change notification settings - Fork 42
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
parse table metadata.configuration as TableProperties
#453
Changes from 2 commits
ba8309e
9d4b599
1b7b193
02d50ee
e4676d6
9f8afa4
ed2c10a
42e6028
f1b9a16
82370b4
00b9d8e
f748f87
af08092
4587794
1e7d286
bd9ac7a
fa48054
ff78623
b667a15
b3cdc61
a891b52
d8a2933
d1ce73d
d8af98c
6d1b466
f18b885
437b8db
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -2,13 +2,16 @@ | |
//! us to relatively simply implement the functionality described in the protocol and expose | ||
//! 'simple' types to the user in the [`TableProperties`] struct. E.g. we can expose a `bool` | ||
//! directly instead of a `BoolConfig` type that we implement `Deserialize` for. | ||
use std::num::NonZero; | ||
use std::time::Duration; | ||
|
||
use super::*; | ||
use crate::expressions::ColumnName; | ||
use crate::table_features::ColumnMappingMode; | ||
use crate::utils::require; | ||
|
||
use tracing::warn; | ||
|
||
const SECONDS_PER_MINUTE: u64 = 60; | ||
const SECONDS_PER_HOUR: u64 = 60 * SECONDS_PER_MINUTE; | ||
const SECONDS_PER_DAY: u64 = 24 * SECONDS_PER_HOUR; | ||
|
@@ -80,10 +83,10 @@ fn try_parse(props: &mut TableProperties, k: &str, v: &str) -> Option<()> { | |
|
||
/// Deserialize a string representing a positive integer into an `Option<u64>`. Returns `Some` if | ||
/// successfully parses, and `None` otherwise. | ||
pub(crate) fn parse_positive_int(s: &str) -> Option<i64> { | ||
pub(crate) fn parse_positive_int(s: &str) -> Option<NonZero<u64>> { | ||
// parse to i64 (then check n > 0) since java doesn't even allow u64 | ||
let n: i64 = s.parse().ok()?; | ||
(n > 0).then_some(n) | ||
NonZero::new(n.try_into().ok()?) | ||
} | ||
|
||
/// Deserialize a string representing a boolean into an `Option<bool>`. Returns `Some` if | ||
|
@@ -99,7 +102,9 @@ pub(crate) fn parse_bool(s: &str) -> Option<bool> { | |
/// Deserialize a comma-separated list of column names into an `Option<Vec<ColumnName>>`. Returns | ||
/// `Some` if successfully parses, and `None` otherwise. | ||
pub(crate) fn parse_column_names(s: &str) -> Option<Vec<ColumnName>> { | ||
ColumnName::parse_column_name_list(s).ok() | ||
ColumnName::parse_column_name_list(s) | ||
.map_err(|e| warn!("column name list failed to parse: {e}")) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This looks like a job for inspect_err? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ah yep good catch thanks - I'll just squeeze that into the next small PR |
||
.ok() | ||
} | ||
|
||
/// Deserialize an interval string of the form "interval 5 days" into an `Option<Duration>`. | ||
|
@@ -185,6 +190,33 @@ fn parse_interval_impl(value: &str) -> Result<Duration, ParseIntervalError> { | |
#[cfg(test)] | ||
mod tests { | ||
use super::*; | ||
use crate::expressions::column_name; | ||
|
||
#[test] | ||
fn test_parse_column_names() { | ||
assert_eq!( | ||
parse_column_names("`col 1`, col.a2,col3").unwrap(), | ||
vec![ | ||
ColumnName::new(["col 1"]), | ||
column_name!("col.a2"), | ||
column_name!("col3") | ||
] | ||
); | ||
} | ||
Comment on lines
+196
to
+205
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. hm yea maybe that was unnecessary - i'll remove since this is basically jsut a pass-through to the column name parsing |
||
|
||
#[test] | ||
fn test_parse_bool() { | ||
assert_eq!(parse_bool("true").unwrap(), true); | ||
assert_eq!(parse_bool("false").unwrap(), false); | ||
assert_eq!(parse_bool("whatever"), None); | ||
} | ||
|
||
#[test] | ||
fn test_parse_positive_int() { | ||
assert_eq!(parse_positive_int("123").unwrap().get(), 123); | ||
assert_eq!(parse_positive_int("0"), None); | ||
assert_eq!(parse_positive_int("-123"), None); | ||
} | ||
|
||
#[test] | ||
fn test_parse_interval() { | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do these custom deserializers have a fallback for cases where the table property exists but cannot parse to the expected type? Ideally, those should land in the
unknown_properties
map along with everything else, with the parsed version beingNone
.Needed because many table properties are not necessarily in the Delta spec, and so different clients may use them differently?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
oh I currently implemented that as failure: see
fail_known_keys
test. You're saying the behavior should instead be something like this?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, exactly. Otherwise I fear we'll eventually add yet another parsing failure ticket to the pile, along with all the CommitInfo issues.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ah yea fair point.. let me see how to do this :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
welp this was the straw that broke
serde
's back.. instead of implementing a customDeserialize
I just decided to do the naive thing and implement a big match by hand. Actually isn't that gross and we can always tryserde
again or just write our own macro to make everything nice