Skip to content

Commit

Permalink
README and comment
Browse files Browse the repository at this point in the history
  • Loading branch information
matthewmturner committed Oct 16, 2024
1 parent 23e3f83 commit bd63680
Show file tree
Hide file tree
Showing 3 changed files with 18 additions and 3 deletions.
3 changes: 2 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ crossterm = { version = "0.28.1", features = ["event-stream"] }
datafusion = "41.0.0"
datafusion-common = "41.0.0"
datafusion-functions-json = { version = "0.41.0", optional = true }
datafusion-functions-parquet = { version = "0.1.0", path = "crates/datafusion-functions-parquet" }
datafusion-functions-parquet = { version = "0.1.0", path = "crates/datafusion-functions-parquet", optional = true }
deltalake = { version = "0.19.0", features = ["datafusion"], optional = true }
directories = "5.0.1"
env_logger = "0.11.5"
Expand Down Expand Up @@ -62,6 +62,7 @@ flightsql = ["dep:arrow-flight", "dep:tonic"]
experimental-flightsql-server = ["dep:arrow-flight", "dep:tonic"]
s3 = ["object_store/aws", "url"]
functions-json = ["dep:datafusion-functions-json"]
functions-parquet = ["dep:datafusion-functions-parquet"]

[[bin]]
name = "dft"
Expand Down
16 changes: 14 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,9 +116,21 @@ Currently, the only supported packaging is on [crates.io](https://crates.io/sear

Once installed you can run `dft` to start the application.

#### Optional Features (Rust Crate Features)
#### Internal Optional Features (Workspace Features)

`dft` has several optional (conditionally compiled features) integrations which are controlled by [Rust Crate Features]
`dft` incubates several optional features in it's `crates` directory. This provides us with the ability to quickly iterate on new features and test them in the main application while at the same time making it easy to export them to their own crates when they are ready.

#### Parquet Functions (`--features=functions-parquet`)

Includes functions from [datafusion-function-parquet] for querying Parquet files in DataFusion in `dft`. For example:

```sql
SELECT * FROM parquet_metadata('my_parquet_file.parquet')
```

#### External Optional Features (Rust Crate Features)

`dft` also has several external optional (conditionally compiled features) integrations which are controlled by [Rust Crate Features]

To build with all features, you can run

Expand Down
2 changes: 2 additions & 0 deletions crates/datafusion-functions-parquet/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,8 @@ use parquet::file::statistics::Statistics;
use std::fs::File;
use std::sync::Arc;

// Copied from https://github.com/apache/datafusion/blob/main/datafusion-cli/src/functions.rs

/// PARQUET_META table function
struct ParquetMetadataTable {
schema: SchemaRef,
Expand Down

0 comments on commit bd63680

Please sign in to comment.