Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update sl-jdbc.md #3953

Merged
merged 26 commits into from
Sep 19, 2023
Merged
Changes from 3 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
8176b66
Update sl-jdbc.md
rpourzand Aug 18, 2023
678daaf
Update website/docs/docs/dbt-cloud-apis/sl-jdbc.md
mirnawong1 Aug 23, 2023
9c850c7
Update website/docs/docs/dbt-cloud-apis/sl-jdbc.md
mirnawong1 Aug 23, 2023
8f458f6
Update website/docs/docs/dbt-cloud-apis/sl-jdbc.md
mirnawong1 Aug 23, 2023
59a4c76
Merge branch 'current' into rpourzand-jdbc-updates-where-filters
mirnawong1 Aug 23, 2023
fd53519
Merge branch 'current' into rpourzand-jdbc-updates-where-filters
mirnawong1 Aug 29, 2023
832a64a
Update sl-jdbc.md
rpourzand Aug 29, 2023
78f4276
Update sl-jdbc.md
rpourzand Aug 29, 2023
d7525ac
Merge branch 'current' into rpourzand-jdbc-updates-where-filters
mirnawong1 Aug 31, 2023
86f9593
Create predicate-pushdown.md
mirnawong1 Aug 31, 2023
33800bb
Update predicate-pushdown.md
mirnawong1 Aug 31, 2023
5d68215
clarify predicate pushdown and add term
mirnawong1 Aug 31, 2023
2bdfdd1
Merge branch 'current' into rpourzand-jdbc-updates-where-filters
mirnawong1 Sep 5, 2023
6f5e46e
Merge branch 'current' into rpourzand-jdbc-updates-where-filters
marcodamore Sep 5, 2023
97bd5a1
Update website/docs/docs/dbt-cloud-apis/sl-jdbc.md
mirnawong1 Sep 5, 2023
ae83b1f
Merge branch 'current' into rpourzand-jdbc-updates-where-filters
mirnawong1 Sep 7, 2023
826b267
Merge branch 'current' into rpourzand-jdbc-updates-where-filters
mirnawong1 Sep 11, 2023
968be37
Merge branch 'current' into rpourzand-jdbc-updates-where-filters
mirnawong1 Sep 18, 2023
e53523c
Update sl-jdbc.md
rpourzand Sep 18, 2023
86ac6dd
Update sl-jdbc.md
rpourzand Sep 18, 2023
de2e325
Update sl-jdbc.md
mirnawong1 Sep 19, 2023
bfe1f3f
Merge branch 'current' into rpourzand-jdbc-updates-where-filters
mirnawong1 Sep 19, 2023
58ba503
Merge branch 'current' into rpourzand-jdbc-updates-where-filters
mirnawong1 Sep 19, 2023
15f9d70
Update sl-jdbc.md
mirnawong1 Sep 19, 2023
35c22e6
Merge branch 'current' into rpourzand-jdbc-updates-where-filters
mirnawong1 Sep 19, 2023
6d5be30
Merge branch 'current' into rpourzand-jdbc-updates-where-filters
mirnawong1 Sep 19, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 18 additions & 6 deletions website/docs/docs/dbt-cloud-apis/sl-jdbc.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,7 @@ To query metric values, here are the following parameters that are available:
| `metrics` | The metric name as defined in your dbt metric configuration | `metrics=['revenue']` | Required |
| `group_by` | Dimension names or entities to group by. We require a reference to the entity of the dimension (other than for the primary time dimension), which is pre-appended to the front of the dimension name with a double underscore. | `group_by=['user__country', 'metric_time']` | Optional |
| `grain` | A parameter specific to any time dimension and changes the grain of the data from the default for the metric. | `group_by=[Dimension('metric_time')` <br/> `grain('week\|day\|month\|quarter\|year')]` | Optional |
| `where` | A where clause that allows you to filter on dimensions and entities using parameters - comes with `TimeDimension`, `Dimension`, and `Entity` objects. Granularity is required with `TimeDimension` | `"{{ where=Dimension('customer__country') }} = 'US')"` | Optional |
| `where` | A where clause that allows you to filter on dimensions and entities using parameters. This takes a filter list OR string. Inputs come with `Dimension`, and `Entity` objects. Granularity is required if the `Dimension` is a time dimension | `"{{ where=Dimension('customer__country') }} = 'US')"` | Optional |
| `limit` | Limit the data returned | `limit=10` | Optional |
|`order` | Order the data returned | `order_by=['-order_gross_profit']` (remove `-` for ascending order) | Optional |
| `explain` | If true, returns generated SQL for the data platform but does not execute | `explain=True` | Optional |
Expand Down Expand Up @@ -215,21 +215,33 @@ select * from {{

### Query with where filters

Where filters have three components:
Where filters in API allow for a filter list or string. We recommend using the filter list for production application as this format will realize all benefits from predicate pushdown where possible.
Copy link
Contributor

@mirnawong1 mirnawong1 Aug 23, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rpourzand , what does predicate mean in this sentence?

Suggested change
Where filters in API allow for a filter list or string. We recommend using the filter list for production application as this format will realize all benefits from predicate pushdown where possible.
In the API, you can use where filters either as a filter list or string. For production applications, it's better to use a filter list. This format helps you take advantage of all the benefits from the predicate pushdown where possible.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So a predicate is just an expression to determine what rows in a database apply to a particular query. For example, if I filter in a WHERE clause on a particular dimension value, the database is basically going to search to determine what values in the DB apply to this query. If this can be pushed down to the database (predicate pushdown)), it's much faster.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could link out some generic docs on predicate pushdown.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Predicate Pushdown points to the where or filter clause which affects the number of rows returned. It basically relates to which rows will be filtered, not which columns. For this reason, while applying the filter on a nested column as ‘library.books’ to merely return records with the values that are not null, the predicate pushdown function will make parquet read blocks that contain no null values for the specified column.


- `TimeDimension()` is used for any time dimension and requires a granularity argument - `TimeDimension('metric_time', 'DAY')`
Where Filters have a few objects that you can use:

- `Dimension()` - This is used for any categorical dimensions - `Dimension('customer__country')`
- `Dimension()` - This is used for any categorical or time dimensions. If used for a time dimension, granularity is required - `Dimension('metric_time').grain('week')` or `Dimension('customer__country')`

- `Entity()` - used for entities like primary and foreign keys - `Entity('order_id')`

Use the following example to query using a `where` filter:
Note: If you prefer more strongly typed `where` clause, you can optionally use `TimeDimension` to separate out categorical dimensions from time ones. The `TimeDimesion` input takes the time dimension name and also requires granularity - an example is `TimeDimension('metric_time', 'MONTH')`.
mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved

Use the following example to query using a `where` filter with the string format:

```bash
select * from {{
semantic_layer.query(metrics=['food_order_amount', 'order_gross_profit'],
group_by=[Dimension('metric_time').grain('month'),'customer__customer_type'],
where="{{ Dimension('metric_time'.grain('month') }} >= '2017-03-09' AND {{ Dimension('customer__customer_type' }} in ('new') AND {{ Entity('order_id') }} = 10")
}}
```

Use the following example to query using a `where` filter with a filter list format:

```bash
select * from {{
semantic_layer.query(metrics=['food_order_amount', 'order_gross_profit'],
group_by=[Dimension('metric_time').grain('month'),'customer__customer_type'],
where="{{ TimeDimension('metric_time', 'MONTH') }} >= '2017-03-09' AND {{ Dimension('customer__customer_type' }} in ('new') AND {{ Entity('order_id') }} = 10")
where=[{{ Dimension('metric_time'.grain('month') }} >= '2017-03-09', {{ Dimension('customer__customer_type' }} in ('new'), {{ Entity('order_id') }} = 10])
}}
```

Expand Down