Skip to content

Commit

Permalink
fixes #556 by documenting all the xtdb stuff
Browse files Browse the repository at this point in the history
Signed-off-by: Sean Corfield <[email protected]>
  • Loading branch information
seancorfield committed Dec 11, 2024
1 parent e4762a1 commit 30d1771
Show file tree
Hide file tree
Showing 8 changed files with 218 additions and 15 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

* 2.6.next in progress
* Address [#558](https://github.com/seancorfield/honeysql/issues/558) by adding `:patch-into` (and `patch-into` helper) for XTDB (but in core).
* Address [#556](https://github.com/seancorfield/honeysql/issues/556) by adding an XTDB section to the documentation with examples.
* Address [#555](https://github.com/seancorfield/honeysql/issues/555) by supporting `SETTING` clause for XTDB.
* Replace `assert` calls with proper validation, throwing `ex-info` on failure (like other existing validation in HoneySQL).
* Experimental `:xtdb` dialect removed (since XTDB no longer supports qualified column names).
Expand Down
3 changes: 2 additions & 1 deletion build/honey/gen_doc_tests.clj
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,8 @@
;;"doc/operator-reference.md"
"doc/options.md"
"doc/postgresql.md"
"doc/special-syntax.md"]
"doc/special-syntax.md"
"doc/xtdb.md"]
regen-reason (if (not (fs/exists? success-marker))
"a previous successful gen result not found"
(let [newer-thans (fs/modified-since target
Expand Down
1 change: 1 addition & 0 deletions doc/cljdoc.edn
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
["SQL Operator Reference" {:file "doc/operator-reference.md"}]
["SQL 'Special Syntax'" {:file "doc/special-syntax.md"}]
["PostgreSQL Support" {:file "doc/postgresql.md"}]
["XTDB Support" {:file "doc/xtdb.md"}]
["New Relic NRQL Support" {:file "doc/nrql.md"}]
["Other Databases" {:file "doc/databases.md"}]]
["All the Options" {:file "doc/options.md"}]
Expand Down
4 changes: 3 additions & 1 deletion doc/databases.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# Other Databases

There is a dedicated section for [PostgreSQL Support](postgres.md).
There are dedicated sections for [New Relic Query Language Support](nrql.md),
[PostgreSQL Support](postgres.md), and
[XTDB Support](xtdb.md).
This section provides hints and tips for generating SQL for other
databases.

Expand Down
3 changes: 2 additions & 1 deletion doc/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -415,7 +415,8 @@ If you want to use a dialect _and_ use the default quoting strategy (automatical
```

Out of the box, as part of the extended ANSI SQL support,
HoneySQL supports quite a few [PostgreSQL extensions](postgresql.md).
HoneySQL supports quite a few [PostgreSQL extensions](postgresql.md)
and [XTDB extensions](xtdb.md).

> Note: the [nilenso/honeysql-postgres](https://github.com/nilenso/honeysql-postgres) library which provided PostgreSQL support for HoneySQL 1.x does not work with HoneySQL 2.x. However, HoneySQL 2.x includes all of the functionality from that library (up to 0.4.112) out of the box!
Expand Down
192 changes: 192 additions & 0 deletions doc/xtdb.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,192 @@
# XTDB Support

As of 2.6.1230, HoneySQL provides support for most of XTDB's SQL
extensions, with additional support being added in subsequent releases.

For the most part, XTDB's SQL is based on
[SQL:2011](https://en.wikipedia.org/wiki/SQL:2011), including the
bitemporal features, but also includes a number of SQL extensions
to support additional XTDB-specific features.

HoneySQL attempts to support all of these XTDB features in the core
ANSI dialect, and this section documents most of those XTDB features.

For more details, see the XTDB documentation:
* [SQL Overview](https://docs.xtdb.com/quickstart/sql-overview.html)
* [SQL Queries](https://docs.xtdb.com/reference/main/sql/queries.html)
* [SQL Transactions/DML](https://docs.xtdb.com/reference/main/sql/txs.html)

## Code Examples

The code examples herein assume:
```clojure
(refer-clojure :exclude '[update set])
(require '[honey.sql :as sql]
'[honey.sql.helpers :refer [select from where
delete-from erase-from
insert-into patch-into values
records]])
```

Clojure users can opt for the shorter `(require '[honey.sql :as sql] '[honey.sql.helpers :refer :all])` but this syntax is not available to ClojureScript users.

## `select` Variations

XTDB allows you to omit `SELECT` in a query. `SELECT *` is assumed if
it is omitted. In HoneySQL, you can simply omit the `:select` clause
from the DSL to achieve this.

```clojure
user=> (sql/format '{select * from foo where (= status "active")})
["SELECT * FROM foo WHERE status = ?" "active"]
user=> (sql/format '{from foo where (= status "active")})
["FROM foo WHERE status = ?" "active"]
```

You can also `SELECT *` and then exclude columns and/or rename columns.

```clojure
user=> (sql/format '{select ((* {exclude _id rename ((title, name))}))})
["SELECT * EXCLUDE _id RENAME title AS name"]
user=> (sql/format '{select ((a.* {exclude _id})
(b.* {rename ((title, name))}))
from ((foo a))
join ((bar b) (= a._id b.foo_id))})
["SELECT a.* EXCLUDE _id, b.* RENAME title AS name FROM foo AS a INNER JOIN bar AS b ON a._id = b.foo_id"]
```

## Nested Sub-Queries

XTDB can produce structured results from `SELECT` queries containing
sub-queries, using `NEST_ONE` and `NEST_MANY`. In HoneySQL, these are
supported as regular function syntax in `:select` clauses.

```clojure
user=> (sql/format '{select (a.*
((nest_many {select * from bar where (= foo_id a._id)})
b))
from ((foo a))})
["SELECT a.*, NEST_MANY (SELECT * FROM bar WHERE foo_id = a._id) AS b FROM foo AS a"]
```

Remember that function calls in `:select` clauses need to be nested three
levels of parentheses (brackets):
`:select [:col-a [:col-b :alias-b] [[:fn-call :col-c] :alias-c]]`.

## `records` Clause

XTDB provides a `RECORDS` clause to specify a list of structured documents,
similar to `VALUES` but specifically for documents rather than a collection
of column values. HoneySQL supports a `:records` clauses and automatically
lifts hash map values to parameters (rather than treating them as DSL fragments).
You can inline a hash map to produce XTDB's inline document syntax.
See also `insert` and `patch` below.

```clojure
user=> (sql/format {:records [{:_id 1 :status "active"}]})
["RECORDS ?" {:_id 1, :status "active"}]
user=> (sql/format {:records [[:inline {:_id 1 :status "active"}]]})
["RECORDS {_id: 1, status: 'active'}"]
```

## `object` (`record`) Literals

While `RECORDS` exists in parallel to the `VALUES` clause, XTDB also provides
a syntax to construct documents in other contexts in SQL, via the `OBJECT`
literal syntax. `RECORD` is a synonym for `OBJECT`. HoneySQL supports both
`:object` and `:record` as special syntax:

```clojure
user=> (sql/format {:select [[[:object {:_id 1 :status "active"}]]]})
["SELECT OBJECT (_id: 1, status: 'active')"]
user=> (sql/format {:select [[[:record {:_id 1 :status "active"}]]]})
["SELECT RECORD (_id: 1, status: 'active')"]
```

## Object Navigation Expressions

In order to deal with nested documents, XTDB provides syntax to navigate
into them, via field names and/or array indices. HoneySQL supports this
via the `:get-in` special syntax, intended to be familiar to Clojure users.

The first argument to `:get-in` is treated as an expression that produces
the document, and subsequent arguments are treated as field names or array
indices to navigate into that document.

```clojure
user=> (sql/format {:select [[[:get-in :doc :field1 :field2]]]})
["SELECT (doc).field1.field2"]
user=> (sql/format {:select [[[:get-in :table.col 0 :field]]]})
["SELECT (table.col)[0].field"]
```

If you want an array index to be a parameter, use `:lift`:

```clojure
user=> (sql/format {:select [[[:get-in :doc [:lift 0] :field]]]})
["SELECT (doc)[?].field" 0]
```

## Temporal Queries

XTDB allows any query to be run in a temporal context via the `SETTING`
clause (ahead of the `SELECT` clause). HoneySQL supports this via the
`:setting` clause. It accepts a sequence of identifiers and expressions.
An identifier ending in `-time` is assumed to be a temporal identifier
(e.g., `:system-time` mapping to `SYSTEM_TIME`). Other identifiers are assumed to
be regular SQL (so `-` is mapped to a space, e.g., `:as-of` mapping to `AS OF`).
A timestamp literal, such as `DATE '2024-11-24'` can be specified in HoneySQL
using `[:inline [:DATE "2024-11-24"]]` (note the literal case of `:DATE`
to produce `DATE`).

See [XTDB's Top-level queries documentation](https://docs.xtdb.com/reference/main/sql/queries.html#_top_level_queries) for more details.

Here's one fairly complex example:

```clojure
user=> (sql/format {:setting [[:basis-to [:inline :DATE "2024-11-24"]]
[:default :valid-time :to :between [:inline :DATE "2022"] :and [:inline :DATE "2023"]]]})
["SETTING BASIS TO DATE '2024-11-24', DEFAULT VALID_TIME TO BETWEEN DATE '2022' AND DATE '2023'"]
```

Table references (e.g., in a `FROM` clause) can also have temporal qualifiers.
See [HoneySQL's `from` clause documentation](clause-reference.md#from) for
examples of that, one of which is reproduced here:

```clojure
user=> (sql/format {:select [:username]
:from [[:user :for :system-time :as-of [:inline "2019-08-01 15:23:00"]]]
:where [:= :id 9]})
["SELECT username FROM user FOR SYSTEM_TIME AS OF '2019-08-01 15:23:00' WHERE id = ?" 9]
```

## `delete` and `erase`

In XTDB, `DELETE` is a temporal deletion -- the data remains in the database
but is no longer visible in queries that don't specify a time range prior to
the deletion. XTDB provides a similar `ERASE` operation that can permanently
delete the data. HoneySQL supports `:erase-from` with the same syntax as
`:delete-from`.

```clojure
user=> (sql/format {:delete-from :foo :where [:= :status "inactive"]})
["DELETE FROM foo WHERE status = ?" "inactive"]
user=> (sql/format {:erase-from :foo :where [:= :status "inactive"]})
["ERASE FROM foo WHERE status = ?" "inactive"]
```

## `insert` and `patch`

XTDB supports `PATCH` as an upsert operation: it will update existing
documents (via merging the new data) or insert new documents if they
don't already exist. HoneySQL supports `:patch-into` with the same syntax
as `:insert-into` with `:records`.

```clojure
user=> (sql/format {:insert-into :foo
:records [{:_id 1 :status "active"}]})
["INSERT INTO foo RECORDS ?" {:_id 1, :status "active"}]
user=> (sql/format {:patch-into :foo
:records [{:_id 1 :status "active"}]})
["PATCH INTO foo RECORDS ?" {:_id 1, :status "active"}]
```
5 changes: 2 additions & 3 deletions src/honey/sql.cljc
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,6 @@
:raw :nest :with :with-recursive :intersect :union :union-all :except :except-all
:table
:select :select-distinct :select-distinct-on :select-top :select-distinct-top
:records
:distinct :expr :exclude :rename
:into :bulk-collect-into
:insert-into :patch-into :replace-into :update :delete :delete-from :erase-from :truncate
Expand All @@ -71,7 +70,7 @@
;; NRQL extension:
:facet
:window :partition-by
:order-by :limit :offset :fetch :for :lock :values
:order-by :limit :offset :fetch :for :lock :values :records
:on-conflict :on-constraint :do-nothing :do-update-set :on-duplicate-key-update
:returning
:with-data
Expand Down Expand Up @@ -1678,7 +1677,6 @@
:select-distinct-on #'format-selects-on
:select-top #'format-select-top
:select-distinct-top #'format-select-top
:records #'format-records
:exclude #'format-selects
:rename #'format-selects
:distinct (fn [k xs] (format-selects k [[xs]]))
Expand Down Expand Up @@ -1727,6 +1725,7 @@
:for #'format-lock-strength
:lock #'format-lock-strength
:values #'format-values
:records #'format-records
:on-conflict #'format-on-conflict
:on-constraint #'format-selector
:do-nothing (fn [k _] (vector (sql-kw k)))
Expand Down
24 changes: 15 additions & 9 deletions test/honey/sql/xtdb_test.cljc
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,9 @@
(is (= ["SELECT (a.b).c"] ; old, partial support:
(sql/format '{select (((. (nest :a.b) :c)))})))
(is (= ["SELECT (a.b).c"] ; new, complete support:
(sql/format '{select (((:get-in :a.b :c)))}))))
(sql/format '{select (((:get-in :a.b :c)))})))
(is (= ["SELECT (a).b.c"] ; the first expression is always parenthesized:
(sql/format '{select (((:get-in :a :b :c)))}))))

(deftest erase-from-test
(is (= ["ERASE FROM foo WHERE foo.id = ?" 42]
Expand Down Expand Up @@ -76,22 +78,26 @@
[:inline {:_id 2 :name "dog"}]]}))))
(testing "insert with records"
(is (= ["INSERT INTO foo RECORDS {_id: 1, name: 'cat'}, {_id: 2, name: 'dog'}"]
(sql/format {:insert-into [:foo
{:records [[:inline {:_id 1 :name "cat"}]
[:inline {:_id 2 :name "dog"}]]}]})))
(sql/format {:insert-into :foo
:records [[:inline {:_id 1 :name "cat"}]
[:inline {:_id 2 :name "dog"}]]})))
(is (= ["INSERT INTO foo RECORDS {_id: 1, name: 'cat'}, {_id: 2, name: 'dog'}"]
(sql/format {:insert-into :foo
:records [[:inline {:_id 1 :name "cat"}]
[:inline {:_id 2 :name "dog"}]]})))
(is (= ["INSERT INTO foo RECORDS ?, ?" {:_id 1 :name "cat"} {:_id 2 :name "dog"}]
(sql/format {:insert-into [:foo
(sql/format {:insert-into [:foo ; as a sub-clause
{:records [{:_id 1 :name "cat"}
{:_id 2 :name "dog"}]}]})))))

(deftest patch-statement
(testing "patch with records"
(is (= ["PATCH INTO foo RECORDS {_id: 1, name: 'cat'}, {_id: 2, name: 'dog'}"]
(sql/format {:patch-into [:foo
{:records [[:inline {:_id 1 :name "cat"}]
[:inline {:_id 2 :name "dog"}]]}]})))
(sql/format {:patch-into [:foo]
:records [[:inline {:_id 1 :name "cat"}]
[:inline {:_id 2 :name "dog"}]]})))
(is (= ["PATCH INTO foo RECORDS ?, ?" {:_id 1 :name "cat"} {:_id 2 :name "dog"}]
(sql/format {:patch-into [:foo
(sql/format {:patch-into [:foo ; as a sub-clause
{:records [{:_id 1 :name "cat"}
{:_id 2 :name "dog"}]}]})))
(is (= ["PATCH INTO foo RECORDS ?, ?" {:_id 1 :name "cat"} {:_id 2 :name "dog"}]
Expand Down

0 comments on commit 30d1771

Please sign in to comment.