Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(bq,sf,rs,pg|quadbin): add function QUADBIN_DISTANCE #457

Merged
merged 8 commits into from
Dec 28, 2023
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions clouds/bigquery/modules/doc/quadbin/QUADBIN_DISTANCE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
## QUADBIN_DISTANCE

```sql:signature
QUADBIN_DISTANCE(origin, destination)
```

**Description**

Returns the **Chebyshev distance** between two quadbin indexes. Returns `null` on invalid inputs.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perhaps "the Chebyshev distance in quadbin cells between two quadbin indexes" ? but I'm not sure, it might be confusing... the Chebyshev distance in number of quadbin cells between two quadbin indexes" ?? I'm not sure yet.

We could also add a comment about both indices needing to be of the same resultion.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On second thought, if the reader understands what Chebyshev distance is (or they have looked it up), they probably have figured out what this that, so we can leave that sentence as it. But I would add:

The origin and destination indices must have the same resolution. Otherwise NULL will be returned.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


* `origin`: `INT64` origin quadbin index.
* `destination`: `INT64` destination quadbin index.

**Return type**

`GEOGRAPHY`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

INT64


**Example**

```sql
SELECT carto.QUADBIN_DISTANCE(5207251884775047167, 5207128739472736255);
-- 1
```
25 changes: 25 additions & 0 deletions clouds/bigquery/modules/sql/quadbin/QUADBIN_DISTANCE.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
----------------------------
-- Copyright (C) 2023 CARTO
----------------------------

CREATE OR REPLACE FUNCTION `@@BQ_DATASET@@.QUADBIN_DISTANCE`
(origin INT64, destination INT64)
RETURNS INT64
AS ((
IF(origin IS NULL OR destination IS NULL,
NULL,
(WITH __quadbin_coords AS (
SELECT
`@@BQ_DATASET@@.QUADBIN_TOZXY`(origin) AS origin_coords,
`@@BQ_DATASET@@.QUADBIN_TOZXY`(destination) AS destination_coords
)
SELECT IF(origin_coords.z != destination_coords.z,
NULL,
GREATEST(
ABS(destination_coords.x - origin_coords.x),
ABS(destination_coords.y - origin_coords.y)
)
)
FROM __quadbin_coords)
)
));
2 changes: 1 addition & 1 deletion clouds/bigquery/modules/test/h3/H3_POLYFILL_TABLE.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,4 @@ UNNEST(\`@@BQ_DATASET@@.__H3_POLYFILL_INIT\`(geom,\`@@BQ_DATASET@@.__H3_POLYFILL
UNNEST(\`@@BQ_DATASET@@.H3_TOCHILDREN\`(parent,12)) AS h3)
SELECT * EXCEPT (geom) FROM __cells
WHERE ST_INTERSECTS(geom, \`@@BQ_DATASET@@.H3_CENTER\`(h3));`.replace(/@@BQ_DATASET@@/g, BQ_DATASET));
});
});
8 changes: 8 additions & 0 deletions clouds/bigquery/modules/test/quadbin/QUADBIN_DISTANCE.test.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
const { runQuery } = require('../../../common/test-utils');

test('QUADBIN_DISTANCE should work', async () => {
const query = 'SELECT `@@BQ_DATASET@@.QUADBIN_DISTANCE`(5207251884775047167, 5207128739472736255) AS output';
const rows = await runQuery(query);
expect(rows.length).toEqual(1);
expect(rows[0].output).toEqual(1);
});
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,4 @@ UNNEST(\`@@BQ_DATASET@@.__QUADBIN_POLYFILL_INIT\`(geom,\`@@BQ_DATASET@@.__QUADBI
UNNEST(\`@@BQ_DATASET@@.QUADBIN_TOCHILDREN\`(parent,12)) AS quadbin)
SELECT * EXCEPT (geom) FROM __cells
WHERE ST_INTERSECTS(geom, \`@@BQ_DATASET@@.QUADBIN_CENTER\`(quadbin));`.replace(/@@BQ_DATASET@@/g, BQ_DATASET));
});
});
23 changes: 23 additions & 0 deletions clouds/postgres/modules/doc/quadbin/QUADBIN_DISTANCE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
## QUADBIN_DISTANCE

```sql:signature
QUADBIN_DISTANCE(origin, destination)
```

**Description**

Returns the **Chebyshev distance** between two quadbin indexes. Returns `null` on invalid inputs.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as BQ:

The origin and destination indices must have the same resolution. Otherwise NULL will be returned.

* `origin`: `BIGINT` origin quadbin index.
* `destination`: `BIGINT` destination quadbin index.

**Return type**

`GEOGRAPHY`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BIGINT


**Example**

```sql
SELECT carto.QUADBIN_DISTANCE(5207251884775047167, 5207128739472736255);
-- 1
```
32 changes: 32 additions & 0 deletions clouds/postgres/modules/sql/quadbin/QUADBIN_DISTANCE.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
----------------------------
-- Copyright (C) 2023 CARTO
----------------------------

CREATE OR REPLACE FUNCTION @@PG_SCHEMA@@.QUADBIN_DISTANCE
(origin BIGINT, destination BIGINT)
RETURNS BIGINT
AS
$BODY$
SELECT
CASE WHEN origin IS NULL OR destination IS NULL THEN
NULL
ELSE
(WITH __quadbin_coords AS (
SELECT
@@PG_SCHEMA@@.QUADBIN_TOZXY(origin) AS origin_coords,
@@PG_SCHEMA@@.QUADBIN_TOZXY(destination) AS destination_coords
)
SELECT
CASE WHEN (origin_coords->>'z')::INT != (destination_coords->>'z')::INT THEN
NULL
ELSE
GREATEST(
ABS((destination_coords->>'x')::DOUBLE PRECISION - (origin_coords->>'x')::DOUBLE PRECISION),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor thing: I guess we could cast the coordinates to to BIGINT as well, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually the DOUBLE PRECISION was intended here. As I took the approach used in QUADBIN_CENTER but BIGINT makes more sense, will add it.

ABS((destination_coords->>'y')::DOUBLE PRECISION - (origin_coords->>'y')::DOUBLE PRECISION)
)
END
FROM __quadbin_coords
)
END
$BODY$
LANGUAGE sql IMMUTABLE PARALLEL SAFE;
10 changes: 10 additions & 0 deletions clouds/postgres/modules/test/quadbin/test_QUADBIN_DISTANCE.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
from test_utils import run_query


def test_quadbin_resolution():
"""Computes distance between quadbins."""
result = run_query(
'SELECT @@PG_SCHEMA@@.QUADBIN_DISTANCE'
'(5207251884775047167, 5207128739472736255)'
)
assert result[0][0] == 1
23 changes: 23 additions & 0 deletions clouds/redshift/modules/doc/quadbin/QUADBIN_DISTANCE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
## QUADBIN_DISTANCE

```sql:signature
QUADBIN_DISTANCE(origin, destination)
```

**Description**

Returns the **Chebyshev distance** between two quadbin indexes. Returns `null` on invalid inputs.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as BQ:

The origin and destination indices must have the same resolution. Otherwise NULL will be returned.

* `origin`: `BIGINT` origin quadbin index.
* `destination`: `BIGINT` destination quadbin index.

**Return type**

`GEOGRAPHY`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BIGINT


**Example**

```sql
SELECT carto.QUADBIN_DISTANCE(5207251884775047167, 5207128739472736255);
-- 1
```
28 changes: 28 additions & 0 deletions clouds/redshift/modules/sql/quadbin/QUADBIN_DISTANCE.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
----------------------------
-- Copyright (C) 2023 CARTO
----------------------------

CREATE OR REPLACE FUNCTION @@RS_SCHEMA@@.QUADBIN_DISTANCE
(origin BIGINT, destination BIGINT)
RETURNS BIGINT
STABLE
AS $$
SELECT CASE WHEN $1 IS NULL OR $2 IS NULL THEN
NULL
ELSE
CASE WHEN @@RS_SCHEMA@@.QUADBIN_RESOLUTION($1) != @@RS_SCHEMA@@.QUADBIN_RESOLUTION($2) THEN
NULL
ELSE
GREATEST(
ABS(
(@@RS_SCHEMA@@.__QUADBIN_TOZXY_X($2) >> (32 - CAST(@@RS_SCHEMA@@.QUADBIN_RESOLUTION($2) AS INT))) -
(@@RS_SCHEMA@@.__QUADBIN_TOZXY_X($1) >> (32 - CAST(@@RS_SCHEMA@@.QUADBIN_RESOLUTION($1) AS INT)))
),
ABS(
(@@RS_SCHEMA@@.__QUADBIN_TOZXY_Y($2) >> (32 - CAST(@@RS_SCHEMA@@.QUADBIN_RESOLUTION($2) AS INT))) -
(@@RS_SCHEMA@@.__QUADBIN_TOZXY_Y($1) >> (32 - CAST(@@RS_SCHEMA@@.QUADBIN_RESOLUTION($1) AS INT)))
)
)
END
END
$$ LANGUAGE sql;
84 changes: 0 additions & 84 deletions clouds/redshift/modules/sql/quadbin/QUADBIN_TOZXY.sql
Original file line number Diff line number Diff line change
Expand Up @@ -2,90 +2,6 @@
-- Copyright (C) 2022 CARTO
----------------------------

CREATE OR REPLACE FUNCTION @@RS_SCHEMA@@.__QUADBIN_TOZXY_XY_INTERLEAVED5
(BIGINT)
-- (quadbin)
RETURNS BIGINT
STABLE
AS $$
SELECT ($1 | ($1 >> 16)) & CAST(FROM_HEX('00000000FFFFFFFF') AS BIGINT)
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION @@RS_SCHEMA@@.__QUADBIN_TOZXY_XY_INTERLEAVED4
(BIGINT)
-- (quadbin)
RETURNS BIGINT
STABLE
AS $$
SELECT @@RS_SCHEMA@@.__QUADBIN_TOZXY_XY_INTERLEAVED5(
($1 | ($1 >> 8)) & CAST(FROM_HEX('0000FFFF0000FFFF') AS BIGINT)
)
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION @@RS_SCHEMA@@.__QUADBIN_TOZXY_XY_INTERLEAVED3
(BIGINT)
-- (quadbin)
RETURNS BIGINT
STABLE
AS $$
SELECT @@RS_SCHEMA@@.__QUADBIN_TOZXY_XY_INTERLEAVED4(
($1 | ($1 >> 4)) & CAST(FROM_HEX('00FF00FF00FF00FF') AS BIGINT)
)
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION @@RS_SCHEMA@@.__QUADBIN_TOZXY_XY_INTERLEAVED2
(BIGINT)
-- (quadbin)
RETURNS BIGINT
STABLE
AS $$
SELECT @@RS_SCHEMA@@.__QUADBIN_TOZXY_XY_INTERLEAVED3(
($1 | ($1 >> 2)) & CAST(FROM_HEX('0F0F0F0F0F0F0F0F') AS BIGINT)
)
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION @@RS_SCHEMA@@.__QUADBIN_TOZXY_XY_INTERLEAVED1
(BIGINT)
-- (quadbin)
RETURNS BIGINT
STABLE
AS $$
SELECT @@RS_SCHEMA@@.__QUADBIN_TOZXY_XY_INTERLEAVED2(
($1 | ($1 >> 1)) & CAST(FROM_HEX('3333333333333333') AS BIGINT)
)
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION @@RS_SCHEMA@@.__QUADBIN_TOZXY_XY_PREINTERLEAVED
(BIGINT)
-- (quadbin)
RETURNS BIGINT
STABLE
AS $$
SELECT @@RS_SCHEMA@@.__QUADBIN_TOZXY_XY_INTERLEAVED1(
$1 & CAST(FROM_HEX('5555555555555555') AS BIGINT)
)
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION @@RS_SCHEMA@@.__QUADBIN_TOZXY_X
(BIGINT)
-- (quadbin)
RETURNS BIGINT
STABLE
AS $$
SELECT @@RS_SCHEMA@@.__QUADBIN_TOZXY_XY_PREINTERLEAVED(
($1 & CAST(FROM_HEX('00FFFFFFFFFFFFF') AS BIGINT)) << 12
)
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION @@RS_SCHEMA@@.__QUADBIN_TOZXY_Y
(BIGINT)
-- (quadbin)
RETURNS BIGINT
STABLE
AS $$
SELECT @@RS_SCHEMA@@.__QUADBIN_TOZXY_X($1 >> 1)
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION @@RS_SCHEMA@@.QUADBIN_TOZXY
(BIGINT)
-- (quadbin)
Expand Down
78 changes: 78 additions & 0 deletions clouds/redshift/modules/sql/quadbin/__QUADBIN_TOZXY_X.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
----------------------------
-- Copyright (C) 2022 CARTO
----------------------------

CREATE OR REPLACE FUNCTION @@RS_SCHEMA@@.__QUADBIN_TOZXY_XY_INTERLEAVED5
(BIGINT)
-- (quadbin)
RETURNS BIGINT
STABLE
AS $$
SELECT ($1 | ($1 >> 16)) & CAST(FROM_HEX('00000000FFFFFFFF') AS BIGINT)
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION @@RS_SCHEMA@@.__QUADBIN_TOZXY_XY_INTERLEAVED4
(BIGINT)
-- (quadbin)
RETURNS BIGINT
STABLE
AS $$
SELECT @@RS_SCHEMA@@.__QUADBIN_TOZXY_XY_INTERLEAVED5(
($1 | ($1 >> 8)) & CAST(FROM_HEX('0000FFFF0000FFFF') AS BIGINT)
)
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION @@RS_SCHEMA@@.__QUADBIN_TOZXY_XY_INTERLEAVED3
(BIGINT)
-- (quadbin)
RETURNS BIGINT
STABLE
AS $$
SELECT @@RS_SCHEMA@@.__QUADBIN_TOZXY_XY_INTERLEAVED4(
($1 | ($1 >> 4)) & CAST(FROM_HEX('00FF00FF00FF00FF') AS BIGINT)
)
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION @@RS_SCHEMA@@.__QUADBIN_TOZXY_XY_INTERLEAVED2
(BIGINT)
-- (quadbin)
RETURNS BIGINT
STABLE
AS $$
SELECT @@RS_SCHEMA@@.__QUADBIN_TOZXY_XY_INTERLEAVED3(
($1 | ($1 >> 2)) & CAST(FROM_HEX('0F0F0F0F0F0F0F0F') AS BIGINT)
)
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION @@RS_SCHEMA@@.__QUADBIN_TOZXY_XY_INTERLEAVED1
(BIGINT)
-- (quadbin)
RETURNS BIGINT
STABLE
AS $$
SELECT @@RS_SCHEMA@@.__QUADBIN_TOZXY_XY_INTERLEAVED2(
($1 | ($1 >> 1)) & CAST(FROM_HEX('3333333333333333') AS BIGINT)
)
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION @@RS_SCHEMA@@.__QUADBIN_TOZXY_XY_PREINTERLEAVED
(BIGINT)
-- (quadbin)
RETURNS BIGINT
STABLE
AS $$
SELECT @@RS_SCHEMA@@.__QUADBIN_TOZXY_XY_INTERLEAVED1(
$1 & CAST(FROM_HEX('5555555555555555') AS BIGINT)
)
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION @@RS_SCHEMA@@.__QUADBIN_TOZXY_X
(BIGINT)
-- (quadbin)
RETURNS BIGINT
STABLE
AS $$
SELECT @@RS_SCHEMA@@.__QUADBIN_TOZXY_XY_PREINTERLEAVED(
($1 & CAST(FROM_HEX('00FFFFFFFFFFFFF') AS BIGINT)) << 12
)
$$ LANGUAGE sql;
12 changes: 12 additions & 0 deletions clouds/redshift/modules/sql/quadbin/__QUADBIN_TOZXY_Y.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
----------------------------
-- Copyright (C) 2022 CARTO
----------------------------

CREATE OR REPLACE FUNCTION @@RS_SCHEMA@@.__QUADBIN_TOZXY_Y
(BIGINT)
-- (quadbin)
RETURNS BIGINT
STABLE
AS $$
SELECT @@RS_SCHEMA@@.__QUADBIN_TOZXY_X($1 >> 1)
$$ LANGUAGE sql;
11 changes: 11 additions & 0 deletions clouds/redshift/modules/test/quadbin/test_QUADBIN_DISTANCE.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
from test_utils import run_query


def test_quadbin_distance():
result = run_query(
'SELECT @@RS_SCHEMA@@.QUADBIN_DISTANCE'
'(5207251884775047167, 5207128739472736255)'
)

assert len(result[0]) == 1
assert result[0][0] == 1
Loading
Loading