-
Notifications
You must be signed in to change notification settings - Fork 44
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(bq,sf,rs,pg|quadbin): add function QUADBIN_DISTANCE #457
Changes from 7 commits
1c44012
7d5e554
5d472cd
3017ba9
a1ed1b3
dfc3bf0
0f7c40d
8c5b0c2
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,23 @@ | ||
## QUADBIN_DISTANCE | ||
|
||
```sql:signature | ||
QUADBIN_DISTANCE(origin, destination) | ||
``` | ||
|
||
**Description** | ||
|
||
Returns the **Chebyshev distance** between two quadbin indexes. Returns `null` on invalid inputs. | ||
|
||
* `origin`: `INT64` origin quadbin index. | ||
* `destination`: `INT64` destination quadbin index. | ||
|
||
**Return type** | ||
|
||
`GEOGRAPHY` | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. INT64 |
||
|
||
**Example** | ||
|
||
```sql | ||
SELECT carto.QUADBIN_DISTANCE(5207251884775047167, 5207128739472736255); | ||
-- 1 | ||
``` |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
---------------------------- | ||
-- Copyright (C) 2023 CARTO | ||
---------------------------- | ||
|
||
CREATE OR REPLACE FUNCTION `@@BQ_DATASET@@.QUADBIN_DISTANCE` | ||
(origin INT64, destination INT64) | ||
RETURNS INT64 | ||
AS (( | ||
IF(origin IS NULL OR destination IS NULL, | ||
NULL, | ||
(WITH __quadbin_coords AS ( | ||
SELECT | ||
`@@BQ_DATASET@@.QUADBIN_TOZXY`(origin) AS origin_coords, | ||
`@@BQ_DATASET@@.QUADBIN_TOZXY`(destination) AS destination_coords | ||
) | ||
SELECT IF(origin_coords.z != destination_coords.z, | ||
NULL, | ||
GREATEST( | ||
ABS(destination_coords.x - origin_coords.x), | ||
ABS(destination_coords.y - origin_coords.y) | ||
) | ||
) | ||
FROM __quadbin_coords) | ||
) | ||
)); |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
const { runQuery } = require('../../../common/test-utils'); | ||
|
||
test('QUADBIN_DISTANCE should work', async () => { | ||
const query = 'SELECT `@@BQ_DATASET@@.QUADBIN_DISTANCE`(5207251884775047167, 5207128739472736255) AS output'; | ||
const rows = await runQuery(query); | ||
expect(rows.length).toEqual(1); | ||
expect(rows[0].output).toEqual(1); | ||
}); |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,23 @@ | ||
## QUADBIN_DISTANCE | ||
|
||
```sql:signature | ||
QUADBIN_DISTANCE(origin, destination) | ||
``` | ||
|
||
**Description** | ||
|
||
Returns the **Chebyshev distance** between two quadbin indexes. Returns `null` on invalid inputs. | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. same as BQ:
|
||
* `origin`: `BIGINT` origin quadbin index. | ||
* `destination`: `BIGINT` destination quadbin index. | ||
|
||
**Return type** | ||
|
||
`GEOGRAPHY` | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. BIGINT |
||
|
||
**Example** | ||
|
||
```sql | ||
SELECT carto.QUADBIN_DISTANCE(5207251884775047167, 5207128739472736255); | ||
-- 1 | ||
``` |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,32 @@ | ||
---------------------------- | ||
-- Copyright (C) 2023 CARTO | ||
---------------------------- | ||
|
||
CREATE OR REPLACE FUNCTION @@PG_SCHEMA@@.QUADBIN_DISTANCE | ||
(origin BIGINT, destination BIGINT) | ||
RETURNS BIGINT | ||
AS | ||
$BODY$ | ||
SELECT | ||
CASE WHEN origin IS NULL OR destination IS NULL THEN | ||
NULL | ||
ELSE | ||
(WITH __quadbin_coords AS ( | ||
SELECT | ||
@@PG_SCHEMA@@.QUADBIN_TOZXY(origin) AS origin_coords, | ||
@@PG_SCHEMA@@.QUADBIN_TOZXY(destination) AS destination_coords | ||
) | ||
SELECT | ||
CASE WHEN (origin_coords->>'z')::INT != (destination_coords->>'z')::INT THEN | ||
NULL | ||
ELSE | ||
GREATEST( | ||
ABS((destination_coords->>'x')::DOUBLE PRECISION - (origin_coords->>'x')::DOUBLE PRECISION), | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. minor thing: I guess we could cast the coordinates to to BIGINT as well, right? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Actually the DOUBLE PRECISION was intended here. As I took the approach used in |
||
ABS((destination_coords->>'y')::DOUBLE PRECISION - (origin_coords->>'y')::DOUBLE PRECISION) | ||
) | ||
END | ||
FROM __quadbin_coords | ||
) | ||
END | ||
$BODY$ | ||
LANGUAGE sql IMMUTABLE PARALLEL SAFE; |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
from test_utils import run_query | ||
|
||
|
||
def test_quadbin_resolution(): | ||
"""Computes distance between quadbins.""" | ||
result = run_query( | ||
'SELECT @@PG_SCHEMA@@.QUADBIN_DISTANCE' | ||
'(5207251884775047167, 5207128739472736255)' | ||
) | ||
assert result[0][0] == 1 |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,23 @@ | ||
## QUADBIN_DISTANCE | ||
|
||
```sql:signature | ||
QUADBIN_DISTANCE(origin, destination) | ||
``` | ||
|
||
**Description** | ||
|
||
Returns the **Chebyshev distance** between two quadbin indexes. Returns `null` on invalid inputs. | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. same as BQ:
|
||
* `origin`: `BIGINT` origin quadbin index. | ||
* `destination`: `BIGINT` destination quadbin index. | ||
|
||
**Return type** | ||
|
||
`GEOGRAPHY` | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. BIGINT |
||
|
||
**Example** | ||
|
||
```sql | ||
SELECT carto.QUADBIN_DISTANCE(5207251884775047167, 5207128739472736255); | ||
-- 1 | ||
``` |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,28 @@ | ||
---------------------------- | ||
-- Copyright (C) 2023 CARTO | ||
---------------------------- | ||
|
||
CREATE OR REPLACE FUNCTION @@RS_SCHEMA@@.QUADBIN_DISTANCE | ||
(origin BIGINT, destination BIGINT) | ||
RETURNS BIGINT | ||
STABLE | ||
AS $$ | ||
SELECT CASE WHEN $1 IS NULL OR $2 IS NULL THEN | ||
NULL | ||
ELSE | ||
CASE WHEN @@RS_SCHEMA@@.QUADBIN_RESOLUTION($1) != @@RS_SCHEMA@@.QUADBIN_RESOLUTION($2) THEN | ||
NULL | ||
ELSE | ||
GREATEST( | ||
ABS( | ||
(@@RS_SCHEMA@@.__QUADBIN_TOZXY_X($2) >> (32 - CAST(@@RS_SCHEMA@@.QUADBIN_RESOLUTION($2) AS INT))) - | ||
(@@RS_SCHEMA@@.__QUADBIN_TOZXY_X($1) >> (32 - CAST(@@RS_SCHEMA@@.QUADBIN_RESOLUTION($1) AS INT))) | ||
), | ||
ABS( | ||
(@@RS_SCHEMA@@.__QUADBIN_TOZXY_Y($2) >> (32 - CAST(@@RS_SCHEMA@@.QUADBIN_RESOLUTION($2) AS INT))) - | ||
(@@RS_SCHEMA@@.__QUADBIN_TOZXY_Y($1) >> (32 - CAST(@@RS_SCHEMA@@.QUADBIN_RESOLUTION($1) AS INT))) | ||
) | ||
) | ||
END | ||
END | ||
$$ LANGUAGE sql; |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,78 @@ | ||
---------------------------- | ||
-- Copyright (C) 2022 CARTO | ||
---------------------------- | ||
|
||
CREATE OR REPLACE FUNCTION @@RS_SCHEMA@@.__QUADBIN_TOZXY_XY_INTERLEAVED5 | ||
(BIGINT) | ||
-- (quadbin) | ||
RETURNS BIGINT | ||
STABLE | ||
AS $$ | ||
SELECT ($1 | ($1 >> 16)) & CAST(FROM_HEX('00000000FFFFFFFF') AS BIGINT) | ||
$$ LANGUAGE sql; | ||
|
||
CREATE OR REPLACE FUNCTION @@RS_SCHEMA@@.__QUADBIN_TOZXY_XY_INTERLEAVED4 | ||
(BIGINT) | ||
-- (quadbin) | ||
RETURNS BIGINT | ||
STABLE | ||
AS $$ | ||
SELECT @@RS_SCHEMA@@.__QUADBIN_TOZXY_XY_INTERLEAVED5( | ||
($1 | ($1 >> 8)) & CAST(FROM_HEX('0000FFFF0000FFFF') AS BIGINT) | ||
) | ||
$$ LANGUAGE sql; | ||
|
||
CREATE OR REPLACE FUNCTION @@RS_SCHEMA@@.__QUADBIN_TOZXY_XY_INTERLEAVED3 | ||
(BIGINT) | ||
-- (quadbin) | ||
RETURNS BIGINT | ||
STABLE | ||
AS $$ | ||
SELECT @@RS_SCHEMA@@.__QUADBIN_TOZXY_XY_INTERLEAVED4( | ||
($1 | ($1 >> 4)) & CAST(FROM_HEX('00FF00FF00FF00FF') AS BIGINT) | ||
) | ||
$$ LANGUAGE sql; | ||
|
||
CREATE OR REPLACE FUNCTION @@RS_SCHEMA@@.__QUADBIN_TOZXY_XY_INTERLEAVED2 | ||
(BIGINT) | ||
-- (quadbin) | ||
RETURNS BIGINT | ||
STABLE | ||
AS $$ | ||
SELECT @@RS_SCHEMA@@.__QUADBIN_TOZXY_XY_INTERLEAVED3( | ||
($1 | ($1 >> 2)) & CAST(FROM_HEX('0F0F0F0F0F0F0F0F') AS BIGINT) | ||
) | ||
$$ LANGUAGE sql; | ||
|
||
CREATE OR REPLACE FUNCTION @@RS_SCHEMA@@.__QUADBIN_TOZXY_XY_INTERLEAVED1 | ||
(BIGINT) | ||
-- (quadbin) | ||
RETURNS BIGINT | ||
STABLE | ||
AS $$ | ||
SELECT @@RS_SCHEMA@@.__QUADBIN_TOZXY_XY_INTERLEAVED2( | ||
($1 | ($1 >> 1)) & CAST(FROM_HEX('3333333333333333') AS BIGINT) | ||
) | ||
$$ LANGUAGE sql; | ||
|
||
CREATE OR REPLACE FUNCTION @@RS_SCHEMA@@.__QUADBIN_TOZXY_XY_PREINTERLEAVED | ||
(BIGINT) | ||
-- (quadbin) | ||
RETURNS BIGINT | ||
STABLE | ||
AS $$ | ||
SELECT @@RS_SCHEMA@@.__QUADBIN_TOZXY_XY_INTERLEAVED1( | ||
$1 & CAST(FROM_HEX('5555555555555555') AS BIGINT) | ||
) | ||
$$ LANGUAGE sql; | ||
|
||
CREATE OR REPLACE FUNCTION @@RS_SCHEMA@@.__QUADBIN_TOZXY_X | ||
(BIGINT) | ||
-- (quadbin) | ||
RETURNS BIGINT | ||
STABLE | ||
AS $$ | ||
SELECT @@RS_SCHEMA@@.__QUADBIN_TOZXY_XY_PREINTERLEAVED( | ||
($1 & CAST(FROM_HEX('00FFFFFFFFFFFFF') AS BIGINT)) << 12 | ||
) | ||
$$ LANGUAGE sql; |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
---------------------------- | ||
-- Copyright (C) 2022 CARTO | ||
---------------------------- | ||
|
||
CREATE OR REPLACE FUNCTION @@RS_SCHEMA@@.__QUADBIN_TOZXY_Y | ||
(BIGINT) | ||
-- (quadbin) | ||
RETURNS BIGINT | ||
STABLE | ||
AS $$ | ||
SELECT @@RS_SCHEMA@@.__QUADBIN_TOZXY_X($1 >> 1) | ||
$$ LANGUAGE sql; |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
from test_utils import run_query | ||
|
||
|
||
def test_quadbin_distance(): | ||
result = run_query( | ||
'SELECT @@RS_SCHEMA@@.QUADBIN_DISTANCE' | ||
'(5207251884775047167, 5207128739472736255)' | ||
) | ||
|
||
assert len(result[0]) == 1 | ||
assert result[0][0] == 1 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
perhaps "the Chebyshev distance in quadbin cells between two quadbin indexes" ? but I'm not sure, it might be confusing... the Chebyshev distance in number of quadbin cells between two quadbin indexes" ?? I'm not sure yet.
We could also add a comment about both indices needing to be of the same resultion.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On second thought, if the reader understands what Chebyshev distance is (or they have looked it up), they probably have figured out what this that, so we can leave that sentence as it. But I would add:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll also had the link to wikipedia as we do in here:
https://github.com/CartoDB/analytics-toolbox-core/blob/bug/sc-318681/missing-quadbin-distance-in-at/clouds/redshift/modules/doc/quadbin/QUADBIN_KRING_DISTANCES.md#L36