diff --git a/app/content/analytics-toolbox-bigquery/release-notes.md b/app/content/analytics-toolbox-bigquery/release-notes.md index d2b83c2b8..0891331a7 100644 --- a/app/content/analytics-toolbox-bigquery/release-notes.md +++ b/app/content/analytics-toolbox-bigquery/release-notes.md @@ -4,33 +4,68 @@ aliases: --- ## Release notes -### October 24, 2022 +### November 15, 2022 -#### Module processing +#### Module retail -Fixed +Fix -- Prevent error in ST_VORONOIPOLYGONS, ST_VORONOILINES, ST_VORONOIPOLYGONS, ST_DELAUNAYLINES when points where too close together by rounding input coordinates to 5 decimal places +- `__KRING_DECAY_GRID` was aggregating the value of the moving window pivoting index instead of neighbour indexes. -### October 7, 2022 +### November 8, 2022 -#### Module clustering +#### Module h3 -Feature +Improvement -- Move ST_CLUSTERKMEANS function to core. +- Add linestrings and points support to function H3_POLYFILL. -#### Module random +### November 7, 2022 + +#### Module retail -Feature +New -- Move ST_GENERATEPOINTS function to core. +- Add `FIND_TWIN_AREAS_WEIGHTED` procedure. + +### November 1, 2022 + +#### All modules + +New + +- Create release workflows + +Improvement + +- Adapt to Semver + +### October 28, 2022 + +#### Module s2 + +New + +- Add S2_RESOLUTION function. +- Add S2_TOCHILDREN function. + +### October 26, 2022 + +#### Module transformations + +Fix + +- Fix ST_BUFFER crashing with geographies close to the poles. + +#### Module tiler + +Improvement -### October 5, 2022 +- Partition tables in spatial index tilesets. #### Module cpg -Feature +New - Add `CUSTOMER_SEGMENTATION_ANALYSIS_DATA` procedure. - Add `GENERATE_TRADE_AREAS` procedure. @@ -38,19 +73,41 @@ Feature #### Module lds -Fixed +Fix - CREATE_ISOLINES output is stored in the column `geom` instead of `__iso_geom`. -Changed +Improvement - CREATE_ISOLINES output is clustered by geometry. +### October 24, 2022 + +#### Module processing + +Fix + +- Prevent error in ST_VORONOIPOLYGONS, ST_VORONOILINES, ST_VORONOIPOLYGONS, ST_DELAUNAYLINES when points where too close together by rounding input coordinates to 5 decimal places. + +### October 7, 2022 + +#### Module clustering + +New + +- Move ST_CLUSTERKMEANS function to core. + +#### Module random + +New + +- Move ST_GENERATEPOINTS function to core. + ### September 30, 2022 #### Module lds -Changed +Improvement - Rename `LDS_API_URL` to `API_BASE_URL`. @@ -58,7 +115,7 @@ Changed #### Module retail -Changed +Improvement - `LINEAR_REG` model is now supported. @@ -66,7 +123,7 @@ Changed #### Module tiler -Changed +Improvement - Fix bug in `__BUILD_REVENUE_MODEL_DATA_GRID`: Output table now contains `competitor_count`. @@ -74,7 +131,7 @@ Changed #### Module tiler -Changed +Improvement - Set `version` field in tilers metadata compliant with AT version. @@ -82,7 +139,7 @@ Changed #### Module tiler -Changed +Improvement - Hotfix: set `generate_feature_id` to false by default due a bug in the generation. @@ -90,7 +147,7 @@ Changed #### Module data -Changed +Improvement - Optimize DATAOBS_ENRICH_GRID, DATAOBS_ENRICH_GRID_RAW when the enrich data is a grid compatible with the input. - Remove support for s2, geohas and quadbinh in all enrich procedures. @@ -100,13 +157,13 @@ Changed #### Module s2 -Feature +New - Add S2_CENTER function. #### Module retail -Changed +Improvement - Return shap_values in PREDICT_REVENUE_AVERAGE. @@ -114,7 +171,7 @@ Changed #### Module tiler -Feature +New - Add `generate_feature_id` parameter in `CREATE_SIMPLE_TILESET`. @@ -122,7 +179,7 @@ Feature #### Module retail -Changed +Improvement - Index parameter admits any type in PREDICT_REVENUE_AVERAGE. - Index from revenue_model_data is not casted in PREDICT_REVENUE_AVERAGE to reduce query data processing. @@ -131,7 +188,7 @@ Changed #### Module tiler -Fixed +Fix - Fixed bug when parsing zoom columns in geography tilers. @@ -139,7 +196,7 @@ Fixed #### Module tiler -Changed +Improvement - Unify `extra_metadata` into `metadata` in tiler metadata. @@ -147,7 +204,7 @@ Changed #### Module h3 -Fixed +Fix - Apply make_valid in H3_BOUNDARY. @@ -155,7 +212,7 @@ Fixed #### Module tiler -Fixed +Fix - Fixed bug in metadata `CREATE_SPATIAL_INDEX_TILESET`. @@ -163,7 +220,7 @@ Fixed #### Module retail -Changed +Improvement - Check fixed model options in BUILD_REVENUE_MODEL procedure. @@ -171,7 +228,7 @@ Changed #### Module retail -Feature +New - Add BUILD_CANNIBALIZATION_DATA procedure. - Add CANNIBALIZATION_OVERLAP procedure. @@ -180,7 +237,7 @@ Feature #### Module lds -Feature +New - Add GEOCODE_TABLE procedure. - Add LDS_QUOTA_INFO procedure. @@ -191,7 +248,7 @@ Feature #### Module statistics -Changed +Improvement - Added HH, LL, LH, HL detection in LOCAL_MORANS_I functions. @@ -199,7 +256,7 @@ Changed #### Module tiler -Fixed +Fix - Fixed bug when passing multiline input to `CREATE_SPATIAL_INDEX_TILESET`. @@ -207,23 +264,23 @@ Fixed #### Module data -Feature +New - Add quadbin support to module. #### Module retail -Feature +New - Add quadbin support #### Module tiler -Feature +New - Support of quadgrid as alias in `CREATE_POINT_AGGREGATION_TILESET`. -Changed +Improvement - Use core functions to convert H3 from int to string. @@ -231,14 +288,14 @@ Changed #### Module h3 -Fixed +Fix - Correctly handle large polygons in H3_POLYFILL. - Fixed wrong uppercase for quadbin and h3 tile ids #### Module statistics -Feature +New - Add SMOOTHING_MRF_QUADBIN procedure. - Add LOCAL_MORANS_I_QUADBIN function. @@ -248,7 +305,7 @@ Feature #### Module tiler -Fixed +Fix - Fixed tablename quoting in BigQuery tiler @@ -256,7 +313,7 @@ Fixed #### Module retail -Fixed +Fix - Fix quadkey support in FIND_WHITESPACE_AREAS @@ -264,7 +321,7 @@ Fixed #### Module retail -Fixed +Fix - Fixing bugs when not providing store variables to BUILD_REVENUE_MODEL_DATA @@ -272,7 +329,7 @@ Fixed #### Module quadbin -Feature +New - Add QUADBIN_BBOX function. - Add QUADBIN_BOUNDARY function. @@ -294,13 +351,13 @@ Feature #### Module data -Fixed +Fix - Fix BQ inplace enrichment with uppercase column names. #### Module tiler -Changed +Improvement - Support H3 integer in `CREATE_SPATIAL_INDEX_TILESET`. @@ -308,7 +365,7 @@ Changed #### Module data -Fixed +Fix - Fix order of `__KRING_DECAY` parameters. @@ -316,11 +373,11 @@ Fixed #### Module tiler -Fixed +Fix - Fix global stats in `CREATE_SPATIAL_INDEX_TILESET`. -Changed +Improvement - Check input resolution limits in `CREATE_SPATIAL_INDEX_TILESET`. @@ -328,7 +385,7 @@ Changed #### Module data -Fixed +Fix - Report missing subscription geographies in DATAOBS_ENRICH_ procedures. @@ -336,7 +393,7 @@ Fixed #### Module data -Fixed +Fix - Fix stability issues in GRIDIFY_ENRICH procedure. @@ -344,11 +401,11 @@ Fixed #### Module retail -Feature +New - Add store features and competitors stores features into BUILD_REVENUE_MODEL_DATA procedure. -Fixed +Fix - Fixing bugs calling other routines @@ -356,7 +413,7 @@ Fixed #### Module tiler -Feature +New - Add spatial index tileset generator `CREATE_SPATIAL_INDEX_TILESET`. @@ -364,7 +421,7 @@ Feature #### Module tiler -Fixed +Fix - Bug fixing when the geometry early collapse @@ -372,13 +429,13 @@ Fixed #### Module data -Feature +New - Add GRIDIFY_ENRICH procedure. #### Module retail -Feature +New - Add FIND_TWIN_AREAS procedure. @@ -386,7 +443,7 @@ Feature #### Module data -Feature +New - Allow using tables as input, not just queries. - Add in place enrichment. @@ -397,7 +454,7 @@ Feature #### Module retail -Fixed +Fix - Fix BUILD_REVENUE_MODEL_DATA k-ring buffer computation. @@ -405,19 +462,19 @@ Fixed #### Module transformations -Changed +Improvement - ST_CONCAVEHULL now allows arrays with one/two points as input. #### Module retail -Feature +New - Add COMMERCIAL_HOTSPOTS procedure. #### Module statistics -Feature +New - Add SMOOTHING_MRF_H3 procedure. - Add SMOOTHING_MRF_QUADKEY procedure. @@ -426,7 +483,7 @@ Feature #### Module statistics -Feature +New - Add VARIOGRAM function. - Add ORDINARY_KRIGING function. @@ -436,14 +493,14 @@ Feature #### Module h3 -Feature +New - Add H3_CENTER function. - Add H3_RESOLUTION function. #### Module retail -Feature +New - Add FIND_WHITESPACE_AREAS procedure. @@ -451,7 +508,7 @@ Feature #### Module retail -Feature +New - Remove features importance tables from BUILD_REVENUE_MODEL procedure. @@ -459,12 +516,12 @@ Feature #### Module statistics -Feature +New - Add LOCAL_MORANS_I_H3 function. - Add LOCAL_MORANS_I_QUADKEY function. -Changed +Improvement - Improve MORANS_I_H3 and MORANS_I_QUADKEY implementation. @@ -472,7 +529,7 @@ Changed #### Module statistics -Fixed +Fix - Replace STDDEV by STDDEV_POP in GETIS_ORD_H3 and GETIS_ORD_QUADKEY. @@ -480,7 +537,7 @@ Fixed #### Module retail -Feature +New - Add BUILD_REVENUE_MODEL_DATA procedure. - Add BUILD_REVENUE_MODEL procedure. @@ -490,7 +547,7 @@ Feature #### Module accessors -Changed +Improvement - Deployment schema "carto" instead of "accessors". @@ -500,7 +557,7 @@ Removed #### Module constructors -Changed +Improvement - Deployment schema "carto" instead of "constructors". @@ -510,7 +567,7 @@ Removed #### Module geohash -Changed +Improvement - Deployment schema "carto" instead of "geohash". @@ -520,7 +577,7 @@ Removed #### Module h3 -Changed +Improvement - Deployment schema "carto" instead of "h3". - Rename ST_ASH3 function to H3_FROMGEOGPOINT. @@ -544,7 +601,7 @@ Removed #### Module measurements -Changed +Improvement - Deployment schema "carto" instead of "measurements". @@ -555,7 +612,7 @@ Removed #### Module placekey -Changed +Improvement - Deployment schema "carto" instead of "placekey". - Rename H3_ASPLACEKEY function to PLACEKEY_FROMH3. @@ -568,7 +625,7 @@ Removed #### Module processing -Changed +Improvement - Deployment schema "carto" instead of "processing". @@ -578,7 +635,7 @@ Removed #### Module s2 -Changed +Improvement - Deployment schema "carto" instead of "s2". - Rename ID_FROMHILBERTQUADKEY function to S2_FROMHILBERTQUADKEY. @@ -597,7 +654,7 @@ Removed #### Module transformations -Changed +Improvement - Deployment schema "carto" instead of "transformations". @@ -607,7 +664,7 @@ Removed #### Module clustering -Changed +Improvement - Deployment schema "carto" instead of "clustering". @@ -617,7 +674,7 @@ Removed #### Module data -Changed +Improvement - Deployment schema "carto" instead of "data". @@ -627,7 +684,7 @@ Removed #### Module random -Changed +Improvement - Deployment schema "carto" instead of "random". @@ -637,7 +694,7 @@ Removed #### Module routing -Changed +Improvement - Deployment schema "carto" instead of "routing". @@ -647,7 +704,7 @@ Removed #### Module statistics -Changed +Improvement - Deployment schema "carto" instead of "statistics". @@ -657,7 +714,7 @@ Removed #### Module tiler -Changed +Improvement - Deployment schema "carto" instead of "tiler". @@ -669,7 +726,7 @@ Removed #### Module tiler -Changed +Improvement - Add support for legacy project names that contain dots or other special characters. @@ -677,7 +734,7 @@ Changed #### Module statistics -Feature +New - Add KNN function. - Add GFUN function. @@ -688,7 +745,7 @@ Feature #### Module tiler -Fixed +Fix - Escape single-quoted properties in the `CREATE_TILESET` metadata when obtaining the geometry type. @@ -696,7 +753,7 @@ Fixed #### Module statistics -Feature +New - Add GWR_GRID function. @@ -704,11 +761,11 @@ Feature #### Module tiler -Fixed +Fix - Use `zoom_min_column` as `zoom_min` and `zoom_max_column` as `zoom_max` when provided. -Changed +Improvement - Optimize simplification method instead of the native BigQuery. @@ -716,14 +773,14 @@ Changed #### Module data -Changed +Improvement - Fields named `dimension`, `total`, `intersection` and `input_area` are now `__carto_dimension`, `__carto_total` and `__carto_intersection` and `__carto_input_area`. Also the column `_carto_enrichment_` is now `__carto_enrichment`. This affects all the _raw_ enrichment procedures: `ENRICH_POINTS_RAW`, `ENRICH_POLYGONS_RAW`, `ENRICH_GRID_RAW`, `DATAOBS_ENRICH_POINTS_RAW`, `DATAOBS_ENRICH_POLYGONS_RAW`, `DATAOBS_ENRICH_GRID_RAW`. -Fixed +Fix - User provided queries can now have columns named `dimension`, `total`, `intersection`, `input_area`, `_nonglobal`, which could have collided previously with internal columns. All internal columns are now prefixed with `__carto_`. This affects all the enrichment procedures: `ENRICH_POINTS`, `ENRICH_POLYGONS`, `ENRICH_GRID`, `DATAOBS_ENRICH_POINTS`, `DATAOBS_ENRICH_POLYGONS`, `DATAOBS_ENRICH_GRID`, `ENRICH_POINTS_RAW`, `ENRICH_POLYGONS_RAW`, `ENRICH_GRID_RAW`, `DATAOBS_ENRICH_POINTS_RAW`, `DATAOBS_ENRICH_POLYGONS_RAW`, `DATAOBS_ENRICH_GRID_RAW`. @@ -731,7 +788,7 @@ Fixed #### Module data -Changed +Improvement - Add `associated_geography_slug` column to `DATAOBS_SUBSCRIPTIONS`. - Some column names have changed in `DATAOBS_SUBSCRIPTIONS`: @@ -743,7 +800,7 @@ Changed - `table` is now `dataset_sample_table`. - In `DATAOBS_SUBSCRIPTION_VARIABLES`, the column `associated_geography_slug` is now NULL for geography datasets. -Fixed +Fix - The output of `DATAOBS_SUBSCRIPTIONS` and `DATAOBS_SUBSCRIPTION_VARIABLES` now omits items that are not Data Observatory products. @@ -751,12 +808,12 @@ Fixed #### Module data -Fixed +Fix - Enrichment results could mix input rows and values, specially for large inputs due to multiple evaluation of nondeterministic ROW_NUMBERS expression. -Changed +Improvement - New API, with consistent naming of procedures ` @@ -764,13 +821,13 @@ Changed #### Module s2 -Changed +Improvement - Rename functions ID_FROMUINT64REPR, UINT64REPR_FROMID to follow convention. #### Module statistics -Feature +New - Create statistics module. - Add GETIS_ORD_H3 function. @@ -783,7 +840,7 @@ Feature #### Module h3 -Changed +Improvement - Review HEXRING, KRING functions. - Change KRING_INDEXED to KRING_DISTANCES. @@ -792,12 +849,12 @@ Changed #### Module tiler -Fixed +Fix - Fix support for DATE in `CREATE_TILESET`. - Fix naming of internal variables to avoid name collisions. -Feature +New - Add `fraction_dropped_per_zoom` in tileset metadata. @@ -818,7 +875,7 @@ Changes #### Module s2 -Fixed +Fix - Avoid keeping planar shape in spherical coordinates in ST_BOUNDARY. @@ -834,7 +891,7 @@ Changes #### Module data -Changed +Improvement - DATAOBS_ENRICH_* procedures are now optimized to reduce the amount of data processed by taking advantage of clusterization by geography in the Data Observatory. - The `input_id_column` parameter has been removed from the following procedures: @@ -851,14 +908,14 @@ Changed #### Module data -Changed +Improvement - ENRICH_GRID and DATAOBS_ENRICH_GRID procedures take `input_query` and `input_index_column` parameters instead of the `indices` array. - DATAOBS_ENRICH_* procedures now report missing subscriptions in a more user-friendly way. #### Module routing -Changed +Improvement - Pass network to JS UDF natively - Support custom speed over LineStrings for GENERATE_NETWORK function. @@ -869,7 +926,7 @@ Changed #### Module tiler -Fixed +Fix - Fix support for TIMESTAMP,DATETIME in CREATE_TILESET and CREATE_SIMPLE_TILESET. - Reduce parallelization to skip the recent error: "Not enough resources for query planning - too many subqueries or query is too complex" @@ -878,7 +935,7 @@ Fixed #### Module h3 -Fixed +Fix - Support GEOMETRYCOLLECTION from ST_ASH3_POLYFILL. @@ -886,7 +943,7 @@ Fixed #### Module data -Feature +New - Add DATAOBS_ENRICH_GRID procedure. - Add ENRICH_GRID procedure. @@ -895,7 +952,7 @@ Feature #### Module h3 -Feature +New - Add KRING_INDEXED function. @@ -903,7 +960,7 @@ Feature #### Module geohash -Feature +New - Create geohash module. - Add VERSION function. @@ -911,7 +968,7 @@ Feature #### Module tiler -Changed +Improvement - Checking if the output table exists before running the tiler. @@ -919,7 +976,7 @@ Changed #### Module data -Changed +Improvement - DATAOBS_ENRICH_* procedures take single `source` parameter with default `carto-customers` project. @@ -927,17 +984,17 @@ Changed #### Module data -Feature +New - Add DATAOBS_SAMPLES procedure. -Changed +Improvement - DATAOBS_SUBSCRIPTIONS and DATAOBS_SUBSCRIPTION_VARIABLES take single `source` parameter with default `carto-customers` project. Views with `sub_` prefix are supported. #### Module routing -Feature +New - Create routing module. - Add GENERATE_NETWORK function. @@ -954,7 +1011,7 @@ Feature #### Module data -Fixed +Fix - Fix DATAOBS_SUBSCRIPTIONS and DATAOBS_SUBSCRIPTION_VARIABLES not working with subscriptions in which the provider id contained underscores. @@ -963,7 +1020,7 @@ Fixed #### Module tiler -Feature +New - Add new automatic tileset generator `CREATE_TILESET`. @@ -971,12 +1028,12 @@ Feature #### Module data -Feature +New - Add DATAOBS_SUBSCRIPTIONS procedure. - Add DATAOBS_SUBSCRIPTION_VARIABLES procedure. -Changed +Improvement - Change spatial catalog structure using two tables. - Improve enrichment procedures: @@ -989,7 +1046,7 @@ Changed #### Module data -Feature +New - Create data module. - Add VERSION function. @@ -1006,7 +1063,7 @@ Feature #### Module s2 -Feature +New - Add TOKEN_FROMID function. - Add ID_FROMTOKEN function. @@ -1017,7 +1074,7 @@ Feature #### Module tiler -Changed +Improvement - Use semantic versioning. - Add `features_count` and `geometry` labels to the tilesets. @@ -1026,7 +1083,7 @@ Changed #### Module clustering -Changed +Improvement - Change ST_CLUSTERKMEANS returning cluster and geom @@ -1034,7 +1091,7 @@ Changed #### Module accessors -Feature +New - Create accessors module. - Add ST_ENVELOPE function. @@ -1042,7 +1099,7 @@ Feature #### Module processing -Feature +New - Create processing module. - Add ST_VORONOIPOLYGONS function. @@ -1054,7 +1111,7 @@ Feature #### Module transformations -Feature +New - Add ST_CONCAVEHULL function. @@ -1062,7 +1119,7 @@ Feature #### Module clustering -Feature +New - Create clustering module. - Add ST_CLUSTERKMEANS function. @@ -1072,14 +1129,14 @@ Feature #### Module constructors -Feature +New - Add ST_BEZIERSPLINE function. - Add ST_MAKEELLIPSE function. #### Module measurements -Feature +New - Create measurements module. - Add ST_ANGLE function. @@ -1088,7 +1145,7 @@ Feature #### Module transformations -Feature +New - Rename module to transformations. - Add ST_CENTERMEAN function. @@ -1108,7 +1165,7 @@ Removed #### Module constructors -Feature +New - Create constructors module. - Add ST_MAKEENVELOPE function. @@ -1119,7 +1176,7 @@ Feature #### Module transformations -Feature +New - Create transformation module. - Add ST_BUFFER function. @@ -1129,7 +1186,7 @@ Feature #### Module random -Changed +Improvement - ST_GENERATEPOINTS now uses a spherically uniform distribution. Previously used to by uniform on projection. @@ -1137,23 +1194,23 @@ Changed #### Module h3 -Changed +Improvement - Use hexadecimal as default type instead of int for h3 indexes. -Fixed +Fix - Fix ST_BOUNDARY generating error when not able to parse geometry. #### Module placekey -Changed +Improvement - Placekey conversions works with hexadecimal h3 indexes instead of int. #### Module random -Feature +New - Create random module. - Add ST_GENERATEPOINTS function. @@ -1163,7 +1220,7 @@ Feature #### Module h3 -Feature +New - Create h3 module. - Add ST_ASH3 function. @@ -1183,7 +1240,7 @@ Feature #### Module placekey -Feature +New - Create placekey module. - Add H3_ASPLACEKEY function. @@ -1193,7 +1250,7 @@ Feature #### Module s2 -Feature +New - Create s2 module. - Add ID_FROMHILBERTQUADKEY function. @@ -1205,7 +1262,7 @@ Feature #### Module tiler -Feature +New - Add QUOTA procedure in order to display the monthly quota and quota available. - Implement support for layer name. @@ -1215,7 +1272,7 @@ Feature - Add new `metadada` input parameter to set the properties `name`, `description`, `legends` and also `extra_metadata`. - Add `zoom_min_column` and `zoom_max_column` to control the zoom level at which particular features are visible. -Changed +Improvement - Change the default and max partition limit to 3999. - Make aggregation tilesets to use the quadkey module UDF LONGLAT_ASQUADINTLIST_RESOLUTION in order to generate quadkey aggregations. @@ -1224,7 +1281,7 @@ Removed - Removed `zoom_step` option. -Fixed +Fix - Fix bug that left empty the generator options in the metadata cell. - Fix CREATE_SIMPLE_TILESET failing with single column tables. @@ -1234,12 +1291,12 @@ Fixed #### Module tiler -Feature +New - Add tileset type to metadata and BigQuery labels. - Add the prefix "aggregation_" to aggregation options: type, resolution and placement. -Changed +Improvement - Moved to the Advanced Spatial Extension. - Change "max_tile_size_strategy" default to "throw_error", matching the other defaults and improve the exception error message in CreatePointAggregationTileset. diff --git a/app/content/analytics-toolbox-bigquery/sql-reference/h3.md b/app/content/analytics-toolbox-bigquery/sql-reference/h3.md index e034d47fe..22906b1d7 100644 --- a/app/content/analytics-toolbox-bigquery/sql-reference/h3.md +++ b/app/content/analytics-toolbox-bigquery/sql-reference/h3.md @@ -368,9 +368,13 @@ carto.H3_POLYFILL(geography, resolution) **Description** -Returns an array with all the H3 cell indexes **with centers** contained in a given polygon. It will return `null` on error (invalid geography type or resolution out of bounds). +Returns an array with all the H3 cell indexes **with centers** contained in a given polygon. It will return `null` on error (invalid geography type or resolution out of bounds). In case of lines, it will return the H3 cell indexes intersecting those lines. For a given point, it will return the H3 index of cell in which that point is contained. -* `geography`: `GEOGRAPHY` **polygon** or **multipolygon** representing the area to cover. +{{% bannerNote type="note" title="warning"%}} +Lines polyfill is calculated by approximating S2 cells to H3 cells, in some cases some cells might be missing. +{{%/ bannerNote %}} + +* `geography`: `GEOGRAPHY` representing the area to cover. * `resolution`: `INT64` number between 0 and 15 with the [H3 resolution](https://h3geo.org/docs/core-library/restable). **Return type** diff --git a/app/content/analytics-toolbox-bigquery/sql-reference/overview.md b/app/content/analytics-toolbox-bigquery/sql-reference/overview.md index fd829815d..04740870d 100644 --- a/app/content/analytics-toolbox-bigquery/sql-reference/overview.md +++ b/app/content/analytics-toolbox-bigquery/sql-reference/overview.md @@ -22,9 +22,9 @@ The CARTO Analytics Toolbox's functions are organized in modules based on the fu | processing | core || | quadbin | core || | random | core || -| retail | advanced || +| retail | advanced || | routing | advanced || -| s2 | core || +| s2 | core || | statistics | advanced || | tiler | advanced || | transformations | core || diff --git a/app/content/analytics-toolbox-bigquery/sql-reference/processing.md b/app/content/analytics-toolbox-bigquery/sql-reference/processing.md index 3d9a58220..645d423c4 100644 --- a/app/content/analytics-toolbox-bigquery/sql-reference/processing.md +++ b/app/content/analytics-toolbox-bigquery/sql-reference/processing.md @@ -69,9 +69,11 @@ SELECT `carto-os`.carto.ST_DELAUNAYLINES( ``` {{% bannerNote type="note" title="ADDITIONAL EXAMPLES"%}} + * [A NYC subway connection graph using Delaunay triangulation](/analytics-toolbox-bigquery/examples/a-nyc-subway-connection-graph-using-delaunay-triangulation/) {{%/ bannerNote %}} + ### ST_DELAUNAYPOLYGONS {{% bannerNote type="code" %}} @@ -132,7 +134,6 @@ SELECT `carto-os`.carto.ST_DELAUNAYPOLYGONS( ``` - ### ST_POLYGONIZE {{% bannerNote type="code" %}} diff --git a/app/content/analytics-toolbox-bigquery/sql-reference/retail.md b/app/content/analytics-toolbox-bigquery/sql-reference/retail.md index 61efde8c9..56a345f26 100644 --- a/app/content/analytics-toolbox-bigquery/sql-reference/retail.md +++ b/app/content/analytics-toolbox-bigquery/sql-reference/retail.md @@ -116,8 +116,7 @@ This procedure is the second step of the Revenue Prediction analysis workflow. I * MAX_ITERATIONS: 50 * DATA_SPLIT_METHOD: 'NO_SPLIT' - This parameter allows using other options compatible with the model used. Models currently supported are [LINEAR_REG](https://cloud.google.com/bigquery-ml/docs/reference/standard-sql/bigqueryml-syntax-create-glm) and [BOOSTED_TREE_REGRESSOR](https://cloud.google.com/bigquery-ml/docs/reference/standard-sql/bigqueryml-syntax-create-boosted-tree). Check the [model documentation](https://cloud.google.com/bigquery-ml/docs/reference/standard-sql/bigqueryml-syntax-create#model_option_list) for more information. Please note that `BOOSTED_TREE_REGRESSOR` is available only on certain regions, as detailed here. - + This parameter allows using other options compatible with the model used. Models currently supported are [LINEAR_REG](https://cloud.google.com/bigquery-ml/docs/reference/standard-sql/bigqueryml-syntax-create-glm) and [BOOSTED_TREE_REGRESSOR](https://cloud.google.com/bigquery-ml/docs/reference/standard-sql/bigqueryml-syntax-create-boosted-tree). Check the [model documentation](https://cloud.google.com/bigquery-ml/docs/reference/standard-sql/bigqueryml-syntax-create#model_option_list) for more information. * `output_prefix`: `STRING` destination prefix for the output tables. It must contain the project, dataset and prefix. For example `..`. **Output** @@ -524,4 +523,78 @@ CALL `carto-un`.carto.CANNIBALIZATION_OVERLAP( {{%/ bannerNote %}} +### FIND_TWIN_AREAS_WEIGHTED + +{{% bannerNote type="code" %}} +carto.FIND_TWIN_AREAS_WEIGHTED(origin_query, target_query, index_column, weights, max_results, output_prefix) +{{%/ bannerNote %}} + +**Description** + +Procedure to obtain the twin areas for a given origin location in a target area. The function is similar to the FIND_TWIN_AREAS(#find_twin_areas) where the full description of the method, based on Principal Component Analysis (PCA), can be found [here](https://carto.com/blog/spatial-data-science-site-planning). Herein, no PCA is performed, but the user has the capability to specify weights for the features and check the similarities between origin and target area. The sum of weights must be less than or equal to 1. Not all them need to be defined. The undefined features are set to the remaining value divided by their number to reach 1. In the case where weights are provided, then no PCA takes place, and the features are standardized. + +The output twin areas are those of the target area considered to be the most similar to the origin location, based on the values of a set of variables. Only variables with numerical values are supported. Both origin and target areas should be provided in grid format (h3, or quadbin) of the same resolution. We recommend using the [carto.GRIDIFY_ENRICH](../#gridify_enrich) procedure to prepare the data in the format expected by this procedure. + +**Input** + +* `origin_query`: `STRING` query to provide the origin cell (`index` column) and its associated data columns. No NULL values should be contained in any of the data columns provided. The cell can be an h3, or a quadbin index. For quadbin, the value should be cast to `STRING` (`CAST(index AS STRING)`). Example origin queries are: + + ```sql + -- When selecting the origin cell from a dataset of gridified data + SELECT * FROM `..` + WHERE index_column = + ``` + + ```sql + -- When the input H3 cell ID is inferred from a (longitude, latitude) pair + SELECT * FROM `..` + WHERE ST_INTERSECTS(`carto-un`.H3_BOUNDARY(index_column), ST_GEOGPOINT(, )) + ``` + + ```sql + -- When the input quadbin cell ID is inferred from a (longitude, latitude) pair + SELECT * FROM `..` + WHERE ST_INTERSECTS(`carto-un`.carto.QUADBIN_BOUNDARY(index_column), ST_GEOGPOINT(, )) + ``` + + ```sql + -- When the cell ID is a quadbin and requires to be cast + SELECT * EXCEPT(index_column), CAST(index_column AS STRING) + FROM `..` + ``` + +* `target_query`: STRING query to provide the target area grid cells (`index` column) and their associated data columns, e.g. `SELECT * FROM ..`. The data columns should be similar to those provided in the `origin_query`, otherwise the procedure will fail. Grid cells with any NULL values will be excluded from the analysis. +* `index_column`: `STRING` name of the index column for both the `origin_query` and the `target_query`. +* `weights`: `ARRAY>` the weights on the features. If set to `NULL`, then all features are treated equally. This parameter is considered only if the length of weights is greater or equal than one. The sum of weights must be less than or equal to 1. If less weights than the number of features are provided, then for the undefined features, the remaining 1 - sum(weights) is distributed evenly. +* `max_results`: `INT64` of the maximum number of twin areas returned. If set to `NULL`, all target cells are returned. +* `output_prefix`: `STRING` destination and prefix for the output tables. It must contain the project, dataset and prefix: `..`. + +**Output** + +The procedure outputs the following: + +* Twin area model, named `.._model`. Please note that the model computation only depends on the `target_query` and therefore the same model can be used if the procedure is re-run for a different `origin_query`. To allow for this scenario in which the model is reused, if the output model already exists, it won't be recomputed. To avoid this behavior, simply choose a different `` in the `output_prefix` parameter. + +* Results table, named `..__results`, containing in each row the index of the target cells (`index_column`) and its associated `similarity_score` and `similarity_skill_score`. The `similarity_score` corresponds to the distance between the origin and target cell taking into account the user defined weights; the `similarity_skill_score` for a given target cell `*t*` is computed as `1 - similarity_score(*t*) / similarity_score(<*t*>)`, where `<*t*>` is the average target cell, computed by averaging each feature for all the target cells. This `similarity_skill_score` represents a relative measure: the score will be positive if and only if the target cell is more similar to the origin than the mean vector data, with a score of 1 meaning perfect matching or zero distance. Therefore, a target cell with a larger score will be more similar to the origin under this scoring rule. + +{{% customSelector %}} +**Example** +{{%/ customSelector %}} + +```sql +CALL `carto-un`.carto.FIND_TWIN_AREAS_WEIGHTED( + -- Input queries + '''SELECT * FROM `cartobq.docs.twin_areas_origin_enriched_quadbin` LIMIT 1''', + '''SELECT * FROM `cartobq.docs.twin_areas_target_enriched_quadbin`''', + -- Twin areas model inputs + 'quadbin', + NULL, + NULL, + 'my-project.my-dataset.my-prefix' +); +-- Table `.._{ID}_results` will be created +-- with the column: quadbin, similarity_score, similarity_skill_score +``` + + {{% euFlagFunding %}} \ No newline at end of file diff --git a/app/content/analytics-toolbox-bigquery/sql-reference/s2.md b/app/content/analytics-toolbox-bigquery/sql-reference/s2.md index f3f5f00d5..8843e294c 100644 --- a/app/content/analytics-toolbox-bigquery/sql-reference/s2.md +++ b/app/content/analytics-toolbox-bigquery/sql-reference/s2.md @@ -198,6 +198,62 @@ SELECT `carto-os`.carto.S2_FROMUINT64REPR('9926595690882924544'); ``` +### S2_RESOLUTION + +{{% bannerNote type="code" %}} +carto.S2_RESOLUTION(index) +{{%/ bannerNote %}} + +**Description** + +Returns the S2 cell resolution as an integer. + +* `index`: `STRING` The S2 cell index. + +**Return type** + +`INT64` + +{{% customSelector %}} +**Example** +{{%/ customSelector %}} + +```sql +SELECT `carto-os`.carto.S2_RESOLUTION(-6432928348669739008); +-- 11 +``` + + +### S2_TOCHILDREN + +{{% bannerNote type="code" %}} +carto.S2_TOCHILDREN(index, resolution) +{{%/ bannerNote %}} + +**Description** + +Returns an array with the S2 indexes of the children/descendents of the given hexagon at the given resolution. + +* `index`: `STRING` The S2 cell index. +* `resolution`: `INT64` number between 0 and 30 with the [S2 resolution](https://S2geo.org/docs/core-library/restable). + +**Return type** + +`ARRAY` + +{{% customSelector %}} +**Example** +{{%/ customSelector %}} + +```sql +SELECT `carto-os`.carto.S2_TOCHILDREN(-6432928348669739008, 12); +-- 6432928554828169216 +-- 6432928417389215744 +-- 6432928279950262272 +-- 6432928142511308800 +``` + + ### S2_TOHILBERTQUADKEY {{% bannerNote type="code" %}} diff --git a/app/content/analytics-toolbox-bigquery/sql-reference/tiler.md b/app/content/analytics-toolbox-bigquery/sql-reference/tiler.md index 11a914491..048f6d8a1 100644 --- a/app/content/analytics-toolbox-bigquery/sql-reference/tiler.md +++ b/app/content/analytics-toolbox-bigquery/sql-reference/tiler.md @@ -19,15 +19,15 @@ Visit the [Overview](/analytics-toolbox-bigquery/overview/tilesets) section to l ### CREATE_POINT_AGGREGATION_TILESET {{% bannerNote type="code" %}} -carto.CREATE_POINT_AGGREGATION_TILESET(source_table, target_table, options) +carto.CREATE_POINT_AGGREGATION_TILESET(input, output_table, options) {{%/ bannerNote %}} **Description** Generates a point aggregation tileset. -* `source_table`: `STRING` that can either be a quoted qualified table name (e.g. ``project-id.dataset-id.table-name``) or a full query wrapped in parentheses (e.g.(Select * FROM \`project-id.dataset-id.table-name`)). -* `target_table`: Where the resulting table will be stored. It must be a `STRING` of the form ``project-id.dataset-id.table-name``. The `project-id` can be omitted (in which case the default one will be used). The dataset must exist and the caller needs to have permissions to create a new table on it. The process will fail if the target table already exists. +* `input`: `STRING` that can either be a quoted qualified table name (e.g. ``project-id.dataset-id.table-name``) or a full query wrapped in parentheses (e.g.(Select * FROM \`project-id.dataset-id.table-name`)). +* `output_table`: Where the resulting table will be stored. It must be a `STRING` of the form ``project-id.dataset-id.table-name``. The `project-id` can be omitted (in which case the default one will be used). The dataset must exist and the caller needs to have permissions to create a new table on it. The process will fail if the target table already exists. * `options`: `STRING` containing a valid JSON with the different options. Valid options are described the table below. | Option | Description | @@ -37,10 +37,10 @@ Generates a point aggregation tileset. |`zoom_max`| Default: `0`. A `NUMBER` that defines the maximum zoom level for tiles. Any zoom level over this level won't be generated.| |`zoom_min_column`| Default: `NULL`. It is the column that each row could have to modify its starting zoom. It can be NULL (then `zoom_min` will be used). When provided, if its value is greater than `zoom_min`, it will take precedence and be used as the actual minimum.| |`zoom_max_column`| Default: `NULL`. It is the column that each row could have to modify its end zoom level. It can be NULL (then zoom_max will be used). When provided, if its value is lower than `zoom_max`, it will be taken as the real maximum zoom level.| -|`target_partitions`| Default: `4000`. Max: `4000`. A `NUMBER` that defines the **maximum** number of partitions to be used in the target table. The partition system, which uses a column named `carto_partition`, divides the available partitions first by zoom level and spatial locality to minimize the cost of tile read requests in web maps. Beware that this does not necessarily mean that all the partitions will be used, as a sparse dataset will leave some of these partitions unused. If you are using [BigQuery BI Engine](https://cloud.google.com/bi-engine/docs/overview) consider that it supports a maximum of 500 partitions per table.| +|`target_partitions`| Default: `3999`. Max: `3999`. A `NUMBER` that defines the **maximum** number of partitions to be used in the target table. The partition system, which uses a column named `carto_partition`, divides the available partitions first by zoom level and spatial locality to minimize the cost of tile read requests in web maps. Beware that this does not necessarily mean that all the partitions will be used, as a sparse dataset will leave some of these partitions unused. If you are using [BigQuery BI Engine](https://cloud.google.com/bi-engine/docs/overview) consider that it supports a maximum of 500 partitions per table.| |`target_tilestats`| Default: `true`. A `BOOLEAN` to determine whether to include statistics of the properties in the metadata. These statistics are based on [mapbox-tilestats](https://github.com/mapbox/mapbox-geostats) and depend on the property type:
  • Number: MIN, MAX, AVG, SUM and quantiles (from 3 to 20 breaks).
  • String / Boolean: List of the top 10 most common values and their count.
For aggregation tilesets, these statistics refer to the cells at the maximum zoom generated.| |`tile_extent`| Default: `4096`. A `NUMBER` defining the extent of the tile in integer coordinates as defined by the MVT spec. -|`tile_buffer`| Default: `0`. A `NUMBER` defining the additional buffer added around the tiles in extent units, which is useful to facilitate geometry stitching across tiles in the renderers. In aggregation tilesets, this property is currently not available and always `0` as no geometries go across tile boundaries.| +|`tile_buffer`| Default: `0` (disabled). A `NUMBER` defining the additional buffer added around the tiles in extent units, which is useful to facilitate geometry stitching across tiles in the renderers. In aggregation tilesets, this property is currently not available and always `0` as no geometries go across tile boundaries.| |`max_tile_size_kb`| Default: `1024`. A `NUMBER` specifying the approximate maximum size for a tile. |`max_tile_size_strategy`| Default: `"throw_error"`. A `STRING` that determines what to do when the maximum size of a tile is reached while it is still processing data. There are three options available:
  • `"return_null"`: The process will return a NULL buffer. This might appear as empty in the map.
  • `"throw_error"`: The process will throw an error cancelling the aggregation, so no tiles or table will be generated.
  • `"drop_fraction_as_needed"`: For every zoom level, this process will drop a consistent fraction of features in every tile to make sure all generated tiles are below the maximum size set in `max_tile_size_kb.` This could lead to features disappearing at different zoom levels. Features will be dropped according to the `tile_feature_order` parameter, which becomes mandatory when using this strategy.
| |`max_tile_features`| Default: `0` (disabled). A `NUMBER` that sets the maximum number of features a tile might contain. This limit is applied before `max_tile_size_kb`, i.e., the tiler will first drop as many features as needed to keep this amount, and then continue with the size limits (if required). To configure in which order features are kept, use in conjunction with `tile_feature_order`.| @@ -51,6 +51,10 @@ Generates a point aggregation tileset. |`metadata`| Default: `{}`. A JSON object to specify the associated metadata of the tileset. Use this to set the name, description and legend to be included in the [TileJSON](https://github.com/mapbox/tilejson-spec/tree/master/2.2.0). Other fields will be included in the object extra_metadata.| |`properties`| Default: `{}`. A JSON object that defines the properties that will be included associated with each cell feature. Each `property` is defined by its name, type (Number, Boolean, String, etc.) and formula to be applied to the values of the points that fall under the cell. This formula can be any SQL formula that uses an [aggregate function](https://cloud.google.com/bigquery/docs/reference/standard-sql/functions-and-operators#aggregate_functions) supported by BigQuery and returns the expected type. Note that every property different from Number will be casted to String.| +{{% bannerNote type="note" title="warning"%}} +There are some cases where [flat-rates](https://cloud.google.com/bigquery/pricing#flat-rate_analysis_pricing) is the only option to create a tileset. Some tables containing huge geographies might trigger a `Query exceeded resource limits` error because of the high CPU usage. +{{%/ bannerNote %}} + {{% customSelector %}} **Examples** {{%/ customSelector %}} @@ -121,7 +125,7 @@ In the example above, for all features we would get a property `"new_column_name ### CREATE_SIMPLE_TILESET {{% bannerNote type="code" %}} -carto.CREATE_SIMPLE_TILESET(source_table, target_table, options) +carto.CREATE_SIMPLE_TILESET(input, output_table, options) {{%/ bannerNote %}} {{% bannerNote type="warning" title="warning"%}} @@ -132,22 +136,22 @@ This procedure is the older version of `carto.CREATE_TILESET` and you can achiev Generates a simple tileset. -* `source_table`: `STRING` that can either be a quoted qualified table name (e.g. \`project-id.dataset-id.table-name\`) or a full query contained by parentheses (e.g.(SELECT * FROM \`project-id.dataset-id.table-name\`)). -* `target_table`: Where the resulting table will be stored. It must be a `STRING` of the form \`projectI-id.dataset-id.table-name\`. The `project-id` can be omitted (in which case the default one will be used). The dataset must exist and the caller needs to have permissions to create a new table on it. The process will fail if the target table already exists. +* `input`: `STRING` that can either be a quoted qualified table name (e.g. \`project-id.dataset-id.table-name\`) or a full query contained by parentheses (e.g.(SELECT * FROM \`project-id.dataset-id.table-name\`)). +* `output_table`: Where the resulting table will be stored. It must be a `STRING` of the form \`projectI-id.dataset-id.table-name\`. The `project-id` can be omitted (in which case the default one will be used). The dataset must exist and the caller needs to have permissions to create a new table on it. The process will fail if the target table already exists. * `options`: `STRING` containing a valid JSON with the different options. Valid options are described the table below. {{% bannerNote type="note" title="tip"%}} -To avoid issues in the process when building the queries that will be executed internally against BigQuery, it is highly recommended to use [raw strings](https://cloud.google.com/bigquery/docs/reference/standard-sql/lexical#string_and_bytes_literals) when passing long queries in the `source_table` that might contain special characters. +To avoid issues in the process when building the queries that will be executed internally against BigQuery, it is highly recommended to use [raw strings](https://cloud.google.com/bigquery/docs/reference/standard-sql/lexical#string_and_bytes_literals) when passing long queries in the `input` that might contain special characters. {{%/ bannerNote %}} | Option | Description | | :----- | :------ | |`geom_column`| Default: `"geom"`. A `STRING` that marks the name of the geography column that will be used. It must be of type `GEOGRAPHY`. | |`zoom_min`| Default: `0`. A `NUMBER` that defines the minimum zoom level for tiles. Any zoom level under this level won't be generated.| -|`zoom_max`| Default: `0`. A `NUMBER` that defines the minimum zoom level for tiles. Any zoom level over this level won't be generated.| +|`zoom_max`| Default: `0`. A `NUMBER` that defines the maximum zoom level for tiles. Any zoom level over this level won't be generated.| |`zoom_min_column`| Default: `NULL`. It is the column that each row could have to modify its starting zoom. It can be NULL (then zoom_min will be used). When provided, if its value is greater than `zoom_min`, it will take precedence and be used as the actual minimum.| |`zoom_max_column`| Default: `NULL`. It is the column that each row could have to modify its end zoom level. It can be NULL (then zoom_max will be used). When provided, if its value is lower than `zoom_max`, it will be taken as the real maximum zoom level.| -|`target_partitions`| Default: `4000`. Max: `4000`. A `NUMBER` that defines the **maximum** amount of partitions to be used in the target table. The partition system, which uses a column named `carto_partition`, divides the available partitions first by zoom level and spatial locality to minimize the cost of tile read requests in web maps. Beware that this does not necessarily mean that all the partitions will be used, as a sparse dataset will leave some of these partitions unused. If you are using [BigQuery BI Engine](https://cloud.google.com/bi-engine/docs/overview) consider that it supports a maximum of 500 partitions per table.| +|`target_partitions`| Default: `3999`. Max: `3999`. A `NUMBER` that defines the **maximum** amount of partitions to be used in the target table. The partition system, which uses a column named `carto_partition`, divides the available partitions first by zoom level and spatial locality to minimize the cost of tile read requests in web maps. Beware that this does not necessarily mean that all the partitions will be used, as a sparse dataset will leave some of these partitions unused. If you are using [BigQuery BI Engine](https://cloud.google.com/bi-engine/docs/overview) consider that it supports a maximum of 500 partitions per table.| |`target_tilestats`| Default: `true`. A `BOOLEAN` to determine whether to include statistics of the properties in the metadata. These statistics are based on [mapbox-tilestats](https://github.com/mapbox/mapbox-geostats) and depend on the property type:
  • Number: MIN, MAX, AVG, SUM and quantiles (from 3 to 20 breaks).
  • String / Boolean: List of the top 10 most common values and their count.
In Simple Tilesets, these statistics are based on the source data.| |`tile_extent`| Default: `4096`. A `NUMBER` defining the extent of the tile in integer coordinates as defined by the [MVT specification](https://github.com/mapbox/vector-tile-spec). |`tile_buffer`| Default: `16`. A `NUMBER` defining the additional buffer added around the tiles in extent units, which is useful to facilitate geometry stitching across tiles in the renderers.| @@ -164,6 +168,10 @@ To avoid issues in the process when building the queries that will be executed i If `drop_fraction_as_needed` is used, a `fraction_dropped_per_zoom` property will be included in the TileJSON, containing an estimate of the percentage of the features that have been dropped per zoom level. Please bear in mind that the exact percentages can be up to 5% higher. {{%/ bannerNote %}} +{{% bannerNote type="note" title="warning"%}} +There are some cases where [flat-rates](https://cloud.google.com/bigquery/pricing#flat-rate_analysis_pricing) is the only option to create a tileset. Some tables containing huge geographies might trigger a `Query exceeded resource limits` error because of the high CPU usage. +{{%/ bannerNote %}} + {{% customSelector %}} **Examples** {{%/ customSelector %}} @@ -237,7 +245,7 @@ R''' ### CREATE_SPATIAL_INDEX_TILESET {{% bannerNote type="code" %}} -carto.CREATE_SPATIAL_INDEX_TILESET(source_table, target_table, options) +carto.CREATE_SPATIAL_INDEX_TILESET(input, output_table, options) {{%/ bannerNote %}} **Description** @@ -246,13 +254,13 @@ Creates a tileset that uses a spatial index (H3 and QUADBIN are currently suppor Aggregated data is computed for all levels between `resolution_min` and `resolution_max`. For each resolution level, all tiles for the area covered by the source table are added, with data aggregated at level `resolution + aggregation resolution`. -* `source_table`: `STRING` that can either be a quoted qualified table name (e.g. \`project-id.dataset-id.table-name\`) or a full query contained by parentheses (e.g.(SELECT * FROM \`project-id.dataset-id.table-name\`)). -* `target_table`: Where the resulting table will be stored. It must be a `STRING` of the form \`project-id.dataset-id.table-name\`. The `project-id` can be omitted (in which case the default one will be used). The dataset must exist and the caller needs to have permissions to create a new table on it. The process will fail if the target table already exists. +* `input`: `STRING` that can either be a quoted qualified table name (e.g. \`project-id.dataset-id.table-name\`) or a full query contained by parentheses (e.g.(SELECT * FROM \`project-id.dataset-id.table-name\`)). +* `output_table`: Where the resulting table will be stored. It must be a `STRING` of the form \`project-id.dataset-id.table-name\`. The `project-id` can be omitted (in which case the default one will be used). The dataset must exist and the caller needs to have permissions to create a new table on it. The process will fail if the target table already exists. * `options`: `STRING` containing a valid JSON with the different options. Valid options are described the table below. | Option | Description | | :----- | :------ | -|`resolution_min`| Default: `2`. A `NUMBER` that defines the minimum resolution level for tiles. Any resolution level under this level won't be generated.| -|`resolution_max`| Default: `15`. A `NUMBER` that defines the maximum resolution level for tiles. Any resolution level over this level won't be generated.| +|`resolution_min`| Default: `0`. A `NUMBER` that defines the minimum resolution level for tiles. Any resolution level under this level won't be generated.| +|`resolution_max`| Default: `0`. A `NUMBER` that defines the maximum resolution level for tiles. Any resolution level over this level won't be generated.| |`spatial_index_column`| A `STRING` in the format `spatial_index_type:column_name`, with `spatial_index_type` being the type of spatial index used in the input table (can be `quadbin` or `h3`), and `column_name` being the name of the column in that input table that contains the tile ids. Notice that the spatial index name is case-sensitive. The type of spatial index also defines the type used in the output table, which will be QUADBIN (for spatial index type `quadbin`) or H3 (for spatial index type `h3`).| |`resolution`| A `NUMBER` defining the resolution of the tiles in the input table.| |`aggregation_resolution`| Defaults: `6` for QUADBIN tilesets, `4` for H3 tilesets. A `NUMBER` defining the resolution to use when aggregating data at each resolution level. For a given `resolution`, data is aggregated at `resolution_level + aggregation resolution`.| @@ -263,6 +271,10 @@ Aggregated data is computed for all levels between `resolution_min` and `resolut Any option left as `NULL` will take its default value if available. {{%/ bannerNote %}} +{{% bannerNote type="note" title="warning"%}} +There are some cases where [flat-rates](https://cloud.google.com/bigquery/pricing#flat-rate_analysis_pricing) is the only option to create a tileset. Some tables containing huge geographies might trigger a `Query exceeded resource limits` error because of the high CPU usage. +{{%/ bannerNote %}} + {{% customSelector %}} **Examples** {{%/ customSelector %}} @@ -308,27 +320,27 @@ CALL carto.CREATE_SPATIAL_INDEX_TILESET( ``` {{% bannerNote type="note" title="warning"%}} -In case of `source_table` being set as a query, it should be taken into account that [CTEs](https://cloud.google.com/bigquery/docs/reference/standard-sql/query-syntax#simple_cte) are not allowed. Also, the query should be as simple as possible in order to avoid BigQuery limitations about the complexity of the final query. +In case of `input` being set as a query, it should be taken into account that [CTEs](https://cloud.google.com/bigquery/docs/reference/standard-sql/query-syntax#simple_cte) are not allowed. Also, the query should be as simple as possible in order to avoid BigQuery limitations about the complexity of the final query. {{%/ bannerNote %}} {{% bannerNote type="note" title="ADDITIONAL EXAMPLES"%}} -* [Creating spatial index tilesets](analytics-toolbox-bigquery/examples/creating-spatial-index-tilesets/) +* [Creating spatial index tilesets](/analytics-toolbox-bigquery/examples/creating-spatial-index-tilesets/) {{%/ bannerNote %}} ### CREATE_TILESET {{% bannerNote type="code" %}} -carto.CREATE_TILESET(source_table, target_table, options) +carto.CREATE_TILESET(input, output_table, options) {{%/ bannerNote %}} **Description** -Creates a simple tileset. It differs from `carto.CREATE_SIMPLE_TILESET` in that the procedure performs a previous analysis in order to find automatically the right options for the tileset. It is done by extracting all the properties to be included within the tileset and sampling the data in order to avoid BigQuery limitations. Therefore, only `source_table` and `target_table` are mandatory and `options` can be set to `NULL`. +Creates a simple tileset. It differs from `carto.CREATE_SIMPLE_TILESET` in that the procedure performs a previous analysis in order to find automatically the right options for the tileset. It is done by extracting all the properties to be included within the tileset and sampling the data in order to avoid BigQuery limitations. Therefore, only `input` and `output_table` are mandatory and `options` can be set to `NULL`. -* `source_table`: `STRING` that can either be a quoted qualified table name (e.g. \`project-id.dataset-id.table-name\`) or a full query contained by parentheses (e.g.(SELECT * FROM \`project-id.dataset-id.table-name\`)). -* `target_table`: Where the resulting table will be stored. It must be a `STRING` of the form \`project-id.dataset-id.table-name\`. The `project-id` can be omitted (in which case the default one will be used). The dataset must exist and the caller needs to have permissions to create a new table on it. The process will fail if the target table already exists. +* `input`: `STRING` that can either be a quoted qualified table name (e.g. \`project-id.dataset-id.table-name\`) or a full query contained by parentheses (e.g.(SELECT * FROM \`project-id.dataset-id.table-name\`)). +* `output_table`: Where the resulting table will be stored. It must be a `STRING` of the form \`project-id.dataset-id.table-name\`. The `project-id` can be omitted (in which case the default one will be used). The dataset must exist and the caller needs to have permissions to create a new table on it. The process will fail if the target table already exists. * `options`: `STRUCT|NULL` containing the different options. Valid options are described in the table below. | Option | Description | @@ -337,7 +349,7 @@ Creates a simple tileset. It differs from `carto.CREATE_SIMPLE_TILESET` in that |`description`| Default: `""`. A `STRING` that contains a description for the tileset to be included in the [TileJSON](https://github.com/mapbox/tilejson-spec/tree/master/2.2.0).| |`legend`| Default: `""`. A `STRING` that contains a legend for the tileset to be included in the [TileJSON](https://github.com/mapbox/tilejson-spec/tree/master/2.2.0).| |`zoom_min`| Default: `0` for `POINTS` datasets and `2` for `POLYGON/LINESTRING` datasets. A `NUMBER` that defines the minimum zoom level for tiles. Any zoom level under this level won't be generated.| -|`zoom_max`| Default: `15`. A `NUMBER` that defines the minimum zoom level for tiles. Any zoom level over this level won't be generated.| +|`zoom_max`| Default: `0`. A `NUMBER` that defines the maximum zoom level for tiles. Any zoom level over this level won't be generated.| |`geom_column_name`| Default: `"geom"`. A `STRING` that contains the name of the geography column that will be used. It must be of type `GEOGRAPHY`. | |`zoom_min_column`| Default: `NULL`. It is the column that each row could have to modify its starting zoom. It can be NULL (then `zoom_min` will be used). It must be a positive number between `zoom_min` and `zoom_max`.| |`zoom_max_column`| Default: `NULL`. It is the column that each row could have to modify its end zoom level. It can be NULL (then `zoom_max` will be used). It must be a positive number between `zoom_min` and `zoom_max`.| @@ -358,6 +370,10 @@ A `fraction_dropped_per_zoom` property will be included in the TileJSON, contain It should be taken into account that `CREATE_TILESET` will not be executed for any level that reaches more than 10 millions tiles. This threshold is set in order to avoid some BigQuery limitations. This could occur if the input dataset is very sparse or `zoom_max` is quite large. {{%/ bannerNote %}} +{{% bannerNote type="note" title="warning"%}} +There are some cases where [flat-rates](https://cloud.google.com/bigquery/pricing#flat-rate_analysis_pricing) is the only option to create a tileset. Some tables containing huge geographies might trigger a `Query exceeded resource limits` error because of the high CPU usage. +{{%/ bannerNote %}} + {{% customSelector %}} **Examples** {{%/ customSelector %}} @@ -407,7 +423,7 @@ CALL `carto-un`.carto.CREATE_TILESET( ``` {{% bannerNote type="note" title="warning"%}} -In case of `source_table` is set as a query, it should be taken into account that [CTEs](https://cloud.google.com/bigquery/docs/reference/standard-sql/query-syntax#simple_cte) are not allowed. Also, the query should be as simple as possible in order to avoid BigQuery limitations about the complexity of the final query. +In case of `input` is set as a query, it should be taken into account that [CTEs](https://cloud.google.com/bigquery/docs/reference/standard-sql/query-syntax#simple_cte) are not allowed. Also, the query should be as simple as possible in order to avoid BigQuery limitations about the complexity of the final query. {{%/ bannerNote %}} {{% bannerNote type="note" title="ADDITIONAL EXAMPLES"%}} diff --git a/app/content/analytics-toolbox-databricks/reference/accessors.md b/app/content/analytics-toolbox-databricks/reference/accessors.md index 3ebee8014..2c49ca4db 100644 --- a/app/content/analytics-toolbox-databricks/reference/accessors.md +++ b/app/content/analytics-toolbox-databricks/reference/accessors.md @@ -1,6 +1,6 @@ ## accessors -
+
This module contains functions that provide information about internal geometries. diff --git a/app/content/analytics-toolbox-databricks/reference/constructors.md b/app/content/analytics-toolbox-databricks/reference/constructors.md index 21da9184a..854a7e47e 100644 --- a/app/content/analytics-toolbox-databricks/reference/constructors.md +++ b/app/content/analytics-toolbox-databricks/reference/constructors.md @@ -1,6 +1,6 @@ ## constructors -
+
This module contains functions that create new geographies from coordinates or already existing geographies. diff --git a/app/content/analytics-toolbox-databricks/reference/formatters.md b/app/content/analytics-toolbox-databricks/reference/formatters.md index da8e8357e..b0ad575de 100644 --- a/app/content/analytics-toolbox-databricks/reference/formatters.md +++ b/app/content/analytics-toolbox-databricks/reference/formatters.md @@ -1,6 +1,6 @@ ## formatters -
+
This module contains functions that export geometries to an external format like WKT. diff --git a/app/content/analytics-toolbox-databricks/reference/indexing.md b/app/content/analytics-toolbox-databricks/reference/indexing.md index e9c7383dd..27cbcdb29 100644 --- a/app/content/analytics-toolbox-databricks/reference/indexing.md +++ b/app/content/analytics-toolbox-databricks/reference/indexing.md @@ -1,6 +1,6 @@ ## indexing -
+
This module contains functions used for indexing diff --git a/app/content/analytics-toolbox-databricks/reference/measurements.md b/app/content/analytics-toolbox-databricks/reference/measurements.md index decc4272e..429b523a7 100644 --- a/app/content/analytics-toolbox-databricks/reference/measurements.md +++ b/app/content/analytics-toolbox-databricks/reference/measurements.md @@ -1,6 +1,6 @@ ## measurements -
+
This module contains functions that compute measurements of distance, area and angles. There are also functions to compute geometry values determined by measurements. diff --git a/app/content/analytics-toolbox-databricks/reference/parsers.md b/app/content/analytics-toolbox-databricks/reference/parsers.md index 55b489071..1a70b499b 100644 --- a/app/content/analytics-toolbox-databricks/reference/parsers.md +++ b/app/content/analytics-toolbox-databricks/reference/parsers.md @@ -1,6 +1,6 @@ ## parsers -
+
This module contains functions that create different geometry types from an external format like WKT. diff --git a/app/content/analytics-toolbox-databricks/reference/predicates.md b/app/content/analytics-toolbox-databricks/reference/predicates.md index 689a2de5a..01f2595d3 100644 --- a/app/content/analytics-toolbox-databricks/reference/predicates.md +++ b/app/content/analytics-toolbox-databricks/reference/predicates.md @@ -1,6 +1,6 @@ ## predicates -
+
This module contains functions that return a boolean based on the relation between 2 geometries or its properties. diff --git a/app/content/analytics-toolbox-databricks/reference/tiler.md b/app/content/analytics-toolbox-databricks/reference/tiler.md index 9e2bea2a3..bdc0e5bc0 100644 --- a/app/content/analytics-toolbox-databricks/reference/tiler.md +++ b/app/content/analytics-toolbox-databricks/reference/tiler.md @@ -4,6 +4,8 @@ We currently provide support to create H3 tilesets: +Visit the [Overview](/analytics-toolbox-bigquery/overview/tilesets) section to learn more about tilesets. + ### create_spatial_index_tileset @@ -18,8 +20,8 @@ Aggregated data is computed for all levels between `resolution_min` and `resolut * `options`: `STRING` containing a valid JSON with the different options. Valid options are described the table below. | Option | Description | | :----- | :------ | -|`resolution_min`| Default: `2`. A `NUMBER` that defines the minimum resolution level for tiles. Any resolution level under this level won't be generated.| -|`resolution_max`| Default: `15`. A `NUMBER` that defines the maximum resolution level for tiles. Any resolution level over this level won't be generated.| +|`resolution_min`| Default: `0`. A `NUMBER` that defines the minimum resolution level for tiles. Any resolution level under this level won't be generated.| +|`resolution_max`| Default: `0`. A `NUMBER` that defines the maximum resolution level for tiles. Any resolution level over this level won't be generated.| |`spatial_index_column`| Default: `h3:h3`. A `STRING` in the format `spatial_index_type:column_name`, with `spatial_index_type` being the type of spatial index used in the input table (for now only `h3` is supported), and `column_name` being the name of the column in that input table that contains the tile ids. The type of spatial index also defines the type used in the output table, which will be or H3 (for spatial index type `h3`).| |`resolution`| A `NUMBER` defining the resolution of the tiles in the input table. If only the `spatial_index_type` is defined, the column_name will be equal to spatial_index_type, `h3`| |`aggregation_resolution`| Default: `4`. A `NUMBER` defining the resolution to use when aggregating data at each resolution level. For a given `resolution`, data is aggregated at `resolution_level + aggregation resolution`.| diff --git a/app/content/analytics-toolbox-databricks/reference/transformations.md b/app/content/analytics-toolbox-databricks/reference/transformations.md index 6d200b717..3163423c6 100644 --- a/app/content/analytics-toolbox-databricks/reference/transformations.md +++ b/app/content/analytics-toolbox-databricks/reference/transformations.md @@ -1,6 +1,6 @@ ## transformations -
+
This module contains functions that compute geometric constructions, or alter geometry size or shape. diff --git a/app/content/analytics-toolbox-databricks/release-notes.md b/app/content/analytics-toolbox-databricks/release-notes.md index 5669f16b3..86d45af48 100644 --- a/app/content/analytics-toolbox-databricks/release-notes.md +++ b/app/content/analytics-toolbox-databricks/release-notes.md @@ -1,10 +1,22 @@ ## Release notes +### November 1, 2022 + +#### All modules + +New + +- Create release workflows + +Improvement + +- Adapt to Semver + ### October 24, 2022 #### Tiler -Feature +New - Add tilestats generation to H3 Tiler @@ -12,11 +24,11 @@ Feature #### All modules -Feature +New - Tasks to build installation packages -Changed +Improvement - Versioning uses YYYY.MM.DD format based on date now @@ -24,7 +36,7 @@ Changed #### Tiler -Feature +New - Add H3 Tiler @@ -32,19 +44,19 @@ Feature #### All modules -Feature +New - Add VERSION_CORE function #### Accessors -Changed +Improvement - Change type returned by ST_X and ST_Y to Double #### All modules -Feature +New - Add VERSION_ADVANCED function @@ -52,7 +64,7 @@ Feature #### All modules -Changed +Improvement - Set default compression codec to snappy @@ -60,13 +72,13 @@ Changed #### All modules -Feature +New - Add headers to functions that didn't have - Add make rule to publis artifact in local and sonatype - Add make rule to check and create headers -Changed +Improvement - change makefiles and sbt file to prepare the at advance @@ -78,7 +90,7 @@ Changed #### All modules -Feature +New - Add structure for databricks in AT advanced repo - Add make rules @@ -88,13 +100,13 @@ Feature #### All modules -Changed +Improvement - The XY precission of the TWKBWriter is set to 5. #### Module formatters -Fixed +Fix - Fix the doc of the transformers functions ST_ASLATLONTEXT and ST_ASTEXT. @@ -102,13 +114,13 @@ Fixed #### Module accessors -Fixed +Fix - Fix the bug in ST_COORDDIM that was adding z coordinate to geometries. #### Module indexing -Fixed +Fix - Fix the cast ClassCastException in ST_GEOMREPROJECT. @@ -116,7 +128,7 @@ Fixed #### All modules -Changed +Improvement - Refactor databricks cloud to adapt it to the new cloud structure @@ -124,7 +136,7 @@ Changed #### Module indexing -Feature +New - Add ST_CRSFROMTEXT function. - Add ST_EXTENTFROMGEOM function. @@ -136,25 +148,25 @@ Feature #### Module formatters -Feature +New - Add ST_ASTWKB function. #### Module parsers -Feature +New - Add ST_GEOMFROMWKT function. #### Module predicates -Feature +New - Add ST_COVERS function. #### Module transformations -Feature +New - Add ST_BUFFERPOINT function. - Add ST_DIFFERENCE function. @@ -164,7 +176,7 @@ Feature #### Module accessors -Feature +New - Add ST_COORDDIM function. - Add ST_DIMENSION function. @@ -185,7 +197,7 @@ Feature #### Module constructors -Feature +New - Add ST_MAKEBBOX function. - Add ST_MAKEBOX2D function. @@ -197,7 +209,7 @@ Feature #### Module formatters -Feature +New - Add ST_ASBINARY function. - Add ST_ASGEOHASH function. @@ -212,7 +224,7 @@ Feature #### Module measurements -Feature +New - Add ST_AREA function. - Add ST_DISTANCE function. @@ -222,7 +234,7 @@ Feature #### Module parsers -Feature +New - Add ST_BOX2DFROMGEOHASH function. - Add ST_GEOMETRYFROMTEXT function. @@ -241,7 +253,7 @@ Feature #### Module predicates -Feature +New - Add ST_CONTAINS function. - Add ST_CROSSES function. @@ -256,7 +268,7 @@ Feature #### Module transformations -Feature +New - Add ST_ANTIMERIDIANSAFEGEOM function. - Add ST_BOUNDARY function. diff --git a/app/content/analytics-toolbox-postgres/release-notes.md b/app/content/analytics-toolbox-postgres/release-notes.md index 876b4db9a..71efbcde3 100644 --- a/app/content/analytics-toolbox-postgres/release-notes.md +++ b/app/content/analytics-toolbox-postgres/release-notes.md @@ -1,10 +1,22 @@ ## Release notes +### November 1, 2022 + +#### All modules + +New + +- Create release workflows + +Improvement + +- Adapt to Semver + ### September 19, 2022 #### Module tiler -Changed +Improvement - Set `version` field in tilers metadata compliant with AT version. @@ -12,7 +24,7 @@ Changed #### Module tiler -Changed +Improvement - Unify `extra_metadata` into `metadata` in tiler metadata. @@ -20,7 +32,7 @@ Changed #### Module tiler -Fixed +Fix - CREATE_SPATIAL_INDEX_TILESET Metadata center is now computed for PostgreSQL as for other providers - Improved performance for quadbin tilesets in `CREATE_SPATIAL_INDEX_TILESET`. @@ -29,7 +41,7 @@ Fixed #### Module tiler -Feature +New - Add `CREATE_SPATIAL_INDEX_TILESET` option. @@ -37,7 +49,7 @@ Feature #### Module quadbin -Feature +New - Add QUADBIN_BBOX function. - Add QUADBIN_BOUNDARY function. @@ -59,7 +71,7 @@ Feature #### Module tiler -Fixed +Fix - Fix metadata tilestats to include only columns in properties - Fix numeric metadata tilestats which were treated as categories @@ -68,7 +80,7 @@ Fixed #### Module tiler -Feature +New - Add `max_tile_features` option. - Add output table existence early check. @@ -81,7 +93,7 @@ Feature #### Module tiler -Feature +New - Add CREATE_POINT_AGGREGATION_TILESET procedure. - Add CREATE_SIMPLE_TILESET procedure. diff --git a/app/content/analytics-toolbox-postgres/sql-reference/tiler.md b/app/content/analytics-toolbox-postgres/sql-reference/tiler.md index 35dd59ddb..b9a03e474 100644 --- a/app/content/analytics-toolbox-postgres/sql-reference/tiler.md +++ b/app/content/analytics-toolbox-postgres/sql-reference/tiler.md @@ -26,9 +26,9 @@ Generates a point aggregation tileset. | Option | Description | | :----- | :------ | -|`geom_column`| Default: `"geom"`. A `STRING` that marks the name of the geography column that will be used. It must be of type `GEOGRAPHY`. | -|`zoom_min`| Default: `2`. An `INTEGER` that defines the minimum zoom level for tiles. Any zoom level under this level won't be generated.| -|`zoom_max`| Default: `NULL`. An `INTEGER` that defines the minimum zoom level for tiles. Any zoom level over this level won't be generated.| +|`geom_column`| Default: `"geom"`. A `STRING` that marks the name of the geometry column that will be used. It must be of type `GEOMETRY`. | +|`zoom_min`| Default: `0`. An `INTEGER` that defines the minimum zoom level for tiles. Any zoom level under this level won't be generated.| +|`zoom_max`| Default: `NULL`. An `INTEGER` that defines the maximum zoom level for tiles. Any zoom level over this level won't be generated.| |`aggregation_resolution`| Default: `6`. An `INTEGER` that specifies the resolution of the spatial aggregation.

Aggregation for zoom `z` is based on quadgrid cells at `z + resolution level`. For example, with resolution `6`, the `z0` tile will be divided into cells that match the `z6` tiles, or the cells contained in the `z10` tile will be the boundaries of the `z16` tiles within them. In other words, each tile is subdivided into `4^resolution` cells, which is the maximum number of resulting features (aggregated) that the tiles will contain..

Note that adding more granularity necessarily means heavier tiles which take longer to be transmitted and processed in the final client, and you are more likely to hit the internal memory limits.| |`aggregation_placement`| Default: `"cell-centroid"`. A `TEXT` that defines what type of geometry will be used to represent the cells generated in the aggregation, which will be the features of the resulting tileset. There are currently four options:
  • `"cell-centroid"`: Each feature will be defined as the centroid of the cell, that is, all points that are aggregated together into the cell will be represented in the tile by a single point positioned at the centroid of the cell.
  • `"cell"`: Each feature will be defined as the entire cell's polygon, thus the final representation in the tile will be a polygon. This provides more precise coordinates but takes more space in the tile and requires more CPU to process it in the renderer.
  • `"features-any"`: The aggregation cell will be represented by any random point from the source data contained within it. That is, if 10 points fall inside a cell, the procedure will randomly choose the location of one of them to represent the aggregation cell.
  • `"features-centroid"`: The feature will be defined as the centroid (point) of the collection of points within the cell.
| |`metadata`| Default: `{}`. A JSON object to specify the associated metadata of the tileset. Use this to set the name, description and legend to be included in the [TileJSON](https://github.com/mapbox/tilejson-spec/tree/master/2.2.0). Other fields will be included in the object extra_metadata.| @@ -110,9 +110,9 @@ Create a simple tileset from a table, with feature dropping. | Option | Description | | :----- | :------ | -|`geom_column`| Default: `"geom"`. A `STRING` that marks the name of the geography column that will be used. It must be of type `GEOGRAPHY`. | -|`zoom_min`| Default: `2`. An `INTEGER` that defines the minimum zoom level for tiles. Any zoom level under this level won't be generated.| -|`zoom_max`| Default: `NULL`. An `INTEGER` that defines the minimum zoom level for tiles. Any zoom level over this level won't be generated. If not provided, the appropriate maximum zoom level is inferred from the size of the features.| +|`geom_column`| Default: `"geom"`. A `STRING` that marks the name of the geometry column that will be used. It must be of type `GEOMETRY`. | +|`zoom_min`| Default: `0`. An `INTEGER` that defines the minimum zoom level for tiles. Any zoom level under this level won't be generated.| +|`zoom_max`| Default: `NULL`. An `INTEGER` that defines the maximum zoom level for tiles. Any zoom level over this level won't be generated. If not provided, the appropriate maximum zoom level is inferred from the size of the features.| |`tile_extent`| Default: `4096`. An `INTEGER` defining the extent of the tile in integer coordinates as defined by the MVT spec. |`tile_buffer`| Default: `16`. An `INTEGER` defining the additional buffer added around the tiles in extent units, which is useful to facilitate geometry stitching across tiles in the renderers. In aggregation tilesets, this property is currently not available and always `0` as no geometries go across tile boundaries.| |`max_tile_size_kb`| Default: `1024`. An `INTEGER` that determines the maximum tile size, in kilobytes, before compression. @@ -180,8 +180,8 @@ Aggregated data is computed for all levels between `resolution_min` and `resolut * `options`: `TEXT` containing a valid JSON with the different options. Valid options are described the table below. | Option | Description | | :----- | :------ | -|`resolution_min`| Default: `2`. An `NUMBER` that defines the minimum resolution level for tiles. Any resolution level under this level won't be generated.| -|`resolution_max`| Default: `15`. A `NUMBER` that defines the maximum resolution level for tiles. Any resolution level over this level won't be generated.| +|`resolution_min`| Default: `0`. An `NUMBER` that defines the minimum resolution level for tiles. Any resolution level under this level won't be generated.| +|`resolution_max`| Default: `0`. A `NUMBER` that defines the maximum resolution level for tiles. Any resolution level over this level won't be generated.| |`spatial_index_column`| A `STRING` in the format `spatial_index_type:column_name`, with `spatial_index_type` being the type of spatial index used in the input table (can only be `quadbin` at the moment), and `column_name` being the name of the column in that input table that contains the tile ids. Notice that the spatial index name is case-sensitive. The type of spatial index also defines the type used in the output table, which will be QUADBIN (for spatial index type `quadbin`).| |`resolution`| A `NUMBER` defining the resolution of the tiles in the input table.| |`aggregation_resolution`| Defaults: `6` for QUADBIN tilesets, `4` for H3 tilesets. A `NUMBER` defining the resolution to use when aggregating data at each resolution level. For a given `resolution`, data is aggregated at `resolution_level + aggregation resolution`.| diff --git a/app/content/analytics-toolbox-redshift/release-notes.md b/app/content/analytics-toolbox-redshift/release-notes.md index 8e927d475..e3ff5ffe8 100644 --- a/app/content/analytics-toolbox-redshift/release-notes.md +++ b/app/content/analytics-toolbox-redshift/release-notes.md @@ -1,10 +1,22 @@ ## Release notes +### November 1, 2022 + +#### All modules + +New + +- Create release workflows + +Improvement + +- Adapt to Semver + ### October 25, 2022 #### Module data -Fixed +Fix - Fix enrichment procedures not working with VARCHAR variables for aggregation types `MIN`, `MAX` and `COUNT`. @@ -12,7 +24,7 @@ Fixed #### Module data -Fixed +Fix - Fix enrichment procedures returning inconsistent results for input queries containing `LIMIT`. - Fix ST_AREA returning NULL by using ST_SIMPLIFY. @@ -22,7 +34,7 @@ Fixed #### Module data -Feature +New - Add DATAOBS_ENRICH_POINTS procedure. - Add DATAOBS_ENRICH_POLYGONS procedure. @@ -33,14 +45,14 @@ Feature #### Module clustering -Feature +New - Move ST_CLUSTERKMEANS function to core. - Move CREATE_CLUSTERKMEANS procedure to core. #### Module random -Feature +New - Move ST_GENERATEPOINTS function to core. @@ -48,11 +60,11 @@ Feature #### Module lds -Changed +Improvement #### Module lds -Changed +Improvement - Rename `LDS_API_URL` to `API_BASE_URL`. @@ -60,7 +72,7 @@ Changed #### Module tiler -Fixed +Fix - Fix CREATE_SPATIAL_INDEX_TILESET returning nested UDFs error when detecting the index type. @@ -68,13 +80,13 @@ Fixed #### Module statistics -Changed +Improvement - Add p-values in the output of GETIS_ORD_QUADBIN function. #### Module tiler -Changed +Improvement - Set `version` field in tilers metadata compliant with AT version. @@ -82,7 +94,7 @@ Changed #### Module data -Feature +New - Add ENRICH_POLYGONS procedure. @@ -90,7 +102,7 @@ Feature #### Module data -Feature +New - Add ENRICH_POINTS procedure. @@ -98,7 +110,7 @@ Feature #### Module statistics -Feature +New - Add GETIS_ORD_QUADBIN function. @@ -106,7 +118,7 @@ Feature #### Module s2 -Fixed +Fix - Fix S2_BOUNDARY inverted coordinates. @@ -114,7 +126,7 @@ Fixed #### Module tiler -Changed +Improvement - Unify `extra_metadata` into `metadata` in tiler metadata. @@ -122,7 +134,7 @@ Changed #### Module statistics -Feature +New - Add P_VALUE function. @@ -130,13 +142,13 @@ Feature #### Module lds -Feature +New - Add GEOCODE_REVERSE_TABLE procedure. #### Module statistics -Feature +New - Create statistics module. - Add MORANS_I_QUADBIN function. @@ -145,7 +157,7 @@ Feature #### Module clustering -Feature +New - Add CREATE_CLUSTERKMEANS procedure. @@ -153,7 +165,7 @@ Feature #### Module quadbin -Changed +Improvement - Update functions volatility. - QUADBIN_FROMZXY accepting BIGINTs as params instead of INTs. @@ -162,7 +174,7 @@ Changed #### Module tiler -Changed +Improvement - Improve speed of CREATE_SPATIAL_INDEX_TILESET. @@ -170,7 +182,7 @@ Changed #### Module quadbin -Changed +Improvement - Release SQL version of QUADBIN_TOZXY. @@ -178,7 +190,7 @@ Changed #### Module tiler -Changed +Improvement - Support h3int from CREATE_SPATIAL_INDEX_TILESET. @@ -186,7 +198,7 @@ Changed #### Module tiler -Changed +Improvement - Support quadbin from CREATE_SPATIAL_INDEX_TILESET instead of quadint. @@ -194,7 +206,7 @@ Changed #### Module quadbin -Feature +New - Add QUADBIN_BBOX function. - Add QUADBIN_BOUNDARY function. @@ -216,7 +228,7 @@ Feature #### Module tiler -Feature +New - Add CREATE_SPATIAL_INDEX_TILESET procedure. @@ -224,7 +236,7 @@ Feature #### Module lds -Fixed +Fix - Fix chunks in CREATE_ISOLINES. @@ -232,7 +244,7 @@ Fixed #### Module lds -Changed +Improvement - Check if the user has enough credits to execute CREATE_ISOLINES. @@ -240,7 +252,7 @@ Changed #### Module lds -Feature +New - Add CREATE_ISOLINES procedure. @@ -248,7 +260,7 @@ Feature #### Module lds -Changed +Improvement - Add optional `language` parameter to GEOCODE_REVERSE function. @@ -256,7 +268,7 @@ Changed #### Module tiler -Feature +New - Add max_simplification_zoom and coordinates_precision parameters. @@ -264,11 +276,11 @@ Feature #### Module tiler -Feature +New - Add max_tile_size_strategy support for CREATE_SIMPLE_TILESET. -Fixed +Fix - Fix tile_feature_order not being taken into account for CREATE_POINT_AGGREGATION_TILESET and CREATE_POINT_AGGREGATION_TILESET. @@ -276,7 +288,7 @@ Fixed #### Module tiler -Fixed +Fix - Escape special characters in generated properties. @@ -284,7 +296,7 @@ Fixed #### Module tiler -Changed +Improvement - Raise improper qualified tables errors. - Limit GeoJSON precision to 6 to comply Redshift VARCHAR size limitations. @@ -294,7 +306,7 @@ Changed #### Module tiler -Feature +New - Add CREATE_SIMPLE_TILESET procedure. @@ -302,7 +314,7 @@ Feature #### Module tiler -Feature +New - Create tiler module. - Add CREATE_POINT_AGGREGATION_TILESET procedure. @@ -311,7 +323,7 @@ Feature #### Module lds -Feature +New - Add GEOCODE_TABLE procedure. - Add GEOCODE function. @@ -323,7 +335,7 @@ Feature #### Module transformations -Fixed +Fix - Refactor of internal __ST_GEOMFROMGEOJSON function to avoid UDFs nestig Redshift limitations @@ -331,7 +343,7 @@ Fixed #### Module constructors -Changed +Improvement - Deployment schema "carto" instead of "constructors". @@ -341,7 +353,7 @@ Removed #### Module placekey -Changed +Improvement - Deployment schema "carto" instead of "placekey". - Rename H3_ASPLACEKEY function to PLACEKEY_FROMH3. @@ -354,7 +366,7 @@ Removed #### Module processing -Changed +Improvement - Deployment schema "carto" instead of "processing". @@ -364,7 +376,7 @@ Removed #### Module s2 -Changed +Improvement - Deployment schema "carto" instead of "s2". - Rename HILBERTQUADKEY_FROMID function to S2_TOHILBERTQUADKEY. @@ -387,7 +399,7 @@ Removed #### Module transformations -Changed +Improvement - Deployment schema "carto" instead of "transformations". @@ -397,7 +409,7 @@ Removed #### Module clustering -Changed +Improvement - Deployment schema "carto" instead of "clustering". @@ -407,7 +419,7 @@ Removed #### Module random -Changed +Improvement - Deployment schema "carto" instead of "random". - ST_GENERATEPOINTS now uses a spherically uniform distribution. Previously used to by uniform on projection. @@ -420,7 +432,7 @@ Removed #### Module clustering -Changed +Improvement - ST_CLUSTERKMEANS returning an Array. @@ -428,7 +440,7 @@ Changed #### Module processing -Feature +New - Create processing module. - Add VERSION function. @@ -440,7 +452,7 @@ Feature #### Module transformations -Feature +New - Create transformations module. - Add VERSION function. @@ -455,7 +467,7 @@ Feature #### Module clustering -Feature +New - Create clustering module. - Add VERSION function. @@ -463,7 +475,7 @@ Feature #### Module random -Feature +New - Create random module. - Add ST_GENERATEPOINTS function. @@ -473,7 +485,7 @@ Feature #### Module s2 -Feature +New - Create s2 module. - Add HILBERTQUADKEY_FROMID function. @@ -495,7 +507,7 @@ Feature #### Module constructors -Feature +New - Create constructors module. - Add ST_BEZIERSPLINE function. @@ -508,7 +520,7 @@ Feature #### Module placekey -Feature +New - Create placekey module. - Add H3_ASPLACEKEY function. diff --git a/app/content/analytics-toolbox-redshift/sql-reference/tiler.md b/app/content/analytics-toolbox-redshift/sql-reference/tiler.md index 6728a7eef..5c94af2d7 100644 --- a/app/content/analytics-toolbox-redshift/sql-reference/tiler.md +++ b/app/content/analytics-toolbox-redshift/sql-reference/tiler.md @@ -28,7 +28,7 @@ Generates a point aggregation tileset. | :----- | :------ | |`geom_column`| Default: `"geom"`. A `VARCHAR` that indicates the name of the geometry column that will be used. This column must be of type `GEOMETRY` with an SRID 4326 and contain only points.| |`zoom_min`| Default: `0`. An `INTEGER` that defines the minimum zoom level at which tiles will be generated. Any zoom level under this level won't be generated.| -|`zoom_max`| Default: `15`; maximum: `20`. An `INTEGER` that defines the maximum zoom level at which tiles will be generated. Any zoom level over this level won't be generated.| +|`zoom_max`| Default: `0`; maximum: `20`. An `INTEGER` that defines the maximum zoom level at which tiles will be generated. Any zoom level over this level won't be generated.| |`aggregation_resolution`| Default: `6`. An `INTEGER` that specifies the resolution of the spatial aggregation.

Aggregation for zoom `z` is based on quadgrid cells at `z + resolution level`. For example, with resolution `6`, the `z0` tile will be divided into cells that match the `z6` tiles, or the cells contained in the `z10` tile will be the boundaries of the `z16` tiles within them. In other words, each tile is subdivided into `4^resolution` cells, which is the maximum number of resulting features (aggregated) that the tiles will contain.

Note that adding more granularity necessarily means heavier tiles which take longer to be transmitted and processed in the final client, and you are more likely to hit the internal memory limits.| |`aggregation_placement`| Default: `"cell-centroid"`. A `VARCHAR` that defines what type of geometry will be used to represent the cells generated in the aggregation, which will be the features of the resulting tileset. There are currently four options:
  • `"cell-centroid"`: Each feature will be defined as the centroid of the cell, that is, all points that are aggregated together into the cell will be represented in the tile by a single point positioned at the centroid of the cell.
  • `"cell"`: Each feature will be defined as the entire cell's polygon, thus the final representation in the tile will be a polygon. This provides more precise coordinates but takes more space in the tile and requires more CPU to process it in the renderer.
  • `"features-any"`: The aggregation cell will be represented by any random point from the source data contained within it. That is, if 10 points fall inside a cell, the procedure will randomly choose the location of one of them to represent the aggregation cell.
  • `"features-centroid"`: The feature will be defined as the centroid (point) of the collection of points within the cell.
| |`metadata`| Default: `{}`. A JSON object to specify the associated metadata of the tileset. Use this to set the name, description and legend to be included in the [TileJSON](https://github.com/mapbox/tilejson-spec/tree/master/2.2.0). Other fields will be included in the object extra_metadata.| @@ -113,7 +113,7 @@ Generates a simple tileset. | :----- | :------ | |`geom_column`| Default: `"geom"`. A `VARCHAR` that specifies the name of the geometry column that will be used. This column must be of type `GEOMETRY` with an SRID 4326.| |`zoom_min`| Default: `0`. A `INTEGER` that defines the minimum zoom level at which tiles will be generated. Any zoom level under this level won't be generated.| -|`zoom_max`| Default: `10`. A `INTEGER` that defines the minimum zoom level at which tiles will be generated. Any zoom level over this level won't be generated.| +|`zoom_max`| Default: `0`. A `INTEGER` that defines the maximum zoom level at which tiles will be generated. Any zoom level over this level won't be generated.| |`metadata`| Default: `{}`. A JSON object to specify the associated metadata of the tileset. Use this to set the name, description and legend to be included in the [TileJSON](https://github.com/mapbox/tilejson-spec/tree/master/2.2.0). Other fields will be included in the object extra_metadata.| |`properties`| Default: `{}`. A JSON object that defines the properties that will be included associated with each cell feature. Each property is defined by its name and type (Number, Boolean, String, etc.). Please note that every property different from Number will be casted to String.| |`max_tile_features`| Default: `10000`. A `NUMBER` that sets the maximum number of features a tile can contain. This limit only applies when the input geometries are points. When this limit is reached, the procedure will stop adding features into the tile. You can configure in which order the features are kept by setting the `tile_feature_order` property.| @@ -160,8 +160,8 @@ Aggregated data is computed for all levels between `resolution_min` and `resolut | Option | Description | | :----- | :------ | -|`resolution_min`| Default: `2`. A `NUMBER` that defines the minimum resolution level for tiles. Any resolution level under this level won't be generated.| -|`resolution_max`| Default: `15`. A `NUMBER` that defines the maximum resolution level for tiles. Any resolution level over this level won't be generated.| +|`resolution_min`| Default: `0`. A `NUMBER` that defines the minimum resolution level for tiles. Any resolution level under this level won't be generated.| +|`resolution_max`| Default: `0`. A `NUMBER` that defines the maximum resolution level for tiles. Any resolution level over this level won't be generated.| |`spatial_index_column`| A `VARCHAR` in the format `spatial_index_type:column_name`, with `spatial_index_type` being the type of spatial index used in the input table (can be `quadbin` or `h3`), and `column_name` being the name of the column in that input table that contains the tile ids. Notice that the spatial index name is case-sensitive. The type of spatial index also defines the type used in the output table, which will be QUADBIN (for spatial index type `quadbin`) or H3 (for spatial index type `h3`).| |`resolution`| A `NUMBER` defining the resolution of the tiles in the input table.| |`aggregation_resolution`| Defaults: `6` for QUADKEY tilesets, `4` for H3 tilesets. A `NUMBER` defining the resolution to use when aggregating data at each resolution level. For a given `resolution`, data is aggregated at `resolution_level + aggregation resolution`.| diff --git a/app/content/analytics-toolbox-snowflake/release-notes.md b/app/content/analytics-toolbox-snowflake/release-notes.md index 11103031b..b84ca4e8b 100644 --- a/app/content/analytics-toolbox-snowflake/release-notes.md +++ b/app/content/analytics-toolbox-snowflake/release-notes.md @@ -8,7 +8,7 @@ aliases: #### Module lds -Feature +New - Add GEOCODE_REVERSE_TABLE procedure. @@ -16,15 +16,27 @@ Feature #### Module lds -Feature +New - Add CREATE_ISOLINES procedure. +### November 1, 2022 + +#### All modules + +New + +- Create release workflows + +Improvement + +- Adapt to Semver + ### October 24, 2022 #### Module processing -Fixed +Fix - Prevent error in ST_VORONOIPOLYGONS, ST_VORONOILINES, ST_VORONOIPOLYGONS, ST_DELAUNAYLINES when points where too close together by rounding input coordinates to 5 decimal places @@ -32,13 +44,13 @@ Fixed #### Module clustering -Feature +New - Move ST_CLUSTERKMEANS function to core. #### Module random -Feature +New - Move ST_GENERATEPOINTS function to core. @@ -46,7 +58,7 @@ Feature #### Module lds -Changed +Improvement - CREATE_ISOLINES output is ordered by geometry to be compliant with Snowflake Search Optimization Service. @@ -54,7 +66,7 @@ Changed #### Module lds -Changed +Improvement - Rename `LDS_API_URL` to `API_BASE_URL`. @@ -62,7 +74,7 @@ Changed #### Module data -Changed +Improvement - Optimize DATAOBS_ENRICH_GRID, DATAOBS_ENRICH_GRID_RAW when the enrich data is a grid compatible with the input. - Remove support for s2, geohash in all enrich procedures. @@ -72,7 +84,7 @@ Changed #### Module tiler -Changed +Improvement - Set `version` field in tilers metadata compliant with AT version. @@ -80,7 +92,7 @@ Changed #### Module tiler -Changed +Improvement - Unify `extra_metadata` into `metadata` in tiler metadata. @@ -88,7 +100,7 @@ Changed #### Module data -Feature +New - Add quadbin support to module. @@ -96,7 +108,7 @@ Feature #### Module tiler -Fixed +Fix - Fix quoting and letter case in spatial index tiler. @@ -104,11 +116,11 @@ Fixed #### Module tiler -Fixed +Fix - Fix wrong uppercase in spatial index tiler. -Changed +Improvement - Use core functions to convert H3 from int to string. @@ -116,7 +128,7 @@ Changed #### Module h3 -Fixed +Fix - Correctly handle large polygons in H3_POLYFILL. - Fixed wrong uppercase for quadbin and h3 tile ids @@ -125,7 +137,7 @@ Fixed #### Module quadbin -Feature +New - Add QUADBIN_BBOX function. - Add QUADBIN_BOUNDARY function. @@ -147,7 +159,7 @@ Feature #### Module tiler -Changed +Improvement - Add CREATE_SPATIAL_INDEX_TILESET procedure. @@ -155,12 +167,12 @@ Changed #### Module data -Changed +Improvement - ENRICH_ procedures require all the enrich data query rows to be of the same geometry kind (points/lines/polygons) . - Various optimizations for performance and preventing out-of-memory errors -Changed +Improvement - Errors used to be reported with a procedure result message. Now they raise exceptions. @@ -168,7 +180,7 @@ Changed #### Module lds -Changed +Improvement - Add optional `language` parameter to GEOCODE_REVERSE function. @@ -176,7 +188,7 @@ Changed #### Module data -Changed +Improvement - Errors used to be reported with a procedure result message. Now they raise exceptions. @@ -184,7 +196,7 @@ Changed #### Module data -Feature +New - Allow using tables as input, not just queries. - Add in place enrichment. @@ -193,7 +205,7 @@ Feature #### Module tiler -Changed +Improvement - Add support for non-uppercase column names @@ -201,7 +213,7 @@ Changed #### Module tiler -Changed +Improvement - Errors used to be reported with a procedure result message. Now they raise exceptions. @@ -209,7 +221,7 @@ Changed #### Module transformations -Feature +New - Add ST_BUFFER function. @@ -217,12 +229,12 @@ Feature #### Module tiler -Changed +Improvement - Run the simple tiler in a single query. - Add `fraction_dropped_per_zoom` to the metadata. -Fixed +Fix - Fix empty properties causing errors. - Fix `drop_fraction_as_needed` in the simple tiler. @@ -232,7 +244,7 @@ Fixed #### Module transformations -Changed +Improvement - ST_CONCAVEHULL now allows arrays with one/two points as input. @@ -240,7 +252,7 @@ Changed #### Module tiler -Feature +New - Add CREATE_SIMPLE_TILESET procedure. @@ -248,7 +260,7 @@ Feature #### Module lds -Feature +New - Add GEOCODE_TABLE procedure. - Add GEOCODE function. @@ -260,7 +272,7 @@ Feature #### Module tiler -Feature +New - Create tiler module. - Add CREATE_POINT_AGGREGATION_TILESET procedure. @@ -269,12 +281,12 @@ Feature #### Module data -Feature +New - Add DATAOBS_SUBSCRIPTIONS procedure. - Add DATABOS_SUBSCRIPTION_VARIABLES procedure. -Changed +Improvement - Read spatial_catalog tables in source. @@ -282,7 +294,7 @@ Changed #### Module data -Changed +Improvement - Missing subscriptions are reported by name in DATAOBS_ENRICH_* procedures @@ -290,7 +302,7 @@ Changed #### Module random -Changed +Improvement - ST_GENERATEPOINTS now uses a spherically uniform distribution. Previously used to by uniform on projection. @@ -298,7 +310,7 @@ Changed #### Module accessors -Changed +Improvement - Deployment schema "carto" instead of "accessors". - Rename ST_ENVELOPE function to ST_ENVELOPE_ARR. @@ -309,7 +321,7 @@ Removed #### Module constructors -Changed +Improvement - Deployment schema "carto" instead of "constructors". @@ -319,7 +331,7 @@ Removed #### Module h3 -Changed +Improvement - Deployment schema "carto" instead of "h3". - Rename ST_ASH3 function to H3_FROMGEOGPOINT. @@ -343,7 +355,7 @@ Removed #### Module measurements -Changed +Improvement - Deployment schema "carto" instead of "measurements". @@ -355,7 +367,7 @@ Removed #### Module placekey -Changed +Improvement - Deployment schema "carto" instead of "placekey". - Rename H3_ASPLACEKEY function to PLACEKEY_FROMH3. @@ -368,7 +380,7 @@ Removed #### Module processing -Changed +Improvement - Deployment schema "carto" instead of "processing". @@ -378,7 +390,7 @@ Removed #### Module s2 -Changed +Improvement - Deployment schema "carto" instead of "s2". - Rename ID_FROMHILBERTQUADKEY function to S2_FROMHILBERTQUADKEY. @@ -393,7 +405,7 @@ Removed #### Module transformations -Changed +Improvement - Deployment schema "carto" instead of "transformations". @@ -403,7 +415,7 @@ Removed #### Module clustering -Changed +Improvement - Deployment schema "carto" instead of "clustering". @@ -413,7 +425,7 @@ Removed #### Module data -Changed +Improvement - Deployment schema "carto" instead of "data". @@ -423,7 +435,7 @@ Removed #### Module random -Changed +Improvement - Deployment schema "carto" instead of "random". @@ -435,21 +447,21 @@ Removed #### Module clustering -Changed +Improvement - Deployment schema "carto" instead of "clustering". - Remove VERSION function. #### Module data -Changed +Improvement - Deployment schema "carto" instead of "data". - Remove VERSION function. #### Module random -Changed +Improvement - Deployment schema "carto" instead of "random". - Remove VERSION function. @@ -458,7 +470,7 @@ Changed #### Module data -Feature +New - Add DATAOBS_ENRICH_POINTS procedure. - Add DATAOBS_ENRICH_POINTS_RAW procedure. @@ -478,7 +490,7 @@ Changes Also the column `_carto_enrichment_` is now `__carto_enrichment`. This affects all the _raw_ enrichment procedures: `ENRICH_POINTS_RAW`, `ENRICH_POLYGONS_RAW`, `ENRICH_GRID_RAW`. -Fixed +Fix - User provided queries can now have columns named `dimension`, `total`, `intersection`, `input_area`, `_nonglobal`, which could have collided previously with internal columns. All internal columns are now prefixed with `__carto_`. This affects all the enrichment procedures: `ENRICH_POINTS`, `ENRICH_POLYGONS`, `ENRICH_GRID`, `ENRICH_POINTS_RAW`, `ENRICH_POLYGONS_RAW`, `ENRICH_GRID_RAW`. @@ -486,7 +498,7 @@ Fixed #### Module data -Feature +New - Create data module. - Add VERSION function. @@ -501,11 +513,11 @@ Feature #### Module h3 -Feature +New - Add KRING_DISTANCES function. -Changed +Improvement - Review HEXRING, KRING functions. @@ -521,7 +533,7 @@ Changes #### Module h3 -Fixed +Fix - Support GEOMETRYCOLLECTION from ST_ASH3_POLYFILL. @@ -529,7 +541,7 @@ Fixed #### Module h3 -Changed +Improvement - Reduce bundle size for every function. @@ -537,7 +549,7 @@ Changed #### Module processing -Feature +New - Create processing module. - Add ST_VORONOIPOLYGONS function. @@ -551,7 +563,7 @@ Feature #### Module clustering -Feature +New - Create clustering module. - Add VERSION function. @@ -559,7 +571,7 @@ Feature #### Module random -Feature +New - Create random module. - Add ST_GENERATEPOINTS function. @@ -569,7 +581,7 @@ Feature #### Module accessors -Feature +New - Create accessors module. - Add ST_ENVELOPE function. @@ -579,7 +591,7 @@ Feature #### Module constructors -Feature +New - Create constructors module. - Add ST_BEZIERSPLINE function. @@ -590,7 +602,7 @@ Feature #### Module measurements -Feature +New - Create measurements module. - Add ST_ANGLE function. @@ -600,7 +612,7 @@ Feature #### Module transformations -Feature +New - Create transformations module. - Add ST_CENTERMEAN function. @@ -616,7 +628,7 @@ Feature #### Module placekey -Feature +New - Create placekey module. - Add H3_ASPLACEKEY function. @@ -628,7 +640,7 @@ Feature #### Module s2 -Feature +New - Create s2 module. - Add ID_FROMHILBERTQUADKEY function. @@ -642,7 +654,7 @@ Feature #### Module h3 -Feature +New - Create h3 module. - Add ST_ASH3 function. diff --git a/app/content/analytics-toolbox-snowflake/sql-reference/tiler.md b/app/content/analytics-toolbox-snowflake/sql-reference/tiler.md index 97e3708fe..90bfcf352 100644 --- a/app/content/analytics-toolbox-snowflake/sql-reference/tiler.md +++ b/app/content/analytics-toolbox-snowflake/sql-reference/tiler.md @@ -34,7 +34,7 @@ Generates a point aggregation tileset. | :----- | :------ | |`geom_column`| Default: `"GEOM"`. A `VARCHAR` that indicates the name of the geography column that will be used. The geography column must be of type `GEOGRAPHY` and contain only points. The capitalization (uppercase/lowercase letters) of the name must match exactly the column name; note that Snowflake by default converts names to uppercase. | |`zoom_min`| Default: `0`. An `INTEGER` that defines the minimum zoom level at which tiles will be generated. Any zoom level under this level won't be generated.| -|`zoom_max`| Default: `15`; maximum: `20`. An `INTEGER` that defines the maximum zoom level at which tiles will be generated. Any zoom level over this level won't be generated.| +|`zoom_max`| Default: `0`; maximum: `20`. An `INTEGER` that defines the maximum zoom level at which tiles will be generated. Any zoom level over this level won't be generated.| |`aggregation_resolution`| Default: `6`. An `INTEGER` that specifies the resolution of the spatial aggregation.

Aggregation for zoom `z` is based on quadgrid cells at `z + resolution level`. For example, with resolution `6`, the `z0` tile will be divided into cells that match the `z6` tiles, or the cells contained in the `z10` tile will be the boundaries of the `z16` tiles within them. In other words, each tile is subdivided into `4^resolution` cells, which is the maximum number of resulting features (aggregated) that the tiles will contain.

Note that adding more granularity necessarily means heavier tiles which take longer to be transmitted and processed in the final client, and you are more likely to hit the internal memory limits.| |`aggregation_placement`| Default: `"cell-centroid"`. A `VARCHAR` that defines what type of geometry will be used to represent the cells generated in the aggregation, which will be the features of the resulting tileset. There are currently four options:
  • `"cell-centroid"`: Each feature will be defined as the centroid of the cell, that is, all points that are aggregated together into the cell will be represented in the tile by a single point positioned at the centroid of the cell.
  • `"cell"`: Each feature will be defined as the entire cell's polygon, thus the final representation in the tile will be a polygon. This provides more precise coordinates but takes more space in the tile and requires more CPU to process it in the renderer.
  • `"features-any"`: The aggregation cell will be represented by any random point from the source data contained within it. That is, if 10 points fall inside a cell, the procedure will randomly choose the location of one of them to represent the aggregation cell.
  • `"features-centroid"`: The feature will be defined as the centroid (point) of the collection of points within the cell.
| |`metadata`| Default: `{}`. A JSON object to specify the associated metadata of the tileset. Use this to set the name, description and legend to be included in the [TileJSON](https://github.com/mapbox/tilejson-spec/tree/master/2.2.0). Other fields will be included in the object extra_metadata.| @@ -119,7 +119,7 @@ Generates a simple tileset. | :----- | :------ | |`geom_column`| Default: `"GEOM"`. A `VARCHAR` that specifies the name of the geography column that will be used. It must be of type `GEOGRAPHY`. The capitalization (uppercase/lowercase letters) of the name must match exactly the column name; note that Snowflake by default will use only uppercase letters for the column names, but this can be altered if column names are quoted in their definition. Do not use quotes here, just mutch the capitalization. | |`zoom_min`| Default: `0`. A `NUMBER` that defines the minimum zoom level at which tiles will be generated. Any zoom level under this level won't be generated.| -|`zoom_max`| Default: `10`. A `NUMBER` that defines the maximum zoom level at which tiles will be generated. Any zoom level over this level won't be generated.| +|`zoom_max`| Default: `0`. A `NUMBER` that defines the maximum zoom level at which tiles will be generated. Any zoom level over this level won't be generated.| |`metadata`| Default: `{}`. A JSON object to specify the associated metadata of the tileset. Use this to set the name, description and legend to be included in the [TileJSON](https://github.com/mapbox/tilejson-spec/tree/master/2.2.0). Other fields will be included in the object extra_metadata.| |`properties`| Default: `{}`. A JSON object that defines the properties that will be included associated with each cell feature. Each property is defined by its name and type (Number, String, etc.). Please note that every property different from Number will be casted to String. Property names must correspond to column names in the input and match it's capitalization exactly; note by default Snowflake will use only uppercase letters for the column names but this can altered if column names where quoted in their definitions. Do not use quotes here, just mutch the capitalization. The properties will appear in the GeoJSON data with the same capitalization as the column names. | |`max_tile_features`| Default: `10000`. A `NUMBER` that sets the maximum number of features a tile can contain. This limit only applies when the input geometries are points. When this limit is reached, the procedure will stop adding features into the tile. You can configure in which order the features are kept by setting the `tile_feature_order` property.| @@ -163,8 +163,8 @@ Aggregated data is computed for all levels between `resolution_min` and `resolut * `options`: `STRING` containing a valid JSON with the different options. Valid options are described the table below. | Option | Description | | :----- | :------ | -|`resolution_min`| Default: `2`. A `NUMBER` that defines the minimum resolution level for tiles. Any resolution level under this level won't be generated.| -|`resolution_max`| Default: `15`. A `NUMBER` that defines the maximum resolution level for tiles. Any resolution level over this level won't be generated.| +|`resolution_min`| Default: `0`. A `NUMBER` that defines the minimum resolution level for tiles. Any resolution level under this level won't be generated.| +|`resolution_max`| Default: `0`. A `NUMBER` that defines the maximum resolution level for tiles. Any resolution level over this level won't be generated.| |`spatial_index_column`| A `STRING` in the format `spatial_index_type:column_name`, with `spatial_index_type` being the type of spatial index used in the input table (can be `quadbin` or `h3`), and `column_name` being the name of the column in that input table that contains the tile ids. Notice that the spatial index name is case-sensitive. The type of spatial index also defines the type used in the output table, which will be QUADBIN (for spatial index type `quadbin`) or H3 (for spatial index type `h3`).| |`resolution`| A `NUMBER` defining the resolution of the tiles in the input table.| |`aggregation_resolution`| Defaults: `6` for QUADBIN tilesets, `4` for H3 tilesets. A `NUMBER` defining the resolution to use when aggregating data at each resolution level. For a given `resolution`, data is aggregated at `resolution_level + aggregation resolution`.|