Skip to content

Commit

Permalink
Merge pull request #120 from PDOK/new-srs-check
Browse files Browse the repository at this point in the history 
updated srs check
  • Loading branch information
@Shalucik
Shalucik authored May 29, 2024
2 parents e1fa9c4 + b42c7e6 commit b82070a
Show file tree
Hide file tree
Showing 6 changed files with 106 additions and 32 deletions.
53 changes: 27 additions & 26 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -84,32 +84,33 @@ geopackage-validator generate-definitions --gpkg-path "/gpkg/${gpkg_path}" > "$s
The Geopackage validator can validate .gkpg files to see if they conform to a set of standards.
The current checks are (see also the 'show-validations' command):

| Validation code** | Description |
|:-----------------:|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| UNKNOWN_ERROR | No unexpected (GDAL) errors must occur. |
| RQ0 | _LEGACY:_ * Geopackage must conform to table names in the given JSON or YAML definitions. |
| RQ1 | Layer names must start with a letter, and valid characters are lowercase a-z, numbers or underscores. |
| RQ2 | Layers must have at least one feature. |
| RQ3 | _LEGACY:_ * Layer features should have an allowed geometry_type (one of POINT, LINESTRING, POLYGON, MULTIPOINT, MULTILINESTRING, or MULTIPOLYGON). |
| RQ4 | The geopackage should have no views defined. |
| RQ5 | Geometry should be valid. |
| RQ6 | Column names must start with a letter, and valid characters are lowercase a-z, numbers or underscores. |
| RQ7 | Tables should have a feature id column with unique index. |
| RQ8 | Geopackage must conform to given JSON or YAML definitions. |
| RQ9 | All geometry tables must have an rtree index. |
| RQ10 | All geometry table rtree indexes must be valid. |
| RQ11 | OGR indexed feature counts must be up to date. |
| RQ12 | Only the following EPSG spatial reference systems are allowed: 28992, 3034, 3035, 3038, 3039, 3040, 3041, 3042, 3043, 3044, 3045, 3046, 3047, 3048, 3049, 3050, 3051, 4258, 4936, 4937, 5730, 7409. |
| RQ13 | It is required to give all GEOMETRY features the same default spatial reference system. |
| RQ14 | The geometry_type_name from the gpkg_geometry_columns table must be one of POINT, LINESTRING, POLYGON, MULTIPOINT, MULTILINESTRING, or MULTIPOLYGON. |
| RQ15 | All table geometries must match the geometry_type_name from the gpkg_geometry_columns table. |
| RQ16 | _LEGACY:_ * All layer and column names shall not be longer than 53 characters. |
| RC17 | It is recommended to name all GEOMETRY type columns 'geom'. |
| RC18 | It is recommended to give all GEOMETRY type columns the same name. |
| RC19 | It is recommended to only use multidimensional geometry coordinates (elevation and measurement) when necessary. |
| RC20 | It is recommended that all (MULTI)POLYGON geometries have a counter-clockwise orientation for their exterior ring, and a clockwise direction for all interior rings. |
| RQ21 | All layer and column names shall not be longer than 57 characters. |
| UNKNOWN_WARNINGS | It is recommended that the unexpected (GDAL) warnings are looked into. |
| Validation code** | Description |
|:-----------------:|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| UNKNOWN_ERROR | No unexpected (GDAL) errors must occur. |
| RQ0 | _LEGACY:_ * Geopackage must conform to table names in the given JSON or YAML definitions. |
| RQ1 | Layer names must start with a letter, and valid characters are lowercase a-z, numbers or underscores. |
| RQ2 | Layers must have at least one feature. |
| RQ3 | _LEGACY:_ * Layer features should have an allowed geometry_type (one of POINT, LINESTRING, POLYGON, MULTIPOINT, MULTILINESTRING, or MULTIPOLYGON). |
| RQ4 | The geopackage should have no views defined. |
| RQ5 | Geometry should be valid. |
| RQ6 | Column names must start with a letter, and valid characters are lowercase a-z, numbers or underscores. |
| RQ7 | Tables should have a feature id column with unique index. |
| RQ8 | Geopackage must conform to given JSON or YAML definitions. |
| RQ9 | All geometry tables must have an rtree index. |
| RQ10 | All geometry table rtree indexes must be valid. |
| RQ11 | OGR indexed feature counts must be up to date. |
| RQ12 | _LEGACY:_ * Only the following EPSG spatial reference systems are allowed: 28992, 3034, 3035, 3038, 3039, 3040, 3041, 3042, 3043, 3044, 3045, 3046, 3047, 3048, 3049, 3050, 3051, 4258, 4936, 4937, 5730, 7409. |
| RQ13 | It is required to give all GEOMETRY features the same default spatial reference system. |
| RQ14 | The geometry_type_name from the gpkg_geometry_columns table must be one of POINT, LINESTRING, POLYGON, MULTIPOINT, MULTILINESTRING, or MULTIPOLYGON. |
| RQ15 | All table geometries must match the geometry_type_name from the gpkg_geometry_columns table. |
| RQ16 | _LEGACY:_ * All layer and column names shall not be longer than 53 characters. |
| RC17 | It is recommended to name all GEOMETRY type columns 'geom'. |
| RC18 | It is recommended to give all GEOMETRY type columns the same name. |
| RC19 | It is recommended to only use multidimensional geometry coordinates (elevation and measurement) when necessary. |
| RC20 | It is recommended that all (MULTI)POLYGON geometries have a counter-clockwise orientation for their exterior ring, and a clockwise direction for all interior rings. |
| RQ21 | All layer and column names shall not be longer than 57 characters. |
| RQ22 | Only the following EPSG spatial reference systems are allowed: 28992, 3034, 3035, 3040, 3041, 3042, 3043, 3044, 3045, 3046, 3047, 3048, 3049, 3857, 4258, 4326, 4936, 4937, 5730, 7409. |
| UNKNOWN_WARNINGS | It is recommended that the unexpected (GDAL) warnings are looked into. |

\* Legacy requirements are only executed with the validate command when explicitly requested in the validation set.
\** Since version 0.8.0 the recommendations are part of the same sequence as the requirements. From now on a check will always maintain the integer part of the code. Even if at a later time the validation type can shift between requirement and recommendation.
Expand Down
25 changes: 24 additions & 1 deletion geopackage_validator/constants.py
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import re

ALLOWED_PROJECTIONS_LIST = [
LEGACY_ALLOWED_PROJECTIONS_LIST = [
28992,
3034,
3035,
Expand All @@ -25,6 +25,29 @@
7409,
]

ALLOWED_PROJECTIONS_LIST = [
28992,
3034,
3035,
3040,
3041,
3042,
3043,
3044,
3045,
3046,
3047,
3048,
3049,
3857,
4258,
4326,
4936,
4937,
5730,
7409,
]

VALID_GEOMETRIES = [
"POINT",
"LINESTRING",
Expand Down
3 changes: 2 additions & 1 deletion geopackage_validator/validate.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,12 @@
RQ0 = "RQ0"
RQ3 = "RQ3"
RQ8 = "RQ8"
RQ12 = "RQ12"
RQ16 = "RQ16"


# Drop legacy requirements
DROP_LEGACY_RQ_FROM_ALL = [RQ0, RQ3, RQ16]
DROP_LEGACY_RQ_FROM_ALL = [RQ0, RQ3, RQ12, RQ16]


def validators_to_use(
Expand Down
7 changes: 6 additions & 1 deletion geopackage_validator/validations/__init__.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,11 @@
from geopackage_validator.validations.layername_check import LayerNameValidator
from geopackage_validator.validations.rtree_present_check import RTreeExistsValidator
from geopackage_validator.validations.rtree_valid_check import ValidRtreeValidator
from geopackage_validator.validations.srs_check import SrsValidator, SrsEqualValidator
from geopackage_validator.validations.srs_check import (
SrsValidator,
SrsValidatorV0,
SrsEqualValidator,
)
from geopackage_validator.validations.geom_column_check import (
GeomColumnNameValidator,
GeomColumnNameEqualValidator,
Expand Down Expand Up @@ -49,6 +53,7 @@
"TableDefinitionValidator",
"TableDefinitionValidatorV0",
"SrsValidator",
"SrsValidatorV0",
"SrsEqualValidator",
"GpkgGeometryTypeNameValidator",
"GeometryTypeEqualsGpkgDefinitionValidator",
Expand Down
48 changes: 46 additions & 2 deletions geopackage_validator/validations/srs_check.py
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
from typing import Iterable, Tuple

from geopackage_validator.constants import ALLOWED_PROJECTIONS_LIST
from geopackage_validator.constants import (
ALLOWED_PROJECTIONS_LIST,
LEGACY_ALLOWED_PROJECTIONS_LIST,
)

from geopackage_validator.validations import validator

Expand Down Expand Up @@ -28,13 +31,54 @@ def srs_equal_check_query(dataset) -> Iterable[str]:
dataset.ReleaseResultSet(srs_list)


class SrsValidator(validator.Validator):
class SrsValidatorV0(validator.Validator):
"""Only the following EPSG spatial reference systems are allowed: 28992, 3034, 3035, 3038, 3039, 3040, 3041, 3042, 3043, 3044, 3045, 3046, 3047, 3048, 3049, 3050, 3051, 4258, 4936, 4937, 5730, 7409."""

code = 12
level = validator.ValidationLevel.ERROR
message = "Found in 'gpkg_spatial_ref_sys' {srs_organisation} {srs_id}. {srs_name} is not allowed."

def check(self) -> Iterable[str]:
srs_metadata = srs_check_query(self.dataset)
return self.srs_check(srs_metadata)

@classmethod
def srs_check(cls, srs_list: Iterable[Tuple[str, str]]):
assert srs_list is not None

results = []

for srs_organisation, srs_id, srs_name in srs_list:

if srs_organisation != "EPSG":
results.append(
cls.message.format(
srs_organisation=srs_organisation,
srs_id=srs_id,
srs_name=srs_name,
)
)
continue # prevent duplicate error message for the same srs entry

if srs_id not in LEGACY_ALLOWED_PROJECTIONS_LIST:
results.append(
cls.message.format(
srs_organisation=srs_organisation,
srs_id=srs_id,
srs_name=srs_name,
)
)

return results


class SrsValidator(validator.Validator):
"""Only the following EPSG spatial reference systems are allowed: 28992, 3034, 3035, 3040, 3041, 3042, 3043, 3044, 3045, 3046, 3047, 3048, 3049, 3857, 4258, 4326, 4936, 4937, 5730, 7409."""

code = 22
level = validator.ValidationLevel.ERROR
message = "Found in 'gpkg_spatial_ref_sys' {srs_organisation} {srs_id}. {srs_name} is not allowed."

def check(self) -> Iterable[str]:
srs_metadata = srs_check_query(self.dataset)
return self.srs_check(srs_metadata)
Expand Down
2 changes: 1 addition & 1 deletion tests/test_validate.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -30,11 +30,11 @@ def test_determine_validations_to_use_none():
"RQ9",
"RQ10",
"RQ11",
"RQ12",
"RQ13",
"RQ14",
"RQ15",
"RQ21",
"RQ22",
"RC17",
"RC18",
"RC19",
Expand Down

0 comments on commit b82070a

Please sign in to comment.