Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to pass all checks #1029

Merged
merged 5 commits into from
Dec 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/code_quality.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ jobs:
with:
node-version: '18'
- run: npm install --global remark-cli remark-validate-links remark-lint-no-dead-urls
- run: remark --version
- run: make remark

Flake_8_syntax_errors:
Expand Down
8 changes: 7 additions & 1 deletion .remarkrc
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,15 @@
},
"lint-no-dead-urls": {
"skipLocalhost": true,
"deadOrAliveOptions": {
"timeout": 20000,
"maxRetries": 2
},
"skipUrlPatterns": [
"https://security.stackexchange.com",
"https://github.com/LayerManager/layman/issues/"
"https://github.com/LayerManager/layman/issues/",
"https://github.com/LayerManager/layman/blob/",
"https://docs.geoserver.org/2.21.x/en/api/"
]
}
}
Expand Down
12 changes: 6 additions & 6 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -210,7 +210,7 @@
- If you are using environment variable [`OAUTH2_CALLBACK_URL`](doc/env-settings.md#oauth2_callback_url), change only its URL path from `/client/authn/oauth2-liferay/callback` to `/client/authn/oauth2-provider/callback`. Keep protocol, domain, and port unchanged.
- Stop using environment variable `LAYMAN_AUTHN_OAUTH2_PROVIDERS`, it has no effect to Layman anymore. There is exactly one OAuth2 provider Python module now, no need to set it.
- Stop using environment variable `FLASK_ENV`, it has no effect to Layman anymore.
- If you used environment variable `FLASK_ENV` with value `development`, add new environment variable [`FLASK_DEBUG`](https://flask.palletsprojects.com/en/2.3.x/config/?highlight=flask_debug#DEBUG):
- If you used environment variable `FLASK_ENV` with value `development`, add new environment variable [`FLASK_DEBUG`](https://flask.palletsprojects.com/en/stable/config/?highlight=flask_debug#DEBUG):
```
FLASK_DEBUG=1
```
Expand Down Expand Up @@ -579,10 +579,10 @@ make timgen-build
### Changes
- [#167](https://github.com/LayerManager/layman/issues/167) Allow publishing also raster geospatial data using [POST Workspace Layers](doc/rest.md#post-workspace-layers) and [PATCH Workspace Layer](doc/rest.md#patch-workspace-layer).
- Following formats are supported:
- [GeoTIFF](https://gdal.org/drivers/raster/gtiff.html)
- [JPEG2000](https://gdal.org/drivers/raster/jp2openjpeg.html)
- [PNG](https://gdal.org/drivers/raster/png.html)
- [JPEG](https://gdal.org/drivers/raster/jpeg.html)
- [GeoTIFF](https://gdal.org/en/stable/drivers/raster/gtiff.html)
- [JPEG2000](https://gdal.org/en/stable/drivers/raster/jp2openjpeg.html)
- [PNG](https://gdal.org/en/stable/drivers/raster/png.html)
- [JPEG](https://gdal.org/en/stable/drivers/raster/jpeg.html)
- Following input combinations of bands and color interpretations are supported:
- 1 band: Gray
- 1 band: Palette
Expand Down Expand Up @@ -742,7 +742,7 @@ make timgen-build
- Parameter *style* accepts also QGIS layer style (QML). Layman Test Client enables to select also `*.qml` files.
- Endpoint [GET Workspace Layer](doc/rest.md#get-workspace-layer) returns in `style` attribute also `type`, either `"sld"` or `"qml"`.
- Endpoint [GET Workspace Layer Style](doc/rest.md#get-workspace-layer-style) returns SLD style or QML style.
- Treat attribute names in QML (also known as '[launder](https://gdal.org/drivers/vector/pg.html#layer-creation-options)').
- Treat attribute names in QML (also known as '[launder](https://gdal.org/en/stable/drivers/vector/pg.html#layer-creation-options)').
- New docker container with QGIS server called `qgis` in demo configuration.
- New directory [LAYMAN_QGIS_DATA_DIR](doc/env-settings.md#LAYMAN_QGIS_DATA_DIR) is used to store [layer QGS files](doc/data-storage.md#filesystem).
- [WMS](doc/endpoints.md#web-map-service) is moved to dedicated [GeoServer workspace](doc/data-storage.md#geoserver) whose name is composed from Layman's [workspace](doc/models.md#workspace) name and suffix `_wms`. [WFS](doc/endpoints.md#web-feature-service) remains in GeoServer workspace whose name is equal to Layman's workspace name.
Expand Down
14 changes: 7 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ Publishing geospatial data online through [REST API](doc/rest.md).
- [**layer**](doc/models.md#layer): visual representation of single vector or raster dataset, including raster [timeseries](doc/models.md#timeseries)
- [**map**](doc/models.md#map): collection of layers
- Accepts **vector** layer data in [GeoJSON](https://en.wikipedia.org/wiki/GeoJSON), [ShapeFile](https://en.wikipedia.org/wiki/Shapefile), or [PostGIS table](https://postgis.net/) identified by [PostgreSQL connection URI](https://www.postgresql.org/docs/15/libpq-connect.html#id-1.7.3.8.3.6)
- Accepts **raster** layer data in [GeoTIFF](https://gdal.org/drivers/raster/gtiff.html), [JPEG2000](https://gdal.org/drivers/raster/jp2openjpeg.html), [PNG](https://gdal.org/drivers/raster/png.html), and [JPEG](https://gdal.org/drivers/raster/jpeg.html) formats
- Accepts layer **style** in [Styled Layer Descriptor](https://ogc.org/standard/sld), [Symbology Encoding](https://ogc.org/standard/se), and [QGIS Style File Format](https://docs.qgis.org/3.16/en/docs/user_manual/appendices/qgis_file_formats.html#qml-the-qgis-style-file-format) (for vector data only) formats
- Accepts **raster** layer data in [GeoTIFF](https://gdal.org/en/stable/drivers/raster/gtiff.html), [JPEG2000](https://gdal.org/en/stable/drivers/raster/jp2openjpeg.html), [PNG](https://gdal.org/en/stable/drivers/raster/png.html), and [JPEG](https://gdal.org/en/stable/drivers/raster/jpeg.html) formats
- Accepts layer **style** in [Styled Layer Descriptor](https://www.ogc.org/publications/standard/sld/), [Symbology Encoding](https://www.ogc.org/publications/standard/se/), and [QGIS Style File Format](https://docs.qgis.org/3.16/en/docs/user_manual/appendices/qgis_file_formats.html#qml-the-qgis-style-file-format) (for vector data only) formats
- Accepts **map** definition in [HSLayers Map Composition](https://github.com/hslayers/map-compositions) format
- Even large files can be easily uploaded from browser thanks to asynchronous chunk upload
- [OAuth2 authentication](doc/security.md#authentication)
Expand All @@ -35,7 +35,7 @@ Publishing geospatial data online through [REST API](doc/rest.md).

You can check your kernel version with `uname -r`.
- Docker Engine v20.10.13+ including Docker Compose v2+
- installation instructions for [centos 7](https://docs.docker.com/install/linux/docker-ce/centos/), including docker-compose-plugin installation
- installation instructions for [centos 7](https://docs.docker.com/engine/install/centos/), including docker-compose-plugin installation

**Optionally**
- linux or any tool to run tasks defined in Makefile using `make` command
Expand Down Expand Up @@ -106,7 +106,7 @@ Then you can log in with automatically provided Wagtail user `layman` and passwo
Layman's source code provides settings suitable for development, testing and demo purposes. Furthermore, there exists [`Makefile`](Makefile) with predefined commands for each purpose including starting all necessary services (both in background and foreground) and stoping it.

Layman's configuration is split into three levels:
- `docker-compose.*.yml` files used as [docker-compose configuration files](https://docs.docker.com/compose/compose-file/) with most general settings of docker containers including volume mappings, port mappings, container names and startup commands
- `docker-compose.*.yml` files used as [docker-compose configuration files](https://docs.docker.com/reference/compose-file/) with most general settings of docker containers including volume mappings, port mappings, container names and startup commands
- `.env*` files with environment settings of both build stage and runtime of docker containers
- `src/layman_settings.py` Python module with settings of Layman's Python modules for runtime

Expand Down Expand Up @@ -170,7 +170,7 @@ When providing **external dependencies**, check their production-related documen
- [PostgreSQL 13.3](https://www.postgresql.org/docs/13/admin.html) & [PostGIS 3.1](http://postgis.net/docs/manual-3.1/performance_tips.html)
- [QGIS Server 3.16.1](https://docs.qgis.org/3.10/en/docs/user_manual/working_with_ogc/server/index.html)
- [GeoServer 2.21.2](https://geoserver.org/release/2.21.2/)
- [Redis 4.0](https://redis.io/topics/admin)
- [Redis 4.0](https://redis.io/docs/latest/operate/oss_and_stack/management/admin/)
- [Micka v2020.014](https://github.com/hsrs-cz/Micka/releases/tag/v2020.014), see also [configuration](deps/micka/sample/confs/config.local.neon) of [dockerized Micka](https://github.com/LayerManager/docker-micka).

Within PostgreSQL, you need to provide one database for Layman and one database for Micka. For Layman, you also need to provide one user [LAYMAN_PG_USER](doc/env-settings.md#LAYMAN_PG_USER) who needs enough privileges to create new schemas in [LAYMAN_PG_DBNAME](doc/env-settings.md#LAYMAN_PG_DBNAME) database. The user also needs access to `public` schema where PostGIS must be installed.
Expand All @@ -190,8 +190,8 @@ After providing external dependencies there is time to provide **internal depend
**System-level** dependencies includes
- python 3.8+
- [python3-lxml](https://lxml.de/installation.html)
- [ogr2ogr](https://gdal.org/programs/ogr2ogr.html) utility of [gdal](https://gdal.org/) 3.3+
- [chromium-browser](https://chromium.org/) 90+ and corresponding version of [chromedriver](https://chromedriver.chromium.org/)
- [ogr2ogr](https://gdal.org/en/stable/programs/ogr2ogr.html) utility of [gdal](https://gdal.org/) 3.3+
- [chromium-browser](https://www.chromium.org/) 90+ and corresponding version of [chromedriver](https://developer.chrome.com/docs/chromedriver/)
- [pipenv](https://pypi.org/project/pipenv/)
- [node.js](https://nodejs.org/) 18 & npm 8 for running Layman Test Client
- [node.js](https://nodejs.org/) 16 & npm 8 for running Timgen
Expand Down
8 changes: 4 additions & 4 deletions doc/dependencies.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,12 +35,12 @@
| --- | --- | --- | --- | --- | --- |
| [flask](http://flask.pocoo.org/) | BSD License | Pipfile | prod | bin | to build REST API |
| [celery](https://github.com/celery/celery) | BSD 3-Clause | Pipfile | prod | bin | asynchronous task runner |
| [redis-py](https://github.com/andymccurdy/redis-py) | MIT | Pipfile | prod | bin | |
| [redis-py](https://github.com/redis/redis-py) | MIT | Pipfile | prod | bin | |
| [unidecode](https://github.com/avian2/unidecode) | GNU GPL v2 | Pipfile | prod | bin | |
| [psycopg2-binary](https://github.com/psycopg/psycopg2) | GNU LGPL | Pipfile | prod | bin | |
| [requests](https://requests.readthedocs.io/) | Apache License 2.0 | Pipfile | prod | bin | |
| [owslib](https://github.com/geopython/OWSLib) | BSD 3-Clause | Pipfile | prod | bin | |
| [jsonschema](https://github.com/Julian/jsonschema) | MIT | Pipfile | prod | bin | |
| [jsonschema](https://github.com/python-jsonschema/jsonschema) | MIT | Pipfile | prod | bin | |
| [flower](https://github.com/mher/flower) | BSD 3-Clause | Pipfile | prod | bin | to monitor celery tasks |
| [selenium](https://www.chromium.org/) | Apache License 2.0 | Dockerfile | prod | bin | for client-side map rendering and integration testing |
| [cacheout](https://github.com/dgilland/cacheout) | MIT | Pipfile | prod | bin | |
Expand All @@ -51,7 +51,7 @@
| [pytest](https://pytest.org/) | MIT License | Pipfile | test | bin | |
| [flake8](https://flake8.pycqa.org/) | MIT | Pipfile | test | bin | code style checker |
| [pycodestyle](https://pycodestyle.pycqa.org/) | MIT | Pipfile | test | bin | code style checker |
| [pylint](https://github.com/PyCQA/pylint) | GNU GPL v2 | Pipfile | test | bin | code style checker |
| [pylint](https://github.com/pylint-dev/pylint) | GNU GPL v2 | Pipfile | test | bin | code style checker |
| [autopep8](https://github.com/hhatto/autopep8) | MIT | Pipfile | test | bin | to automatically fix code style |
| [pytest-rerunfailures](https://github.com/pytest-dev/pytest-rerunfailures) | MPL | Pipfile | test | bin | to automatically rerun flaky tests |
| [pytest-timeout](https://pypi.org/project/pytest-timeout/) | MIT | Pipfile | test | bin | to automatically stop tests after given timeout |
Expand Down Expand Up @@ -89,5 +89,5 @@
| [layermanager/docker-postgis](https://github.com/layermanager/docker-postgis) | MIT | docker-compose.yml | dev | bin | |
| [jirikcz/micka](https://github.com/LayerManager/docker-micka) | GNU GPL v3 | docker-compose.yml | prod-external | bin | |
| [samtux/micka](https://github.com/samtux/docker-micka) | GNU GPL v3 | jirikcz/micka | prod-external | src | |
| [docker-library/redis](https://github.com/docker-library/redis) | BSD 3-Clause | docker-compose.yml | prod | bin | |
| [docker-library/redis](https://github.com/redis/docker-library-redis) | BSD 3-Clause | docker-compose.yml | prod | bin | |
| [plantuml/plantuml](https://hub.docker.com/r/plantuml/plantuml) | GNU GPL v3 | Makefile | dev | bin | render PlantUML images |
6 changes: 3 additions & 3 deletions doc/endpoints.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
To use [headers for GeoServer's Proxy URL](https://docs.geoserver.org/2.21.x/en/user/configuration/globalsettings.html#use-headers-for-proxy-url) see [client proxy documentation](client-proxy.md).

## Web Map Service
[Web Map Service (WMS)](https://ogc.org/standard/wms) endpoint is implemented using combination of Layman's authentication proxy, Layman's WMS proxy, and [GeoServer](https://docs.geoserver.org/2.21.x/en/user/services/wms/reference.html).
[Web Map Service (WMS)](https://www.ogc.org/publications/standard/wms/) endpoint is implemented using combination of Layman's authentication proxy, Layman's WMS proxy, and [GeoServer](https://docs.geoserver.org/2.21.x/en/user/services/wms/reference.html).

The authentication proxy understands same [authentication credentials](security.md#authentication) as Layman REST API (e.g. OAuth2 credentials) and passes the request to GeoServer with credentials understandable by GeoServer.

Expand All @@ -15,7 +15,7 @@ WMS respects [publication access rights](security.md#publication-access-rights).
GetLegendGraphic query is answered directly by GeoServer for layers with SLD style and can be parametrized according to [GeoServer documentation](https://docs.geoserver.org/latest/en/user/services/wms/get_legend_graphic/index.html). For layers with QML style is such query redirected by GeoServer to QGIS server and can be parametrized according to [QGIS documentation](https://docs.qgis.org/3.16/en/docs/server_manual/services.html?highlight=getlegendgraphic#getlegendgraphics).

## Web Feature Service
[Web Feature Service (WFS)](https://ogc.org/standard/wfs) endpoint is implemented using combination of Layman's authentication proxy, Layman's WFS proxy, and [GeoServer](https://docs.geoserver.org/2.21.x/en/user/services/wfs/reference.html).
[Web Feature Service (WFS)](https://www.ogc.org/publications/standard/wfs/) endpoint is implemented using combination of Layman's authentication proxy, Layman's WFS proxy, and [GeoServer](https://docs.geoserver.org/2.21.x/en/user/services/wfs/reference.html).

The authentication proxy behaves in the same way as in case of [WMS](#web-map-service).

Expand All @@ -31,4 +31,4 @@ WFS respects [publication access rights](security.md#publication-access-rights).
For layers in `EPSG:5514` and WFS requests in `CRS:84`, the features may have wrong coordinates by hundreds of meters. For requests in `EPSG:4326`, coordinates are correct.

## Catalogue Service
[Catalogue Service (CSW)](https://ogc.org/standard/cat) is implemented using [Micka](https://github.com/hsrs-cz/Micka).
[Catalogue Service (CSW)](https://www.ogc.org/publications/standard/cat/) is implemented using [Micka](https://github.com/hsrs-cz/Micka).
14 changes: 7 additions & 7 deletions doc/env-settings.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ Internal URL of REST API [Current User](rest.md#current-user) endpoint.
Internal URL (only protocol & host & port, without path) of Layman's REST API.

### LTC_REDIS_URL
URL of [Redis logical database](https://redis.io/commands/select) including database number where Layman Test Client stores user sessions including authentication credentials.
URL of [Redis logical database](https://redis.io/docs/latest/commands/select/) including database number where Layman Test Client stores user sessions including authentication credentials.

### LTC_SESSION_SECRET
See [`secret` at express-session documentation](https://www.npmjs.com/package/express-session#secret).
Expand All @@ -126,7 +126,7 @@ To enable not-so-secure SSL communication (UnsafeLegacyRenegotiation), set `NODE
## Connection to Redis

### LAYMAN_REDIS_URL
URL of [Redis logical database](https://redis.io/commands/select) including database number. Layman stores internal data about publications and users in this database. By default, Layman flushes the whole logical database on every startup! See also [LAYMAN_SKIP_REDIS_LOADING](#LAYMAN_SKIP_REDIS_LOADING).
URL of [Redis logical database](https://redis.io/docs/latest/commands/select/) including database number. Layman stores internal data about publications and users in this database. By default, Layman flushes the whole logical database on every startup! See also [LAYMAN_SKIP_REDIS_LOADING](#LAYMAN_SKIP_REDIS_LOADING).

## Connection to PostgreSQL

Expand Down Expand Up @@ -197,10 +197,10 @@ Filesystem directory where data published on QGIS are stored, including styles.
HTTP Basic Authentication credentials for communication with [CSW](#CSW_URL) encoded as `user:password`.

### CSW_URL
Internal URL of [OGC Catalogue Service v2.0.2](https://ogc.org/standard/cat) endpoint. Tested with [Micka](http://micka.bnhelp.cz/).
Internal URL of [OGC Catalogue Service v2.0.2](https://www.ogc.org/publications/standard/cat/) endpoint. Tested with [Micka](https://www.bnhelp.cz/produkty/metadata/).

### CSW_PROXY_URL
Public URL of [OGC Catalogue Service v2.0.2](https://ogc.org/standard/cat) endpoint. Tested with [Micka](http://micka.bnhelp.cz/).
Public URL of [OGC Catalogue Service v2.0.2](https://www.ogc.org/publications/standard/cat/) endpoint. Tested with [Micka](https://www.bnhelp.cz/produkty/metadata/).

### MICKA_ACCEPTED_VERSION
Version of Micka that Layman will accept on startup encoded as `version:revision`, e.g. `2020.014:2020-04-15.01`. Also, on one of '>=' or '==' prefixes can be used with obvious meaning, `e.g. >=2020.014:2020-04-15.01`. For prefix '>=', version and revision are compared independently as strings. If the variable is not set, a version defined in [`src/layman_settings.py`](../src/layman_settings.py) will be accepted. If none prefix is used, value is compared as with '=='.
Expand All @@ -211,13 +211,13 @@ String with public domain and optionally port, e.g. `<domain>` or `<domain>:<por
## Flask settings

### FLASK_APP
See [Flask documentation](https://flask.palletsprojects.com/en/2.3.x/cli/?highlight=flask_app#environment-variables-from-dotenv).
See [Flask documentation](https://flask.palletsprojects.com/en/stable/cli/?highlight=flask_app#environment-variables-from-dotenv).

### FLASK_DEBUG
See [Flask documentation](https://flask.palletsprojects.com/en/2.3.x/config/?highlight=flask_debug#DEBUG).
See [Flask documentation](https://flask.palletsprojects.com/en/stable/config/?highlight=flask_debug#DEBUG).

### FLASK_SECRET_KEY
See [Flask documentation](https://flask.palletsprojects.com/en/2.3.x/config/?highlight=secret_key#SECRET_KEY).
See [Flask documentation](https://flask.palletsprojects.com/en/stable/config/?highlight=secret_key#SECRET_KEY).

## Docker settings

Expand Down
2 changes: 1 addition & 1 deletion doc/metadata.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Metadata

Layman is able to publish partial metadata records to [OGC Catalogue Service](https://ogc.org/standard/cat) [Micka](http://micka.bnhelp.cz/). Records are partial because Layman does not know all metadata properties. Below are listed
Layman is able to publish partial metadata records to [OGC Catalogue Service](https://www.ogc.org/publications/standard/cat/) [Micka](https://www.bnhelp.cz/produkty/metadata/). Records are partial because Layman does not know all metadata properties. Below are listed
- [metadata properties that are known to Layman](#metadata-properties-known-to-layman)
- [metadata properties unknown to Layman](#metadata-properties-unknown-to-layman), that Layman is aware of.

Expand Down
Loading
Loading