v1.23.2

Upgrade requirements

• #1021 After upgrade and start of Layman, it's needed to change GeoServer configuration by following steps:
  1. Log in to GeoServer Admin
  2. Click on Security > Authentication
  3. Find part "Authentication Filters" and click on "laymanHttpHeader"
  4. In part "Role source" choose "Role service" in left dropdown and "layman_role_service" in right dropdown.
  5. Click on Save button.

Changes

• #1021 Fix WFS-T error by manually changing GeoServer configuration as described in upgrade requirements.
• #1022 Fix login in demo configuration with authentication.

v1.23.1

Known issues

• #1021 WFS-T error "<layer> is read-only".
  • Fixed in v1.23.2.
• #1022 Login not working in demo configuration with authentication
  • Fixed in v1.23.2.

Changes

• #1016 Flower and Celery worker containers used to wait for all dependencies and also check role service, from now on it waits only for Layman API to be ready.
• #165 Roles in REST endpoint GET Roles result are in alphabetical order.

v1.23.0

Known issues

• #1016 Flower container did not start if LAYMAN_ROLE_SERVICE_URI points to host.docker.internal.
  • Fixed in v1.23.1.
• #1021 WFS-T error "<layer> is read-only".
  • Fixed in v1.23.2.
• #1022 Login not working in demo configuration with authentication
  • Fixed in v1.23.2.

Upgrade requirements

• Change environment variable LAYMAN_CLIENT_VERSION:

  LAYMAN_CLIENT_VERSION=v1.18.0
  

• Stop using environment variable LAYMAN_GS_ROLE_SERVICE, it has no effect to Layman anymore. Layman now uses role service identified by new environment variable LAYMAN_ROLE_SERVICE_URI. The service is called layman_role_service on GeoServer.
• Set new environment variable LAYMAN_ROLE_SERVICE_URI
• Stop using environment variable OAUTH2_INTROSPECTION_SUB_KEY, it has no effect to Layman anymore.
• If you are using Wagtail as OAuth2 provider
  • After running make upgrade-demo or make upgrade-demo-full, run also script v1_23_change_oauth2_sub_username_to_user_id.py:

    docker compose -f docker-compose.deps.demo.yml -f docker-compose.demo.yml run --rm --no-deps -u root -e LAYMAN_WAGTAIL_DB_URI=<URI_of_Wagtail_db> layman bash -c "cd src && python3 -B v1_23_change_oauth2_sub_username_to_user_id.py"

    • URI_of_Wagtail_db is PostgreSQL connection URI to Wagtail database, e.g. postgresql://user:[email protected]:5432/wagtail_db_name
    • The script changes OAuth2 "sub" values in Layman prime DB schema from Wagtail usernames to Wagtail user IDs. See 940.

Migrations and checks

Schema migrations

• #165 Add column role_name to table rights in prime DB schema. Add constraint that exactly one of columns role_name and id_user is not null.
• #165 Create DB schema _role_service that can be used as role service.
• #165 Column name in table workspaces in prime DB schema length is changed to 59 characters.
• Drop DB table right_types.

Data migrations

• #165 Delete technical roles and user-role relations in GeoServer default role service, which is now replaced by JDBC role service.

Changes

• #165 Prior to this version, Layman enabled to use usernames and pseudo-role EVERYONE in access rights. From now on, Layman accepts also role names.
• #165 Roles (except of EVERYONE) are managed by role service.
• #165 New REST endpoint GET Roles with list of all roles registered in role service, that can be used in access rights.
  • This new endpoint was added to Test Client into tab "Others".
• #165 POST Workspace Layers/Maps and PATCH Workspace Layer/Map saves role names mentioned in access_rights.read and access_rights.write parameters into prime DB schema.
• #165 Many requests respect roles in access rights:
  • GET/PATCH/DELETE Workspace Layer
  • GET Workspace Layer Thumbnail/Style/Metadata Comparison/Chunk
  • GET/PATCH/DELETE Workspace Map
  • GET Workspace Map File/Thumbnail/Metadata Comparison
  • GET Workspace Layers/Maps
  • GET Layers/Maps/Publications
  • DELETE Workspace Layers/Maps
  • requests to WMS and WFS endpoints
• #165 POST Workspace Layers/Maps respects roles in GRANT_CREATE_PUBLIC_WORKSPACE and GRANT_PUBLISH_IN_PUBLIC_WORKSPACE
• #165 Many endpoints return previously associated role names in access_rights.read and access_rights.write keys:
  • GET/PATCH Workspace Layer
  • GET/PATCH Workspace Map
  • GET Workspace Layers/Maps
  • GET Layers/Maps/Publications
• #165 Name of users and public workspaces are from now on restricted to a maximum length of 59 characters.
• 940 Use userId as OAuth2 "sub" instead of username. This is suitable for Wagtail.
• 940 Stop supporting Liferay as OAuth2 provider.
• 941 Wagtail database is now persistent when restarting Layman or Wagtail.
• All changes from v1.22.1, v1.22.2 and v1.22.3.
• #960 Handle WMS requests with HTTP error more efficiently in timgen.
• #962 Make values of layman_metadata.publication_status and status key(s) more consistent in responses of PATCH Workspace Layer/Map and GET Workspace Layer/Map.
• Timgen handles better some edge cases:
  • If map's internal layer points to non-existent workspace, Timgen is faster and does not fail.
  • If map's file was not obtained from Layman, Timgen fails quickly.
• Upgrade Python dependencies
  • werkzeug 2.3.7 -> 3.0.7 (suggested by dependabot)
  • urllib3 1.26.16 -> 1.26.18 (suggested by dependabot)
  • pillow 10.0.0 -> 10.1.0 (suggested by dependabot)
  • cacheout 0.14.1 -> 0.15.0
  • flask 2.3.3 -> 3.0.0
  • jsonschema 4.19.0 -> 4.20.0
  • owslib 0.29.2 -> 0.29.3
  • importlib-metadata 6.8.0 -> 7.0.0
  • psycopg2-binary 2.9.7 -> 2.9.9
  • redis 5.0.0 -> 5.0.1
  • unidecode 1.3.6 -> 1.3.7
  • platformdirs 3.10.0 -> 4.1.0
  • pycodestyle 2.11.0 -> 2.11.1
  • pytest 7.4.2 -> 7.4.3
  • pytest-rerunfailures 12.0 -> 13.0
  • pytest-timeout 2.1.0 -> 2.2.0
• Add development dependency plantuml/plantuml for rendering PlantUML diagrams.

v1.22.3

Known issues

• #1022 Login not working in demo configuration with authentication
  • Fixed in v1.23.2.

Upgrade requirements

• Change environment variable LAYMAN_CLIENT_VERSION:

  LAYMAN_CLIENT_VERSION=v1.17.2
  

Changes

• Improve logging in Layman Test Client related to Passport.js.
• 968 Enable to use not-so-secure SSL communication (UnsafeLegacyRenegotiation) in Layman Test Client. It can be activated by environment variable NODE_OPTIONS:

  NODE_OPTIONS="--openssl-config=/code/unsafe_openssl.cnf"
  

v1.22.2

Known issues

• #1022 Login not working in demo configuration with authentication
  • Fixed in v1.23.2.

Upgrade requirements

• Change environment variable LAYMAN_CLIENT_VERSION:

  LAYMAN_CLIENT_VERSION=v1.17.1
  

Changes

• #958 Fix X-Forwarded-Proto on LTC proxy with https protocol in demo environment.
• #956 Fix login to GeoServer with https protocol.
• #952 Fix patch saving publication files before access right validation.

v1.22.1

Known issues

• #956 Unable to login into GeoServer with https protocol
  • Fixed in v1.22.2.
• #958 Request through LTC proxy returns URLs with http although request came to https
  • Fixed in v1.22.2.
• #1022 Login not working in demo configuration with authentication
  • Fixed in v1.23.2.

Changes

• #949 Fix reading map-layer relations from map composition that includes also other than WMS and Vector layers.

v1.22.0

Known issues

• #949 Reading map-layer relations fails for map composition that includes also other than WMS and Vector layers.
  • Fixed in v1.22.1.
• #956 Unable to login into GeoServer with https protocol
  • Fixed in v1.22.2.
• #958 Request through LTC proxy returns URLs with http although request came to https
  • Fixed in v1.22.2.
• #1022 Login not working in demo configuration with authentication
  • Fixed in v1.23.2.

Upgrade requirements

• Change environment variable LAYMAN_CLIENT_VERSION:

  LAYMAN_CLIENT_VERSION=v1.17.0
  

• Stop using environment variable LAYMAN_GS_PROXY_BASE_URL, it has no effect to Layman anymore.
  • GeoServer's Proxy Base URL is now automatically set by Layman on each start. Value is automatically derived from environment variables LAYMAN_CLIENT_PUBLIC_URL (protocol) and LAYMAN_PROXY_SERVER_NAME (domain and port). URL path is always /geoserver/.
• Stop using environment variable LAYMAN_GS_PATH, it has no effect to Layman anymore.
  • GeoServer's URL path must be always /geoserver/ (that is true for GeoServer shipped with Layman).

Migrations and checks

Schema migrations

• #868 Create new table map_layer in prime DB schema.

Data migrations

• #887 Drop unique DB index on users table on sub column. Create unique DB index on users table on (sub, issuer_id) columns.
• #765 Fix issuer_id values in users table that were broken since v1.21.0.
• #765 Remove authn.txt files from workspace directories. The same information as in authn.txt files is saved in prime DB schema.
• #868 Fill table map_layer with relations between maps and internal layers (layers published on this Layman instance). Relations to external layers (layers of other servers) are not imported into the table.

Changes

• #868 Responses to many requests respect HTTP X-Forwarded headers of the request. Those requests are:
  • GET Publications, Layers, Workspace Layers, Maps, and Workspace Maps
  • GET, PATCH, and DELETE Workspace Layer
  • GET, PATCH, and DELETE Workspace Map
  • GET Workspace Map File
  • POST and DELETE Workspace Layers
  • POST and DELETE Workspace Maps
  • requests to WMS and WFS endpoints
• #868 Responses to GET Workspace Layer Metadata Comparison and GET Workspace Map Metadata Comparison do not respect HTTP X-Forwarded headers of the request intentionally, in order to keep URLs in canonical form.
• #868 Relations between map and internal layers are updated in map_layer table when calling POST Workspace Maps, PATCH Workspace Map, DELETE Workspace Map, and DELETE Workspace Maps.
• #901 Endpoints POST and PATCH accept map compositions in version 3.x.x.
• #927 SLD styles are internally published to GeoServer with raw parameter set to True.
• #880 Use Docker Compose v2 (docker compose) in Makefile without compatibility flag and remove Makefile_docker-compose_v1 file. Docker containers are named according to Docker Compose v2 and may have different name after upgrade.
• #765 Stop saving OAuth2 claims in filesystem, use prime DB schema only.
• #893 It is possible to specify logging level by new environment variable LAYMAN_LOGLEVEL. Default level is INFO.
• Use psycopg2.pool.ThreadedConnectionPool to share DB connections.
• Add new test Python dependency:
  • jsonpath-ng 1.6.0
• Upgrade Python dependencies
  • certifi 2023.5.7 -> 2023.7.22 (suggested by dependabot)
  • tornado 6.3.2 -> 6.3.3 (suggested by dependabot)
  • flask 2.3.2 -> 2.3.3
  • jsonschema 4.17.3 -> 4.19.0
  • lxml 4.9.2 -> 4.9.3
  • owslib 0.28.1 -> 0.29.2
  • psycopg2-binary 2.9.5 -> 2.9.7
  • redis 4.5.5 -> 5.0.0
  • autopep8 2.0.1 -> 2.0.4
  • flake8 6.0.0 -> 6.1.0
  • pillow 9.3.0 -> 10.0.0
  • pycodestyle 2.10.0 -> 2.11.0 (to be consistent with GitHub Actions)
  • pytest 7.2.0 -> 7.4.2
  • pytest-rerunfailures 10.3 -> 12.0
  • watchdog 2.2.0 -> 3.0.0

v1.21.1

Known issues

• #1022 Login not working in demo configuration with authentication
  • Fixed in v1.23.2.

Migrations and checks

Data migrations

• #887 Creating unique DB index on users.sub column.

Changes

• #887 Fix creating new user workspaces for existing users that happened in v1.21.0.

v1.21.0

Known issues

• #887 New user workspaces are created for already existing users.
  • Fixed in v1.21.1.
• #1022 Login not working in demo configuration with authentication
  • Fixed in v1.23.2.

Upgrade requirements

• Change environment variable LAYMAN_CLIENT_VERSION:

  LAYMAN_CLIENT_VERSION=v1.16.0
  

• Rename environment variable OAUTH2_LIFERAY_SECRET to OAUTH2_CLIENT_SECRET.
• Rename all environment variables OAUTH2_LIFERAY_SECRET<n> to OAUTH2_CLIENT<n>_SECRET. For example, variable OAUTH2_LIFERAY_SECRET4 becomes OAUTH2_CLIENT4_SECRET.
• Rename all other OAUTH2_LIFERAY_<postfix> environment variables to OAUTH2_<postfix>. For example, variable OAUTH2_LIFERAY_AUTH_URL becomes OAUTH2_AUTH_URL.
• If you are using environment variable OAUTH2_CALLBACK_URL, change only its URL path from /client/authn/oauth2-liferay/callback to /client/authn/oauth2-provider/callback. Keep protocol, domain, and port unchanged.
• Stop using environment variable LAYMAN_AUTHN_OAUTH2_PROVIDERS, it has no effect to Layman anymore. There is exactly one OAuth2 provider Python module now, no need to set it.
• Stop using environment variable FLASK_ENV, it has no effect to Layman anymore.
  • If you used environment variable FLASK_ENV with value development, add new environment variable FLASK_DEBUG:

    FLASK_DEBUG=1
    

• Stop using environment variable CSW_RECORD_URL, it has no effect to Layman anymore. Value is derived from variable CSW_PROXY_URL.
• Stop using HTTP header AuthorizationIssUrl when authenticating by OAuth. The header has no effect to Layman anymore. There is exactly one OAuth2 provider now, no need to distinguish it. Now, the only distinguished HTTP header when authenticating by OAuth2 is Authorization header.
• If you are running Layman with development settings (e.g. starting it with make start-dev)
  • change values of environment variables:

    OAUTH2_CLIENT_ID=VECGuQb00tWt8HZNkA4cxu6dnoQD5pF6Up3daAoK
    OAUTH2_CLIENT_SECRET=aY14rwkEKasNqBEZX8OnhpRk8lpHAfT7oKTlf4LriEK8oMZxhnGKcnt4bZ72pceNEl83B6LtBvhKr3BqBLFA80Pd6Ugav2rkc8bk7TE4LkaoB2qcBQmjiOiEpizsgZGx
    OAUTH2_AUTH_URL=http://localhost:8083/o/authorize
    OAUTH2_TOKEN_URL=http://wagtail:8000/o/token/
    OAUTH2_INTROSPECTION_URL=http://wagtail:8000/o/introspect/
    OAUTH2_INTROSPECTION_SUB_KEY=username
    OAUTH2_USER_PROFILE_URL=http://wagtail:8000/profile
    

  • unset environment variable OAUTH2_SCOPE (previously OAUTH2_LIFERAY_SCOPE)
  • after usual dev upgrade commands run also

    make wagtail-build
    

• If your upgrade command ends with message Error response from daemon: invalid IP address in add-host: "host-gateway", try to uninstall Docker Engine completely and install it again with Docker Compose plugin (see e.g. manual for Centos). Then run upgrade command again.

Migrations and checks

Schema migrations

• #528 Add new data type enum_wfs_wms_status and create new string column wfs_wms_status in publications table in prime DB schema.

Data migrations

• #528 Fill column wfs_wms_status in publications table in prime DB schema. Set value AVAILABLE for each vector layer that is fully available in WFS and WMS and for each raster layer that is fully available in WMS. Set NOT_AVAILABLE for all other layers and null for all existing maps.
• #520 Set MetadataURL for each layer in WFS and WMS workspace in GeoServer.

Changes

• #769 New request GET Publications was added. It enables querying both layers and maps by single request.
• #769 New key publication_type was added to responses of requests GET Publications, GET Layers, GET Workspace Layers, GET Maps, and GET Workspace Maps. Possible values of the key are layer and map.
• #528 New key wfs_wms_status was added to layer items in responses of requests GET Layers, GET Workspace Layers, and GET Publications.
• #520 New element MetadataURL was added for each layer to GetCapabilities response of WFS 2.0.0 and WMS 1.3.0. The element contains URL of CSW metadata record of the layer.
• #800 Requests POST Workspace Layers and PATCH Workspace Layer support new parameter time_regex_format. Its value is later accessible in the new subkey wms.time.regex_format in responses of GET Workspace Layer and PATCH Workspace Layer requests.
• #764, #860 Layman accepts new types of QML styles:
  • labels without symbology
  • point clustering
• #857 Requests POST Workspace Layers and PATCH Workspace Layer accept host.docker.internal in external_table_uri parameter to reach localhost of host server.
• #847 Fix publishing external table layers with @ character or other dangerous characters in the username or in the password.
• #833 Make Timgen WMS requests more robust (handle WMS errors, delayed retry, add timestamp to each request).
• #877 Use Docker Compose v2 (docker compose) in Makefile. As of now, all containers are named in the same way as previously. Old Makefile using Docker Compose v1 (docker-compose) is archived as Makefile_docker-compose_v1. It will be removed in the next minor release.
• #815 Propagate LAYMAN_PROXY_SERVER_NAME value to GeoServer environment variable GEOSERVER_CSRF_WHITELIST.
• #765 Remove Liferay from dev stack, use Wagtail CRX + Django OAuth Toolkit as new OAuth2 provider (authorization server).
• Upgrade Python dependencies
  • flask 2.2.2 -> 2.3.2
  • redis 4.5.1 -> 4.5.4
  • owslib 0.27.2 -> 0.28.1
  • requests 2.28.1 -> 2.31.0
• Upgrade Node.js Timgen dependencies
  • vite 3.2.5 -> 3.2.7
• Document that temporal part of timeseries datetime dimension extracted by time_regex parameter is cut off, so the smallest possible unit of datetime dimension is one day.

v1.20.1

Known issues

• #847 Publishing layers from an external table with @ in username or password fails
  • Fixed in v1.21.0.

Changes

• #818 Fix publishing QML layer from external DB with geo-column name other than wkb_geometry.
• #812 Fix processing of WFS-T queries with implicit namespace.
• Fix time_regex sample values in documentation and tests.