v1.20.0

Known issues

• #847 Publishing layers from an external table with @ in username or password fails
  • Fixed in v1.21.0.
• #818 Publishing QML layer from external DB with non-default geo-column name fails.
  • Fixed in v1.20.1.
• #812 Implicit namespace in WFS-T fails.
  • Fixed in v1.20.1.

Upgrade requirements

• Change environment variable LAYMAN_CLIENT_VERSION:

  LAYMAN_CLIENT_VERSION=v1.15.0
  

Migrations and checks

Schema migrations

• #703 Create new json column external_table_uri in publications table in prime DB schema.
• #703 Rename column file_type to geodata_type in publications table in prime DB schema.

Data migrations

• #703 Fill column external_table_uri in publications table in prime DB schema. Value is set to null for all existing publications.

Changes

• #703 Endpoints POST Workspace Layers and PATCH Workspace Layer support new body parameter external_table_uri.
• #703 Endpoints GET Workspace Layer and PATCH Workspace Layer return new keys:
  • original_data_source with values file or database_table
  • geodata_type which replaces key file.file_type that is deprecated now
  • db which replaces key db_table that is deprecated now
• #703 Attribute names in WFS-T requests must match to regex ^[a-zA-Z_][a-zA-Z_0-9]*$, otherwise Layman error is raised. It applies to attributes of both internal and external tables, and only to attributes that not exist in database yet.
• #703 Endpoint PATCH Workspace Layer raises exception if parameter crs is used without file parameter. It's the same behaviour as behaviour of POST Workspace Layers endpoint.
• #772 Speed up endpoints GET Workspace Layer Thumbnail, GET Workspace Layer Style, GET Workspace Map Thumbnail and GET Workspace Map File.
• #755 Fix generation of some map thumbnails by downgrading Node.js of Timgen from v18 to v16.
• #755 Change Node.js dependencies of Timgen:
  • http-server -> express 4
  • cors-anywhere -> http-proxy-middleware 2

v1.19.0

Known issues

• #755 If map contains WMS layer that is running on domain without TLS1.3 support (e.g. https://geoportal.kraj-lbc.cz/ ), generating thumbnail fails.
  • Fixed in v1.20.0.

Upgrade requirements

• Change environment variable LAYMAN_CLIENT_VERSION:

  LAYMAN_CLIENT_VERSION=v1.14.0
  

• If you are running Layman with development settings, run

make build-dev
make timgen-build
make client-build

Migrations and checks

Schema migrations

Data migrations

Changes

• #348 Upgrade GeoServer to 2.21.2.
  • #613 Workspace-specific WMS GetCapabilities documents includes LegendURL element for every style of every layer. Previously vector layers with QML style did not have it. GetLegendGraphic queries can be parametrized depending on layer style.
  • In workspace-specific WMS GetCapabilities documents, style name consists only of style name without <workspace>: prefix. For example, formerly it was testuser_wms:blue_style, now it is only blues_style.
  • #681 Enable to publish layer with specific SLD style.
  • #681 Endpoints POST Workspace Layers and PATCH Workspace Layer normalize grayscale float raster files with alpha channel to grayscale without it with internal mask 0/1.
  • Layman now uses official GeoServer docker image for demo and development purpose.
• #720 Upgrade Python dependencies
  • celery 5.0.5 -> 5.2.7
  • flask 2.0.2 -> 2.2.2
  • unidecode 1.3.2 -> 1.3.6
  • psycopg2-binary 2.9.3 -> 2.9.5
  • owslib 0.22.0 -> 0.27.2
  • requests 2.27.0 -> 2.28.1
  • jsonschema 4.3.3 -> 4.17.3
  • flower 1.0.0 -> 1.2.0
  • selenium 4.1.0 -> 4.7.2
  • cacheout 0.13.1 -> 0.14.1
  • kombu 5.2.3 -> 5.2.4
• #720 Remove Python dependency "pyproj".
• #720 Upgrade Python dev dependencies
  • pytest 6.2.5 -> 7.2.0
  • watchdog 2.1.6 -> 2.2.0
  • flake8 4.0.1 -> 6.0.0
  • pycodestyle 2.8.0 -> 2.10.0
  • pylint 2.7.4 -> 2.15.9
  • autopep8 1.6.0 -> 2.0.1
  • pytest-rerunfailures 10.2 -> 10.3
  • pytest-timeout 2.0.2 -> 2.1.0
• #726 Upgrade Node.js of Laymen Test Client from v12 to v18 and dependencies:
  • connect-redis 3 -> 6
  • dotenv 8 -> 16
  • http-proxy-middleware 0.21 -> 2
  • isomorphic-unfetch 2 -> 3
  • next 9 -> 13
  • react 16 -> 18
  • react-dom 16 -> 18
  • redis 3 -> 4
  • semantic-ui-react 0.88 -> 2
  • xml-formatter 2 -> 3
• #732 Upgrade Node.js of Timgen from v10 to v18 and dependencies:
  • ol 5 -> 7
  • http-server 0.11 -> 14
• #732 Use "vite" instead of "parcel" and "babel" for dev & build of Timgen.

v1.18.0

Upgrade requirements

• Change environment variable LAYMAN_CLIENT_VERSION:

  LAYMAN_CLIENT_VERSION=v1.13.0
  

Migrations and checks

Schema migrations

• #635 Create new boolean column image_mosaic in publication table.

Data migrations

• #635 Fill column image_mosaic in publications table in prime DB schema for all publications. Value of each publication is set to false.

Changes

• #635 Endpoints POST Workspace Layers and PATCH Workspace Layer support publishing timeseries raster layers. Temporal information is read from file names using new body parameter time_regex. Timeseries data files keep their original slugified names in both Layman and GeoServer data directories (instead of renaming to <layer_name>.<extension>). Each timeseries is published to GeoServer as one ImageMosaic coverage store.
• #446 If endpoint POST Workspace Layers receives grayscale input raster file (with or without alpha band) and if no input style was sent with the raster file, then Layman will automatically create and use customized SLD style to stabilize contrast of the layer in WMS.
• #446 Transparency of paletted GeoTIFF with transparent data values is respected in WMS. No custom style is needed. It was probably fixed in v1.16.0 by upgrade of GeoServer.
• #635 Endpoints GET Workspace Layer and PATCH Workspace Layer returns new subkeys:
  • file.paths with list of paths to all main data files
  • image_mosaic for raster layers stating that layer was published to GeoServer using ImageMosaic coverage store (true value for timeseries and false otherwise)
  • wms.time for timeseries with list of available time instants and regular expression used to extract them from file names
• #635 Subkey file.path is marked deprecated for endpoints GET Workspace Layer and PATCH Workspace Layer. Use file.paths instead.
• #635 Metadata sources returns new key 'temporal_extent'.
• #635 Endpoints POST Workspace Layers and PATCH Workspace Layer do not support combination of zip file and uncompressed main file.
• #697 Normalized GeoTIFF files are created as BigTIFF to enable publishing raster files greater than 4 GB.
• #660 Vector data files with invalid byte sequence (e.g. ShapeFile with invalid byte sequence in UTF-8 encoding) are first converted to GeoJSON, then cleaned with iconv, and finally imported to database.
• #667 Fix broken statistics during normalization of float rasters with big nodata value.
• #668 Fix broken size of raster in EPSG:3034 during normalization.
• #669 Fix slow publication of vector layers metadata to Micka. The reason was slow guessing of spatial_resolution.scale_denominator metadata property.
• #701 After publishing to GeoServer, Layman checks that Layer is available in WMS & WFS GetCapabilities to prevent situation when GeoServer hides publishing error. It may happen when data file with wrong CRS is published.

v1.17.0

Upgrade requirements

• Only versions 1.14.0 and newer can be upgraded to this version. For older versions, please upgrade to last 1.16.x first.
• Change environment variable LAYMAN_CLIENT_VERSION:

  LAYMAN_CLIENT_VERSION=v1.12.0
  

Migrations and checks

Schema migrations

• #576 Create new column file_type in publication table.
• #541 Rename vector data DB tables to layer_<uuid> format.

Data migrations

• #576 Fill column file_type in publications table in prime DB schema for all publications. Value of each map will be NULL. Value of each layer will be same as value of file.file_type in GET Workspace Layer response (i.e. vector, raster, or unknown).

Changes

• #551 Endpoints POST Workspace Layers and PATCH Workspace Layer support new body parameter overview_resampling.
• #576 Endpoints GET Layers and GET Workspace Layers returns new file.file_type key with the same value as file.file_type in GET Workspace Layer response (i.e. vector, raster, or unknown).
• #541 Layer name and map name can start with numbers.
• Maximum length of layer and map name is 210 characters.
• #606 Fix filtering and ordering publications by bounding box in case of publication with whole world bounding box in database.
• New environment variable OAUTH2_LIFERAY_SCOPE. Introduced in v1.16.2.
• New environment variable OAUTH2_LIFERAY_INTROSPECTION_SUB_KEY. Introduced in v1.16.1.
• #599 Layman supports uploading data files with upper or mixed case extensions. Introduced in v1.16.1.
• #541 Vector layers are stored in DB table with name in form layer_<UUID>, e.g. layer_96b918c6_d88c_42d8_b999_f3992b826958, previously the name of the table was the same as name of the layer.

v1.16.3

Known issues

• Endpoints GET Layers, GET Workspace Layers, GET Maps and GET Workspace Maps return HTTP error code 500 when there is layer/map in EPSG:4326 and bounding box including point with latitude 90 or -90 and query includes bbox_filter or ordering_bbox in EPSG:3857. #606
  • Fixed in v1.17.0.

Changes

• Stop Firefox instances if map thumbnail generation times out. Previously Firefox processes keeps running and were never stopped.
• Add logging for Micka response in case of exception.

v1.16.2

Known issues

• Layman can consume so many sources (CPU, RAM) that it leads to failure of async tasks or infinite running of async tasks. The reason is not stopping Firefox instances in some cases.
  • Fixed in v1.16.3.
• Endpoints GET Layers, GET Workspace Layers, GET Maps and GET Workspace Maps return HTTP error code 500 when there is layer/map in EPSG:4326 and bounding box including point with latitude 90 or -90 and query includes bbox_filter or ordering_bbox in EPSG:3857. #606
  • Fixed in v1.17.0.

Upgrade requirements

• Change environment variable LAYMAN_CLIENT_VERSION:

  LAYMAN_CLIENT_VERSION=v1.11.0
  

• If you are using Liferay as OAuth2 provider, set new environment variable OAUTH2_LIFERAY_SCOPE:

  OAUTH2_LIFERAY_SCOPE=liferay-json-web-services.everything.read.userprofile
  

  If you are using Wagtail, do not set this variable at all (not even to empty string).
• If you are running Layman with development settings, run

  make client-build
  

Changes

• New environment variable OAUTH2_LIFERAY_SCOPE.

v1.16.1

Known issues

• Layman can consume so many sources (CPU, RAM) that it leads to failure of async tasks or infinite running of async tasks. The reason is not stopping Firefox instances in some cases.
  • Fixed in v1.16.3.
• Endpoints GET Layers, GET Workspace Layers, GET Maps and GET Workspace Maps return HTTP error code 500 when there is layer/map in EPSG:4326 and bounding box including point with latitude 90 or -90 and query includes bbox_filter or ordering_bbox in EPSG:3857. #606
  • Fixed in v1.17.0.

Changes

• Fix infinity loop when generating map thumbnail. One of consequences was that such infinity loops consumed all celery workers and it was not possible to complete POST/PATCH map or layer.
• Fix empty map thumbnail. In some cases, map thumbnail was generated as if anonymous user asks for the map. Now the thumbnail is generated as if user with writing rights asks for the map.
• New environment variable OAUTH2_LIFERAY_INTROSPECTION_SUB_KEY.
• #599 Layman supports uploading data files with upper or mixed case extensions.

v1.16.0

Known issues

• Generating map thumbnail can ends up in infinity loop and consume all celery workers.
  • Fixed in v1.16.1.
• Layman can consume so many sources (CPU, RAM) that it leads to failure of async tasks or infinite running of async tasks. The reason is not stopping Firefox instances in some cases.
  • Fixed in v1.16.3.
• Endpoints GET Layers, GET Workspace Layers, GET Maps and GET Workspace Maps return HTTP error code 500 when there is layer/map in EPSG:4326 and bounding box including point with latitude 90 or -90 and query includes bbox_filter or ordering_bbox in EPSG:3857. #606
  • Fixed in v1.17.0.

Upgrade requirements

• Only versions 1.12.0 and newer can be upgraded to this version. For older versions, please upgrade to last 1.15.x first.
• Due to GeoServer upgrade, it's possible that make upgrade-demo fails with following error:

Waiting for GeoServer REST API, user=layman, url=http://geoserver:8080/geoserver/rest/workspaces/
Traceback (most recent call last):
  File "src/wait_for_deps.py", line 158, in <module>
    main()
  File "src/wait_for_deps.py", line 88, in main
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 960, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error:  for url: http://geoserver:8080/geoserver/rest/workspaces/

If you encounter such error, you can use script upgrade_v1_16_fix_gs.sh for fixing this issue. Be aware, that you will lose some security GeoServer settings, like Layman user and admin password, so you need to set them again after. The script needs to be run from Layman`s root directory:

make stop-demo
sh src/layman/upgrade/upgrade_v1_16_fix_gs.sh

After the script finishes, either set GEOSERVER_ADMIN_PASSWORD or create LAYMAN_GS_USER in GeoServer GUI.

• Set new environment variable LAYMAN_INPUT_SRS_LIST
• Change LAYMAN_CLIENT_VERSION to v1.10.0
• Unset environment variable LAYMAN_SETTINGS_MODULE, it has no effect anymore.
• If you are running Layman with development settings, run

make geoserver-build
make timgen-build
make client-build

Migrations and checks

Schema migrations

• #64 Create new column srid in publication table.

Data migrations

• #64 Native CRS of previously uploaded layers is set to EPSG:3857.
• #64 Native CRS of previously uploaded maps is set according their composition file (either EPSG:3857 or EPSG:4326) and their composition file is upgraded to version 2.0.0.

Changes

• #64 Upgrade GeoServer to 2.15.2, because 2.13.0 had serious problem with transformations of EPSG:5514.
• #64 Responses of GET Layers, GET Workspace Layers, GET Workspace Layer, PATCH Workspace Layer, GET Maps, GET Workspace Maps, GET Workspace Map, PATCH Workspace Map contains new attributes
  • native_crs with native CRS in form "EPSG:<code>", e.g. "EPSG:4326"
  • native_bounding_box with coordinates in native CRS [minx, miny, maxx, maxy]
• #64 New environment variable LAYMAN_INPUT_SRS_LIST
• #64 Layman supports import of layers in EPSG:3034, EPSG:3035, EPSG:5514, EPSG:32633, EPSG:32634 and EPSG:3059.
• #64 New raster layers are normalized in native CRS. New vector layers are imported into DB also in native CRS. Existing layers (normalized raster files, vector tables in DB) are kept in EPSG:3857 until they are patched with another file, or deleted.
• #519 Endpoints GET Layers, GET Workspace Layers, GET Maps, GET Workspace Maps support new query parameters bbox_filter_crs and ordering_bbox_crs.
• #64 Layer thumbnails are generated in native CRS of the layer.
• #64 WMS proxy was added to WMS endpoint. In case of some special WMS GetMap requests, it changes requested CRS to fix some GeoServer issues.
• #64 For layers in EPSG:5514 and WFS requests in CRS:84, the features may have wrong coordinates by hundreds of meters. For requests in EPSG:4326, coordinates are correct.
• #572 Endpoints POST Workspace Layers and PATCH Workspace Layer accept also raster files with .jpeg extension .
• #64 Map compositions are validated against map-composition-schema defined in describedBy key of map composition data JSON. Layman now supports only map compositions in version 2.
• #489 Error responses from Micka and GeoServer are logged into log and also propagated as part of raised exception, so they can be seen from flower.
• #548 Suppress GeoServer HTTP error 409 when setting layer access rights if they already have the same value.
• #548 If Micka returns HTTP error 500 on CSW/SOAP Insert/Update/Delete, retry the request.
• #548 If GeoServer returns HTTP error 500 on GetCapabilities, retry the request.
• Remove LAYMAN_SETTINGS_MODULE, import src/layman_settings.py directly.
• #555 Upgrade Selenium from 3 to 4
• #555 Use Firefox instead of Chrome in Selenium for map thumbnail generation and Layman Test Client tests.

v1.15.1

Changes

• #525 Keep NoData value in normalized raster files. Also NoData values are normalized as transparent always, even if Alpha channel is available.

v1.15.0

Known issues

• Normalized raster file created by Layman can be 100x bigger then uploaded raster, because compression is not used.
  • Fixed in v1.15.0.
• NoData values are changed to 0 during normalization. #525
  • Fixed in v1.15.1.

Changes

• #169 POST Workspace Layers accepts also compressed data files in ZIP format (*.zip) in file parameter. PATCH Workspace Layer accepts also data file in ZIP format (*.zip) in file parameter. ZIP archives can be also uploaded by chunks.
• #503 Raster data (e.g. GeoTIFF, JPEG2000, PNG, JPEG) sent on POST Workspace Layers and PATCH Workspace Layer are compressed during normalization to decrease occupied disk space.
• #232 Prefixes '>=' or '==' can be used in MICKA_ACCEPTED_VERSION environment variable.
• Documentation describes how to use external images in QML styles in POST Workspace Layers and PATCH Workspace Layer requests; see style parameter.
• #169 GET Workspace Layer returns path to main file inside archive if zipped file was sent (key file.path).
• #465 Fix situation, when Layman does not start if *.qgis file of the first layer with QML style does not exist. It was already fixed in v1.14.1.
• #464 Fix publishing layers with unusual attribute names (e.g. x, or Číslo) and QML styles. It was already fixed in v1.14.1.
• #459 Fix situation, when sometimes publication stayed in PENDING status after failure. It was already fixed in v1.14.1.
• #502 Fix error message, when invalid raster file is sent.
• Fix: Layers, which were created in Layman, but not published in GeoServer due to any validation/error can now be patched. Previously internal error was raised for every PATCH Workspace Layer call.
• Rename item username to workspace in error 41.
• Detailed test-related documentation for developers.
• #487 Upgrade
  • flask from 2.0 to 2.0.2+
  • celery from 4.4.7 to 5.0.5
  • kombu from 4.6.10 to 5.1.0
  • flower from 0.9.7 to 1.0.0
  • jsonschema from 3.2.0 to 4.0.1
  • psycopg2-binary from 2.8.6 to 2.9.1
  • requests from 2.25.1 to 2.26.0
  • unidecode from 1.2.0 to 1.3.2