ODC EP 005 Support WMTS API based dataset locations

ODC Enhancement: Support WMTS/Other Image API based dataset locations

Overview

There are a couple of use cases where expecting file or cloud blob-storage based access to datasets is unreasonable or not performant enough.

1. Sensor fusion with data from proprietary sources.
2. Large amounts IO based processing e.g. via WPS running outside the boundaries of the cloud account where the better solution would be say S3 + FSx.

Proposed By

Tisham Dhar

State

• Under Discussion
• In Progress
• Completed
• Rejected
• Deferred

Motivation

For comparison of IO performance and limits.

• Cloudfront offers 250,000 requests per second.
• S3 offers 5,500 requests per second.

The WMTS (or other similar API) based access to data would unlock.

• Google Earth Engine sources as Arrays
• Microsoft Planetary Computer sources (though Azure blob store support does this)
• Geoscience Australia NLI elevation sources while meeting reporting needs for data usage
• Aerometrex Aerial Imagery and Elevation sources while meeting commercial needs
• Planet Satellite Imagery sources while meeting commercial needs

Limitations of WMTS specifically

• Can only access a maximum of three uint8 bands per requests. Additional bands only possible if exposed by the source service as styles - and even then there is no standardised support in Capabilities documents to advertise which bands are available in each style and how they are mapped to the RGB bands in returned image. WCS would be a more natural protocol choice given this.
• Requests restricted to defined tile matrix sets. WMS or WCS would be a more natural protocol choice given this.

Key Question/Challenges

• Which protocols do the target proprietary providers offer?
• Is there a "lowest common denominator" protocol all providers support, or would we need to support independent protocols for each provider?
• Do the providers map their products to the protocol(s) in a consistent manner, or would we need special handling for each provider?
• We would presumably at least have to handle authentication specially for each targetted provider.