Skip to content

Commit

Permalink
Create Sitemap Process and XML Formatter (#5)
Browse files Browse the repository at this point in the history
* Rename provider tests

* Add XML Formatter

* Create pygeoapi sitemap processor

* Update sitemap.py

* Add SitemapProcess testing

* Update README.md
  • Loading branch information
webb-ben authored Jul 31, 2023
1 parent 736987f commit 2852459
Show file tree
Hide file tree
Showing 12 changed files with 806 additions and 10 deletions.
9 changes: 7 additions & 2 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,9 @@ jobs:
with:
postgresql password: ${{ env.POSTGRESQL_PASSWORD }}
postgresql db: "test"
- name: Run pygeoapi
run: |
docker run -d -it --rm -p 5000:80 $(docker build -q .)
- name: Install requirements 📦
run: |
pip3 install https://github.com/geopython/pygeoapi/archive/refs/heads/master.zip
Expand All @@ -33,9 +36,11 @@ jobs:
env:
POSTGRESQL_PASSWORD: ${{ env.POSTGRESQL_PASSWORD }}
run: |
pytest tests/test_ckan.py
pytest tests/test_ckan_provider.py
pytest tests/test_postgresql_provider.py
pytest tests/test_sparql.py
pytest tests/test_sitemap_process.py
pytest tests/test_sparql_provider.py
pytest tests/test_xml_formatter.py
- name: run flake8 ⚙️
run: |
find . -type f -name "*.py" | xargs flake8
2 changes: 2 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
FROM geopython/pygeoapi:latest

ADD ./docker/pygeoapi.config.yml /pygeoapi/local.config.yml

ADD . /pygeoapi_plugins

RUN pip3 install -e /pygeoapi_plugins
Expand Down
23 changes: 20 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ pygeoapi plugins developed by the Center for Geospatial Solutions

## OGC API - Features

CGS additional feature providers are listed below, along with a matrix of supported query parameters.
Additional OGC API - Feature providers are listed below, along with a matrix of supported query parameters.

| Provider | Property Filters/Display | Result Type | BBox | Datetime | Sort By | Skip Geometry | CQL | Transactions | CRS |
| ------------------ | ------------------------ | ------------ | ---- | -------- | ------- | ------------- | --- | ------------ | --- |
Expand Down Expand Up @@ -80,7 +80,11 @@ The SPARQL Provider only uses variables prefixed with sparql\_ in the configurat

## OGC API - Processes

CGS provides an intersection process, using OGC API - Features Part 3: Filtering to return CQL intersections of features.
Additional OGC API - Process are listed below

### Intersector

The intersection process uses OGC API - Features Part 3: Filtering to return CQL intersections of features.
An example configuration in a pygeoapi configuration is below.

```
Expand All @@ -90,4 +94,17 @@ An example configuration in a pygeoapi configuration is below.
name: pygeoapi_plugins.process.intersect.IntersectionProcessor
```

This plugin is used in https://nhdpv2-census.internetofwater.app/.
This plugin is used in https:/reference.geoconnex.us/.

### Sitemap Generator

The Sitemap Generator process makes use of the XML formatter and OGC API - Features to generate a sitemap of the pygeoapi instance.
This can be used with the python package [sitemap-generator](https://github.com/cgs-earth/sitemap-generator) to generate a sitemap index.
An example configuration in a pygeoapi configuration is below.

```
sitemap-generator:
type: process
processor:
name: pygeoapi_plugins.process.sitemap.SitemapProcessor
```
5 changes: 0 additions & 5 deletions docker/examples/sparql/sparql.config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -184,8 +184,3 @@ resources:
homepage: foaf:homepage
wikipedia_link: foaf:isPrimaryTopicOf
time_zone: dbp:timezone

# intersector:
# type: process
# processor:
# name: pygeoapi_plugins.process.intersect.IntersectionProcessor
206 changes: 206 additions & 0 deletions docker/pygeoapi.config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,206 @@
server:
bind:
host: 0.0.0.0
port: 80
url: http://localhost:5000
mimetype: application/json; charset=UTF-8
encoding: utf-8
gzip: false
language: en-US
cors: true
pretty_print: true
limit: 10
# templates: /path/to/templates
map:
url: https://maps.wikimedia.org/osm-intl/{z}/{x}/{y}.png
attribution: '<a href="https://wikimediafoundation.org/wiki/Maps_Terms_of_Use">Wikimedia maps</a> | Map data &copy; <a href="https://openstreetmap.org/copyright">OpenStreetMap contributors</a>'
ogc_schemas_location: /schemas.opengis.net

logging:
level: ERROR
#logfile: /tmp/pygeoapi.log

metadata:
identification:
title: pygeoapi Demo instance - running latest GitHub version
description: pygeoapi provides an API to geospatial data
keywords:
- geospatial
- data
- api
keywords_type: theme
terms_of_service: https://creativecommons.org/licenses/by/4.0/
url: https://github.com/geopython/pygeoapi
license:
name: CC-BY 4.0 license
url: https://creativecommons.org/licenses/by/4.0/
provider:
name: pygeoapi Development Team
url: https://pygeoapi.io
contact:
name: Kralidis, Tom
position: Lead Dev
address: Mailing Address
city: City
stateorprovince: Administrative Area
postalcode: Zip or Postal Code
country: Canada
phone: +xx-xxx-xxx-xxxx
fax: +xx-xxx-xxx-xxxx
email: [email protected]
url: Contact URL
hours: Hours of Service
instructions: During hours of service. Off on weekends.
role: pointOfContact

resources:
places:
type: collection
title: Places
description: Cities around the world and their DBpedia context
keywords:
- sparql
- pygeoapi
- rdf
context:
- name: schema:name
description: schema:description
subjectOf: schema:subjectOf
links:
- type: application/html
rel: canonical
title: data source
href: http://dbpedia.org/
hreflang: en-US
extents:
spatial:
bbox: [-180, -90, 180, 90]
crs: http://www.opengis.net/def/crs/OGC/1.3/CRS84
temporal:
begin: null
end: null
providers:
- type: feature
name: pygeoapi_plugins.provider.sparql.SPARQLProvider
data: /pygeoapi_plugins/tests/data/places.csv
id_field: index
geometry:
x_field: lon
y_field: lat
sparql_provider: CSV
sparql_endpoint: https://dbpedia.org/sparql
sparql_subject: uri
sparql_predicates:
population: dbo:populationTotal
country: <http://dbpedia.org/ontology/country>
leader: dbpedia2:leaderName
populated:
type: collection
title: Populated Places
description: Populated places, public domain and their DBpedia context
keywords:
- sparql
- pygeoapi
- rdf
links:
- type: text/html
rel: canonical
title: information
href: http://www.naturalearthdata.com/
hreflang: en-US
extents:
spatial:
bbox: [-180, -90, 180, 90]
crs: http://www.opengis.net/def/crs/OGC/1.3/CRS84
temporal:
begin: 2011-11-11
end: null # or empty (either means open ended)
providers:
- type: feature
name: pygeoapi_plugins.provider.sparql.SPARQLProvider
data: /pygeoapi_plugins/tests/data/ne_110m_populated_places_simple.geojson
id_field: id
sparql_provider: GeoJSON
sparql_endpoint: https://dbpedia.org/sparql
sparql_subject: uri
sparql_predicates:
leader: dbpedia2:leaderName|dbp:leaderName
population: dbo:populationTotal|dbp:populationCensus
states:
type: collection
title: States
description: U.S. States and their DBpedia context
keywords:
- States
- Census
geojsonld: false
links:
- type: application/html
rel: canonical
title: data source
href: https://www.hydroshare.org/resource/3295a17b4cc24d34bd6a5c5aaf753c50/data/contents/states.gpkg
hreflang: en-US
extents:
spatial:
bbox: [-170, 15, -51, 72]
crs: http://www.opengis.net/def/crs/OGC/1.3/CRS84
temporal:
begin: null
end: null
providers:
- type: feature
name: pygeoapi_plugins.provider.sparql.SPARQLProvider
data: /pygeoapi_plugins/tests/data/states.gpkg
id_field: GEOID
table: states
sparql_provider: SQLiteGPKG
sparql_endpoint: https://dbpedia.org/sparql
sparql_subject: " :NAME"
sparql_predicates:
senator: dbp:senators
motto: dbo:motto|dbp:motto
capital: dbo:capital
homepage: foaf:homepage
wikipedia_link: foaf:isPrimaryTopicOf
time_zone: dbp:timezone
reservoirs:
type: collection
title: New Mexico Reservoirs
description: This is a point coverage of dams in the New Mexico, which originally was derived from the national inventory of dams data base (U.S. Army Corps of Engineers, 1982)
keywords:
- pygeoapi
- ckan
- api
context:
- name: schema:name
description: schema:description
subjectOf: schema:subjectOf
links:
- type: application/html
rel: canonical
title: data source
href: https://catalog.newmexicowaterdata.org/mn_MN/dataset/new-mexico-reservoirs/resource/08369d21-520b-439e-97e3-5ecb50737887
hreflang: en-US
extents:
spatial:
bbox: [-109, 31, -103, 37]
crs: http://www.opengis.net/def/crs/OGC/1.3/CRS84
temporal:
begin: null
end: null
providers:
- type: feature
name: pygeoapi_plugins.provider.ckan.CKANProvider
data: https://catalog.newmexicowaterdata.org/api/3/action/datastore_search
resource_id: 08369d21-520b-439e-97e3-5ecb50737887
id_field: _id
x_field: LONDD
y_field: LATDD
intersector:
type: process
processor:
name: pygeoapi_plugins.process.intersect.IntersectionProcessor
sitemap-generator:
type: process
processor:
name: pygeoapi_plugins.process.sitemap.SitemapProcessor
30 changes: 30 additions & 0 deletions pygeoapi_plugins/formatter/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# =================================================================
#
# Author: Benjamin Webb <[email protected]>
#
# Copyright (c) 2023 Center for Geospatial Solutions
#
# Permission is hereby granted, free of charge, to any person
# obtaining a copy of this software and associated documentation
# files (the "Software"), to deal in the Software without
# restriction, including without limitation the rights to use,
# copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the
# Software is furnished to do so, subject to the following
# conditions:
#
# The above copyright notice and this permission notice shall be
# included in all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
# OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
# HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
# WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
# OTHER DEALINGS IN THE SOFTWARE.
#
# =================================================================

"""Output formatter package"""
Loading

0 comments on commit 2852459

Please sign in to comment.