Skip to content

Commit

Permalink
major update
Browse files Browse the repository at this point in the history
  • Loading branch information
debpal committed Sep 24, 2024
1 parent 0081626 commit 62e2520
Show file tree
Hide file tree
Showing 14 changed files with 1,392 additions and 216 deletions.
18 changes: 15 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,16 +3,28 @@
SuomiGeoData is a Python package whose concept originated on September 11, 2024. It is designed to simplify the process of downloading and extracting geospatial data from Suomi, that is Finland. The package offers the following features:


* [Paituli](https://paituli.csc.fi/download.html) website
* [Paituli integration](https://paituli.csc.fi/download.html)

- Provides access to vector format index maps for downloading DEM and the topographic database.
- Downloads DEM as raster files and the topographic database as shapefiles based on label names from the index maps.
- Downloads all DEM labels intersected with a given vector format area.
- Downloads clipped DEM data that matches a given vector format area.

* [Syke integration](https://www.syke.fi/en-US/Open_information/Spatial_datasets/Downloadable_spatial_dataset)

- Downloads CORINE land cover 2018 raster.
- Downloads vector files of latest subcatchment divisions, ranging from level 1 to 5.
- Extracts individual or merged subcatchments by identifier and uses these areas to download DEM.

* Geoprocessing

- Simplified merging and clipping of raster files.


## Roadmap

* Enable downloading DEM for a specified area using a shapefile.
* Enable downloading the topographic database for a specified area using a shapefile.
* Implement searching and merging of features from the downloaded topographic database.


## Easy Installation
Expand Down Expand Up @@ -55,6 +67,6 @@ For detailed information, see the [documentation](http://suomigeodata.readthedoc
| **PyPI**| ![PyPI - Version](https://img.shields.io/pypi/v/SuomiGeoData) ![PyPI - Status](https://img.shields.io/pypi/status/SuomiGeoData) ![PyPI - Python Version](https://img.shields.io/pypi/pyversions/SuomiGeoData) ![PyPI - Wheel](https://img.shields.io/pypi/wheel/SuomiGeoData) ![PyPI - Downloads](https://img.shields.io/pypi/dm/SuomiGeoData) |
| **GitHub** | ![GitHub last commit](https://img.shields.io/github/last-commit/debpal/SuomiGeoData) [![flake8](https://github.com/debpal/SuomiGeoData/actions/workflows/linting.yml/badge.svg)](https://github.com/debpal/SuomiGeoData/actions/workflows/linting.yml) [![mypy](https://github.com/debpal/SuomiGeoData/actions/workflows/typing.yml/badge.svg)](https://github.com/debpal/SuomiGeoData/actions/workflows/typing.yml) [![pytest](https://github.com/debpal/SuomiGeoData/actions/workflows/testing.yml/badge.svg)](https://github.com/debpal/SuomiGeoData/actions/workflows/testing.yml) ![GitHub repo size](https://img.shields.io/github/repo-size/debpal/SuomiGeoData) |
| **Codecov** | [![codecov](https://codecov.io/gh/debpal/SuomiGeoData/graph/badge.svg?token=ORFQKXO96C)](https://codecov.io/gh/debpal/SuomiGeoData) |
| **Read**_the_**Docs** | [![Documentation Status](https://readthedocs.org/projects/suomigeodata/badge/?version=latest)](https://suomigeodata.readthedocs.io/en/latest/?badge=latest) |
| **Read** _the_ **Docs** | [![Documentation Status](https://readthedocs.org/projects/suomigeodata/badge/?version=latest)](https://suomigeodata.readthedocs.io/en/latest/?badge=latest) |
| **License** | ![PyPI - License](https://img.shields.io/pypi/l/SuomiGeoData) |

6 changes: 4 additions & 2 deletions SuomiGeoData/__init__.py
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
from .paituli import Paituli
from .syke import Syke


__all__ = [
'Paituli'
'Paituli',
'Syke'
]


__version__ = '0.1.0'
__version__ = '1.0.0'
193 changes: 191 additions & 2 deletions SuomiGeoData/core.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,11 @@
import os
import typing
import pyogrio
import rasterio
import rasterio.merge
import rasterio.drivers
import rasterio.mask
import geopandas


class Core:
Expand All @@ -7,7 +14,7 @@ class Core:
Core functionality of :mod:`SuomiGeoData` module.
'''

def is_valid_write_shape_driver(
def is_valid_ogr_driver(
self,
file_path: str
) -> bool:
Expand All @@ -34,6 +41,33 @@ def is_valid_write_shape_driver(

return output

def is_valid_raster_driver(
self,
file_path: str
) -> bool:

'''
Returns whether the given file path is a valid raster file.
Parameters
----------
file_path : str
File path to save the raster.
Returns
-------
bool
True if the file path is valid, False otherwise.
'''

try:
rasterio.drivers.driver_from_extension(file_path)
output = True
except Exception:
output = False

return output

@property
def _url_prefix_paituli_dem_tdb(
self,
Expand All @@ -60,9 +94,164 @@ def default_http_headers(
output = {
'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7',
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36',
'Host': 'www.nic.funet.fi',
'Accept-Encoding': 'gzip, deflate, br, zstd',
'Connection': 'keep-alive'
}

return output

def raster_merging(
self,
folder_path: str,
output_file: str,
raster_ext: str = '.tif',
**kwargs: typing.Any
) -> str:

'''
Merges raster files and returns a confirmation message.
Parameters
----------
folder_path : str
Folder path containing input raster files.
output_file : str
File path to save the output raster.
raster_ext : str, optional
Extension of input raster files. Defaults to '.tif' if not provided.
**kwargs : optional
Additional keyword arguments for updating the dictionary of
:attr:`rasterio.profile` attribute.
Returns
-------
str
A confirmation message indicating that the raster merging is complete.
'''

# file paths
if os.path.isdir(folder_path):
file_paths = filter(
lambda x: os.path.isfile(os.path.join(folder_path, x)),
os.listdir(folder_path)
)
else:
raise Exception(
'The folder path does not exist.'
)

# extract raster files
raster_files = filter(
lambda x: x.endswith(raster_ext),
file_paths
)

# raster merging
check_file = self.is_valid_raster_driver(output_file)
# output file check fail
if check_file is False:
raise Exception(
'Could not retrieve driver from the file path.'
)
else:
# open the split rasters
split_rasters = [
rasterio.open(os.path.join(folder_path, file)) for file in raster_files
]
# merge the split rasters
profile = split_rasters[0].profile
output_array, output_transform = rasterio.merge.merge(
datasets=split_rasters
)
# update merged raster profile
profile.update(
{
'height': output_array.shape[1],
'width': output_array.shape[2],
'transform': output_transform
}
)
for key, value in kwargs.items():
profile[key] = value
# save the merged raster
with rasterio.open(output_file, 'w', **profile) as output_raster:
output_raster.write(output_array)
# close the split rasters
for raster in split_rasters:
raster.close()

return 'Merging of rasters completed.'

def raster_clipping_by_mask(
self,
input_file: str,
mask_area: str | geopandas.GeoDataFrame,
output_file: str,
**kwargs: typing.Any
) -> str:

'''
Clips a raster file using a mask and returns a confirmation message.
Parameters
----------
input_file : str
File path to the input raster.
mask_area : str or GeoDataFrame
Mask area either as a file path or a GeoDataFrame.
output_file : str
File path to save the output raster.
**kwargs : optional
Additional keyword arguments for updating the dictionary of
:attr:`rasterio.profile` attribute.
Returns
-------
str
A confirmation message indicating that the raster clipping is complete.
'''

# mask area
if isinstance(mask_area, str):
mask_geometry = geopandas.read_file(mask_area).geometry.to_list()
elif isinstance(mask_area, geopandas.GeoDataFrame):
mask_geometry = mask_area.geometry.to_list()
else:
raise Exception('Input area must be either file or GeoDataFrame format.')

# raster clipping
check_file = self.is_valid_raster_driver(output_file)
# output file check fail
if check_file is False:
raise Exception(
'Could not retrieve driver from the file path.'
)
else:
# raster clipping
with rasterio.open(input_file) as input_raster:
profile = input_raster.profile
output_array, output_transform = rasterio.mask.mask(
dataset=input_raster,
shapes=mask_geometry,
all_touched=True,
crop=True
)
# update clipped raster profile
profile.update(
{'height': output_array.shape[1],
'width': output_array.shape[2],
'transform': output_transform}
)
for key, value in kwargs.items():
profile[key] = value
# save the clipped raster
with rasterio.open(output_file, 'w', **profile) as output_raster:
output_raster.write(output_array)

return 'Raster clipping completed.'
1 change: 1 addition & 0 deletions SuomiGeoData/data/example_area.cpg
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
UTF-8
Binary file added SuomiGeoData/data/example_area.dbf
Binary file not shown.
1 change: 1 addition & 0 deletions SuomiGeoData/data/example_area.prj
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
PROJCS["EUREF_FIN_TM35FIN",GEOGCS["GCS_ETRS_1989",DATUM["D_ETRS_1989",SPHEROID["GRS_1980",6378137.0,298.257222101]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]],PROJECTION["Transverse_Mercator"],PARAMETER["False_Easting",500000.0],PARAMETER["False_Northing",0.0],PARAMETER["Central_Meridian",27.0],PARAMETER["Scale_Factor",0.9996],PARAMETER["Latitude_Of_Origin",0.0],UNIT["Meter",1.0]]
Binary file added SuomiGeoData/data/example_area.shp
Binary file not shown.
Binary file added SuomiGeoData/data/example_area.shx
Binary file not shown.
Loading

0 comments on commit 62e2520

Please sign in to comment.