Raster operations

[msbarry]

I recently tweaked planetiler to generate an mbtiles of terrarium RGB tiles from https://registry.opendata.aws/terrain-tiles/. I just did this in a one-off branch but I'm curious if people are interested in using planetiler for generating raster tilesets?

A few possibilities I could think of

• copy from one raster tile source (http) to another (mbtiles), applying transformations like reducing the precision, compressing, transforming from png to webp, or merging 256px tiles into 512px one zoom-level lower.
• generate DEM tiles from raw sources (like jaxa, srtm, skadi, etc.)

There are other tools in this space, just curious if there's any appetite for planetiler to do them outside my one-off hacking?

[wipfli]

Yeah I would love to use planetiler to create terrain or other raster tiles. I know nothing about raster sources and having a simple way with planetiler to get frontend ready tiles would be great...

[tyrauber]

@msbarry, I am very interested in this as well. I actually need to work up an Outdoor theme. I've got the NASA ASTER images on an EC2 Volume. I was actually going to run through a series of GDAL contour transformations to produce contour lines, hillshade, etc, and assemble those tiles into a shapefiles, and then convert those to mbiles, etc, and then use them as layers in a Planetiler build profile. But perhaps there is a better way? Is dem-download the branch? Can I help in anyway?

[bdon]

@tyrauber The DEM would need to be warped into Web Mercator before slicing Z/X/Y.

@msbarry how much code would this share with the rest of Planetiler? I think contour line generation could be interesting as that's inherently vector, but maybe it would make sense for the raster operations to live in a separate codebase?

[msbarry]

Thanks @tyrauber - For # 2, I think the goal would be to convert the raster landcover layer from ESA to vector shapes.

I agree gdal bindings seem like a good compromise to get started fast. For contour lines for example, the workflow would be to take a 1°x1° elevation tile, use gdal to generate contour isoline shapes, then pass those through the rest of planetiler's vector shape handling, with some slight tweaks to ensure that contours meet properly at intersections since I think those 1° tiles are slightly overlapping.

That's a great overview of DEM sources. The gzipped 1° "skadi" tiles on AWS are about 200gb total and while that source is 30m, it's built from sources that vary in resolution a lot more. @acalcutt also pointed me to the JAXA 30m dataset https://www.eorc.jaxa.jp/ALOS/en/aw3d30/data/index.htm although I'm not sure how the quality compares to those others and terrarium?

@bdon here are the categories I see:

• raw raster source->vector tiles processing would share the entire processing pipeline and you may want the layers to end up in the same tileset as layers processed from OSM so definitely makes sense to have that in planetiler
• raw raster source->raster tiles would share tile writing pipeline, and if you are overlaying multiple sources then possibly feature sorting/merging too
• raw raster tile -> raster tile would just share the tile reading/writing code and parallel processing so less of a case to include - except now that we have the unified tile archive writer/readers I was thinking of building some small tools that just copy from readers to writers, optionally applying transformations like merging, stripping vector layers/attributes, and these would be similar to that category.

[tyrauber]

Yeah @bdon, we'd need to do a srid translation operation first with either gdalwarp -s_srs EPSG:4269 -t_srs EPSG:3857 or gdalwarp -s_srs EPSG:4269 -t_srs EPSG:3857.

@msbarry I had the same initial thought as @bdon, regarding whether this operation should be done outside of planetiler and pass the layer in as a shapefile or mbtile? Couldn't the tool prep and output a file that Planetiler could still use in the processing pipeline, with addShapefileSource or something like it?

Each one of these gdal operations is going to require some configuration.

For example, contours will require at least the interval (-i) and polygon (-p) flag. Hillshading, Slope and Color Relief will require even more configuration including color text files.

Here is some pseudo code of the gdal steps for producing contour lines, and hillshade, slope and color relief.

// 3857 projection
gdalwarp -s_srs EPSG:4269 -t_srs EPSG:3857 -r bilinear source.tif source_3857.tif

// contour lines

gdal_contour -a elev source_3857.tif source_contour.shp -i 10.0

// hillshade
gdaldem hillshade -az 45 -z 1.3 source_3857.tif hillshade_3857.tif

// color relief
gdaldem color-relief source_3857.tif color-relief.txt relief_3857.tif

// Slope
gdaldem slope source_3857.tif slope_3857.tif
gdaldem color-relief slope.tif color-slope.txt slope-shade.tif

// Combine
gdalwarp hillshade_3857.tif relief_3857.tif slope_3857.tif output.tif -srcnodata -dstnodata

source_contour.shp would need to be processed into a vector layer, combining and optimizing the polygons across tiles.

output.tif would need to be translated to a raster layer. As much as I'd love hillshade, slope and color relief as a vector, for client side styling, I just don't know how that would be possible.

References:

• tranform gdaltransform
• warp gdalwarp
• contour lines gdal_contour
• hillshade, relief and slope gdaldem

[wipfli]

Landcover and hillshading is nicely combined in a vector layer by google maps for example:

[tyrauber]

👍 @wipfli, if we could accomplish that, that would be stellar.