GDAL in Geoserver Cloud

[ppradela]

Do these environment variables LD_LIBRARY_PATH and GDAL_DATA work in the cloud version? I have GDAL installed in my system and downloaded GDAL plugin (jar files). Will it work if I put valid paths to these variables during the containers deployment and upload the GDAL plugin using webui importer?

[groldan]

Not that easy:

• we need to install gdal-bin on the docker images. In order to do so, it'd be better if all the geoserver service images shared a common base image so they can share as many layers as possible (WIP at Use common base docker images to share layers across service images #396)
• The gdal-java package is not available anymore from the apt repositories, so we need to install gdal.jar and gdalalljni.so by some other means
• With that in place it'd be just a matter of including the geoserver GDAL and OGR extensions as maven dependencies

[ppradela]

Thanks for the reply. On the normal version of Geoserver when I used it and I wanted to have GDAL I built the Geoserver image based on the GDAL image. As far I know the Geoserver image and the GDAL image are based on the Ubuntu image.

[groldan]

yeah, I've been experimenting with options. Of which, using osgeo/gdal:ubuntu-full-3.6.3 as base image brings up the issue of it being 1.64GB, so we'd end up with ~2GB images.

[groldan]

note right now we have

geoservercloud/geoserver-cloud-rest             1.6-SNAPSHOT             101c2da87680   6 days ago      637MB
geoservercloud/geoserver-cloud-gwc              1.6-SNAPSHOT             c65e10edabdd   6 days ago      641MB
geoservercloud/geoserver-cloud-wps              1.6-SNAPSHOT             553f62f0ecc2   6 days ago      643MB
geoservercloud/geoserver-cloud-wms              1.6-SNAPSHOT             379088befc8b   6 days ago      640MB
geoservercloud/geoserver-cloud-webui            1.6-SNAPSHOT             6e7ccef21fde   6 days ago      656MB
geoservercloud/geoserver-cloud-wfs              1.6-SNAPSHOT             cbe7d3b93465   6 days ago      622MB
geoservercloud/geoserver-cloud-wcs              1.6-SNAPSHOT             f06868de7437   6 days ago      604MB

where all of them share the layers of the base image

geoservercloud/gs-cloud-base-geoserver-image    1.6-SNAPSHOT             843f4accd820   6 days ago      463MB

which is built but not pushed

[jemacchi]

Check this PR
It includes code for a GDAL base image which would facilitate multistage docker images generation.

[jemacchi]

Just for record, and some manual testing (if required).
The following Dockerfile builds a GDAL image based on the same temurin base image as in GSCloud.
And then on top of that deploys a Geoserver vanilla 2.25.3 with GRIB and NETCDF plugins included (along others, not relevant).

FROM eclipse-temurin:11-jdk as builder

ENV GDAL_VERSION 3.8.4
ENV GDAL_PATH /usr/share/gdal
ENV GDAL_DATA $GDAL_PATH/3.8.4
ENV PATH $GDAL_PATH:$PATH
ENV LD_LIBRARY_PATH $LD_LIBRARY_PATH:/usr/lib/jni:/usr/share/java:/usr/local/lib/:/usr/lib/
ENV ANT_VERSION 1.9.16

#
# GDAL INSTALLATION
#
# Manual compilation - https://stackoverflow.com/questions/76913667/unable-to-locate-package-libgdal-java
RUN apt update && apt-get install -y wget build-essential libuv1-dev g++ libstdc++6 make linux-headers-generic \
   libssl-dev swig zlib1g-dev proj-bin proj-data libproj-dev sqlite3 libsqlite3-dev \ 
   libtiff-dev libcurl4-openssl-dev cmake libcrypto++-dev ant libeccodes-dev libnetcdf-dev
# Install GDAL
RUN wget http://download.osgeo.org/gdal/${GDAL_VERSION}/gdal-${GDAL_VERSION}.tar.gz -O /tmp/gdal.tar.gz && \
   tar xzf /tmp/gdal.tar.gz -C /tmp && \
   cd /tmp/gdal-${GDAL_VERSION} && \
   sed -i 's/source="7"/source="8"/' /tmp/gdal-${GDAL_VERSION}/swig/java/build.xml && \
   sed -i 's/target="7"/target="8"/' /tmp/gdal-${GDAL_VERSION}/swig/java/build.xml
RUN cd /tmp/gdal-${GDAL_VERSION} && mkdir build && cd build && cmake .. && cmake --build . && cmake --build . --target install

RUN cp /tmp/gdal-3.8.4/build/swig/java/gdal.jar /tmp/gdal.jar
RUN cp /tmp/gdal-3.8.4/build/swig/java/libgdalalljni.so /tmp/libgdalalljni.so

# for geoserver vanilla use eclipse-temurin:11-jre
FROM eclipse-temurin:11-jre  

LABEL maintainer="GeoServer PSC <[email protected]>"

COPY --from=builder /tmp/gdal.jar /usr/local/lib/gdal.jar
COPY --from=builder /tmp/libgdalalljni.so /usr/local/lib/jni/libgdalalljni.so

ENV JAVA_TOOL_OPTS="\
--add-exports=java.desktop/sun.awt.image=ALL-UNNAMED \
--add-opens=java.base/java.lang=ALL-UNNAMED \
--add-opens=java.base/java.util=ALL-UNNAMED \
--add-opens=java.base/java.lang.reflect=ALL-UNNAMED \
--add-opens=java.base/java.text=ALL-UNNAMED \
--add-opens=java.desktop/java.awt.font=ALL-UNNAMED \
--add-opens=java.desktop/sun.awt.image=ALL-UNNAMED \
--add-opens=java.naming/com.sun.jndi.ldap=ALL-UNNAMED \
-Djava.awt.headless=true"

ENV GDAL_VERSION 3.8.4
ENV GDAL_PATH /usr/share/gdal
ENV GDAL_DATA $GDAL_PATH/3.8.4
ENV PATH $JAVA_HOME/bin:$GDAL_PATH:$PATH
ENV GEOSERVER_VERSION 2.25.3
ENV GEOSERVER_HOME /opt/geoserver
ENV GEOSERVER_DATA_DIR /var/local/geoserver
ENV LD_LIBRARY_PATH $LD_LIBRARY_PATH:/usr/lib/jni:/usr/share/java:/usr/local/lib/:/usr/lib/:/usr/local/lib/jni
ENV JAVA_OPTS "-server -Xms768m -Xmx1560m"
ENV CLASSPATH=/usr/local/lib/gdal.jar:$CLASSPATH

RUN apt-get update && apt-get install -y gdal-bin unzip wget
#
# GEOSERVER INSTALLATION
# 
# Get Geoserver
RUN wget -c http://downloads.sourceforge.net/project/geoserver/GeoServer/$GEOSERVER_VERSION/geoserver-$GEOSERVER_VERSION-bin.zip -O ~/geoserver.zip
RUN mkdir -p /opt/geoserver && unzip ~/geoserver.zip -d /opt/geoserver
# Create data_dir and move Geoserver data_dir sample
RUN mkdir $GEOSERVER_DATA_DIR && mv /opt/geoserver/data_dir/* $GEOSERVER_DATA_DIR && rm -rf /opt/geoserver/data_dir && \
# Get OGR plugin
   wget -c http://downloads.sourceforge.net/project/geoserver/GeoServer/$GEOSERVER_VERSION/extensions/geoserver-$GEOSERVER_VERSION-ogr-wfs-plugin.zip -O ~/geoserver-ogr-plugin.zip &&\
   unzip -o ~/geoserver-ogr-plugin.zip -d /opt/geoserver/webapps/geoserver/WEB-INF/lib/ && \
   rm ~/geoserver-ogr-plugin.zip && \
# Get GDAL plugin
   wget -c http://downloads.sourceforge.net/project/geoserver/GeoServer/$GEOSERVER_VERSION/extensions/geoserver-$GEOSERVER_VERSION-gdal-plugin.zip -O ~/geoserver-gdal-plugin.zip &&\
   unzip -o ~/geoserver-gdal-plugin.zip -d /opt/geoserver/webapps/geoserver/WEB-INF/lib/ && \
   rm ~/geoserver-gdal-plugin.zip && \
# Get import plugin
   wget -c http://downloads.sourceforge.net/project/geoserver/GeoServer/$GEOSERVER_VERSION/extensions/geoserver-$GEOSERVER_VERSION-importer-plugin.zip -O ~/geoserver-importer-plugin.zip &&\
   unzip -o ~/geoserver-importer-plugin.zip -d /opt/geoserver/webapps/geoserver/WEB-INF/lib/ && \
   rm ~/geoserver-importer-plugin.zip && \
# Replace GDAL Java bindings
   rm -rf $GEOSERVER_HOME/webapps/geoserver/WEB-INF/lib/imageio-ext-gdal-bindings-*.jar && \
   rm -rf $GEOSERVER_HOME/webapps/geoserver/WEB-INF/lib/gdal-3.2.0.jar && \
   cp /usr/local/lib/gdal.jar $GEOSERVER_HOME/webapps/geoserver/WEB-INF/lib/gdal-3.8.4.jar && \
# Plugins
   wget -c http://downloads.sourceforge.net/project/geoserver/GeoServer/$GEOSERVER_VERSION/extensions/geoserver-$GEOSERVER_VERSION-netcdf-plugin.zip -O ~/geoserver-netcdf-plugin.zip &&\
   unzip -o ~/geoserver-netcdf-plugin.zip -d /opt/geoserver/webapps/geoserver/WEB-INF/lib/ && \
   rm ~/geoserver-netcdf-plugin.zip && \
   wget -c http://downloads.sourceforge.net/project/geoserver/GeoServer/$GEOSERVER_VERSION/extensions/geoserver-$GEOSERVER_VERSION-grib-plugin.zip -O ~/geoserver-grib-plugin.zip &&\
   unzip -o ~/geoserver-grib-plugin.zip -d /opt/geoserver/webapps/geoserver/WEB-INF/lib/ && \
   rm ~/geoserver-grib-plugin.zip && \
   wget -c http://downloads.sourceforge.net/project/geoserver/GeoServer/$GEOSERVER_VERSION/extensions/geoserver-$GEOSERVER_VERSION-printing-plugin.zip -O ~/geoserver-printing-plugin.zip &&\
   unzip -o ~/geoserver-printing-plugin.zip -d /opt/geoserver/webapps/geoserver/WEB-INF/lib/ && \
   rm ~/geoserver-printing-plugin.zip && \
   wget -c http://downloads.sourceforge.net/project/geoserver/GeoServer/$GEOSERVER_VERSION/extensions/geoserver-$GEOSERVER_VERSION-css-plugin.zip -O ~/geoserver-css-plugin.zip &&\
   unzip -o ~/geoserver-css-plugin.zip -d /opt/geoserver/webapps/geoserver/WEB-INF/lib/ && \
   rm ~/geoserver-css-plugin.zip && \
   wget -c http://downloads.sourceforge.net/project/geoserver/GeoServer/$GEOSERVER_VERSION/extensions/geoserver-$GEOSERVER_VERSION-querylayer-plugin.zip -O ~/geoserver-querylayer-plugin.zip &&\
   unzip -o ~/geoserver-querylayer-plugin.zip -d /opt/geoserver/webapps/geoserver/WEB-INF/lib/ && \
   rm ~/geoserver-querylayer-plugin.zip && \
   wget -c http://downloads.sourceforge.net/project/geoserver/GeoServer/$GEOSERVER_VERSION/extensions/geoserver-$GEOSERVER_VERSION-sldservice-plugin.zip -O ~/gs-sldservice-plugin.zip &&\
   unzip -o ~/gs-sldservice-plugin.zip -d /opt/geoserver/webapps/geoserver/WEB-INF/lib/  &&\
   rm ~/gs-sldservice-plugin.zip

# Expose GeoServer's default port
EXPOSE 8080
CMD ["/opt/geoserver/bin/startup.sh"]