User Manual refresh using Markdown and MkDocs (geonetwork#7329)

* online help ported from hnap manual

* translate module from hnap prototype

Expanded to convert from sphinx

* Update help to match discussion

* some sample pages to test translate script from rst to md

* Testing translate pandoc rst to markdown

* Changes to style and theme:
- add browser window style
- colors
- add extension and features

* pre/post

* overview section

* overview section

* Changes to style and theme:
- add browser window style
- colors
- add extensions and features

* update translate rst command for directory processing

* bulk covert change-log

* api content

* fix quickstart formatting

* user-guide landing page

* anchors lookup from rst files

* example of fixing an broken anchor link

* include version text to avoid i18n issue

* Document include plugin for use to inline content from other files

* Handle rst fenced blocks like note and warning (and deprecated and see also)

* admin guide > managing metadata standards

* Remaining admin guide pages

* Review of admin-guide fixing common indenting rst mistakes

These introduce random block quotes when none is intended

* install-guide

* contributing

* tutorials

* customizing application

* special logic needed for admonition

* maintainer guide

* user guide update

* Some updates to files corrected by hand earlier

* anenxes

* integrate into maven build

* Work on reducing download size of manual

* Fixing indents for lists and code blocks, and thus removing blockquotes in the resulting pages

* Improve indents and TOC, add tabs, add copy to clipboard for code blocks and smart symbols, add extra text to the footer

* Improve order and titles of pages

* work on appearance

# Conflicts:
#	docs/manual/mkdocs.yml

* work on translating doc references

# Conflicts:
#	docs/manual/docs/overview/index.md

* work on index headings

* back convert to rst any manual changes perofrmed to markdown

* switch to preprocessing references

* Second check of script generated conversion

* restore change log pages, requires update of mkdocs-static-i18n>=1.0.5

* Fold in online help and clean up nav structure

* update doc links to index.rst files

* Update script to better handle links to index.rst files

* small corrections from review with jose

* recover mef docs from 2.10.x branch

* index rst files for doc conversion

* update anchors.txt index based on improved rst scan

* remove help module, added as distinct assembly to manual

* no parameter doc conversion

* Version management with mike

* Move translate script

* update to toctree conversion with document titles

* process reference links using anchors_txt index

* script fixes for heading length, rst_epilog substitutions, and reference titles

* clearn conversion, although warning while serving due to absolute links

* change to generating relative links

* clean up warnings

* include version changes in title index

* regenerat version changes now that we have references working

* clean up gallery warnings

* adjust nav

* Fix numbering and admonitions on some pages, fixing some edge cases, adding language to code blocks

* Fix arrows (-->) with `Smart Symbols` and add grid to startup page

* Use less lines in the footer and fix the version toc

* Fix the grid on the first page - remove the uneven gap

* Update content structure and address feedback

geonetwork#7363

* translate script removed to its own repository

* langauge chooser by default

* publish french to mike versions by default

* Add suite_url so that language chooser works with mike versioning

* update deployment and translation instructions

* remove development feedback banner

* Fix for the grid on the starting page, make it responsive

* Change to major.minor examples and use of remote for docs.geonetwork-opensource.org publishing

We will need to test edit link and other features once this is deployed

* geonetwork icons

* Add a gitignore include so that overrides/.icons is version controlled correctly

* remove manuals submodule

This was no longer included in the build

* describe the repo as core-geonetwork in header

* troubleshooting mike deploy

* continue to troubleshoot mike deploy

* working on mike instructions

---------

Co-authored-by: Michel Gabriel <[email protected]>

.gitmodules

@@ -7,6 +7,3 @@
 [submodule "web-ui/src/main/resources/catalog/lib/bootstrap-table"]
 	path = web-ui/src/main/resources/catalog/lib/bootstrap-table
 	url = https://github.com/wenzhixin/bootstrap-table.git
-[submodule "docs/manuals"]
-	path = docs/manuals
-	url = https://github.com/geonetwork/doc.git

docs/README.md

@@ -1,9 +1,11 @@
-Documentation
-=============
+# Documentation
+
 
 This module contains:
-* Documentation in RST format
-* Change logs
-* Copyright template
-* Licenses for library
-* Utility folder with scripts to generate extra docs
+* `manual` in Markdown format for mkdocs
+* `manuals` in RST format for sphinx-build (vai https://github.com/geonetwork/doc)
+* Change logs txt files
+* [copyright.txt](copyright.txt) Copyright template
+* `licenses` for library
+* Utility folder with scripts to generate extra docs
+* `translate` python module for help with sphinx-build to mkdocs 

docs/manual/README.md

@@ -0,0 +1,156 @@
+# Geonetwork Manual and Help
+
+Documentation for GeoNetwork opensource is available via https://geonetwork-opensource.org.
+
+This documentation is written under the creative commons license [Attribution-ShareAlike 3.0 Unported (CC BY-SA 3.0)](LICENSE.md).
+
+Reference:
+
+* [Documentation Writing Guide](docs/devel/docs/docs.md)
+
+## Communication
+
+The [project issue tracker](https://github.com/geonetwork/core-geonetwork/issues) is used for communication, with ongoing topics tagged [documentation](https://github.com/geonetwork/core-geonetwork/issues?q=is%3Aissue+label%3Adocumenation).
+
+## Material for MkDocs
+
+Documentation is [mkdocs-material](https://squidfunk.github.io/mkdocs-material/) which is a Markdown documentation framework written on top of [MkDocs](https://www.mkdocs.org/).
+
+If you are using python3:
+
+1. Install using ``pip3`` and build:
+
+   ```bash
+   pip3 install -r requirements.txt
+   ```
+
+2. Use ***mkdocs** to preview locally:
+
+   ```bash
+   mkdocs serve
+   ```
+      
+3. Preview: http://localhost:8000
+
+   Preview uses a single version, so expect some warnings from version chooser:
+   ```
+   "GET /versions.json HTTP/1.1" code 404
+   ```
+
+4. Optional: Preview online help:
+
+   ```bash
+   mkdocs serve --config-file help.yml  
+   ```
+
+### VirtualEnv
+
+If you use a python virtual environment:
+
+1. Activate virtual environment:
+
+   ```bash
+   virtualenv venv
+   source venv/bin/activate
+   pip install -r requirements.txt
+   ```
+
+2. Use ***mkdocs*** to preview from virtual environment:
+
+   ```bash
+   mkdocs serve
+   ```
+
+3. Preview: http://localhost:8000
+
+### Docker
+
+If you are not familiar with python the mkdocs-material website has instructions for docker:
+
+1. Run mkdocs in Docker environment:
+
+   ```
+   docker pull squidfunk/mkdocs-material
+   docker run --rm -it -p 8000:8000 -v ${PWD}:/docs squidfunk/mkdocs-material
+   ```
+
+2. Preview: http://localhost:8000
+
+## Maven Integration
+
+1. Build documentation with ``compile`` phase:
+   ```
+   mvn compile
+   ```
+
+2. Assemble ``zip`` with ``package`` phase:
+   ```bash
+   mvn package
+   ```
+
+3. Both ``install`` and ``deploy`` are skipped (so ``mvn clean install`` is fine).
+
+4. Use default profile to only build the default english docs:
+
+   ```
+   mvn install -Pdefault
+   ```
+
+## Deploy
+
+We use ``mike`` for publishing to https://geonetwork.github.io using `<major>.<minor>` version:
+
+1. Define the remote `website`:
+
+   ```bash
+   git remote add website https://github.com/geonetwork/geonetwork.github.io
+   git fetch website
+   git checkout -b gh-pages website/gh-pages
+   git branch --set-upstream-to=websigte/gh-pages
+   ```
+
+   The file ``mkdocs.yml`` setting for ``remote_name`` is set to ``website``.
+   To test with your own for, use ``--remote origion`` in the examples below.
+
+2. To deploy SNAPSHOT development docs from the `main` branch to website `gh-pages` branch:
+
+   ```bash
+   mike deploy --push --update-aliases 4.4 devel
+   ```
+
+3. To deploy documentation for a new release:
+
+   ```bash
+   mike deploy --push --update-aliases 4.2 stable
+   ```
+
+   When starting a new branch you can make it the default:
+
+   ```bash
+   mike set-default --push 4.2
+   ```
+
+4. To publish documentation for a maintenance release:
+
+   ```bash
+   mike deploy --push --update-aliases 3.12 maintenance
+   ```
+
+5. To show published versions:
+
+   ```bash
+
+   mike list
+   ```
+
+4. To preview things locally (uses your local ``gh-pages`` branch):
+
+   ```bash
+   mike serve
+   ```
+
+Reference:
+
+* https://squidfunk.github.io/mkdocs-material/setup/setting-up-versioning/
+* https://github.com/squidfunk/mkdocs-material-example-versioning
+* https://github.com/jimporter/mike

docs/manual/docs/administrator-guide/configuring-the-catalog/css-configuration.md

@@ -0,0 +1,13 @@
+# Configuring CSS & Style {#css-configuration}
+
+To get to the CSS & Style configuration, you must be logged on as administrator. Open 'Settings' from the Administration page and select ``CCS & Style`` configuration.
+
+![](img/css-screen.png)
+
+This page allows you to change various colors of page elements. Note that 'background' refers to the background color of elements, and 'color' to the font color used in the element. The above configuration will result in the visualization below:
+
+![](img/css-result.png)
+
+There is also an option to link to a background image. When linking to an image, make sure to use (the full web address to) an image from a https source if you're running GeoNetwork in https. Placing the image (upload as logo) on the same server is the best option to prevent potential browser blocks.
+
+Note that you can save a configuration locally for reuse later or on alternative servers. Saving a configuration will take a long time, because all scripts and styles will be rebuilt from sources. Temporary files are stored on the server in an application folder, you may need to revisit this page after an upgrade the system (or redeploy of a docker container).

docs/manual/docs/administrator-guide/configuring-the-catalog/csw-configuration.md

@@ -0,0 +1,108 @@
+# Configuring CSW {#csw-configuration}
+
+To get to the CSW server configuration, you must be logged on as administrator first. Open 'Settings' from the Administration page and select CSW Server configuration.
+
+![](img/csw.png)
+
+The CSW service provides a description of itself, the human who administers it, and other information through a `es` request (eg. <http://localhost:8080/geonetwork/srv/en/csw?request=GetCapabilities&service=CSW&version=2.0.2>). This form allows you to configure the CSW server and fill out some of the properties returned in response to a GetCapabilities request.
+
+Configuration options:
+
+-   *CSW Enabled*: By default CSW endpoint is enabled, but it can be disabled.
+
+    ![](img/csw-off.png)
+
+    The Open Geospatial Catalogue Service for the Web (OGC-CSW) service, is a self-describing service that allows query, update and insertion of metadata records.
+
+-   *Record to use for GetCapabilities*: The capabilities document is created using the record selected here.
+
+-   *Inserted metadata is public*: By default, metadata inserted with the CSW Harvest and CSW Transaction operations is not publicly viewable. A user with the appropriate access rights could do this after the CSW Harvest and CSW Transaction operations, but this is not always convenient. If this option is checked all metadata inserted using the CSW Harvest and CSW Transaction operations will be publicly viewable.
+
+-   *Create element if it does not exist when using XPath in CSW transaction.*: If not checked, only existing elements can be updated.
+
+## Service Metadata Record {#csw-configuration_service_record}
+
+In order to create a custom capabilities, it is recommended to create a dedicated service metadata record (using ISO19115-3 or ISO19139 standards). This record is used to build the capabilities document using the template `web/src/main/webapp/xml/csw/capabilities.xml`.
+
+When creating such record, the following information will be used to create the capabilities:
+
+-   title
+-   abstract
+-   keywords
+-   fees (from Distribution --> ordering instructions field)
+-   Access constraints (from Access constraints --> Other constraints field)
+-   Contact (from Identification --> First point of contact)
+
+The service record MUST be public.
+
+If an error occurred while building the capabilities document from the service record, a WARNING is reported using a comment and the default capabilities is used:
+
+![](img/csw-error.png)
+
+## CSW Configuration for INSPIRE {#csw-configuration_inspire}
+
+If your catalogue also focus on INSPIRE (see [Configuring for the INSPIRE Directive](inspire-configuration.md)), it may be relevant to also populate the following to properly configure your discovery service:
+
+-   metadata language (+ additional languages if any) (use for SupportedLanguages, DefaultLanguage)
+-   INSPIRE themes
+-   temporal coverage
+-   metadata contact
+-   metadata conformity regarding Commission regulation 1089/2010
+
+With this information the CSW can be validated using the INSPIRE validator.
+
+## CSW post processing
+
+In some situation, user may want to modify XML encoding of records when retrieved using CSW. For example:
+
+-   to make record more consistent
+-   to adapt encoding in order to validate records with INSPIRE rules (eg. as a workaround for workaround for <https://github.com/inspire-eu-validation/community/issues/95>)
+
+For this, user can add extra conversion step during output of GetRecords and GetRecordById operation:
+
+-   Adding post process to srv/eng/csw by creating for each output schema a file eg. for gmd, present/csw/gmd-csw-postprocessing.xsl
+-   Adding post process to a portal inspire/eng/csw by creating for each output schema a file eg. for gmd, present/csw/gmd-inspire-postprocessing.xsl
+
+For a portal INSPIRE, create an "inspire" portal in the admin --> settings --> sources. User can then setup a post processing phase for the INSPIRE CSW of the portal. Create a post processing file gmd-inspire-postprocessing.xsl in yourschema/present/csw folder. The following example, remove contact with no email and link not starting with HTTP to conform to INSPIRE validator rules.
+
+``` xml
+<?xml version="1.0" encoding="UTF-8"?>
+<xsl:stylesheet version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
+                xmlns:gn="http://www.fao.org/geonetwork"
+                xmlns:gmd="http://www.isotc211.org/2005/gmd"
+                xmlns:gco="http://www.isotc211.org/2005/gco"
+                xmlns:srv="http://www.isotc211.org/2005/srv"
+                exclude-result-prefixes="#all">
+
+  <!-- Remove all contact not having an email -->
+  <xsl:template match="*[gmd:CI_ResponsibleParty
+                         and count(gmd:CI_ResponsibleParty/gmd:contactInfo/*/gmd:address/*/gmd:electronicMailAddress[*/text() != '']) = 0]"
+                priority="2"/>
+
+  <!-- Remove all online source not using HTTP to conform with
+  https://github.com/inspire-eu-validation/community/issues/95
+  -->
+  <xsl:template match="*[gmd:CI_OnlineResource
+                         and count(gmd:CI_OnlineResource/gmd:linkage/gmd:URL[not(starts-with(text(), 'http'))]) > 0]"
+                priority="2"/>
+
+
+  <!-- Remove geonet:* elements. -->
+  <xsl:template match="gn:*" priority="2"/>
+
+  <!-- Copy everything. -->
+  <xsl:template match="@*|node()">
+    <xsl:copy>
+      <xsl:apply-templates select="@*|node()"/>
+    </xsl:copy>
+  </xsl:template>
+</xsl:stylesheet>
+```
+
+The service can be tested with:
+
+``` shell
+curl 'http://localhost:8080/geonetwork/inspire/eng/csw' \
+  -H 'Content-type: application/xml' \
+  --data-binary $'<csw:GetRecordById xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" service="CSW" version="2.0.2"                   outputSchema="http://www.isotc211.org/2005/gmd"><csw:ElementSetName>full</csw:ElementSetName><csw:Id>3de9790e-529f-431f-ac4f-e86d827bde8e</csw:Id>\n</csw:GetRecordById>'
+```

docs/manual/docs/administrator-guide/configuring-the-catalog/index.md

@@ -0,0 +1,9 @@
+# Configuring the catalog
+
+-   [System configuration](system-configuration.md)
+-   [User Interface Configuration](user-interface-configuration.md)
+-   [Configuring CSS & Style](css-configuration.md)
+-   [Configuring CSW](csw-configuration.md)
+-   [Portal configuration](portal-configuration.md)
+-   [Configuring for the INSPIRE Directive](inspire-configuration.md)
+-   [Map servers configuration for geopublication](map-server-configuration.md)