Support UUID with URL special characters.

[fxprunayre]

eg. info:doi:10.24396/ORDAR-56 or http://dada.moo/ORDAR-56

In order to support UUID with character like / or ; in it, you need
to disable default Spring HTTP Firewall behavior which consider those characters unsecured.
Error would look like URL contained a potentially malicious String "%2F"

Client side URL encode UUIDs and spring will not
decode path before matching URL (which would cause issue with request mapping).

Use -Dgeonetwork.security.coreconfig=encodeduuid to enable the security configuration for the StrictHttpFirewall and the filterChainProxy (see config-security-core-encodeduuid.xml).

If encodeduuid is enabled, on Tomcat it will also require -Dorg.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=true
and if using an Apache reverse proxy

  AllowEncodedSlashes On
  ProxyPass /geonetwork http://localhost:8080/geonetwork nocanon
  ProxyPassReverse /geonetwork http://localhost:8080/geonetwork

By default, this is not active and has to be enabled if needed.

This also fix UUID containing "." with part of the API operations not matching them.

On Elasticsearch side, document can also be accessed using URL encoded UUID eg. http://localhost:9200/gn-records/_doc/https%3A%2F%2Fdoi.org%2F10.13155%2F77514

Related to #3501

[josegar74]

Tested to import a metadata with that type of uuid, display the metadata detail page, edit it and add online resources, export to xml and pdf, all these looking fine.

But switching to the advanced view in the metadata detail page doesn't seem to work:

The record with identifier was not found or is not shared with you. Try to sign in if you've an account.

[fxprunayre]

But switching to the advanced view in the metadata detail page doesn't seem to work:

Fixed @josegar74

[josegar74]

Doesn't really work for me, the option doesn't fail now, but the page is displaying the default view, not the full view.

In the metadata detail page, I see these 2 failing requests:

• http://localhost:8080/geonetwork/srv/api/records/info%3Adoi%3A10.24396%2FORDAR-56/extents.png (error 400)
• http://localhost:8080/geonetwork/srv/api/records/info:doi:10.24396/ORDAR-56/popularity (error 404)

I noticed also that searching for info:doi:10.24396/ORDAR-56 shows a popup with this error message: Query returned an error. Check the console for details., the search request returns this error:

{
"servlet":"spring",
"message":"Error is: Bad Request.\nRequest:\n{"from":0,"size":30,"sort":["_score"],"query":{"function_score":{"boost":"5","functions":[{"filter":{"exists":{"field":"parentUuid"}},"weight":0.3},{"filter":{"match":{"cl_status.key":"obsolete"}},"weight":0.3},{"gauss":{"dateStamp":{"scale":"365d","offset":"90d","decay":0.5}}}],"score_mode":"multiply","query":{"bool":{"must":[{"query_string":{"query":"(any:(info\\\\:doi\\\\:10.24396/ORDAR\\\\-56) resourceTitleObject.default:(info\\\\:doi\\\\:10.24396/ORDAR\\\\-56)^2)"}},{"terms":{"isTemplate":["n"]}}],"filter":{"query_string":{"query":"* AND (draft:n OR draft:e)"}}}}}},"aggregations":{"cl_hierarchyLevel.key":{"terms":{"field":"cl_hierarchyLevel.key"},"aggs":{"format":{"terms":{"field":"format"}}}},"cl_spatialRepresentationType.key":{"terms":{"field":"cl_spatialRepresentationType.key","size":10}},"availableInServices":{"filters":{"filters":{"availableInViewService":{"query_string":{"query":"+linkProtocol:/OGC:WMS.*/"}},"availableInDownloadService":{"query_string":{"query":"+linkProtocol:/OGC:WFS.*/"}}}}},"th_gemet_tree.default":{"terms":{"field":"th_gemet_tree.default","size":100,"order":{"_key":"asc"},"include":"[^^]+^?[^^]+"}},"th_httpinspireeceuropaeumetadatacodelistPriorityDataset-PriorityDataset_tree.default":{"terms":{"field":"th_httpinspireeceuropaeumetadatacodelistPriorityDataset-PriorityDataset_tree.default","size":100,"order":{"_key":"asc"}}},"tag.default":{"terms":{"field":"tag.default","include":".*","size":10},"meta":{"caseInsensitiveInclude":true}},"th_regions_tree.default":{"terms":{"field":"th_regions_tree.default","size":100,"order":{"_key":"asc"}}},"resolutionScaleDenominator":{"histogram":{"field":"resolutionScaleDenominator","interval":10000,"keyed":true,"min_doc_count":1},"meta":{"collapsed":true}},"creationYearForResource":{"histogram":{"field":"creationYearForResource","interval":5,"keyed":true,"min_doc_count":1},"meta":{"collapsed":true}},"OrgForResource":{"terms":{"field":"OrgForResource","include":".*","size":15},"meta":{"caseInsensitiveInclude":true}},"cl_maintenanceAndUpdateFrequency.key":{"terms":{"field":"cl_maintenanceAndUpdateFrequency.key","size":10},"meta":{"collapsed":true}}},"_source":{"includes":["uuid","id","creat*","group*","logo","category","topic*","inspire*","resource*","draft","overview.*","owner*","link*","image*","status*","rating","tag*","geom","contact*","*Org*","hasBoundingPolygon","isTemplate","valid","isHarvested","dateStamp","documentStandard","cl_status*","mdStatus*","recordLink","op*"]},"track_total_hits":true}\n.\nError:\n{"error":{"root_cause":[{"type":"query_shard_exception","reason":"failed to create query: end-of-string expected at position 9","index_uuid":"TGw-bPQSSZSh1uxDF-0Pow","index":"gn-records"}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"gn-records","node":"UrW-0pr0TQqbzuTymcFpXg","reason":{"type":"query_shard_exception","reason":"failed to create query: end-of-string expected at position 9","index_uuid":"TGw-bPQSSZSh1uxDF-0Pow","index":"gn-records","caused_by":{"type":"illegal_argument_exception","reason":"end-of-string expected at position 9"}}}]},"status":400}.",
"url":"/geonetwork/srv/api/search/records/_search",
"status":"400"
}

[fxprunayre]

Doesn't really work for me, the option doesn't fail now, but the page is displaying the default view, not the full view.

Did not pushed last changes :/ Fixed

I noticed also that searching for info:doi:10.24396/ORDAR-56 shows a popup with this error message: Query returned an error. Check the console for details., the search request returns this error:

You forgot to uncomment <property name="firewall" ref="httpFirewall"/> no ?

[josegar74]

@fxprunayre, the full view works fine now, but the search doesn't. I have uncomment <property name="firewall" ref="httpFirewall"/>.

Code changes I have in config-security-core.xml to check:

[fxprunayre]

For supporting encoded / in UUID, on Tomcat it will also require
-Dorg.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=true

and if using a reverse proxy

        AllowEncodedSlashes On
        ProxyPass /geonetwork http://localhost:8080/geonetwork nocanon
        ProxyPassReverse /geonetwork http://localhost:8080/geonetwork

[fgravin]

Excellent, thanks @josegar74 for pointing this out !

Would you please tell me what is the status of this PR ?

• Is it used in production somewhere ?
• Does it miss some devs ?
• What energy would it need to be merged in main ?

Thanks for the work @fxprunayre !

[josegar74]

@fgravin, I guess @fxprunayre can tell that better, but apart of resolving the conflicts, I think that needs more testing.

[CLAassistant]

All committers have signed the CLA.