Support for other grid specifications

[hetland]

This post is based on an email exchange between some folks (@kthyng, @rljacob, @ChrisBarker-NOAA, @rajeeja)
that I proposed to move here, so all can participate

There are a number of regional (FVCOM, SUNTANS, Delft3D) and global (ICON) models that would benefit from uxarray, however, there is of course a limit to the support that uxarray can provide for these many different models. Better to make it easy to read grids that don't conform to ugrid-conventions, so other groups can build off of uxarray.

What's the best way to expose the grid reading functions to make it easy for people two make a wrapper for their own model output?

[ChrisBarker-NOAA]

NOTE: There are two levels of non-compliance:

1. The easier one -- all the info is there, in the right format, but the meta-data is not. This .is the case with FVCOM (except for the boundaries, I think)

2. The grid is specified in a non-compatible way -- you would need to transform the grid specification to produce conforming data structure -- this is a harder problem.

Note: we've faces all this in the gridded project:

https://github.com/NOAA-ORR-ERD/gridded

And did not originally have a good architecture to do all that (and still only halfway there). Don't make the same mistakes we did :-)

My suggestion:

Have determining what the grid is, and what variables map to what roles in the grid structure as one step, and then actually building the grid object from that specification as a separate step -- then you bypass the discovery, and simply pass the specification it to the grid-building step for non-compliant files.

Even better, have a way to pass a transformation function in to create each part of the Grid object, and then it could be totally customizable.

I've been hoping to refactor gridded into the structure, but haven't had the chance yet.

[philipc2]

Even better, have a way to pass a transformation function in to create each part of the Grid object, and then it could be totally customizable.

This is a fantastic idea. We already have support for constructing grids from inputted vertices, but being able to manually pass through a set of coordinates and connectictivies would be extremely useful, assuming the user is of course willing to modify their data in a way fits the criteria. At the most basic level, we only need the coordinates (x, y) and the face node connectivity and this would allow for a "work-around" if a grid format isn't currently supported.

from uxarray import Grid

# assuming some x, y, face_nodes are appropriate inputs 
uxgrid = Grid.from_connectivity({"node_x" : some_x, "node_y" : some_x, "face_nodes" : some_face_nodes})

[philipc2]

The approach that we've taken in UXarray is similar to what @ChrisBarker-NOAA mentioned in his suggestion

Grid Representation

We have chosen to internally represent all unstructured grid formats in the UGRID conventions, which means that any dataset that is loaded in needs to first be converted into the UGRID conventions.

As long as the unstructured grid can be loaded in with Xarray, it can be passed through as an input into the construction of our Grid object, which handles the parsing, construction, representation, etc

Parsing

Our parsing right now is very basic, except for the _is_ugrid() check which we'd like to implement for each support method. However, the inputted dataset is parsed in order to detect which mesh_type it is

uxarray/uxarray/utils/helpers.py

Lines 34 to 49 in 85f803d

	# exodus with coord or coordx
	if "coord" in dataset:
	mesh_type = "exo"
	elif "coordx" in dataset:
	mesh_type = "exo"
	# scrip with grid_center_lon
	elif "grid_center_lon" in dataset:
	mesh_type = "scrip"
	# ugrid topology
	elif _is_ugrid(dataset):
	mesh_type = "ugrid"
	elif "verticesOnCell" in dataset:
	mesh_type = "mpas"
	else:
	raise RuntimeError(f"Could not recognize dataset format.")
	return mesh_type

Reading & Encoding

Once we know the mesh_type, we then call the corresponding read function which goes through the dataset and converts its variables into the UGRID conventions, stored under self._ds.

Example (MPAS)

I've recently written a new reader to support MPAS meshes. MPAS was an interesting format since we needed to provide support for reading in both the Primal and Dual meshes. I believe going through this implementation, specifically _primal_to_ugrid or _dual_to_ugrid will provide a decent idea of how we handle reading in and encoding meshes

https://github.com/UXARRAY/uxarray/blob/main/uxarray/io/_mpas.py

[rljacob]

Just noting that a model doesn't have to adopt UGRID if its not currently following a convention in its mesh description. SCRIP (or MPAS) might be an easier first step and are also supported.