C++ Equivalent of SolutionArray

[ischoegl]

In the Python API, SolutionArray objects hold multiple states, which allows for efficient manipulation of time series, spatial profiles etc. The implementation of this feature uses Python (Cython) exclusively, where an equivalent capability does not exist in the C++ core. Beyond manipulation of data sets, SolutionArray objects also offers a convenient vehicle to save and load data (e.g. CSV, HDF5), where interoperability between different interfaces would be beneficial.

This discussion is meant to collect some ideas within this context.

[ischoegl]

HDF5 file input/output

One desirable feature would presumably be to have HDF5 input/output implemented in C++ (at the moment, only the Python API implements this feature based on the h5py library).

Per @speth in Cantera/cantera#1112 (an effort to replace legacy XML file I/O with a YAML format):

While I'd love to have the ability to write HDF5 output files directly, my previous experience with using the HDF5 C++ library was that it was a beastly dependency to add to a project, especially when trying to build on Windows where there isn't (or at least wasn't) an easy way to install the library and headers. That was the main driver for choosing to base this on the already-available YAML input/output capabilities, even if HDF5 is better suited for this mostly numeric array-centric data.

One workaround may be to use more user-friendly HDF5 C++ wrappers, e.g. BlueBrain/HighFive ... this is just the result of a quick search, where I mainly picked the one with the largest number of forks (in other words: there may be better options).

[bryanwweber]

One way to look at it, since Windows seems to be the problem per @speth, is what's available on conda-forge?

[speth]

I would imagine that using the HDF5 library from Anaconda / conda-forge would be fine, but even then I think we would want it to be an optional dependency - I don't think we want to be too dependent on conda.

The HighFive wrapper looks pretty interesting, and might help alleviate another issue with HDF5, which is that when working with it's C/C++ API, it is very easy to make mistakes and end up with a segfault or invalid memory access if you don't do everything just right.

[bryanwweber]

I would imagine that using the HDF5 library from Anaconda / conda-forge would be fine, but even then I think we would want it to be an optional dependency - I don't think we want to be too dependent on conda.

Yes, naturally. I was just thinking from the standpoint of making it easy for developers on Windows to install the dependency, conda is a nice way to do that quickly.

[ischoegl]

I second the Anaconda comment ... that's imho the only way other dependencies (boost?) are manageable. But vanilla HDF5 itself appears to be tricky, so a wrapper may be a way to ensure that the effort of maintaining this feature can be minimized. Segfaults are notoriously hard to track down, especially for new contributors.

[ischoegl]

Closing as the feature is now implemented (per Cantera/cantera#1426)