I/O Abstraction

[rburghol]

• Protocols are defined in src/hsp2/hsp2io/protocols.py
  • Defines what things should look like, ex: write_ts, read_ts
  • Bu they don't actually do anything
• Classes actually implement the interfaces
  • Ex: src/hsp2/hsp2io/io.py
  • Defines the code that implements the function, such as write_ts()

    HSPsquared/src/hsp2/hsp2io/io.py

    Line 58 in 6f40cc3

    def write_ts(self,

Caching

• IO class (read hdf) allows caching of timeseries, so that there is no need to reload a series that has already been loaded from disk.
• KO class does not currently have the ability to store data in memory, save_timeseries pushes the data to disk, and then, that seems to incur large overhead.

[aufdenkampe]

@rburghol, thanks for recording these notes in an issue!

I have one clarification/correction, however. The implementation of the file interface is actually done by file type in modules such as src/hsp2/hsp2io/hdf.py. So if you wanted to implement IO for a new file type (i.e. Parquet), you would want to copy the pattern in hdf.py.

The io.py module only implements the IOManager() class that orchestrates different IO protocols for the IO interface you pass into it.

To see an example of how that is used, look at the Create an instance of the HSP2 IO Manager section in the examples/1_Intro_to_HSP2.ipynb notebook. NOTE that this notebook appears to have gotten broken with some of the repo reorganization that's happened in the last year.

This PR describes it all reasonably well too:

• IO Abstraction #68

@ptomasula, anything else to add?

[rburghol]

Thanks @aufdenkampe -- I was flying by the seat of my pants!

[aufdenkampe]

@rburghol, with PR #184, I fixed the Example 1 notebook, so you should be able to use it to explore the hsp2io.io.IOManager() class.

[rburghol]

Thanks @aufdenkampe