Could experiment/checkpoint tables just be pandas dataframes?

[bfirsh]

This is something neat that this does: https://github.com/richardliaw/track

[rsokl]

Replicate looks great! I just happened across this issue and thought I would share some experience I had with metric logging.

I would recommend using xarray instead of pandas dataframes; xarray was born out of the awkwardness that was extending pandas dataframes beyond two dimensions.

This makes it much easier to do things like align or concatenate metrics across experiments, handle the difference between train/test and batch/epoch level-metrics. All of these tasks benefit from xarray's usage of coordinate systems.

I made a live-logging/plotting library that makes use of xarray to this effect: https://github.com/rsokl/noggin

• brief discussion of accessing metrics as an xarray Dataset
• convenience function for aligning metrics across experiments

These are just for reference; I don't plan on maintaining noggin any further, so I don't mean to recommend that you actually leverage noggin (although there are probably nuggets to be taken from it).