Compress final output files

[klamike]

• Uses the deflate/zlib integration in HDF5.jl to compress the datasets in the final output files. The highest compression level is enabled by default.
• Uses gzip compression via OPFGenerator.save_json by default for the reference case.json.gz file.
• Run some benchmarks with various compression options

[mtanneau]

Do you have a sense of how much space we save with this?

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

[klamike]

Unfortunately the current approach does not work for the Vector{String} datasets (termination/result status codes). Since these are really enums we can store them as integers instead, then compress works well. We can store the mapping from Integer(instance) to String(instance) in the HDF5 dataset's attributes for readers to be able to convert back even if MOI changes the mapping. We also currently store the formulation name for each sample, which can be dropped.

Storing Vector{Enum} feels like something HDF5 would support natively, so I looked into it... there does seem to be a relevant datatype in HDF5 itself but neither the Julia nor Python interfaces have nice support for it.

With the enum -> Integer change and level 9 compression, an 89_pegase dataset gets about 30% smaller.

[mtanneau]

Unfortunately the current approach does not work for the Vector{String} datasets (termination/result status codes).

Is it (technically) possible to compress only the numerical datasets? e.g., doing a an eltype check and compressing only if it's a numerical value.

We can store the mapping from Integer(instance) to String(instance) in the HDF5 dataset's attributes for readers to be able to convert back even if MOI changes the mapping.

Not against storing the integer codes instead of the Strings. It will also save a (tiny) bit of space.
We can ask the JuMP devs whether changing the integer codes of an enum would be considered a breaking change.

With the enum -> Integer change and level 9 compression, an 89_pegase dataset gets about 30% smaller.

That's not bad! For the record: this 30% is to be compared (/combined) with merging some fields (e.g. merging duals of lower/upper bound constraints).