Expose zran functionality at module level

[forrestfwilliams]

I'm interested in using the zran functionality that is present within indexed_gzip with files that are not gzips, but were also compressed using zlib (e.g., zlib compressed zip files). However, the zran functionality (see here) is not available from the installed package. Would it be possible to expose this functionality at the module level?

Related to issue #109.

[forrestfwilliams]

@martindurant any thoughts on this?

[pauldmccarthy]

Hi @forrestfwilliams, this shouldn't be too difficult, as there is already a Cython interface definition file exposing the main zran functions. So actually - if you code in Cython, the zran module is accessible via indexed_gzip.zran. Otherwise it would be fairly straightforward to add a new Cython module which exposes its contents to pure Python code.

On a related note, currently the indexed_gzip Python package is the only formal component of the public API of this library, so exposing the zran library would be a fairly major change in terms of the promises that this package provides (this is not to say that I'm against the idea!).

Also related - a few months ago I did some refactoring (still active on the enh/libigzip branch) to a) support zlib-ng in addition to zlib, and b) expose indexed_gzip functionality as a C/C++ library. But progress has stalled on this, and I'm not sure when I'll get back to it.

[forrestfwilliams]

Thanks for your input on this! I'm interested in making this a reality but I don't have any experience with Cython so it may take me a while to get up to speed.

[martindurant]

were also compressed using zlib

I'm pretty sure that it should be possible to add gzip headers to any bytestream, so that you can handle deflate/ZIP or zlib with the library as-is, with a little manipulation of the file-like that gets passed. That's probably how I would do it, rather than reach into the low-level functions.

OTOH, I don't see an equivalent project in rust ( the original zran was of course C), so #124 with a clean C API is I think still interesting.

[forrestfwilliams]

Hey @martindurant wanted to mention that I ended up creating the zran Python library which is heavily inspired by this project, but exposes more of the low-level functionality. I hope it's useful for you!

[martindurant]

Thanks @forrestfwilliams . When I have the chance to work with it, I'll leave an issue on your repo with muy thoughts.