Add control over compression level

[zamazan4ik]

Hi!

Now the library allows enabling compression before sending to Pulsar but does not allow to choose compression level - the defaults are used. Could you please add the possibility to control compression level for the compression algorithms? It will allow to configure more precisely tradeoff between CPU and size of the messages.

Thanks in advance!

[zamazan4ik]

@tisonkun @CleverAkanoa What do you think about the idea?

[tisonkun]

@zamazan4ik It seems that the compression level is a zstd specific config. Do you propose to add a Producer config CompressionLevel as Pulsar Go client does so that when zstd is used, we can configure the compression level also?

[tisonkun]

@zamazan4ik If so, you can review #240 and if you agree with the implementation, I can schedule a 4.3.0 release to include this feature.

[zamazan4ik]

@tisonkun Why we are not able to add compression-level support for other algorithms?

• flate2: https://docs.rs/flate2/0.2.20/flate2/write/struct.ZlibEncoder.html#method.new
• lz4: https://docs.rs/lz4/latest/lz4/block/enum.CompressionMode.html

As far as I see, only Snappy does not support compression levels.

So I suggest enabling compression-level parameters for all other compression algorithms.

[tisonkun]

OK. These configs seem heterogeneous and I need a few time to find a proper data model.

If you have good idea, feel free to comment on the PR #240 or send your patch.

[tisonkun]

@zamazan4ik released as https://crates.io/crates/pulsar 5.0.0.

You may take a look at #224 to help triage tasks for a compatible release against Pulsar 2.10.x (whether the tasks are valid, how to reproduce, what's their priority, etc.)

[zamazan4ik]

Thanks a lot!