[User Poll] Migration to Kafka 4.0 - removal of Zookeeper based Kafka clusters

[k-wall]

Kafka 4.0 is not too far away now so we need to think about how this project will transition to Kafka 4.0.

This project currently has the ability to produce Kafka clusters for testing purposes. It can produce either KRaft clusters or Kafka Clusters with Zookeeper. The cluster can either run in-VM, or with Kafka Broker(s)/Zookeeper as containers.

The project realises its in-VM support using compile time dependencies on the Kafka libraries. It literally creates
instances of kafka.server.KafkaServer or kafka.server.KafkaRaftServer depending on your wishes.

When Kafka 4.0 comes along, inevitably some of the symbols this library depends upon will be gone when Kafka 4.x JARs are on the classpath.

It might be possible to retain the extension's support for Zookeeper based Kafka clusters when Kafka 3.x jars are on the classpath. Reflective code might be one way or forcing a fallback onto the Container path might be another. It is probably going to be a fair amount of work.

Poll

We'd like to straw poll users of this library to gauge the importance of ability this library to produce Zookeeper based clusters (i.e. those clusters produced with the @ZooKeeperCluster annotation).

You might have code like this:

    @Test
    void myFunkyTest(@BrokerCluster @ZooKeeperCluster KafkaCluster cluster)
            throws Exception {
        // do some stuff with cluster
    }

If we were to remove this feature, would this impact your test-cases?

Reply with emojis on to this comment:

😊 = content for Zookeeper based Kafka clusters support to be removed
🙁 = prefer Zookeeper based Kafka clusters support to continue.

More expansive thoughts welcomed too, just leave comments.

This poll is indicative. We'll do our best to support the wishes of the majority but can't make a promise.

This poll with remain open until 2024-10-01.