Kafka 2.1.0 on Java 11

See the 5.0 milestone for details. There are quite some breaking changes, now that we'd destabilize anyway, such as a new zookeeper storageclass and auto.create.topics.enable changed to false.

Kafka 2.1.0 is the first Kafka release with Java 11 support. Previously it was Java 8, while support for Docker containers was greatly improved with Java 10. See https://medium.com/adorsys/jvm-memory-settings-in-a-container-environment-64b0840e1d9e and https://blog.docker.com/2018/04/improved-docker-container-integration-with-java-10/ for background.

We should take the leap sooner rather than later because memory limits with java containers is a huge improvement. Actually it's quite difficult to maintain a production cluster without resource limits on containers. Thus we encourage everyone to try to stabilize on Kafka 2.1.0 + Java 11 in test environments as soon as possible.

If you need a more well tested setup or rely on other tools in the Kafka ecosystem that are not yet available with 2.1+ support, see the v4.3.0 release of this repo.

Proper shutdown command

This release makes shutdown graceful. See #207 for details.

Kafka 1.0.2 + utilities

• Kafka upgraded to 1.0.2
• Added Mailgun Pixy
• Upgraded Burrow to 1.1.0
• Switched events-kube from curl+kafkacat to Eventrouter (#186)

Like v4.0.0 but from master

See https://github.com/Yolean/kubernetes-kafka/tree/v4.0.0

New recommended server.properties and init script

• Config changes: #170 #148
• Init script updated for robustness, lower risk for partial success: #167
• Kafka 1.0.1, not #172

For Kubernetes 1.8

Apart from updating some yml to new apiVersions, this release also includes:

• Adds additional listener on port 9094 for clients outside the cluster (#78)
• Adds bootstrap.kafka service, for clients' --bootstrap.servers args to be independent of the current status of broker pods (#52)
• Dynamically provisioned volumes are now Retained by default (#85)
• Example kafka-manager manifests, to list topics w/ stats (#83).
• The readiness tests now stay online, to show the effect of broker/node disruptions (#79)
• The Kafka readiness probe has been simplified (#81)

Kafka 0.11.0.0

Highlights:

• Removes the kafka.kafka service, to encourage use of bootstrap.servers and thus actual "listener" names (#21)
• Adds support for Balancing Replicas Across availability zones (#41)
• Uses dynamically provisioned volumes
• ZooKeeper too uses persistent volumes (#33)
• Support for RBAC (#59)
• Tested with Kubernetes 1.6 and 1.7

Additional features as "addons", notably:

• Support for pull-based monitoring such as Prometheus (#49, known issue #61)
• Schema Registry and REST Proxy (#45)
• Single node scale (#44)