Skip to content

Releases: linkedin/kafka

2.3.0.29

12 May 23:06
48d70e3
Compare
Choose a tag to compare
[LI-HOTFIX] add configuration to control least loaded node logic in c…

2.4.1.22: [LI-HOTFIX] make GZIP decompression use BufferSupplier (#158)

11 May 22:14
074c432
Compare
Choose a tag to compare

Add a backwards compatible constructor signature in KafkaServer

11 May 19:03
e4977d3
Compare
Choose a tag to compare
2.4.1.21

Add a backwards compatiable constructor signature in KafkaServer (#160)

KafkaActions interface to conditionally invoke environment-dependent actions

10 May 17:38
9dbf990
Compare
Choose a tag to compare
2.4.1.20

Add KafkaActions interface to conditionally invoke environment-depend…

Skip shutdown safety check at epoch already checked

07 May 22:31
1980522
Compare
Choose a tag to compare
[LI-HOTFIX] Skip shutdown safety check at epoch already checked (#157)

TICKET = N/A
LI_DESCRIPTION =
This makes a couple of changes to address some deficiencies
discovered with this safety check when using it in a large, busy
cluster.

* Skip the check if already approved for the requested epoch
* Define redundancy factor property as the required result of
  a broker shutdown, not the required state before shutdown,
  allowing the shutdown of lagging replicas.
* Change default redundancy factor from 2 => 1

In c337e4752 (PR #103) a shutdown safety check was introduced. In
testing this in a large cluster, we found a problem. If the controller
sees that brokerId 1 is safe to shut down at brokerEpoch 99, it will
add that broker to `controllerContext.shuttingDownBrokerIds`. But it
might still not allow shutdown because some leadership transitions
must happen first.

In the process of performing those leadership transitions, brokerId 1
may receive StopReplica, which would cause it to leave the
ISR. Subsequent ControlledShutdownRequest attempts would then fail the
`safeToShutdown` safety check, so the broker would never be allowed to
shut down.

This patch adjusts the implementation of `safeToShutdown`
slightly. Now, if the controller has said that a broker is safe to
shut down at a given epoch, it will always allow that shutdown for
that epoch without performing any additional checks. This allows the
full controlled shutdown process to complete which in a large, busy
cluster may take several rounds of `ControlledShutdownRequest`.

In addition, Lucas pointed out that if a broker which is attempting to
shutdown is already out of ISR, then it is probably pretty harmless to
allow it to shut down if the safety check would have allowed it to when
there are no under-replicated partitions. So we changed the logic a
little further.

ControlledShutdownSafetyCheckEnableProp is now defined as,
 "Shutdown will be allowed if removing the candidate broker will leave
min.insync.replicas + ControlledShutdownSafetyCheckEnableProp replicas
in the ISR."

The default redundancy factor is changed to 1.

EXIT_CRITERIA = N/A

Ignore the flaky test testIllegalRequiredAcks

03 May 22:49
b908973
Compare
Choose a tag to compare

Ignore the flaky test testIllegalRequiredAcks

Adding async fetcher threads and controller request merging

03 May 15:17
b1a6c08
Compare
Choose a tag to compare

This release mainly adds two new features on the Kafka broker side

  1. async fetcher threads
  2. merging of controller requests using the LiCombinedControl RPC

CI changes and Backport Add brokerid log for admin client listTopics failure and Fixing tests (#136)

30 Apr 00:32
5497b76
Compare
Choose a tag to compare

This release includes 2 CI related changes to help with release, and 2 backports for features:
CI:
Migrate 2.3-li-1 build to Github Actions (#150)
Remove travis config (#151)
Features:
Add log brokerid for admin client listTopics failure (#131) on 2.4
Fixing tests (#136)

Backport Add brokerid log for admin client listTopics failure and Fixing tests (#136)

27 Apr 23:19
416a544
Compare
Choose a tag to compare

This release includes 2 backports:
Add log brokerid for admin client listTopics failure (#131) on 2.4
Fixing tests (#136)

Backport Add brokerid log for admin client listTopics failure and Fixing tests (#136)

27 Apr 23:00
6582d0a
Compare
Choose a tag to compare

This release includes 2 backports:
Add log brokerid for admin client listTopics failure (#131) on 2.4
Fixing tests (#136)