RabbitMQ 4.0.0-rc.2

[michaelklishin]

RabbitMQ 4.0.0-rc.2 is a candidate of a new major release.

Starting June 1st, 2024, community support for this series will only be provided to regularly contributing users
and those who hold a valid commercial support license.

Highlights

Some key improvements in this release are listed below.

• Khepri, an alternative schema data store developed to replace Mnesia,
  has matured and is now fully supported (it previously was an experimental feature)
• AMQP 1.0 is now a core protocol that is always enabled. Its plugin is now a no-op that only exists to simplify upgrades.
• The AMQP 1.0 implementation is now significantly more efficient: its peak throughput is more than double than that of 3.13.x
  on some workloads
• Efficient sub-linear quorum queue recovery on node startup using checkpoints
• Quorum queues now support priorities (but not exactly the same way as classic queues)
• AMQP 1.0 clients now can manage topologies similarly to how AMQP 0-9-1 clients do it
• The AMQP 1.0 convention (address format) used for interacting with with AMQP 0-9-1 entities is now easier to reason about
• Mirroring (replication) of classic queues was removed after several years of deprecation. For replicated messaging data types,
  use quorum queues and/or streams. Non-replicated classic queues remain and their development continues
• Classic queue storage efficiency improvements, in particular recovery time and storage of multi-MiB messages
• Nodes with multiple enabled plugins and little on disk data to recover now start up to 20-30% faster
• New exchange type: Local Random Exchange

See Compatibility Notes below to learn about breaking or potentially breaking changes in this release.

Breaking Changes and Compatibility Notes

Classic Queues is Now a Non-Replicated Queue Type

After three years of deprecated, classic queue mirroring was completely removed in this version.
Quorum queues and streams are two mature
replicated data types offered by RabbitMQ 4.x. Classic queues continue being supported without any breaking changes
for client libraries and applications but they are now a non-replicated queue type.

After an upgrade to 4.0, all classic queue mirroring-related parts of policies will have no effect.
Classic queues will continue to work like before but with only one replica.

Clients will be able to connect to any node to publish to and consume from any non-replicated classic queues.
Therefore applications will be able to use the same classic queues as before.

See Mirrored Classic Queues Migration to Quorum Queues for guidance
on how to migrate to quorum queues for the parts of the system that really need to use replication.

Quorum Queues Now Have a Default Redelivery Limit

Quorum queues now have a default redelivery limit set to 20.
Messages that are redelivered 20 times or more will be dead-lettered or dropped (removed).

This limit is necessary to protect nodes from consumers that run into infinite fail-requeue-fail-requeue loops. Such
consumers can drive a node out of disk space by making a quorum queue Raft log grow forever without allowing compaction
of older entries to happen.

If 20 deliveries per message is a common scenario for a queue, a dead-lettering target or a higher limit must be configured
for such queues. The recommended way of doing that is via a policy.
See the Position Messaging Handling section
in the quorum queue documentation guide.

Note that increasing the limit is recommended against: usually the presence of messages that have been redelivered 20 times or more suggests
that a consumer has entered a fail-requeue-fail-requeue loop, in which case even a much higher limit
won't help avoid the dead-lettering.

For specific cases where the RabbitMQ configuration cannot be updated to include a dead letter policy
the delivery limit can be disabled by setting a delivery limit configuration of -1. However, the RabbitMQ team
strongly recommends keeping the delivery limit in place to ensure cluster availability isn't
accidentally sacrificed.

CQv1 Storage Implementation was Removed

CQv1, the original classic queue storage layer, was removed
except for the part that's necessary for upgrades to CQv2 (the 2nd generation).

In case rabbitmq.conf explicitly sets classic_queue.default_version to 1 like so

# this configuration value is no longer supported,
# remove this line or set the version to 2
classic_queue.default_version = 1

nodes will now fail to start. Removing the line will make the node start and perform
the migration from CQv1 to CQv2.

Settings cluster_formation.randomized_startup_delay_range.* were Removed

The following two deprecated rabbitmq.conf settings were removed:

cluster_formation.randomized_startup_delay_range.min
cluster_formation.randomized_startup_delay_range.max

RabbitMQ 4.0 will fail to boot if these settings are configured in rabbitmq.conf.

Several Disk I/O-Related Metrics were Removed

Several I/O-related metrics are dropped, they should be monitored at the infrastructure and kernel layers

Default Maximum Message Size Reduced to 16 MiB

Default maximum message size is reduced to 16 MiB (from 128 MiB).

The limit can be increased via a rabbitmq.conf setting:

# 32 MiB
max_message_size = 33554432

However, it is recommended that such large multi-MiB messages are put into a blob store, and their
IDs are passed around in messages instead of the entire payload.

AMQP 1.0

RabbitMQ 3.13 rabbitmq.conf setting rabbitmq_amqp1_0.default_vhost is unsupported in RabbitMQ 4.0.

Instead default_vhost will be used to determine the default vhost an AMQP 1.0 client connects to(i.e. when the AMQP 1.0 client
does not define the vhost in the hostname field of the open frame).

MQTT

RabbitMQ 3.13 rabbitmq.conf settings mqtt.default_user, mqtt.default_password,
and amqp1_0.default_user are unsupported in RabbitMQ 4.0.

Instead, set the new RabbitMQ 4.0 settings anonymous_login_user and anonymous_login_pass (both values default to guest).
For production scenarios, disallow anonymous logins.

TLS Client (LDAP, Shovels, Federation) Defaults

Starting with Erlang 26, client side TLS peer certificate chain verification settings are enabled by default in most contexts:
from federation links to shovels to TLS-enabled LDAP client connections.

If using TLS peer certificate chain verification is not practical or necessary, it can be disabled.
Please refer to the docs of the feature in question, for example,
this one on TLS-enabled LDAP client connections,
two others on TLS-enabled dynamic shovels and dynamic shovel URI query parameters.

Shovels

RabbitMQ Shovels will be able connect to a RabbitMQ 4.0 node via AMQP 1.0 only when the Shovel runs on a RabbitMQ node >= 3.13.7.

TLS-enabled Shovels will be affected by the TLS client default changes in Erlang 26 (see above).

Erlang/OTP Compatibility Notes

This release requires Erlang 26.2.

Provisioning Latest Erlang Releases explains
what package repositories and tools can be used to provision latest patch versions of Erlang 26.x.

Release Artifacts

RabbitMQ releases are distributed via GitHub.
Debian and RPM packages are available via
repositories maintained by the RabbitMQ Core Team.

Community Docker image, Chocolatey package, and the Homebrew formula
are other installation options. They are updated with a delay.

Upgrading to 4.0

Documentation guides on upgrades

See the Upgrading guide for documentation on upgrades and GitHub releases
for release notes of individual releases.

This release series only supports upgrades from 3.13.x.

This release requires all feature flags in the 3.x series (specifically 3.13.x) to be enabled before upgrading,
there is no upgrade path from 3.12.14 (or a later patch release) straight to 4.0.0.

Required Feature Flags

This release graduates all feature flags introduced up to 3.13.0.

All users must enable all stable [feature flags] before upgrading to 4.0 from
the latest available 3.13.x patch release.

Mixed version cluster compatibility

RabbitMQ 4.0.0 nodes can run alongside 3.13.x nodes. 4.0.x-specific features can only be made available when all nodes in the cluster
upgrade to 4.0.0 or a later patch release in the new series.

While operating in mixed version mode, some aspects of the system may not behave as expected. The list of known behavior changes will be covered in future updates.
Once all nodes are upgraded to 4.0.0, these irregularities will go away.

Mixed version clusters are a mechanism that allows rolling upgrade and are not meant to be run for extended
periods of time (no more than a few hours).

Recommended Post-upgrade Procedures

Configure Dead Lettering or Increase the Limit for Frequently Redelivered Messages

In environments where messages can experience 20 redeliveries, the affected queues should have dead lettering
configured (usually via a policy) to make sure
that messages that are redelivered 20 times are moved to a separate queue (or stream) instead of
being dropped (removed) by the crash-requeue-redelivery loop protection mechanism.

Alternatively, the limit can be increased using a policy.
This option is recommended against: usually the presence of messages that have been redelivered 20 times or more suggests
that a consumer has entered a fail-requeue-fail-requeue loop, in which case even a much higher limit
won't help avoid the dead-lettering.

Changes Worth Mentioning

This section is incomplete and will be expanded as 4.0 approaches its release candidate stage.

Core Server

Enhancements

• Efficient sub-linear quorum queue recovery on node startup using checkpoints.

  GitHub issue: #10637

• Classic queue storage v2 (CQv2) optimizations. For example, CQv2 recovery time on node boot
  is now twice as fast for some data sets.

  GitHub issue: #11112

• Node startup time improvements. For some environments, nodes with very small on disk data sets
  now start about 25% quicker.

  GitHub issue: #10989

• Quorum queues now support priorities. However,
  there are difference with how priorities work in classic queues.

  GitHub issue: #10637

• Per-message metadata stored in the quorum queue Raft log now uses less disk space.

  GitHub issue: #8261

• Single Active Consumer (SAC) implementation of quorum queues now respects consumer priorities.

  GitHub issue: #8261

• rabbitmq.conf now supports encrypted values
  with a prefix:

  default_user = bunnies-444
  default_pass = encrypted:F/bjQkteQENB4rMUXFKdgsJEpYMXYLzBY/AmcYG83Tg8AOUwYP7Oa0Q33ooNEpK9

  GitHub issue: #11989

• All feature flags up to 3.13.0 have graduated and are now mandatory.

  GitHub issue: #11659

• Quorum queues now use a default redelivery limit of 20.

  GitHub issue: #11937

• queue_master_locator queue setting has been deprecated in favor of queue_leader_locator used by quorum queues
  and streams.

  GitHub issue: #10702

AMQP 1.0

Bug Fixes

• AMQP 0-9-1 to AMQP 1.0 string data type conversion improvements.

  GitHub issue: #11715

Enhancements

• AMQP 1.0 is now a core protocol that is always enabled.
  Its plugin is now a no-op that only exists to simplify upgrades.

  GitHub issues: #9022, #10662

• The AMQP 1.0 implementation is now significantly more efficient: its peak throughput is more than double than that of 3.13.x
  on some workloads.

  GitHub issue: #9022

• For AMQP 1.0, resource alarms only block inbound TRANSFER frames instead of blocking all traffic.

  GitHub issue: #9022

• AMQP 1.0 clients now can manage topologies (queues, exchanges, bindings).

  GitHub issue: #10559

• AMQP 1.0 implementation now supports a new (v2) address format for referencing queues, exchanges, and so on.

  GitHub issues: #11604, #11618

• AMQP 1.0 implementation now supports consumer priorities.

  GitHub issue: #11705

• Client-provided connection name will now be logged for AMQP 1.0 connections.

  GitHub issue: #11958

Streams

Enhancements

• Stream filtering is now supported for AMQP 1.0 clients.

  GitHub issue: #10098

Prometheus Plugin

Enhancements

• Detailed memory breakdown metrics are now exposed via the Prometheus scraping endpoint.

  GitHub issue: #11743

• New per-exchange and per-queue metrics.

  Contributed by @LoisSotoLopez.

  GitHub issue: #11559

• Shovel and Federation metrics are now available via two new plugins: rabbitmq_shovel_prometheus and rabbitmq_federation_prometheus.

  Contributed by @SimonUnge.

  GitHub issue: #11942

Shovel Plugin

Enhancements

• Shovels now can be configured to use pre-declared topologies. This is primarily useful in environments where
  schema definition comes from definitions.

  GitHub issue: #10501

Local Random Exchange Plugin

This is an initial release that includes Local Random Exchange.

GitHub issues: #8334, #10091.

STOMP Plugin

Enhancements

• STOMP now supports consumer priorities.

  GitHub issue: #11947

Dependency Changes

• Ra was upgraded to 2.14.0
• Khepri was upgraded to 0.16.0
• Cuttlefish was upgraded to 3.4.0

Source Code Archives

To obtain source code of the entire distribution, please download the archive named rabbitmq-server-4.0.0-rc.2.tar.xz
instead of the source tarball produced by GitHub.

---

This discussion was created from the release RabbitMQ 4.0.0-rc.2.