Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Outbox tweaks #6176

Merged
merged 2 commits into from
Aug 9, 2023
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 7 additions & 9 deletions nservicebus/outbox/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,9 @@ Most message queues, and some data stores, do not support distributed transactio

The NServiceBus **outbox** feature ensures consistency between business data and messages. It simulates an atomic transaction, distributed across both the data store used for business data and the message queue used for messaging.

Note: Messages dispatched to the transport as part of the outbox dispatch stage will not be batched and each message is sent in isolation.
danielmarbach marked this conversation as resolved.
Show resolved Hide resolved

Note: Messages sent with [immediate dispatch](/nservicebus/messaging/send-a-message.md#dispatching-a-message-immediately) will be sent immediately and won't be handled by the outbox.

Note: Messages sent using `IMessageSession` won't be handled by the outbox. The outbox requires an incoming message context. Use the `IMessageHandlerContext` instance to dispatch messages handled by the outbox.
Note: Messages sent using `IMessageSession` won't be handled by the outbox. The outbox requires an incoming message context. Use the `IMessageHandlerContext` instance to dispatch messages handled by the outbox. Use the [Transactional Session](/nservicebus/transactional-session) when requiring outbox behavior without an incoming message.

## The consistency problem

Expand All @@ -42,15 +40,15 @@ Consider a message handler that creates a `User` in the business database, and a
To avoid these problems, developers of distributed systems have two options:

1. Ensure all message handlers are [idempotent](https://en.wikipedia.org/wiki/Idempotence). This means each message handler can handle the same message multiple times without adverse side effects. This is often difficult to achieve.
2. Implement infrastructure which guarantees consistency between business data and messages. This avoids the need for all messages handlers to be idempotent.
2. Implement infrastructure which guarantees consistency between business data and messages. This avoids the need for all message handlers to be idempotent.

The outbox feature is the infrastructure described in the second option.

## How it works

The outbox feature guarantees that each message is processed once and only once, using the database transaction used to store business data.

Returning to the earlier example of a message handler which creates a `User` and then publishes a `UserCreated` event, the following process occurs. Details are described following the diagram.
Returning to the earlier example of a message handler that creates a `User` and then publishes a `UserCreated` event, the following process occurs. Details are described following the diagram.

```mermaid
graph TD
Expand Down Expand Up @@ -143,7 +141,7 @@ MSDTC uses a chatty protocol due to the need for multiple resource managers to c

The more resource managers involved and/or the higher the latency between them, the worse MSDTC performs. Cloud environments, in particular, have much higher latency and the message queue and the data store are often not even located in the same server rack.

The rise of cloud infrastructure and [decline of MSMQ](https://particular.net/blog/msmq-is-dead) have contributed to the overall decline in use of MSDTC in the software development industry.
The rise of cloud infrastructure and [decline of MSMQ](https://particular.net/blog/msmq-is-dead) have contributed to the overall decline in the use of MSDTC in the software development industry.

The outbox feature is designed to provide the same level of consistency between business data and messages provided by MSDTC, without the need to coordinate multiple resource managers.

Expand Down Expand Up @@ -177,11 +175,11 @@ The outbox only uses the incoming [message identifier](/nservicebus/messaging/me

## Outbox expiration duration

To determine if a message has been processed before, the identification data for each outbox record is retained. The duration that this data is retained for varies depending on the persistence chosen for the outbox. The default duration, as well as the frequency of data removal, can be overridden for all outbox persisters.
To determine if a message has been processed before, the identification data for each outbox record is retained. The duration that this data is retained varies depending on the persistence chosen for the outbox. The default duration, as well as the frequency of data removal, can be overridden for all outbox persisters.

After the outbox data retention period has lapsed, a retried message will be seen as the first of its kind and will be reprocessed. It is important to ensure that the retention period of outbox data is longer than the maximum time the message can be retried, including delayed retries and manual retries via ServicePulse.

Depending on the throughput of the system's endpoints, the outbox cleanup interval may need to be run more frequently. Increased frequency will allow each cleanup operation to purge the fewest records possible each time it runs. Purging fewer records will make the purge operation run faster which will ensure that it completes before the next purge operation is due to start.
Depending on the throughput of the system's endpoints, the outbox cleanup interval may need to be run more frequently. The increased frequency will allow each cleanup operation to purge the fewest records possible each time it runs. Purging fewer records will make the purge operation run faster which will ensure that it completes before the next purge operation is due to start.

## Storage requirements

Expand All @@ -195,7 +193,7 @@ NOTE: If the system is processing a high volume of messages, having a long dedup

## Persistence

The outbox feature requires persistence in order to perform deduplication and to store outgoing downstream messages.
The outbox feature requires persistent storage in order to perform deduplication and store outgoing downstream messages.

For more information on the outbox persistence options available refer to the dedicated persistence pages:

Expand Down