diff --git a/docs/en/ingest-management/elastic-agent/configuration/outputs/output-elasticsearch.asciidoc b/docs/en/ingest-management/elastic-agent/configuration/outputs/output-elasticsearch.asciidoc index c768f76c2..2a5207209 100644 --- a/docs/en/ingest-management/elastic-agent/configuration/outputs/output-elasticsearch.asciidoc +++ b/docs/en/ingest-management/elastic-agent/configuration/outputs/output-elasticsearch.asciidoc @@ -55,6 +55,8 @@ category. Many of these settings have sensible defaults that allow you to run * <> +* <> + * <> [[output-elasticsearch-commonly-used-settings]] @@ -477,6 +479,56 @@ exports the API under a custom prefix. |=== +[[output-elasticsearch-memory-queue-settings]] +== Memory queue settings + +The memory queue keeps all events in memory. + +The memory queue waits for the output to acknowledge or drop events. If +the queue is full, no new events can be inserted into the memory queue. Only +after the signal from the output will the queue free up space for more events to be accepted. + +The memory queue is controlled by the parameters `flush.min_events` and `flush.timeout`. If +`flush.timeout` is `0s` or `flush.min_events` is `0` or `1` then events can be sent by the output as +soon as they are available. If the output supports a `bulk_max_size` parameter it controls the +maximum batch size that can be sent. + +If `flush.min_events` is greater than `1` and `flush.timeout` is greater than `0s`, events will only +be sent to the output when the queue contains at least `flush.min_events` events or the +`flush.timeout` period has expired. In this mode the maximum size batch that that can be sent by the +output is `flush.min_events`. If the output supports a `bulk_max_size` parameter, values of +`bulk_max_size` greater than `flush.min_events` have no effect. The value of `flush.min_events` +should be evenly divisible by `bulk_max_size` to avoid sending partial batches to the output. + +This sample configuration forwards events to the output if 512 events are available or the oldest +available event has been waiting for 5s in the queue: + +[source,yaml] +------------------------------------------------------------------------------ +queue.mem: + events: 4096 + flush.min_events: 512 + flush.timeout: 5s +------------------------------------------------------------------------------ + +[cols="2*> +* <> + * <> [[output-logstash-commonly-used-settings]] @@ -162,6 +164,56 @@ use SSL/TLS. For more information, refer to <>. +[[output-logstash-memory-queue-settings]] +== Memory queue settings + +The memory queue keeps all events in memory. + +The memory queue waits for the output to acknowledge or drop events. If +the queue is full, no new events can be inserted into the memory queue. Only +after the signal from the output will the queue free up space for more events to be accepted. + +The memory queue is controlled by the parameters `flush.min_events` and `flush.timeout`. If +`flush.timeout` is `0s` or `flush.min_events` is `0` or `1` then events can be sent by the output as +soon as they are available. If the output supports a `bulk_max_size` parameter it controls the +maximum batch size that can be sent. + +If `flush.min_events` is greater than `1` and `flush.timeout` is greater than `0s`, events will only +be sent to the output when the queue contains at least `flush.min_events` events or the +`flush.timeout` period has expired. In this mode the maximum size batch that that can be sent by the +output is `flush.min_events`. If the output supports a `bulk_max_size` parameter, values of +`bulk_max_size` greater than `flush.min_events` have no effect. The value of `flush.min_events` +should be evenly divisible by `bulk_max_size` to avoid sending partial batches to the output. + +This sample configuration forwards events to the output if 512 events are available or the oldest +available event has been waiting for 5s in the queue: + +[source,yaml] +------------------------------------------------------------------------------ +queue.mem: + events: 4096 + flush.min_events: 512 + flush.timeout: 5s +------------------------------------------------------------------------------ + +[cols="2*