From 2f681b37a7390aca5df2fd023f98cb01aa3e0c5a Mon Sep 17 00:00:00 2001 From: Ben Wheatley Date: Sun, 14 Apr 2019 02:36:31 +0100 Subject: [PATCH] parser_json: Document new stream_buffer_size param See fluent/fluentd#2381 In addition to documenting the parameter, update the `in_exec` and `out_exec_filter` articles to provide a workaround for users encountering this problem. This note exists in the v0.12 version of the `in_exec` page but was absent in the v1.0 docs, so add the whole note. Closes #624 Signed-off-by: Ben Wheatley --- docs/v1.0/in_exec.txt | 2 ++ docs/v1.0/out_exec_filter.txt | 2 +- docs/v1.0/parser_json.txt | 11 +++++++++++ 3 files changed, 14 insertions(+), 1 deletion(-) diff --git a/docs/v1.0/in_exec.txt b/docs/v1.0/in_exec.txt index e552d6fe..a592a4b5 100644 --- a/docs/v1.0/in_exec.txt +++ b/docs/v1.0/in_exec.txt @@ -25,6 +25,8 @@ You can run a program periodically or permanently. To run periodically, please u NOTE: Please see the Config File article for the basic structure and syntax of the configuration file. +NOTE: When using the json format in <parse> section, this plugin uses the Yajl library to parse the program output. Yajl buffers data internally so the output isn't always instantaneous. See the LINK:[parser_json](parser_json#stream_buffer_size) documentation to adjust this. + ## Plugin helpers * [compat_parameters](api-plugin-helper-compat_parameters) diff --git a/docs/v1.0/out_exec_filter.txt b/docs/v1.0/out_exec_filter.txt index 32ce295f..7f72999d 100644 --- a/docs/v1.0/out_exec_filter.txt +++ b/docs/v1.0/out_exec_filter.txt @@ -27,7 +27,7 @@ The `out_exec_filter` Buffered Output plugin (1) executes an external program us NOTE: Please see the LINK:[Config File](config-file) article for the basic structure and syntax of the configuration file. -NOTE: When using the json format in <parse> section, this plugin uses the Yajl library to parse the program output. Yajl buffers data internally so the output isn't always instantaneous. +NOTE: When using the json format in <parse> section, this plugin uses the Yajl library to parse the program output. Yajl buffers data internally so the output isn't always instantaneous. See the LINK:[parser_json](parser_json#stream_buffer_size) documentation to adjust this. ## Supported modes diff --git a/docs/v1.0/parser_json.txt b/docs/v1.0/parser_json.txt index 4cb63a45..de9186f4 100644 --- a/docs/v1.0/parser_json.txt +++ b/docs/v1.0/parser_json.txt @@ -14,6 +14,17 @@ See [Parse section configurations](parse-section) Set JSON parser. +### stream_buffer_size + +| type | default | version | +|:-------:|:-------:|:-------:| +| integer | 8092 | 1.4.3 | + +The amount of data, in bytes, that the Yajl parser will wait for before parsing and emitting JSON objects. +See the [Yajl documentation](https://www.rubydoc.info/github/brianmario/yajl-ruby/Yajl%2FParser:parse) for further details. + +This will only take effect when the parser is being to process an IO stream, e.g. in the context of the `out_exec_filter` or `in_exec` plugins. + ## Example :::text