fluentd

Fluentd Input Plugin

The fluentd plugin gathers metrics from plugin endpoint provided by in_monitor plugin. This plugin understands data provided by /api/plugin.json resource (/api/config.json is not covered).

You might need to adjust your fluentd configuration, in order to reduce series cardinality in case whene your fluentd restarts frequently. Every time when fluentd starts, plugin_id value is given a new random value. According to fluentd documentation, you are able to add @id parameter for each plugin to avoid this behaviour and define custom plugin_id.

example configuratio with @id parameter for http plugin:

<source>
  @type http
  @id http
  port 8888
</source>

Configuration:

# Read metrics exposed by fluentd in_monitor plugin
[[inputs.fluentd]]
  ## This plugin reads information exposed by fluentd (using /api/plugins.json endpoint).
  ##
  ## Endpoint:
  ## - only one URI is allowed
  ## - https is not supported
  endpoint = "http://localhost:24220/api/plugins.json"

  ## Define which plugins have to be excluded (based on "type" field - e.g. monitor_agent)
  exclude = [
	  "monitor_agent",
	  "dummy",
  ]

Measurements & Fields:

Fields may vary depends on type of the plugin

• fluentd
  • retry_count (float, unit)
  • buffer_queue_length (float, unit)
  • buffer_total_queued_size (float, unit)

Tags:

• All measurements have the following tags:
  • plugin_id (unique plugin id)
  • plugin_type (type of the plugin e.g. s3)
  • plugin_category (plugin category e.g. output)

Example Output:

$ telegraf --config fluentd.conf --input-filter fluentd --test
* Plugin: inputs.fluentd, Collection 1
> fluentd,host=T440s,plugin_id=object:9f748c,plugin_category=input,plugin_type=dummy buffer_total_queued_size=0,buffer_queue_length=0,retry_count=0 1492006105000000000
> fluentd,plugin_category=input,plugin_type=dummy,host=T440s,plugin_id=object:8da98c buffer_queue_length=0,retry_count=0,buffer_total_queued_size=0 1492006105000000000
> fluentd,plugin_id=object:820190,plugin_category=input,plugin_type=monitor_agent,host=T440s retry_count=0,buffer_total_queued_size=0,buffer_queue_length=0 1492006105000000000
> fluentd,plugin_id=object:c5e054,plugin_category=output,plugin_type=stdout,host=T440s buffer_queue_length=0,retry_count=0,buffer_total_queued_size=0 1492006105000000000
> fluentd,plugin_type=s3,host=T440s,plugin_id=object:bd7a90,plugin_category=output buffer_queue_length=0,retry_count=0,buffer_total_queued_size=0 1492006105000000000