kubernetes fluentd

Collects logs from your nodejs application, assumed you use winston like this project.

How it works

It parse logs generated by your nodejs application (using winston), these logs exist in /var/log/containers/*, which is managed by kubernetes.

your log message will look like {"log":"message","stream":"stdout","time":"2016-10-04T11:31:02.492951101Z"}. but log entry must have a json object, otherwise it will be ignored by current fluentd configuration.

build & Usage

  $ docker build -t k8s-fluentd-nodejs-winston .
  $ docker run -d -p 24224:24224 -p 24231:24231 \
      --name fluentd \
      -e ELASTICSEARCH_HOST=elasticsearch \
      -e ELASTICSEARCH_PORT=9200 \
      -e SLACK_CHANNEL=general \
      -e SLACK_WEBHOOK_URL=https://hooks.slack.com/services/XXX/XXX/XXX \
      -e KIBANA_HOST=http://kibana.internal.domain.com \
      k8s-fluentd-nodejs-winston

Notes

• Elasticsearch Host and port are configured with environment variables ELASTICSEARCH_HOST default is elasticsearch ELASTICSEARCH_INDEX default is logstash ELASTICSEARCH_SCHEMA default is http ELASTICSEARCH_SSL_VERIFY default is false ELASTICSEARCH_PORTdefault is 9200
• ELASTICSEARCH_USER and ELASTICSEARCH_PASSWORD env may contains credentials to your elasticsearch
• Slack is configured by SLACK_WEBHOOK_URL and SLACK_CHANNEL
• KIBANA_HOST should contain link for your kibana host.
• Fluentd configuration is based on gcp-fluentd and fabric8io/docker-fluentd-kubernetes
• Fluentd exclude kubernetes logs which start with kube or k8s
• json_in_string parser is based on fluentd-elasticsearch-container