The short_message field has been populated with array, not a string

[eyudkin]

Hi guys!
Im using logstash with logstash-output-gelf plugin via docker, there is my docker-compose config:

  logstash:
    image: docker.elastic.co/logstash/logstash-oss:6.6.1
    volumes:
      - ./logstash/config:/usr/share/logstash/config
      - ./logstash/pipeline:/usr/share/logstash/pipeline
    ports:
      - "5000:5000"
      - "9600:9600"
    environment:
      LS_JAVA_OPTS: "-Xmx256m -Xms256m"
    links:
      - graylog
    depends_on:
      - graylog
    command: bash -c "logstash-plugin install logstash-output-gelf && /usr/local/bin/docker-entrypoint --log.level debug --config.debug"

And Im trying to send my logs to graylog using the following config:

input {
  http {
    host => "0.0.0.0"
    port => 5000
  }
}

filter {
  if ![message] or [message] == "" {
    mutate {
      add_field => { "message" => "Hello, world!" }
    }
  }
}

output {
  gelf {
    host => "graylog"
    port => 12201
    protocol => "TCP"
    short_message => "myshortmessage"
  }
  stdout {}
}

Graylog seems connected (I can see this connection in its UI) and I can successfully send messages to it using netcat.
Unfortunately logstash-to-graylog messaging does not work, it seems that graylog declines my messages.
There is my log output from logstash:

Sending GELF event {:event=>{"short_message"=>["", "Hello, world!"], "full_message"=>",Hello, world!", "host"=>"my-ip-was-here", "_headers_http_host"=>"myhost", "_headers_http_version"=>"HTTP/1.1", "_headers_http_user_agent"=>"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36", "_headers_cache_control"=>"no-cache", "_headers_http_accept"=>"image/webp,image/apng,image/*,*/*;q=0.8", "_headers_accept_encoding"=>"gzip, deflate", "_headers_content_length"=>"0", "_headers_accept_language"=>"en-US,en;q=0.9,ru-RU;q=0.8,ru;q=0.7", "_headers_request_method"=>"GET", "_headers_request_path"=>"/favicon.ico", "_headers_pragma"=>"no-cache", "_headers_connection"=>"keep-alive", "_headers_referer"=>"somereferer", "level"=>6}}

---

And If I modify my logstash config with:

filter {
  if ![message] or [message] == "" {
    mutate {
      add_field => { "message" => "Hello, world!" }
    }
  }
  if ![short_message] or [short_message] == "" {
    mutate {
      add_field => { "short_message" => "%{message}"}
    }
  }
}

Then it successfully works, graylog receives and shows me all my messages. Logstash debug output is following in this case:

Sending GELF event {:event=>{"short_message"=>",Hello, world!-,Hello, world!", "full_message"=>",Hello, world!", "host"=>"my-ip-was-here", "_headers_upgrade_insecure_requests"=>"1", "_headers_http_version"=>"HTTP/1.1", "_headers_request_method"=>"GET", "_headers_http_accept"=>"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3", "_headers_content_length"=>"0", "_headers_cache_control"=>"max-age=0", "_headers_http_host"=>"my-host-was-here", "_headers_accept_encoding"=>"gzip, deflate", "_headers_connection"=>"keep-alive", "_headers_http_user_agent"=>"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36", "_headers_request_path"=>"/", "_headers_accept_language"=>"en-US,en;q=0.9,ru-RU;q=0.8,ru;q=0.7", "level"=>6}}

---

Please, check the difference:
:event=>{"short_message"=>["", "Hello, world!"] vs :event=>{"short_message"=>"Hello, world!"

It looks like "short_message" option doesnt work as expected and adds array instead of string which is invalid/unexpected format for gelf/graylog.

---

Btw, "short_message" is the required field for gelf (please check gelf docs) but there are no information about it in logstash-gelf docs here or here.
I suggest to make this field required.

[dmuensterer]

+1 Any progress here? I just encountered the same issue.
The short_message field is not set correctly, thus making the GELF output not working as expected.
If you need any more details to fix, please let me know.