Missing headers, hostname and syslog tag validation can cause data to be ommited

[JPvRiel]

When metadata processing is enabled and the advanced (now overly complicated) configuration applies a lot of conditionals and tricks to check if the syslog RFC3164 parser had to deal with a message that hasn't got a proper header. If it seems the message lacks a header, it will distrust the header properties the RFC 3164 parser module set. This helps avoid getting junk populated in as hostnames or app-names. From production experience, vulnerability scanners and Cisco network devices can cause a lot of bogus data to show up in the $hostname or $app-name properties.

While extra work is done to try remap or validate the properties, in certain circumstances, the $msg property used in templates will not include the distrusted content rsyslog found as part of the assumed syslogtag. Output templates using $msg will then possibly miss parts of the original message. Templates that include the $rawmsg property don't have to worry as much about this issue, e.g. TmplJSONRawMsg, but other templates, such as TmplRFC5424EndMetaShort might cut out "misinterpreted" pieces at the start of the message when it gets forwarded.

In 60-ruleset.conf.tmpl, there has been this placeholder:

      # If we wanted to correct a misaligned $msg due to a bad header... Failing to do this may swallow part of the malformed message where the parsed values were untrusted and discarded.
      #set $.msg = $rawmsg-after-pri;

However, it's performance impact could be significant and this hasn't been tested, so for now, it's left as a known concern.