syslog.socket: Failed with result 'trigger-limit-hit'.

[gkeishin]

Build: https://openpower.xyz/job/openbmc-build/1927/

BMC Firmware level:

ID="openbmc-phosphor"
NAME="Phosphor OpenBMC (Phosphor OpenBMC Project Reference Distro)"
VERSION="2.6.0-rc1"
VERSION_ID="2.6.0-rc1-31-g0cab252"
PRETTY_NAME="Phosphor OpenBMC (Phosphor OpenBMC Project Reference Distro) 2.6.0-rc1"
BUILD_ID="2.6.0-rc1"

Journald snippet

Jan 14 08:35:23 witherspoon systemd[1]: syslog.socket: Trigger limit hit, refusing further activation.
Jan 14 08:35:23 witherspoon systemd[1]: syslog.socket: Failed with result 'trigger-limit-hit'.

root@witherspoon:~# systemctl status syslog.socket | cat
● syslog.socket - Syslog Socket
   Loaded: loaded (/lib/systemd/system/syslog.socket; static; vendor preset: disabled)
   Active: failed (Result: trigger-limit-hit) since Mon 2019-01-14 08:50:34 UTC; 15min ago
     Docs: man:systemd.special(7)
           https://www.freedesktop.org/wiki/Software/systemd/syslog
   Listen: /run/systemd/journal/syslog (Datagram)

Jan 14 08:50:34 witherspoon systemd[1]: Listening on Syslog Socket.
Jan 14 08:50:34 witherspoon systemd[1]: syslog.socket: Trigger limit hit, refusing further activation.
Jan 14 08:50:34 witherspoon systemd[1]: syslog.socket: Failed with result 'trigger-limit-hit'.
root@witherspoon:~#
```

This is a pervasive failure and seen on Witherspoon / Romulus and likely on other as well.

This has an impact on rsyslog module and probably because of this even when the config was created the server config file is not created by the app

```
cat: can't open '/etc/rsyslog.d/server.conf': No such file or directory
``` 

[dkodihal]

https://gerrit.openbmc-project.xyz/#/c/openbmc/phosphor-logging/+/17320/ fixes this.

[gkeishin]

We are still seeing the syslog exiting other remote logging associated with it are fixed and clean.

[maximyep]

Few attempts to do a patch leads to the error as described above

Mar 21 09:18:55 romulus systemd[1]: Started Hostname Service.
Mar 21 09:18:55 PatchName systemd-resolved[764]: System hostname changed to 'PatchName'.
Mar 21 09:18:55 PatchName systemd-hostnamed[1383]: Changed static host name to 'PatchName'
Mar 21 09:18:55 PatchName systemd[1]: Listening on Syslog Socket.
Mar 21 09:18:56 PatchName systemd[1]: syslog.socket: Trigger limit hit, refusing further activation.
Mar 21 09:18:56 PatchName systemd[1]: syslog.socket: Failed with result 'trigger-limit-hit'.

Log:
qemu-log-2019-03-21.txt