Making Logrus/logging defaults pluggable?

[afoltzm]

I'm trying to improve observability in Argo as I upgrade, and the way logs show up in Stackdriver is one of the first major hurdles I've encountered with it. I understand the default behavior in Logrus is to send all logging to stderr; I may not agree with that design choice, but it's not my library and it is too late for Logrus to change something like that without inflicting a ton of churn on everyone who relies on that default. However, Logrus does allow for an override of the default behavior:

func init() {
  // Log as JSON instead of the default ASCII formatter.
  log.SetFormatter(&log.JSONFormatter{})

  // Output to stdout instead of the default stderr
  // Can be any io.Writer, see below for File example
  log.SetOutput(os.Stdout)

  // Only log the warning severity or above.
  log.SetLevel(log.WarnLevel)
}

I'm not super familiar with the Argo codebase, but it looks like the init code for Logrus in the workflow controller is just as simple as the example init code above:

func initConfig() {
	log.SetFormatter(&log.TextFormatter{
		TimestampFormat: "2006-01-02T15:04:05.000Z",
		FullTimestamp:   true,
	})
}

To me, it seems like adding in a line for log.SetOutput() in initConfig() would be a straightforward way of overriding the defaults for Argo. Can we make the log output target configurable via an environment variable / ConfigMap so that users who wish to override the default can do so? It would help improve the signal-to-noise ratio of logs for those of us relying on the defaults of Stackdriver and other tools that make assumptions about stderr that don't match the convention used by Logrus.

[afoltzm]

It also occurs to me that additional customizations for Logrus could be surfaced to Argo administrators via a similar mechanism, such as logging to structured JSON with log.SetFormatter(&log.JSONFormatter{}), which would be similarly helpful for searching/filtering on log levels.

[terrytangyuan]

Structured JSON logging is already supported via --log-format. I don't think other customization is supported yet besides logging level.