Unexpectedly-flat JSON output for nested, opaque objects

[gpind]

Using these deps:

• cambium.core 1.1.0
• cambium.codec-cheshire 1.0.0
• cambium.logback.core 0.4.4
• cambium.logback.json 0.4.4

Logging an opaque object (which sometimes happens unintentionally) under a top-level key works (output formatted for legibility):

=> (require '[cambium.core :as log] '[cambium.codec :as codec] '[cambium.logback.json.flat-layout :as flat])
nil
=> (flat/set-decoder! codec/destringify-val)
nil
=> (log/info {:obj (Object.)} "top-level object")
{"timestamp":"2021-06-22T23:50:33.175Z",
 "level":"INFO",
 "thread":"nREPL-session-6f233f7b-f9a8-4308-8554-041007120b46",
 "ns":"cambium-test.core",
 "line":1,
 "obj":"java.lang.Object@6156744d",
 "column":1,
 "logger":"cambium-test.core",
 "message":"top-level object",
 "context":"default"}
nil

Whereas if that object is nested under another key, it results in the surrounding map being flattened in a strange way:

=> (log/info {:nest {:obj (Object.)}} "nested object")
{"timestamp":"2021-06-22T23:50:45.719Z",
 "level":"INFO",
 "thread":"nREPL-session-6f233f7b-f9a8-4308-8554-041007120b46",
 "ns":"cambium-test.core",
 "line":1,
 "column":1,
 "nest":"{\"obj\" #object[java.lang.Object 0x3baa6181 \"java.lang.Object@3baa6181\"]}",
 "logger":"cambium-test.core",
 "message":"nested object",
 "context":"default"}
nil

My logback.xml:

<configuration>
  <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
    <encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
      <layout class="cambium.logback.json.FlatJsonLayout">
        <jsonFormatter class="ch.qos.logback.contrib.jackson.JacksonJsonFormatter" />
        <timestampFormat>yyyy-MM-dd'T'HH:mm:ss.SSS'Z'</timestampFormat>
        <timestampFormatTimezoneId>UTC</timestampFormatTimezoneId>
        <appendLineSeparator>true</appendLineSeparator>
      </layout>
    </encoder>
  </appender>

  <root level="debug">
    <appender-ref ref="STDOUT" />
  </root>
</configuration>

[kumarshantanu]

@gpind Thanks for the report! Are you sure you ran (flat/set-decoder! codec/destringify-val) before logging the nested object? Unfortunately, non-JSON-compliant data (e.g. arbitrary object) is not guaranteed to be deserialized back as data, because the data we pass is encoded as JSON-string before it is decoded back and logged by Logback.

[kumarshantanu]

One potential solution is to exclude the arbitrary objects by rebinding cambium.core/transform-context.

[gpind]

Yes, I ran the second command in the same REPL as the first. It's totally expected that serializing an arbitrary object would result in a string like "java.lang.Object@48eb8577", but what's unfortunate here is that it breaks the surrounding map too.

Ideally we'd be able to remove/stringify these opaque objects generically, perhaps using clojure.walk plus transform-context. Unfortunately (satisfies? cheshire.generate/JSONable ...) doesn't work on the types whose encoders are built into Cheshire, so AFAICT we'd have to whitelist all those types. Not a huge deal, but FWIW it this feels to me like a minor footgun that could/should be eliminated by cambium itself.

[kumarshantanu]

but FWIW it this feels to me like a minor footgun that could/should be eliminated by cambium itself.

Sounds reasonable. Let me think how best to address this in Cambium - also, PRs welcome.