Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configuration for formatting Throwable stack traces #103

Open
HaloFour opened this issue Nov 10, 2020 · 6 comments
Open

Configuration for formatting Throwable stack traces #103

HaloFour opened this issue Nov 10, 2020 · 6 comments
Labels
agent-java community Issues and PRs created by the community

Comments

@HaloFour
Copy link
Contributor

I'd like to see more options for how stack traces are formatted in the ECS logs to help in reducing the size of logs.

Some examples:

  1. Maximum depth
  2. Include/exclude stack frames based on a pattern
  3. Shorten class names by abbreviating package names to fit within a specified length
  4. Root cause first or last

Ideally a configuration option could override the formatting of the stack trace through an interface implementation provided by the application. That implementation could return a String[] or Stream<String> of the formatted stack frames which are then added to the ECS log either as a JSON string or an array of strings.

I'd be happy to work on a PR for this if there is interest in it being included.

@felixbarny felixbarny linked a pull request Nov 11, 2020 that will close this issue
@michaelmcfadyensky
Copy link
Contributor

I have the same requirement as this and would preferably use the ecs encoder with our logback.xml, if possible. I'm currently using the logback logstash encoder along with a ShortenedThrowableConverter to achieve this requirement at the moment.

Would it be possible to expose a property in the ecs-encoder to allow the user to configure any ThrowableConverter in their logback file?
For example:

<encoder class="co.elastic.logging.logback.EcsEncoder">
    <throwableConverter class="net.logstash.logback.stacktrace.ShortenedThrowableConverter">
        <exclude>some.package.to.exclude.*</exclude>
        <maxLength>13400</maxLength>
    </throwableConverter>
</encoder>

@felixbarny
Copy link
Member

That sounds like a useful feature!

The easiest way to integrate that would be to filter the output of Throwable#printStackTrace on a line-by-line basis, using a custom PrintWriter, similar to this

private static void formatThrowableAsArray(final StringBuilder jsonBuilder, final Throwable throwable) {
final StringBuilder buffer = getMessageStringBuilder();
final PrintWriter pw = new PrintWriter(new StringBuilderWriter(buffer), true) {
@Override
public void println() {
flush();
jsonBuilder.append("\t\"");
JsonUtils.quoteAsString(buffer, jsonBuilder);
jsonBuilder.append("\",");
jsonBuilder.append(NEW_LINE);
buffer.setLength(0);
}
};
throwable.printStackTrace(pw);
removeIfEndsWith(jsonBuilder, NEW_LINE);
removeIfEndsWith(jsonBuilder, ",");
}

Some suggestions for configuration options:

  • excludeContaining: using StringBuilder.indexOf(String)
  • excludePattern: slower, but more powerful, using regix
  • maxLines: the max number of lines of the output. Does not chop off a line as maxLength would do.

@michaelmcfadyensky
Copy link
Contributor

I found the easiest way to integrate this is to reuse the existing ThrowableHandlingConverter interface that comes with logback. This allows us to plug in any existing implementation of this interface with out having the re-implement it for this specific encoder. Though the downside is that this solution is specific to the logback encoder.

This commit solved the issue I highlighted in my comment above for the logback encoder. I know this may not be the solution you guys would want to go with but it would be good to get feedback on it.

@felixbarny
Copy link
Member

While this is indeed only a solution that works for logback and no generic one, I think there's great value in being able to re-use existing ThrowableHandlingConverters. This also doesn't restrict us from adding a more generic/logger implementation agnostic way of filtering stack frames in the future.

I'd happily accept a PR adding support for custom ThrowableHandlingConverters.

@rdifrango
Copy link
Contributor

I also would like to see this feature and I wonder if there's a way to tap into the existing capabilities of logback and log4j2 as they already support syntax that is roughly:

ex{depth}
exception{depth}
throwable{depth}

ex{depth, evaluator-1, ..., evaluator-n}
exception{depth, evaluator-1, ..., evaluator-n}
throwable{depth, evaluator-1, ..., evaluator-n}

or

xEx{depth}
xException{depth}
xThrowable{depth}

xEx{depth, evaluator-1, ..., evaluator-n}
xException{depth, evaluator-1, ..., evaluator-n}
xThrowable{depth, evaluator-1, ..., evaluator-n}

See:

https://logback.qos.ch/manual/layouts.html
https://logging.apache.org/log4j/2.x/manual/layouts.html

@HaloFour
Copy link
Contributor Author

HaloFour commented Apr 5, 2022

@rdifrango

I also would like to see this feature and I wonder if there's a way to tap into the existing capabilities of logback and log4j2 as they already support syntax that is roughly:

I've started this PR to allow specifying a pattern to be applied when rendering error.stack_trace that allows you to use one of the built-in converters or to use your own: #177

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
agent-java community Issues and PRs created by the community
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants