Bundling AWS X-Ray Agent with -javaagent causes JVM metrics to stop being reported

[uziyonasi-nice]

Following the official documentation we noticed the the JVM heap size and max threads count has stopped being recorded after adding the X-Ray agent via JVM arguments.

Below screenshot shows before and after removing the -javaagent parameter from the JAVA_OPTS:

I've included the AWS X-Ray agent using maven POM dependencies after removing the disco jar library from the JVM arguments and it did not interfere with reporting the JVM metrics (it can be seen in the screenshot in the relevant part of the graph starting at 11:35).

I tried to reproduce it locally by configuring the IDE to start the microservice the same way that the ECS task definition is loading the disco agent jar but couldn't see the same issue.
I've also managed to get the metric value by enabling spring actuator and retrieved it from the EC2 instance itself:

But when calling the same metric via the ALB I was getting that the metric was not found.

Please let me know if agent/disco logs are needed and I will try to generate them.

[mxiamxia]

we noticed the the JVM heap size and max threads count has stopped being recorded

How does your service generate these JVM metrics today? Based on your description, it seems like there are possible conflicts that caused JVM metrics failed be generated if you setup X-Ray Disco Agent via JVM parameter?

BTW, X-Ray DiscoAgent is in the deprecation path. If you want to leverage Auto-Instrumentation to send traces to X-Ray, pls use ADOT/OTel SDK. https://aws-otel.github.io/docs/getting-started/java-sdk

[uziyonasi-nice]

How does your service generate these JVM metrics today? Based on your description, it seems like there are possible conflicts that caused JVM metrics failed be generated if you setup X-Ray Disco Agent via JVM parameter?

The metrics are being generated and reported using Spring Boot Actuator and Micrometer.

BTW, X-Ray DiscoAgent is in the deprecation path. If you want to leverage Auto-Instrumentation to send traces to X-Ray, pls use ADOT/OTel SDK. https://aws-otel.github.io/docs/getting-started/java-sdk

I'm aware of the ADOT SDK but since our infrastructure currently doesn't support OTEL we decided to use the Disco agent instead.