OpenSearch Coordinating Nodes memory exhaustion after upgrade 2.9 to 2.12

[rlevytskyi]

Describe the bug

Hello OpenSearch Team,
We’ve just updated our OpenSearch cluster from version 2.9.0 to 2.12.0.
Among other issues, we’ve noticed that Opensearch coordinating node is now consuming all the heap it is provided with.

Related component

Other

To Reproduce

1. Have an OpenSearch cluster version 2.9
2. Upgrade it to 2.12
3. Provide coordinating node with 18 GB heap instead of 4GB
4. See JVM eats all it get.

Expected behavior

We didn't expect significant memory usage increase after upgrade

Additional Details

Plugins

• security

Screenshots
Please note JVM young is high now

Host/Environment (please complete the following information):

• OS: Oracle Linux 8.9
• OS: Docker image opensearchproject/opensearch-dashboards:2.12.0

[peternied]

[Triage - attendees 1 2 3 4 5]
@rlevytskyi Thank for filing, as there we don't have enough details to act on this issue. Without a root cause its hard to know how this would be addressed. It looks like this is directly related to [1] and we are going to close this issue to funnel discussion in this space to a single location.

• [1] OpenSearch Data Nodes memory exhaustion after upgrade from 2.9 to 2.12 (JDK 21 upgrade) #12454