[Resource Management] Compress traffic

[kavilla]

Is your feature request related to a problem? Please describe.

OpenSearch Dashboards Server supports compressed traffic. OpenSearch Core APIs support compression. OpenSearch JS Client supports the compression header but OpenSearch Dashboards does not currently hook into the available compression algorithm.

Describe the solution you'd like

Hook into the the clients compression algorithm (currently only gzip) and allow the feature to be configurable at an cluster admin level (opensearch_dashboards.yml config). Ideally the benefit would help request-response time overall with the compression on the client side. It should improve resource management if too much physical limitations of machines prevent successful response (one example being test runners).

Requirements:

• Benchmark before enabling compression
• Implementation
• Benchmark after enabling compression
• Update relevant workflows and CI

Describe alternatives you've considered

n/a

Additional context

Compression on sample data indices can be improvement. Better documentation on node API compression support (saved objects supports compression).

Hook into existing benchmark project can assist with this.

Audit requests and response are correctly compressed and decompressed. For example, concurrent search feature tends to not support the total max hits header

Example of implementation the solution above: https://github.com/opensearch-project/OpenSearch-Dashboards/pull/5223/files

We saw an issue with of reaching the total disk allocation on 2.11 being hit easily and want to be more conscience of hitting this limit. Ideally the more aware of resource needs to run OpenSearch Dashboards, the less resources required by folks who run the application. We will have to think about if the engine supporting the request and generating the responses fail gracefully when compression is not supported.

Do we need to think about how plugins can hook into a framework of compression. For example, if OpenSearch compression is still Lucene based then likely document ID are generated specifically to support compression.