Add read/write bytes metrics

[ykmr1224]

Description

• Add read/write bytes metrics
  • It uses SparkListener to collect read/write bytes/records for each task, and emit the sum once query completes.
• Added HistoricGauge to emit each data points separately with timestamp.
  • Codahale stores metrics and those are emitted periodically. If application emit metrics multiple times within a period, those datapoints would be summarized or simply ignored other than last value. It is not ideal when we want to track each datapoints (such as when we want to gather metrics for each API call, etc.).
  • HistoricGausge stores each data point with timestamp, and emit each data points to CW using those timestamps.
• Replaced QueryExecutionTimeMetric with HistoricGauge

Sample metrics:

Related Issues

n/a

Check List

• Updated documentation (docs/ppl-lang/README.md)
• Implemented unit tests
• Implemented tests for combination with other commands
• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[seankao-az]

LGTM

[seankao-az]

might be a trivial question..
what could go wrong if use a regular list instead of synchronized list?

[ykmr1224]

As metric emit is run in separate thread, there could be concurrency issue.

[opensearch-trigger-bot]

The backport to 0.5-nexus failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/opensearch-spark/backport-0.5-nexus 0.5-nexus
# Navigate to the new working tree
pushd ../.worktrees/opensearch-spark/backport-0.5-nexus
# Create a new branch
git switch --create backport/backport-803-to-0.5-nexus
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 2a647d43264cdcbb65f95b22c8373dee50497937
# Push it to GitHub
git push --set-upstream origin backport/backport-803-to-0.5-nexus
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/opensearch-spark/backport-0.5-nexus

Then, create a pull request where the base branch is 0.5-nexus and the compare/head branch is backport/backport-803-to-0.5-nexus.