Add interactive job metrics

[noCharger]

Description

Metrics included in this PR:

"s3.error.count";
"session.running.count";
"session.failed.count";
"session.success.count";
"session.processingTime";
"statement.running.count";
"statement.failed.count";
"statement.success.count";
"statement.processingTime";

There are two options to calculate session and statement metrics:
Option 1 (taken in this PR): Utilizes direct instrumentation, offering simplicity. However, it lacks resilience against manual modifications to request/result indices by users.
Option 2: Implements asynchronous data retrieval from the request index, ensuring data integrity from the primary source. This introduces additional overhead and dependency, given the external nature of the index.

Issues Resolved

#117

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[penghuo]

Description

Metrics included in this PR:

"s3.error.count";
"session.running.count";
"session.failed.count";
"session.success.count";
"session.processingTime";
"statement.running.count";
"statement.failed.count";
"statement.success.count";
"statement.processingTime";

There are two options to calculate session and statement metrics: Option 1 (taken in this PR): Utilizes direct instrumentation, offering simplicity. However, it lacks resilience against manual modifications to request/result indices by users. Option 2: Implements asynchronous data retrieval from the request index, ensuring data integrity from the primary source. This introduces additional overhead and dependency, given the external nature of the index.

Issues Resolved

#117

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

which option is preferred? option1 and option2?

[noCharger]

Description

Metrics included in this PR:

"s3.error.count";
"session.running.count";
"session.failed.count";
"session.success.count";
"session.processingTime";
"statement.running.count";
"statement.failed.count";
"statement.success.count";
"statement.processingTime";

There are two options to calculate session and statement metrics: Option 1 (taken in this PR): Utilizes direct instrumentation, offering simplicity. However, it lacks resilience against manual modifications to request/result indices by users. Option 2: Implements asynchronous data retrieval from the request index, ensuring data integrity from the primary source. This introduces additional overhead and dependency, given the external nature of the index.

Issues Resolved

#117
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

which option is preferred? option1 and option2?

option1 which taken in this PR

[penghuo]

what are the dimension of these metrics?

[penghuo]

by definition, A counter is a cumulative metric that represents a single monotonically increasing counter whose value can only increase or be reset to zero on restart.

https://prometheus.io/docs/concepts/metric_types/#counter

[noCharger]

by definition, A counter is a cumulative metric that represents a single monotonically increasing counter whose value can only increase or be reset to zero on restart.

https://prometheus.io/docs/concepts/metric_types/#counter

Not sure about Prometheus, decrementing is possible with the Spark metrics library: https://metrics.dropwizard.io/4.2.0/manual/core.html#Counters