Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[core.metrics] Add support for multiple processes in ops metrics & stats API; deprecate process field #107900

Closed
wants to merge 14 commits into from

Conversation

Bamieh
Copy link
Member

@Bamieh Bamieh commented Aug 9, 2021

WIP. Remaining items until ready:

  • Deprecate process field
  • Add name field for each process. So far we only have main_thread.
  • Move event loop delay histogram to core
  • Telemetry collectors to use core's histogram
  • Core.metrics use histogram.mean for event_loop_delay.
  • Core.metrics expose histogram to be used for stats and status api.
  • Update unit tests
  • Add integration tests

Summary

/api/status

{
  "processes": 
     [
	    {
	      "name": "main_thread",
	      "memory": {
	        "heap": {
	          "total_in_bytes": 387874816,
	          "used_in_bytes": 332095280,
	          "size_limit": 12935233536
	        },
	        "resident_set_size_in_bytes": 540487680
	      },
	      "pid": 29897,
	      "event_loop_delay": 11417136.036613273,
	      "event_loop_delay_histogram": {
	        "min": 9469952,
	        "max": 19021823,
	        "mean": 11417136.036613273,
	        "exceeds": 0,
	        "stddev": 1132985.4515301601,
	        "fromTimestamp": "2021-08-09T12:24:51.190Z",
	        "lastUpdatedAt": "2021-08-09T12:24:56.185Z",
	        "percentiles": {
	          "50": 11247615,
	          "75": 12599295,
	          "95": 12656639,
	          "99": 14434303
	        }
	      },
	      "uptime_in_millis": 101736.272458
	    }
	  ],
  "process": {
      "name": "main_thread",
      "memory": {
        "heap": {
          "total_in_bytes": 387874816,
          "used_in_bytes": 332095280,
          "size_limit": 12935233536
        },
        "resident_set_size_in_bytes": 540487680
      },
      "pid": 29897,
      "event_loop_delay": 11417136.036613273,
      "event_loop_delay_histogram": {
        "min": 9469952,
        "max": 19021823,
        "mean": 11417136.036613273,
        "exceeds": 0,
        "stddev": 1132985.4515301601,
        "fromTimestamp": "2021-08-09T12:24:51.190Z",
        "lastUpdatedAt": "2021-08-09T12:24:56.185Z",
        "percentiles": {
          "50": 11247615,
          "75": 12599295,
          "95": 12656639,
          "99": 14434303
        }
      },
      "uptime_in_millis": 101736.272458
    }
}

Closes #104031

@Bamieh Bamieh added v8.0.0 release_note:skip Skip the PR/issue when compiling release notes auto-backport Deprecated - use backport:version if exact versions are needed v7.16.0 labels Aug 9, 2021
Copy link
Contributor

@joshdover joshdover left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

General direction here looking good!

src/core/server/metrics/collectors/types.ts Outdated Show resolved Hide resolved
src/core/server/metrics/collectors/process.ts Outdated Show resolved Hide resolved
Comment on lines 40 to 41
process: mainProcess,
processes,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kibanamachine
Copy link
Contributor

kibanamachine commented Aug 24, 2021

💔 Build Failed

Failed CI Steps

Metrics [docs]

Unknown metric groups

API count

id before after diff
core 2246 2263 +17

API count missing comments

id before after diff
core 997 1011 +14

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@Bamieh
Copy link
Member Author

Bamieh commented Aug 24, 2021

Closing in favor of #109820

@Bamieh Bamieh closed this Aug 24, 2021
@Bamieh Bamieh deleted the core/process_ops_multi_process branch August 24, 2021 11:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto-backport Deprecated - use backport:version if exact versions are needed release_note:skip Skip the PR/issue when compiling release notes v7.16.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[core.metrics] Add support for multiple processes in ops metrics & stats API; deprecate process field
3 participants