Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metrics framework #926

Merged
merged 20 commits into from
Nov 26, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions common/metrics/config.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
package metrics

// Config provides configuration for a Metrics instance.
type Config struct {
// Namespace is the namespace for the metrics.
Namespace string

// HTTPPort is the port to serve metrics on.
HTTPPort int
}
101 changes: 101 additions & 0 deletions common/metrics/count_metric.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
package metrics

import (
"fmt"
"github.com/Layr-Labs/eigensdk-go/logging"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promauto"
)

var _ CountMetric = &countMetric{}

// countMetric a standard implementation of the CountMetric.
type countMetric struct {
Metric

// logger is the logger used to log errors.
logger logging.Logger

// name is the name of the metric.
name string

// description is the description of the metric.
description string

// counter is the prometheus counter used to report this metric.
vec *prometheus.CounterVec

// labeler is the label maker used to create labels for this metric.
labeler *labelMaker
}

// newCountMetric creates a new CountMetric instance.
func newCountMetric(
logger logging.Logger,
registry *prometheus.Registry,
namespace string,
name string,
description string,
labelTemplate any) (CountMetric, error) {

labeler, err := newLabelMaker(labelTemplate)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we only create labeler when labelTemplate is not nil?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The labeler becomes a no-op when the label template is nil. The purpose of using this pattern was to simplify the business logic a little. Instead of wrapping each use of the labeler in an if statement depending on whether the labeler is enabled or not, we can instead use the labeler in the same way regardless of whether or not we have a non-nil template.

This being said, if you don't like this pattern, let me know and I'll make the suggested change.

if err != nil {
return nil, err
}

vec := promauto.With(registry).NewCounterVec(
prometheus.CounterOpts{
Namespace: namespace,
Name: fmt.Sprintf("%s_count", name),
},
labeler.getKeys(),
)

return &countMetric{
logger: logger,
name: name,
description: description,
vec: vec,
labeler: labeler,
}, nil
}

func (m *countMetric) Name() string {
return m.name
}

func (m *countMetric) Unit() string {
return "count"
}

func (m *countMetric) Description() string {
return m.description
}

func (m *countMetric) Type() string {
return "counter"
}

func (m *countMetric) LabelFields() []string {
return m.labeler.getKeys()
}

func (m *countMetric) Increment(label ...any) {
m.Add(1, label...)
}

func (m *countMetric) Add(value float64, label ...any) {
var l any
if len(label) > 0 {
l = label[0]
}

values, err := m.labeler.extractValues(l)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what if metric has no labels?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The labeler handles this edge case.

  • When l is nil, m.labeler.extractValues(l) returns a list of empty strings with length equal to the number of flags in the template.
  • If the template is nil, m.labeler.extractValues(l) returns an empty list.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. Does m.vec.WithLabelValues(values...) handle empty values gracefully?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

m.vec.WithLabelValues() requires that the number of provided values be exactly equal to the number of registered keys. It's ok if a value is an empty string, but the number of strings must match.

if err != nil {
m.logger.Errorf("error extracting values from label for metric %s: %v", m.name, err)
return
}

observer := m.vec.WithLabelValues(values...)
observer.Add(value)
}
103 changes: 103 additions & 0 deletions common/metrics/gauge_metric.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
package metrics

import (
"fmt"
"github.com/Layr-Labs/eigensdk-go/logging"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promauto"
)

var _ GaugeMetric = &gaugeMetric{}

// gaugeMetric is a standard implementation of the GaugeMetric interface via prometheus.
type gaugeMetric struct {
Metric

// logger is the logger used to log errors.
logger logging.Logger

// name is the name of the metric.
name string

// unit is the unit of the metric.
unit string

// description is the description of the metric.
description string

// gauge is the prometheus gauge used to report this metric.
vec *prometheus.GaugeVec

// labeler is the label maker used to create labels for this metric.
labeler *labelMaker
}

// newGaugeMetric creates a new GaugeMetric instance.
func newGaugeMetric(
logger logging.Logger,
registry *prometheus.Registry,
namespace string,
name string,
unit string,
description string,
labelTemplate any) (GaugeMetric, error) {

labeler, err := newLabelMaker(labelTemplate)
if err != nil {
return nil, err
}

vec := promauto.With(registry).NewGaugeVec(
prometheus.GaugeOpts{
Namespace: namespace,
Name: fmt.Sprintf("%s_%s", name, unit),
},
labeler.getKeys(),
)

return &gaugeMetric{
logger: logger,
name: name,
unit: unit,
description: description,
vec: vec,
labeler: labeler,
}, nil
}

func (m *gaugeMetric) Name() string {
return m.name
}

func (m *gaugeMetric) Unit() string {
return m.unit
}

func (m *gaugeMetric) Description() string {
return m.description
}

func (m *gaugeMetric) Type() string {
return "gauge"
}

func (m *gaugeMetric) LabelFields() []string {
return m.labeler.getKeys()
}

func (m *gaugeMetric) Set(value float64, label ...any) {
var l any
if len(label) > 0 {
l = label[0]
}

values, err := m.labeler.extractValues(l)
if err != nil {
m.logger.Errorf("failed to extract values from label: %v", err)
return
}

observer := m.vec.WithLabelValues(values...)

observer.Set(value)
}
73 changes: 73 additions & 0 deletions common/metrics/label_maker.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
package metrics

import (
"fmt"
"reflect"
)

// labelMaker encapsulates logic for creating labels for metrics.
type labelMaker struct {
keys []string
emptyValues []string
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how is emptyValues used?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a label is set up with a non-null template, but no labels are provided at runtime, then this emptyValues list is passed go prometheus. Prometheus returns an error if you don't pass in the expected number of flags.

func (l *labelMaker) extractValues(label any) ([]string, error) {
	// ...

	if label == nil {
		return l.emptyValues, nil
	}

We could create a new empty list each time, but I thought it would be more resource efficient to just reuse the same empty list over and over.

templateType reflect.Type
labelCount int
}

// newLabelMaker creates a new labelMaker instance given a label template. The label template may be nil.
func newLabelMaker(labelTemplate any) (*labelMaker, error) {
labeler := &labelMaker{
keys: make([]string, 0),
}

if labelTemplate == nil {
return labeler, nil
}

v := reflect.ValueOf(labelTemplate)
if v.Kind() != reflect.Struct {
return nil, fmt.Errorf("label template must be a struct")
}

t := v.Type()
labeler.templateType = t
for i := 0; i < t.NumField(); i++ {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This panics if labelTemplate is not a struct
We should check if v.Kind() == reflect.Struct

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea, done.

if v.Kind() != reflect.Struct {
    return nil, fmt.Errorf("label template must be a struct")
}

As an aside, add the reflection library to the list of things that make me annoyed at the people who designed golang. One of the core principals is that things should return errors, not panic. This is exactly the sort of situation where returning errors would be way better than panicking.


fieldType := t.Field(i).Type
if fieldType.Kind() != reflect.String {
return nil, fmt.Errorf(
"field %s has type %v, only string fields are supported", t.Field(i).Name, fieldType)
}

labeler.keys = append(labeler.keys, t.Field(i).Name)
}

labeler.emptyValues = make([]string, len(labeler.keys))
labeler.labelCount = len(labeler.keys)

return labeler, nil
}

// getKeys provides the keys for the label struct.
func (l *labelMaker) getKeys() []string {
return l.keys
}

// extractValues extracts the values from the given label struct.
func (l *labelMaker) extractValues(label any) ([]string, error) {
if l.templateType == nil || label == nil {
return l.emptyValues, nil
}

if l.templateType != reflect.TypeOf(label) {
return nil, fmt.Errorf(
"label type mismatch, expected %v, got %v", l.templateType, reflect.TypeOf(label))
}

values := make([]string, 0, l.labelCount)
for i := 0; i < l.labelCount; i++ {
v := reflect.ValueOf(label)
values = append(values, v.Field(i).String())
}

return values, nil
}
102 changes: 102 additions & 0 deletions common/metrics/latency_metric.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
package metrics

import (
"fmt"
"github.com/Layr-Labs/eigensdk-go/logging"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promauto"
"time"
)

var _ LatencyMetric = &latencyMetric{}

// latencyMetric is a standard implementation of the LatencyMetric interface via prometheus.
type latencyMetric struct {
Metric

// logger is the logger used to log errors.
logger logging.Logger

// name is the name of the metric.
name string

// description is the description of the metric.
description string

// vec is the prometheus summary vector used to report this metric.
vec *prometheus.SummaryVec

// lm is the label maker used to create labels for this metric.
labeler *labelMaker
}

// newLatencyMetric creates a new LatencyMetric instance.
func newLatencyMetric(
logger logging.Logger,
registry *prometheus.Registry,
namespace string,
name string,
description string,
objectives map[float64]float64,
labelTemplate any) (LatencyMetric, error) {

labeler, err := newLabelMaker(labelTemplate)
if err != nil {
return nil, err
}

vec := promauto.With(registry).NewSummaryVec(
prometheus.SummaryOpts{
Namespace: namespace,
Name: fmt.Sprintf("%s_ms", name),
Objectives: objectives,
},
labeler.getKeys(),
)

return &latencyMetric{
logger: logger,
name: name,
description: description,
vec: vec,
labeler: labeler,
}, nil
}

func (m *latencyMetric) Name() string {
return m.name
}

func (m *latencyMetric) Unit() string {
return "ms"
}

func (m *latencyMetric) Description() string {
return m.description
}

func (m *latencyMetric) Type() string {
return "latency"
}

func (m *latencyMetric) LabelFields() []string {
return m.labeler.getKeys()
}

func (m *latencyMetric) ReportLatency(latency time.Duration, label ...any) {
var l any
if len(label) > 0 {
l = label[0]
}

values, err := m.labeler.extractValues(l)
if err != nil {
m.logger.Errorf("error extracting values from label: %v", err)
}

observer := m.vec.WithLabelValues(values...)

nanoseconds := float64(latency.Nanoseconds())
milliseconds := nanoseconds / float64(time.Millisecond)
observer.Observe(milliseconds)
}
Loading
Loading