[Infra][ECO] Show a callout to prompt users to ingest metrics data

[crespocarlos]

Summary

We want to display a callout to prompt users to ingest metrics data.

The API created in #193701 will provide the information needed to make the decision on when to show the callout.

The callout will be displayed on the following tabs

Hosts

• Overview
• Metrics
• Processes

Containers

• Overview
• Metrics

We might want to consider making the ServiceTabEmptyState component in the APM plugin callout code a shared component.

AC

• The callout is displayed when there is no metrics data detected for a given entity
• The callout is displayed below the date picker
• "Add metrics" points to (see example here):
  • Hosts: /app/observabilityOnboarding/?category=logs using ObservabilityOnboardingLocatorParams
  • Containers: /app/observabilityOnboarding/?category=infra using ObservabilityOnboardingLocatorParams
• "Try it now in our demo cluster" is displayed only for hosts and points to Hosts View (we need a shortened URL pointing to it)
• On the Overview tab, the callout is dismissible
• On the Overview tab, the KPI section is hidden in favour of the callout
• Custom telemetry events will be added
• For containers that aren’t Docker or Kubernetes, the callout will not be shown on any tab

Copy for promo

Hosts

• Overview

View core metrics to understand your host performance
Collect metrics such as CPU and memory usage to identify performance bottlenecks that could be affecting your users.

Add Metrics (point to metrics onboarding)
Try it now in our demo cluster
Learn More

• Metrics

View core metrics to understand your host performance
Collect metrics such as CPU and memory usage to identify performance bottlenecks that could be affecting your users.

Add Metrics (point to metrics onboarding)
Try it now in our demo cluster
Learn More

• Processes

View host processes to identify performance bottlenecks
Collect process data to understand what is consuming resource on your hosts.

Add Metrics (point to metrics onboarding)
Try it now in our demo cluster
Learn More

Containers

• Overview

View core metrics to understand your container performance
Collect metrics such as CPU and memory usage to identify performance bottlenecks that could be affecting your users.

Add Metrics (point to metrics onboarding)
[NO DEMO CLUSTER LINK]
Learn More

• Metrics

View core metrics to understand your container performance
Collect metrics such as CPU and memory usage to identify performance bottlenecks that could be affecting your users.

Add Metrics (point to metrics onboarding)
[NO DEMO CLUSTER LINK]
Learn More

Custom events

• Add Metrics Callout Add Metrics Clicked
• Add Metrics Callout Try It Clicked
• Add Metrics Callout Learn More Clicked
• Add Metrics Callout Dismissed

The properties.view will be "add_metrics_cta" for all of them.

[elasticmachine]

Pinging @elastic/obs-ux-infra_services-team (Team:obs-ux-infra_services)

[crespocarlos]

@roshan-elastic we'll need the copy and the URL that the "Learn More" link will point to before moving this into Ready. The suggestion is to use the same design from APM's callout, but It'd be nice to review it (APM's Overview callout shows a screenshot, but I don't know if we need the same for infra).

We also need to evaluate how to leverage the demo cluster for this. For hosts we could point to the hosts view, but what should we do for containers? Point to Infra's inventory? If we keep the "Try it now in the demo cluster" button we'll need short links.

[roshan-elastic]

@roshan-elastic we'll need the copy and the URL that the "Learn More" link will point to before moving this into Ready. The suggestion is to use the same design from APM's callout, but It'd be nice to review it (APM's Overview callout shows a screenshot, but I don't know if we need the same for infra).

Sounds good - I'll put it on my list.

We also need to evaluate how to leverage the demo cluster for this. For hosts we could point to the hosts view, but what should we do for containers? Point to Infra's inventory? If we keep the "Try it now in the demo cluster" button we'll need short links.

Would it be possible to just not show the 'try it in our demo cluster' for containers? I think we'd have to point to the old inventory filtered by containers and I think that experience just isn't worth showing about.

@crespocarlos

[crespocarlos]

Hey @roshan-elastic

Would it be possible to just not show the 'try it in our demo cluster' for containers? I think we'd have to point to the old inventory filtered by containers and I think that experience just isn't worth showing about.

This is good idea. Let's do that.

[roshan-elastic]

Cool man - I'll update this issue with the copy/behaviour/links for each one as per the AC tomorrow (hopefully AM)

[roshan-elastic]

Hey @crespocarlos - updated the copy in the description at the top.

[iblancof]

I see that this task is marked as Ready and @crespocarlos is assigned since he was refining it.

I’ll assign it to myself to start working on it as he is currently on PTO.

[cauemarcondes]

@iblancof you'll need what I've done on this PR #194612

[iblancof]

@iblancof you'll need what I've done on this PR #194612

Thanks for sharing @cauemarcondes!

I planned to begin with everything unrelated to the data streams, so I wouldn't be blocked.

[iblancof]

Hey @roshan-elastic,
could you clarify what specific telemetry events and data we need for this?

Here's what we did in Services for reference:

[roshan-elastic]

Hey @iblancof,

Sure - here you go:

Add Metrics

{
	"timestamp": "2024-07-11T10:12:40.495Z",
	"event_type": "entity_inventory_add_data"
	"context": {
             ...
	},
	"properties": {
		"view": "add_metrics_cta",
	}
}

Try it now

{
	"timestamp": "2024-07-11T10:12:40.495Z",
	"event_type": "try it" // this is how we'll denote 'trying'
	"context": {
            ...
	},
	"properties": {
		"view": "add_metrics_cta"
	}
}

Learn More

{
	"timestamp": "2024-07-11T10:12:40.495Z",
	"event_type": "learn_more_click"
	"context": {
            ...
	},
	"properties": {
		"view": "add_metrics_cta"
	}
}

Dismiss

{
	"timestamp": "2024-07-11T10:12:40.495Z",
	"event_type": "dismiss"
	"context": {
            ...
	},
	"properties": {
		"view": "add_metrics_cta"
	}
}

[crespocarlos]

Just linking a doc with best practices on how to name custom events.

Based on that, It might be better to create them with more meaningful names, eg:

• entity_inventory_add_data -> Hosts Add Data Prompt Add Data Click
• dismiss -> Hosts Add Data Prompt Dismiss Click
• try it -> Hosts Add Data Prompt Try It Click

[roshan-elastic]

Nice @crespocarlos - I'm happy to let engineers pick the names using best practice. I think it'll be faster in the long run (I'll be a bottleneck).

[crespocarlos]

100% @roshan-elastic . I shared that more for engineers. I'll socialize this with the team.

[iblancof]

Just linking a doc with best practices on how to name custom events.

Thanks a lot for sharing this, @crespocarlos!
Yesterday, I added a point to the agenda to discuss this in our weekly because I felt like we weren’t following clear guidelines. Now that I know there's documentation, I'll remove the point I added, but it might be a good idea for you to share it with the team.

I’ll think about naming the events according to the guidelines and will add it to the issue description.

[iblancof]

Following @crespocarlos suggestions and the best practices the event names will be:

• Add Metrics Callout Add Metrics Clicked
• Add Metrics Callout Try It Clicked
• Add Metrics Callout Learn More Clicked
• Add Metrics Callout Dismissed

I will keep the properties.view: "add_metrics_cta" for all of them as @roshan-elastic suggested.

---

I'll add this, but it’s not meant to change what we’re doing in this task. It's more of a comment for future consideration.

I have the feeling that we don't have a clear idea of what should be included in the event name and what should be extracted from the properties attributes.

For example, should the location from which the event is triggered be included in the name or as a property? Should we reuse some events and filter them based on properties? Should we push for making automatic events like "click" more queryable to avoid the need for custom events?

I think it would be helpful to understand how this data is being used so we can format events consistently, making it easier for everyone.

[crespocarlos]

hey @iblancof

I have the feeling that we don't have a clear idea of what should be included in the event name and what should be extracted from the properties attributes.

I also find this particular case confusing

For example, should the location from which the event is triggered be included in the name or as a property?

IIRC the location goes in the context attribute

Should we reuse some events and filter them based on properties?

Not sure what you meant here.

Should we push for making automatic events like "click" more queryable to avoid the need for custom events?

That'd be preferable. After looking at the events, they could perhaps be automatic events instead. We could add data attributes to enrich the click events. eg: data-component-name="infra_hosts_view_add_metrics_cta" data-button="dismiss|try_it|add_data"

I don't know if it would make it harder for @roshan-elastic to get the data, though.

For reference, we already use data-component-name in the asset details.

[iblancof]

I don't think we should make a quick decision right now. This needs a good discussion (and I don’t believe it’s the best idea to start this in this issue thread) to define a clear strategy, at least for use within our areas of ownership. This way, we can ensure some consistency and facilitate the consumption of this data as much as possible.

Regarding what we’ll do for this issue, I’ll stick with what we proposed here and move forward with it to avoid blocking the issue.

[crespocarlos]

100% Agree. Again, I'm not proposing quick decisions for strategies around telemetry, just sharing what's possible for automatic collection. Good that we have the doc, now we need to add more details to it. eg: when to use custom vs automatic collection.

[roshan-elastic]

That'd be preferable. After looking at the events, they could perhaps be automatic events instead. We could add data attributes to enrich the click events. eg: data-component-name="infra_hosts_view_add_metrics_cta" data-button="dismiss|try_it|add_data"

I don't know if it would make it harder for @roshan-elastic to get the data, though.

The problem with passing in attributes is that is that whilst it does provide some structure to the data - querying against it in ES is insanely slow/painful (because they all get lumped into a field called properties.target which has insane cardinality).

Here's an example of the content in the properties.target field - it's the entire DOM (each delimited value is treated as a unique 'value' so the cardinality is ridiculous and unworkably slow almost all of the time):

sample telemetry event

Where there are explicit telemetry requirements, I'd much prefer to emit them as standard events that have specific fields.

This is what is nice about the event_type field, the properties object and the context object:

• event_type makes it very easy to isolate the events
• properties makes it very easy to slice and dice the event by the properties we care about
• context make it very easy to associate common data with the event (e.g. licence type etc)

I think ideally, the properties or context objects would be populated with the key attributes like data-component-name="infra_hosts_view_add_metrics_cta". For example:

{
 "event_type" : "my event",
 "properties" : {
     "component-name":"infra_hosts_view_add_metrics",
    "view":"hosts view",
    "view_type":"EEM",
    "entity_signals":["metrics","logs"]
    ...
 ,
 "context": {
    "licence_type":"enterprise",
    ...
 }

The above is just an example to think about in the future this would be very easy to work with in the data.

For this issue

Could we try and use the same approach we used for the APM click throughs and separate out finding an efficient way to declare the data in a way which is easy to declare + leverage?

[iblancof]

For this issue

Could we try and use the same approach we used for the APM click throughs and separate out finding an efficient way to declare the data in a way which is easy to declare + leverage?

Yeah sure, my last proposal was to have the custom events I shared in this comment:

Following @crespocarlos suggestions and the best practices the event names will be:

• Add Metrics Callout Add Metrics Clicked
• Add Metrics Callout Try It Clicked
• Add Metrics Callout Learn More Clicked
• Add Metrics Callout Dismissed

I will keep the properties.view: "add_metrics_cta" for all of them as @roshan-elastic suggested.

Does that work for you, @roshan-elastic? We're using event_type as the main identifier for the event being triggered, and for page-specific details, we have the URL info in the context, just like we did for APM.

[roshan-elastic]

Sounds good @iblancof !

[iblancof]

Hey @roshan-elastic,

I noticed that we have a check in the code to hide the container metrics tab content when the container is neither Docker nor Kubernetes.

Hidden	Normal

Does this mean we also don't want to show the callout for non-Docker/Kubernetes containers in either the Overview or Metrics tabs?

cc: @crespocarlos

[roshan-elastic]

Hey @iblancof - I would go for whatever is simplest and most consistent for now.

As a user, I'd expect this to be hidden if adding metrics won't show me metrics. I think the impact of this will be relatively low either way so I think it would be good to do whatever is simplest (knowing we'll look at this kind of thing more holistically when we evolve the entity view template).