Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Clean up OpenTelemetry docs #11882

Merged
merged 8 commits into from
Nov 1, 2023
Merged

docs: Clean up OpenTelemetry docs #11882

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
1cca6cd
reduce content on aws lambda support
colleenmcginnis Oct 17, 2023
8957ca5
update otel links
colleenmcginnis Oct 17, 2023
51287a1
clarify env var example
colleenmcginnis Oct 17, 2023
ddf75de
use lens instead of tsvb
colleenmcginnis Oct 17, 2023
0f20167
remove resource attributes link
colleenmcginnis Oct 17, 2023
79585d7
Merge branch 'main' into obs-docs-issue-3291
colleenmcginnis Oct 26, 2023
2e96a85
apply suggestions from code review
colleenmcginnis Oct 31, 2023
70a8de6
Merge branch 'main' into obs-docs-issue-3291
colleenmcginnis Nov 1, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions docs/otel-direct.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,8 @@ https://github.com/open-telemetry/opentelemetry-java-instrumentation[OpenTelemet
See the https://opentelemetry.io/docs/instrumentation/[OpenTelemetry Instrumentation guides] to download the
OpenTelemetry Agent or SDK for your language.

Define the following environment variables to configure the OpenTelemetry agent and enable communication with Elastic APM.
Define environment variables to configure the OpenTelemetry agent and enable communication with Elastic APM.
For example, if you are instrumenting a Java app, define the following environment variables:

[source,bash]
----
Expand Down Expand Up @@ -114,7 +115,9 @@ For information on how to format an API key, see

Please note the required space between `Bearer` and `an_apm_secret_token`, and `APIKey` and `an_api_key`.

| `OTEL_EXPORTER_OTLP_CERTIFICATE` | The trusted certificate used to verify the TLS credentials of the client. (optional)
| `OTEL_METRICS_EXPORTER` | Metrics exporter to use. See https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/#exporter-selection[exporter selection] for more information.

| `OTEL_LOGS_EXPORTER` | Logs exporter to use. See https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/#exporter-selection[exporter selection] for more information.

|===

Expand Down Expand Up @@ -142,5 +145,4 @@ https://github.com/elastic/apm-server/blob/main/dev_docs/otel.md#muxing-grpc-and
==== Next steps

* <<open-telemetry-collect-metrics>>
* Add <<open-telemetry-resource-attributes>>
* Learn about the <<open-telemetry-known-limitations,limitations of this integration>>
10 changes: 6 additions & 4 deletions docs/otel-limitations.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,10 +33,12 @@ APM Server does not yet support JSON Encoding for OTLP/HTTP.
[[open-telemetry-collector-exporter]]
==== OpenTelemetry Collector exporter for Elastic

The https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/elasticexporter#legacy-opentelemetry-collector-exporter-for-elastic[OpenTelemetry Collector exporter for Elastic]
was deprecated in 7.13 and replaced by the native support of the OpenTelemetry Line Protocol in
Elastic {observability} (OTLP). To learn more, see
https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/elasticexporter#migration[migration].
The https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/v0.57.2/exporter/elasticsearchexporter#elasticsearch-exporter[OpenTelemetry Collector exporter for Elastic]
colleenmcginnis marked this conversation as resolved.
Show resolved Hide resolved
has been deprecated and replaced by the native support of the OpenTelemetry Line Protocol in Elastic Observability (OTLP).
// To learn more, see https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/v0.57.2/exporter/elasticsearchexporter#migration[migration].

The https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/elasticsearchexporter#elasticsearch-exporter[OpenTelemetry Collector exporter for Elastic]
colleenmcginnis marked this conversation as resolved.
Show resolved Hide resolved
(which is different from the legacy exporter mentioned above) is not intended to be used with Elastic APM and Elastic Observability. Use <<open-telemetry-direct>> instead.

[float]
[[open-telemetry-tbs]]
Expand Down
43 changes: 2 additions & 41 deletions docs/otel-metrics.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,45 +46,6 @@ only OpenTelemetry metrics documents.
[[open-telemetry-visualize]]
==== Visualize in {kib}

TSVB within {kib} is the recommended visualization for OpenTelemetry metrics. TSVB is a time series data visualizer that allows you to use the
{es} aggregation framework's full power. With TSVB, you can combine an infinite number of aggregations to display complex data.
Use *Lens* to create visualizations for OpenTelemetry metrics. Lens enables you to build visualizations by dragging and dropping data fields. It makes smart visualization suggestions for your data, allowing you to switch between visualization types.

// lint ignore ecommerce
In this example eCommerce OpenTelemetry dashboard, there are four visualizations: sales, order count, product cache, and system load. The dashboard provides us with business
KPI metrics, along with performance-related metrics.


[role="screenshot"]
image::./images/ecommerce-dashboard.png[OpenTelemetry visualizations]

Let's look at how this dashboard was created, specifically the Sales USD and System load visualizations.

. Open the main menu, then click *Dashboard*.
. Click *Create dashboard*.
. Click *Save*, enter the name of your dashboard, and then click *Save* again.
. Let’s add a Sales USD visualization. Click *Edit*.
. Click *Create new* and then select *TSVB*.
. For the label name, enter Sales USD, and then select the following:
+
* Aggregation: `Counter Rate`.
* Field: `order_sum`.
* Scale: `auto`.
* Group by: `Everything`
. Click *Save*, enter Sales USD as the visualization name, and then click *Save and return*.
. Now let's create a visualization of load averages on the system. Click *Create new*.
. Select *TSVB*.
. Select the following:
+
* Aggregation: `Average`.
* Field: `system.cpu.load_average.1m`.
* Group by: `Terms`.
* By: `host.ip`.
* Top: `10`.
* Order by: `Doc Count (default)`.
* Direction: `Descending`.
. Click *Save*, enter System load per host IP as the visualization name, and then click *Save and return*.
+
Both visualizations are now displayed on your custom dashboard.

IMPORTANT: By default, Discover shows data for the last 15 minutes. If you have a time-based index
and no data displays, you might need to increase the time range.
For more information on using Lens, refer to the {kibana-ref}/lens.html[Lens documentation].
168 changes: 1 addition & 167 deletions docs/otel-other.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,177 +1,11 @@
[[open-telemetry-other-env]]
=== Other execution environments

** <<open-telemetry-aws-lambda>>
** <<open-telemetry-aws-lambda-java-terraform>>
** <<open-telemetry-aws-lambda-nodejs>>
** <<open-telemetry-aws-lambda-nodejs-terraform>>

[float]
[[open-telemetry-aws-lambda]]
=== AWS Lambda Support

[[open-telemetry-aws-lambda]]
AWS Lambda functions can be instrumented with OpenTelemetry and monitored with Elastic {observability}.

To get started, follow the official AWS Distro for OpenTelemetry Lambda https://aws-otel.github.io/docs/getting-started/lambda[getting started documentation] and configure the OpenTelemetry Collector to output traces and metrics to your Elastic cluster.

[float]
[[open-telemetry-aws-lambda-java]]
==== Instrumenting AWS Lambda Java functions

NOTE: For a better startup time, we recommend using SDK-based instrumentation, i.e. manual instrumentation of the code, rather than auto instrumentation.

To instrument AWS Lambda Java functions, follow the official https://aws-otel.github.io/docs/getting-started/lambda/lambda-java[AWS Distro for OpenTelemetry Lambda Support For Java].

Noteworthy configuration elements:

* AWS Lambda Java functions should extend `com.amazonaws.services.lambda.runtime.RequestHandler`,
+
[source,java]
----
public class ExampleRequestHandler implements RequestHandler<APIGatewayProxyRequestEvent, APIGatewayProxyResponseEvent> {
public APIGatewayProxyResponseEvent handleRequest(APIGatewayProxyRequestEvent event, Context context) {
// add your code ...
}
}
----

* When using SDK-based instrumentation, frameworks you want to gain visibility of should be manually instrumented
** The below example instruments https://square.github.io/okhttp/4.x/okhttp/okhttp3/-ok-http-client/[OkHttpClient] with the OpenTelemetry instrument https://search.maven.org/artifact/io.opentelemetry.instrumentation/opentelemetry-okhttp-3.0/1.3.1-alpha/jar[io.opentelemetry.instrumentation:opentelemetry-okhttp-3.0:1.3.1-alpha]
+
[source,java]
----
import io.opentelemetry.instrumentation.okhttp.v3_0.OkHttpTracing;

OkHttpClient httpClient = new OkHttpClient.Builder()
.addInterceptor(OkHttpTracing.create(GlobalOpenTelemetry.get()).newInterceptor())
.build();
----

* The configuration of the OpenTelemetry Collector, with the definition of the Elastic {observability} endpoint, can be added to the root directory of the Lambda binaries (e.g. defined in `src/main/resources/opentelemetry-collector.yaml`)
+
[source,yaml]
----
# Copy opentelemetry-collector.yaml in the root directory of the lambda function
# Set an environment variable 'OPENTELEMETRY_COLLECTOR_CONFIG_FILE' to '/var/task/opentelemetry-collector.yaml'
receivers:
otlp:
protocols:
http:
grpc:

exporters:
logging:
loglevel: debug
otlp/elastic:
# Elastic APM server https endpoint without the "https://" prefix
endpoint: "${ELASTIC_OTLP_ENDPOINT}" <1>
headers:
# Elastic APM Server secret token
Authorization: "Bearer ${ELASTIC_OTLP_TOKEN}" <1>

service:
pipelines:
traces:
receivers: [otlp]
exporters: [logging, otlp/elastic]
metrics:
receivers: [otlp]
exporters: [logging, otlp/elastic]
logs:
receivers: [otlp]
exporters: [logging, otlp/elastic]
----
<1> Environment-specific configuration parameters can be conveniently passed in as environment variables: `ELASTIC_OTLP_ENDPOINT` and `ELASTIC_OTLP_TOKEN`

* Configure the AWS Lambda Java function with:
** https://docs.aws.amazon.com/lambda/latest/dg/API_Layer.html[Function
layer]: The latest https://aws-otel.github.io/docs/getting-started/lambda/lambda-java[AWS
Lambda layer for OpenTelemetry] (e.g. `arn:aws:lambda:eu-west-1:901920570463:layer:aws-otel-java-wrapper-ver-1-2-0:1`)
** https://docs.aws.amazon.com/lambda/latest/dg/API_TracingConfig.html[`TracingConfig` / Mode] set to `PassTrough`
** https://docs.aws.amazon.com/lambda/latest/dg/API_FunctionConfiguration.html[`FunctionConfiguration` / Timeout] set to more than 10 seconds to support the longer cold start inherent to the Lambda Java Runtime
** Export the environment variables:
*** `AWS_LAMBDA_EXEC_WRAPPER="/opt/otel-proxy-handler"` for wrapping handlers proxied through the API Gateway (see https://aws-otel.github.io/docs/getting-started/lambda/lambda-java#enable-auto-instrumentation-for-your-lambda-function[here])
*** `OTEL_PROPAGATORS="tracecontext, baggage"` to override the default setting that also enables X-Ray headers causing interferences between OpenTelemetry and X-Ray
*** `OPENTELEMETRY_COLLECTOR_CONFIG_FILE="/var/task/opentelemetry-collector.yaml"` to specify the path to your OpenTelemetry Collector configuration

[float]
[[open-telemetry-aws-lambda-java-terraform]]
==== Instrumenting AWS Lambda Java functions with Terraform

We recommend using an infrastructure as code solution like Terraform or Ansible to manage the configuration of your AWS Lambda functions.

Here is an example of AWS Lambda Java function managed with Terraform and the https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_function[AWS Provider / Lambda Functions]:

* Sample Terraform code: https://github.com/cyrille-leclerc/my-serverless-shopping-cart/tree/main/checkout-function/deploy
* Note that the Terraform code to manage the HTTP API Gateway (https://github.com/cyrille-leclerc/my-serverless-shopping-cart/tree/main/utils/terraform/api-gateway-proxy[here]) is copied from the official OpenTelemetry Lambda sample https://github.com/open-telemetry/opentelemetry-lambda/tree/e72467a085a2a6e57af133032f85ac5b8bbbb8d1/utils[here]

[float]
[[open-telemetry-aws-lambda-nodejs]]
==== Instrumenting AWS Lambda Node.js functions

NOTE: For a better startup time, we recommend using SDK-based instrumentation for manual instrumentation of the code rather than auto instrumentation.

To instrument AWS Lambda Node.js functions, see https://aws-otel.github.io/docs/getting-started/lambda/lambda-js[AWS Distro for OpenTelemetry Lambda Support For JavaScript].

The configuration of the OpenTelemetry Collector, with the definition of the Elastic {observability} endpoint, can be added to the root directory of the Lambda binaries: `src/main/resources/opentelemetry-collector.yaml`.

[source,yaml]
----
# Copy opentelemetry-collector.yaml in the root directory of the lambda function
# Set an environment variable 'OPENTELEMETRY_COLLECTOR_CONFIG_FILE' to '/var/task/opentelemetry-collector.yaml'
receivers:
otlp:
protocols:
http:
grpc:

exporters:
logging:
loglevel: debug
otlp/elastic:
# Elastic APM server https endpoint without the "https://" prefix
endpoint: "${ELASTIC_OTLP_ENDPOINT}" <1>
headers:
# Elastic APM Server secret token
Authorization: "Bearer ${ELASTIC_OTLP_TOKEN}" <1>

service:
pipelines:
traces:
receivers: [otlp]
exporters: [logging, otlp/elastic]
metrics:
receivers: [otlp]
exporters: [logging, otlp/elastic]
logs:
receivers: [otlp]
exporters: [logging, otlp/elastic]
----
<1> Environment-specific configuration parameters can be conveniently passed in as environment variables: `ELASTIC_OTLP_ENDPOINT` and `ELASTIC_OTLP_TOKEN`

Configure the AWS Lambda Node.js function:

* https://docs.aws.amazon.com/lambda/latest/dg/API_Layer.html[Function
layer]: The latest https://aws-otel.github.io/docs/getting-started/lambda/lambda-js[AWS
Lambda layer for OpenTelemetry]. For example, `arn:aws:lambda:eu-west-1:901920570463:layer:aws-otel-nodejs-ver-0-23-0:1`)
* https://docs.aws.amazon.com/lambda/latest/dg/API_TracingConfig.html[`TracingConfig` / Mode] set to `PassTrough`
* https://docs.aws.amazon.com/lambda/latest/dg/API_FunctionConfiguration.html[`FunctionConfiguration` / Timeout] set to more than 10 seconds to support the cold start of the Lambda JavaScript Runtime
* Export the environment variables:
** `AWS_LAMBDA_EXEC_WRAPPER="/opt/otel-handler"` for wrapping handlers proxied through the API Gateway. See https://aws-otel.github.io/docs/getting-started/lambda/lambda-js#enable-auto-instrumentation-for-your-lambda-function[enable auto instrumentation for your lambda-function].
** `OTEL_PROPAGATORS="tracecontext"` to override the default setting that also enables X-Ray headers causing interferences between OpenTelemetry and X-Ray
** `OPENTELEMETRY_COLLECTOR_CONFIG_FILE="/var/task/opentelemetry-collector.yaml"` to specify the path to your OpenTelemetry Collector configuration.
** `OTEL_TRACES_SAMPLER="AlwaysOn"` define the required sampler strategy if it is not sent from the caller. Note that `Always_on` can potentially create a very large amount of data, so in production set the correct sampling configuration, as per the https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/sdk.md#sampling[specification].

[float]
[[open-telemetry-aws-lambda-nodejs-terraform]]
==== Instrumenting AWS Lambda Node.js functions with Terraform

To manage the configuration of your AWS Lambda functions, we recommend using an infrastructure as code solution like Terraform or Ansible.

Here is an example of AWS Lambda Node.js function managed with Terraform and the https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_function[AWS Provider / Lambda Functions]:

* https://github.com/michaelhyatt/terraform-aws-nodejs-api-worker-otel/tree/v0.24[Sample Terraform code]

[float]
[[open-telemetry-lambda-next]]
==== Next steps
Expand Down