Skip to content

Commit

Permalink
docs: Clean up OpenTelemetry docs (#11882)
Browse files Browse the repository at this point in the history 
* reduce content on aws lambda support

* update otel links

* clarify env var example

* use lens instead of tsvb

* remove resource attributes link

* apply suggestions from code review

Co-authored-by: Alexander Wert <[email protected]>

---------

Co-authored-by: Alexander Wert <[email protected]>
(cherry picked from commit 691b526)
  • Loading branch information
@colleenmcginnis @mergify
colleenmcginnis authored and mergify[bot] committed Nov 1, 2023
1 parent 10b3c43 commit 73d8e69
Show file tree
Hide file tree
Showing 4 changed files with 14 additions and 215 deletions.
8 changes: 5 additions & 3 deletions docs/otel-direct.asciidoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,8 @@ https://github.com/open-telemetry/opentelemetry-java-instrumentation[OpenTelemet
See the https://opentelemetry.io/docs/instrumentation/[OpenTelemetry Instrumentation guides] to download the
OpenTelemetry Agent or SDK for your language.

Define the following environment variables to configure the OpenTelemetry agent and enable communication with Elastic APM.
Define environment variables to configure the OpenTelemetry agent and enable communication with Elastic APM.
For example, if you are instrumenting a Java app, define the following environment variables:

[source,bash]
----
Expand Down Expand Up @@ -114,7 +115,9 @@ For information on how to format an API key, see

Please note the required space between `Bearer` and `an_apm_secret_token`, and `APIKey` and `an_api_key`.

| `OTEL_EXPORTER_OTLP_CERTIFICATE` | The trusted certificate used to verify the TLS credentials of the client. (optional)
| `OTEL_METRICS_EXPORTER` | Metrics exporter to use. See https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/#exporter-selection[exporter selection] for more information.

| `OTEL_LOGS_EXPORTER` | Logs exporter to use. See https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/#exporter-selection[exporter selection] for more information.

|===

Expand Down Expand Up @@ -142,5 +145,4 @@ https://github.com/elastic/apm-server/blob/main/dev_docs/otel.md#muxing-grpc-and
==== Next steps

* <<open-telemetry-collect-metrics>>
* Add <<open-telemetry-resource-attributes>>
* Learn about the <<open-telemetry-known-limitations,limitations of this integration>>
10 changes: 6 additions & 4 deletions docs/otel-limitations.asciidoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -33,10 +33,12 @@ APM Server does not yet support JSON Encoding for OTLP/HTTP.
[[open-telemetry-collector-exporter]]
==== OpenTelemetry Collector exporter for Elastic

The https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/elasticexporter#legacy-opentelemetry-collector-exporter-for-elastic[OpenTelemetry Collector exporter for Elastic]
was deprecated in 7.13 and replaced by the native support of the OpenTelemetry Line Protocol in
Elastic {observability} (OTLP). To learn more, see
https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/elasticexporter#migration[migration].
The https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/v0.57.2/exporter/elasticexporter[OpenTelemetry Collector exporter for Elastic]
has been deprecated and replaced by the native support of the OpenTelemetry Line Protocol in Elastic Observability (OTLP).
// To learn more, see https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/v0.57.2/exporter/elasticsearchexporter#migration[migration].

The https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/elasticsearchexporter#elasticsearch-exporter[Elasticsearch exporter for the OpenTelemetry Collector]
(which is different from the legacy exporter mentioned above) is not intended to be used with Elastic APM and Elastic Observability. Use <<open-telemetry-direct>> instead.

[float]
[[open-telemetry-tbs]]
Expand Down
43 changes: 2 additions & 41 deletions docs/otel-metrics.asciidoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -46,45 +46,6 @@ only OpenTelemetry metrics documents.
[[open-telemetry-visualize]]
==== Visualize in {kib}

TSVB within {kib} is the recommended visualization for OpenTelemetry metrics. TSVB is a time series data visualizer that allows you to use the
{es} aggregation framework's full power. With TSVB, you can combine an infinite number of aggregations to display complex data.
Use *Lens* to create visualizations for OpenTelemetry metrics. Lens enables you to build visualizations by dragging and dropping data fields. It makes smart visualization suggestions for your data, allowing you to switch between visualization types.

// lint ignore ecommerce
In this example eCommerce OpenTelemetry dashboard, there are four visualizations: sales, order count, product cache, and system load. The dashboard provides us with business
KPI metrics, along with performance-related metrics.


[role="screenshot"]
image::./images/ecommerce-dashboard.png[OpenTelemetry visualizations]

Let's look at how this dashboard was created, specifically the Sales USD and System load visualizations.

. Open the main menu, then click *Dashboard*.
. Click *Create dashboard*.
. Click *Save*, enter the name of your dashboard, and then click *Save* again.
. Let’s add a Sales USD visualization. Click *Edit*.
. Click *Create new* and then select *TSVB*.
. For the label name, enter Sales USD, and then select the following:
+
* Aggregation: `Counter Rate`.
* Field: `order_sum`.
* Scale: `auto`.
* Group by: `Everything`
. Click *Save*, enter Sales USD as the visualization name, and then click *Save and return*.
. Now let's create a visualization of load averages on the system. Click *Create new*.
. Select *TSVB*.
. Select the following:
+
* Aggregation: `Average`.
* Field: `system.cpu.load_average.1m`.
* Group by: `Terms`.
* By: `host.ip`.
* Top: `10`.
* Order by: `Doc Count (default)`.
* Direction: `Descending`.
. Click *Save*, enter System load per host IP as the visualization name, and then click *Save and return*.
+
Both visualizations are now displayed on your custom dashboard.

IMPORTANT: By default, Discover shows data for the last 15 minutes. If you have a time-based index
and no data displays, you might need to increase the time range.
For more information on using Lens, refer to the {kibana-ref}/lens.html[Lens documentation].
168 changes: 1 addition & 167 deletions docs/otel-other.asciidoc
View file
Original file line number Diff line number Diff line change
@@ -1,177 +1,11 @@
[[open-telemetry-other-env]]
=== Other execution environments

** <<open-telemetry-aws-lambda>>
** <<open-telemetry-aws-lambda-java-terraform>>
** <<open-telemetry-aws-lambda-nodejs>>
** <<open-telemetry-aws-lambda-nodejs-terraform>>

[float]
[[open-telemetry-aws-lambda]]
=== AWS Lambda Support

[[open-telemetry-aws-lambda]]
AWS Lambda functions can be instrumented with OpenTelemetry and monitored with Elastic {observability}.

To get started, follow the official AWS Distro for OpenTelemetry Lambda https://aws-otel.github.io/docs/getting-started/lambda[getting started documentation] and configure the OpenTelemetry Collector to output traces and metrics to your Elastic cluster.

[float]
[[open-telemetry-aws-lambda-java]]
==== Instrumenting AWS Lambda Java functions

NOTE: For a better startup time, we recommend using SDK-based instrumentation, i.e. manual instrumentation of the code, rather than auto instrumentation.

To instrument AWS Lambda Java functions, follow the official https://aws-otel.github.io/docs/getting-started/lambda/lambda-java[AWS Distro for OpenTelemetry Lambda Support For Java].

Noteworthy configuration elements:

* AWS Lambda Java functions should extend `com.amazonaws.services.lambda.runtime.RequestHandler`,
+
[source,java]
----
public class ExampleRequestHandler implements RequestHandler<APIGatewayProxyRequestEvent, APIGatewayProxyResponseEvent> {
public APIGatewayProxyResponseEvent handleRequest(APIGatewayProxyRequestEvent event, Context context) {
// add your code ...
}
}
----

* When using SDK-based instrumentation, frameworks you want to gain visibility of should be manually instrumented
** The below example instruments https://square.github.io/okhttp/4.x/okhttp/okhttp3/-ok-http-client/[OkHttpClient] with the OpenTelemetry instrument https://search.maven.org/artifact/io.opentelemetry.instrumentation/opentelemetry-okhttp-3.0/1.3.1-alpha/jar[io.opentelemetry.instrumentation:opentelemetry-okhttp-3.0:1.3.1-alpha]
+
[source,java]
----
import io.opentelemetry.instrumentation.okhttp.v3_0.OkHttpTracing;
OkHttpClient httpClient = new OkHttpClient.Builder()
.addInterceptor(OkHttpTracing.create(GlobalOpenTelemetry.get()).newInterceptor())
.build();
----

* The configuration of the OpenTelemetry Collector, with the definition of the Elastic {observability} endpoint, can be added to the root directory of the Lambda binaries (e.g. defined in `src/main/resources/opentelemetry-collector.yaml`)
+
[source,yaml]
----
# Copy opentelemetry-collector.yaml in the root directory of the lambda function
# Set an environment variable 'OPENTELEMETRY_COLLECTOR_CONFIG_FILE' to '/var/task/opentelemetry-collector.yaml'
receivers:
otlp:
protocols:
http:
grpc:
exporters:
logging:
loglevel: debug
otlp/elastic:
# Elastic APM server https endpoint without the "https://" prefix
endpoint: "${ELASTIC_OTLP_ENDPOINT}" <1>
headers:
# Elastic APM Server secret token
Authorization: "Bearer ${ELASTIC_OTLP_TOKEN}" <1>
service:
pipelines:
traces:
receivers: [otlp]
exporters: [logging, otlp/elastic]
metrics:
receivers: [otlp]
exporters: [logging, otlp/elastic]
logs:
receivers: [otlp]
exporters: [logging, otlp/elastic]
----
<1> Environment-specific configuration parameters can be conveniently passed in as environment variables: `ELASTIC_OTLP_ENDPOINT` and `ELASTIC_OTLP_TOKEN`

* Configure the AWS Lambda Java function with:
** https://docs.aws.amazon.com/lambda/latest/dg/API_Layer.html[Function
layer]: The latest https://aws-otel.github.io/docs/getting-started/lambda/lambda-java[AWS
Lambda layer for OpenTelemetry] (e.g. `arn:aws:lambda:eu-west-1:901920570463:layer:aws-otel-java-wrapper-ver-1-2-0:1`)
** https://docs.aws.amazon.com/lambda/latest/dg/API_TracingConfig.html[`TracingConfig` / Mode] set to `PassTrough`
** https://docs.aws.amazon.com/lambda/latest/dg/API_FunctionConfiguration.html[`FunctionConfiguration` / Timeout] set to more than 10 seconds to support the longer cold start inherent to the Lambda Java Runtime
** Export the environment variables:
*** `AWS_LAMBDA_EXEC_WRAPPER="/opt/otel-proxy-handler"` for wrapping handlers proxied through the API Gateway (see https://aws-otel.github.io/docs/getting-started/lambda/lambda-java#enable-auto-instrumentation-for-your-lambda-function[here])
*** `OTEL_PROPAGATORS="tracecontext, baggage"` to override the default setting that also enables X-Ray headers causing interferences between OpenTelemetry and X-Ray
*** `OPENTELEMETRY_COLLECTOR_CONFIG_FILE="/var/task/opentelemetry-collector.yaml"` to specify the path to your OpenTelemetry Collector configuration

[float]
[[open-telemetry-aws-lambda-java-terraform]]
==== Instrumenting AWS Lambda Java functions with Terraform

We recommend using an infrastructure as code solution like Terraform or Ansible to manage the configuration of your AWS Lambda functions.

Here is an example of AWS Lambda Java function managed with Terraform and the https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_function[AWS Provider / Lambda Functions]:

* Sample Terraform code: https://github.com/cyrille-leclerc/my-serverless-shopping-cart/tree/main/checkout-function/deploy
* Note that the Terraform code to manage the HTTP API Gateway (https://github.com/cyrille-leclerc/my-serverless-shopping-cart/tree/main/utils/terraform/api-gateway-proxy[here]) is copied from the official OpenTelemetry Lambda sample https://github.com/open-telemetry/opentelemetry-lambda/tree/e72467a085a2a6e57af133032f85ac5b8bbbb8d1/utils[here]

[float]
[[open-telemetry-aws-lambda-nodejs]]
==== Instrumenting AWS Lambda Node.js functions

NOTE: For a better startup time, we recommend using SDK-based instrumentation for manual instrumentation of the code rather than auto instrumentation.

To instrument AWS Lambda Node.js functions, see https://aws-otel.github.io/docs/getting-started/lambda/lambda-js[AWS Distro for OpenTelemetry Lambda Support For JavaScript].

The configuration of the OpenTelemetry Collector, with the definition of the Elastic {observability} endpoint, can be added to the root directory of the Lambda binaries: `src/main/resources/opentelemetry-collector.yaml`.

[source,yaml]
----
# Copy opentelemetry-collector.yaml in the root directory of the lambda function
# Set an environment variable 'OPENTELEMETRY_COLLECTOR_CONFIG_FILE' to '/var/task/opentelemetry-collector.yaml'
receivers:
otlp:
protocols:
http:
grpc:
exporters:
logging:
loglevel: debug
otlp/elastic:
# Elastic APM server https endpoint without the "https://" prefix
endpoint: "${ELASTIC_OTLP_ENDPOINT}" <1>
headers:
# Elastic APM Server secret token
Authorization: "Bearer ${ELASTIC_OTLP_TOKEN}" <1>
service:
pipelines:
traces:
receivers: [otlp]
exporters: [logging, otlp/elastic]
metrics:
receivers: [otlp]
exporters: [logging, otlp/elastic]
logs:
receivers: [otlp]
exporters: [logging, otlp/elastic]
----
<1> Environment-specific configuration parameters can be conveniently passed in as environment variables: `ELASTIC_OTLP_ENDPOINT` and `ELASTIC_OTLP_TOKEN`

Configure the AWS Lambda Node.js function:

* https://docs.aws.amazon.com/lambda/latest/dg/API_Layer.html[Function
layer]: The latest https://aws-otel.github.io/docs/getting-started/lambda/lambda-js[AWS
Lambda layer for OpenTelemetry]. For example, `arn:aws:lambda:eu-west-1:901920570463:layer:aws-otel-nodejs-ver-0-23-0:1`)
* https://docs.aws.amazon.com/lambda/latest/dg/API_TracingConfig.html[`TracingConfig` / Mode] set to `PassTrough`
* https://docs.aws.amazon.com/lambda/latest/dg/API_FunctionConfiguration.html[`FunctionConfiguration` / Timeout] set to more than 10 seconds to support the cold start of the Lambda JavaScript Runtime
* Export the environment variables:
** `AWS_LAMBDA_EXEC_WRAPPER="/opt/otel-handler"` for wrapping handlers proxied through the API Gateway. See https://aws-otel.github.io/docs/getting-started/lambda/lambda-js#enable-auto-instrumentation-for-your-lambda-function[enable auto instrumentation for your lambda-function].
** `OTEL_PROPAGATORS="tracecontext"` to override the default setting that also enables X-Ray headers causing interferences between OpenTelemetry and X-Ray
** `OPENTELEMETRY_COLLECTOR_CONFIG_FILE="/var/task/opentelemetry-collector.yaml"` to specify the path to your OpenTelemetry Collector configuration.
** `OTEL_TRACES_SAMPLER="AlwaysOn"` define the required sampler strategy if it is not sent from the caller. Note that `Always_on` can potentially create a very large amount of data, so in production set the correct sampling configuration, as per the https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/sdk.md#sampling[specification].

[float]
[[open-telemetry-aws-lambda-nodejs-terraform]]
==== Instrumenting AWS Lambda Node.js functions with Terraform

To manage the configuration of your AWS Lambda functions, we recommend using an infrastructure as code solution like Terraform or Ansible.

Here is an example of AWS Lambda Node.js function managed with Terraform and the https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_function[AWS Provider / Lambda Functions]:

* https://github.com/michaelhyatt/terraform-aws-nodejs-api-worker-otel/tree/v0.24[Sample Terraform code]

[float]
[[open-telemetry-lambda-next]]
==== Next steps
Expand Down

0 comments on commit 73d8e69

Please sign in to comment.