From 1cca6cdec279106c413888ad764ee39fa79d82eb Mon Sep 17 00:00:00 2001 From: Colleen McGinnis Date: Tue, 17 Oct 2023 17:24:55 -0500 Subject: [PATCH 1/6] reduce content on aws lambda support --- docs/otel-other.asciidoc | 168 +-------------------------------------- 1 file changed, 1 insertion(+), 167 deletions(-) diff --git a/docs/otel-other.asciidoc b/docs/otel-other.asciidoc index 7da74573877..f896b832154 100644 --- a/docs/otel-other.asciidoc +++ b/docs/otel-other.asciidoc @@ -1,177 +1,11 @@ [[open-telemetry-other-env]] -=== Other execution environments - -** <> -** <> -** <> -** <> - -[float] -[[open-telemetry-aws-lambda]] === AWS Lambda Support +[[open-telemetry-aws-lambda]] AWS Lambda functions can be instrumented with OpenTelemetry and monitored with Elastic {observability}. To get started, follow the official AWS Distro for OpenTelemetry Lambda https://aws-otel.github.io/docs/getting-started/lambda[getting started documentation] and configure the OpenTelemetry Collector to output traces and metrics to your Elastic cluster. -[float] -[[open-telemetry-aws-lambda-java]] -==== Instrumenting AWS Lambda Java functions - -NOTE: For a better startup time, we recommend using SDK-based instrumentation, i.e. manual instrumentation of the code, rather than auto instrumentation. - -To instrument AWS Lambda Java functions, follow the official https://aws-otel.github.io/docs/getting-started/lambda/lambda-java[AWS Distro for OpenTelemetry Lambda Support For Java]. - -Noteworthy configuration elements: - -* AWS Lambda Java functions should extend `com.amazonaws.services.lambda.runtime.RequestHandler`, -+ -[source,java] ----- -public class ExampleRequestHandler implements RequestHandler { - public APIGatewayProxyResponseEvent handleRequest(APIGatewayProxyRequestEvent event, Context context) { - // add your code ... - } -} ----- - -* When using SDK-based instrumentation, frameworks you want to gain visibility of should be manually instrumented -** The below example instruments https://square.github.io/okhttp/4.x/okhttp/okhttp3/-ok-http-client/[OkHttpClient] with the OpenTelemetry instrument https://search.maven.org/artifact/io.opentelemetry.instrumentation/opentelemetry-okhttp-3.0/1.3.1-alpha/jar[io.opentelemetry.instrumentation:opentelemetry-okhttp-3.0:1.3.1-alpha] -+ -[source,java] ----- -import io.opentelemetry.instrumentation.okhttp.v3_0.OkHttpTracing; - -OkHttpClient httpClient = new OkHttpClient.Builder() - .addInterceptor(OkHttpTracing.create(GlobalOpenTelemetry.get()).newInterceptor()) - .build(); ----- - -* The configuration of the OpenTelemetry Collector, with the definition of the Elastic {observability} endpoint, can be added to the root directory of the Lambda binaries (e.g. defined in `src/main/resources/opentelemetry-collector.yaml`) -+ -[source,yaml] ----- -# Copy opentelemetry-collector.yaml in the root directory of the lambda function -# Set an environment variable 'OPENTELEMETRY_COLLECTOR_CONFIG_FILE' to '/var/task/opentelemetry-collector.yaml' -receivers: - otlp: - protocols: - http: - grpc: - -exporters: - logging: - loglevel: debug - otlp/elastic: - # Elastic APM server https endpoint without the "https://" prefix - endpoint: "${ELASTIC_OTLP_ENDPOINT}" <1> - headers: - # Elastic APM Server secret token - Authorization: "Bearer ${ELASTIC_OTLP_TOKEN}" <1> - -service: - pipelines: - traces: - receivers: [otlp] - exporters: [logging, otlp/elastic] - metrics: - receivers: [otlp] - exporters: [logging, otlp/elastic] - logs: - receivers: [otlp] - exporters: [logging, otlp/elastic] ----- -<1> Environment-specific configuration parameters can be conveniently passed in as environment variables: `ELASTIC_OTLP_ENDPOINT` and `ELASTIC_OTLP_TOKEN` - -* Configure the AWS Lambda Java function with: -** https://docs.aws.amazon.com/lambda/latest/dg/API_Layer.html[Function -layer]: The latest https://aws-otel.github.io/docs/getting-started/lambda/lambda-java[AWS -Lambda layer for OpenTelemetry] (e.g. `arn:aws:lambda:eu-west-1:901920570463:layer:aws-otel-java-wrapper-ver-1-2-0:1`) -** https://docs.aws.amazon.com/lambda/latest/dg/API_TracingConfig.html[`TracingConfig` / Mode] set to `PassTrough` -** https://docs.aws.amazon.com/lambda/latest/dg/API_FunctionConfiguration.html[`FunctionConfiguration` / Timeout] set to more than 10 seconds to support the longer cold start inherent to the Lambda Java Runtime -** Export the environment variables: -*** `AWS_LAMBDA_EXEC_WRAPPER="/opt/otel-proxy-handler"` for wrapping handlers proxied through the API Gateway (see https://aws-otel.github.io/docs/getting-started/lambda/lambda-java#enable-auto-instrumentation-for-your-lambda-function[here]) -*** `OTEL_PROPAGATORS="tracecontext, baggage"` to override the default setting that also enables X-Ray headers causing interferences between OpenTelemetry and X-Ray -*** `OPENTELEMETRY_COLLECTOR_CONFIG_FILE="/var/task/opentelemetry-collector.yaml"` to specify the path to your OpenTelemetry Collector configuration - -[float] -[[open-telemetry-aws-lambda-java-terraform]] -==== Instrumenting AWS Lambda Java functions with Terraform - -We recommend using an infrastructure as code solution like Terraform or Ansible to manage the configuration of your AWS Lambda functions. - -Here is an example of AWS Lambda Java function managed with Terraform and the https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_function[AWS Provider / Lambda Functions]: - -* Sample Terraform code: https://github.com/cyrille-leclerc/my-serverless-shopping-cart/tree/main/checkout-function/deploy -* Note that the Terraform code to manage the HTTP API Gateway (https://github.com/cyrille-leclerc/my-serverless-shopping-cart/tree/main/utils/terraform/api-gateway-proxy[here]) is copied from the official OpenTelemetry Lambda sample https://github.com/open-telemetry/opentelemetry-lambda/tree/e72467a085a2a6e57af133032f85ac5b8bbbb8d1/utils[here] - -[float] -[[open-telemetry-aws-lambda-nodejs]] -==== Instrumenting AWS Lambda Node.js functions - -NOTE: For a better startup time, we recommend using SDK-based instrumentation for manual instrumentation of the code rather than auto instrumentation. - -To instrument AWS Lambda Node.js functions, see https://aws-otel.github.io/docs/getting-started/lambda/lambda-js[AWS Distro for OpenTelemetry Lambda Support For JavaScript]. - -The configuration of the OpenTelemetry Collector, with the definition of the Elastic {observability} endpoint, can be added to the root directory of the Lambda binaries: `src/main/resources/opentelemetry-collector.yaml`. - -[source,yaml] ----- -# Copy opentelemetry-collector.yaml in the root directory of the lambda function -# Set an environment variable 'OPENTELEMETRY_COLLECTOR_CONFIG_FILE' to '/var/task/opentelemetry-collector.yaml' -receivers: - otlp: - protocols: - http: - grpc: - -exporters: - logging: - loglevel: debug - otlp/elastic: - # Elastic APM server https endpoint without the "https://" prefix - endpoint: "${ELASTIC_OTLP_ENDPOINT}" <1> - headers: - # Elastic APM Server secret token - Authorization: "Bearer ${ELASTIC_OTLP_TOKEN}" <1> - -service: - pipelines: - traces: - receivers: [otlp] - exporters: [logging, otlp/elastic] - metrics: - receivers: [otlp] - exporters: [logging, otlp/elastic] - logs: - receivers: [otlp] - exporters: [logging, otlp/elastic] ----- -<1> Environment-specific configuration parameters can be conveniently passed in as environment variables: `ELASTIC_OTLP_ENDPOINT` and `ELASTIC_OTLP_TOKEN` - -Configure the AWS Lambda Node.js function: - -* https://docs.aws.amazon.com/lambda/latest/dg/API_Layer.html[Function -layer]: The latest https://aws-otel.github.io/docs/getting-started/lambda/lambda-js[AWS -Lambda layer for OpenTelemetry]. For example, `arn:aws:lambda:eu-west-1:901920570463:layer:aws-otel-nodejs-ver-0-23-0:1`) -* https://docs.aws.amazon.com/lambda/latest/dg/API_TracingConfig.html[`TracingConfig` / Mode] set to `PassTrough` -* https://docs.aws.amazon.com/lambda/latest/dg/API_FunctionConfiguration.html[`FunctionConfiguration` / Timeout] set to more than 10 seconds to support the cold start of the Lambda JavaScript Runtime -* Export the environment variables: -** `AWS_LAMBDA_EXEC_WRAPPER="/opt/otel-handler"` for wrapping handlers proxied through the API Gateway. See https://aws-otel.github.io/docs/getting-started/lambda/lambda-js#enable-auto-instrumentation-for-your-lambda-function[enable auto instrumentation for your lambda-function]. -** `OTEL_PROPAGATORS="tracecontext"` to override the default setting that also enables X-Ray headers causing interferences between OpenTelemetry and X-Ray -** `OPENTELEMETRY_COLLECTOR_CONFIG_FILE="/var/task/opentelemetry-collector.yaml"` to specify the path to your OpenTelemetry Collector configuration. -** `OTEL_TRACES_SAMPLER="AlwaysOn"` define the required sampler strategy if it is not sent from the caller. Note that `Always_on` can potentially create a very large amount of data, so in production set the correct sampling configuration, as per the https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/sdk.md#sampling[specification]. - -[float] -[[open-telemetry-aws-lambda-nodejs-terraform]] -==== Instrumenting AWS Lambda Node.js functions with Terraform - -To manage the configuration of your AWS Lambda functions, we recommend using an infrastructure as code solution like Terraform or Ansible. - -Here is an example of AWS Lambda Node.js function managed with Terraform and the https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_function[AWS Provider / Lambda Functions]: - -* https://github.com/michaelhyatt/terraform-aws-nodejs-api-worker-otel/tree/v0.24[Sample Terraform code] - [float] [[open-telemetry-lambda-next]] ==== Next steps From 8957ca5f5517ee5df68fc9e14c932474b4f602bc Mon Sep 17 00:00:00 2001 From: Colleen McGinnis Date: Tue, 17 Oct 2023 17:51:03 -0500 Subject: [PATCH 2/6] update otel links --- docs/otel-limitations.asciidoc | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/docs/otel-limitations.asciidoc b/docs/otel-limitations.asciidoc index dd2e0ee9280..15791afb6a2 100644 --- a/docs/otel-limitations.asciidoc +++ b/docs/otel-limitations.asciidoc @@ -33,10 +33,12 @@ APM Server does not yet support JSON Encoding for OTLP/HTTP. [[open-telemetry-collector-exporter]] ==== OpenTelemetry Collector exporter for Elastic -The https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/elasticexporter#legacy-opentelemetry-collector-exporter-for-elastic[OpenTelemetry Collector exporter for Elastic] -was deprecated in 7.13 and replaced by the native support of the OpenTelemetry Line Protocol in -Elastic {observability} (OTLP). To learn more, see -https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/elasticexporter#migration[migration]. +The https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/v0.57.2/exporter/elasticsearchexporter#elasticsearch-exporter[OpenTelemetry Collector exporter for Elastic] +has been deprecated and replaced by the native support of the OpenTelemetry Line Protocol in Elastic Observability (OTLP). +// To learn more, see https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/v0.57.2/exporter/elasticsearchexporter#migration[migration]. + +The https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/elasticsearchexporter#elasticsearch-exporter[OpenTelemetry Collector exporter for Elastic] +(which is different from the legacy exporter mentioned above) is not intended to be used with Elastic APM and Elastic Observability. Use <> instead. [float] [[open-telemetry-tbs]] From 51287a1d43d581b2f01d75b857b9af1c3d220398 Mon Sep 17 00:00:00 2001 From: Colleen McGinnis Date: Tue, 17 Oct 2023 18:00:02 -0500 Subject: [PATCH 3/6] clarify env var example --- docs/otel-direct.asciidoc | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/docs/otel-direct.asciidoc b/docs/otel-direct.asciidoc index baf06727bf9..7a986240f7c 100644 --- a/docs/otel-direct.asciidoc +++ b/docs/otel-direct.asciidoc @@ -85,7 +85,8 @@ https://github.com/open-telemetry/opentelemetry-java-instrumentation[OpenTelemet See the https://opentelemetry.io/docs/instrumentation/[OpenTelemetry Instrumentation guides] to download the OpenTelemetry Agent or SDK for your language. -Define the following environment variables to configure the OpenTelemetry agent and enable communication with Elastic APM. +Define environment variables to configure the OpenTelemetry agent and enable communication with Elastic APM. +For example, if you are instrumenting a Java app, define the following environment variables: [source,bash] ---- @@ -114,7 +115,9 @@ For information on how to format an API key, see Please note the required space between `Bearer` and `an_apm_secret_token`, and `APIKey` and `an_api_key`. -| `OTEL_EXPORTER_OTLP_CERTIFICATE` | The trusted certificate used to verify the TLS credentials of the client. (optional) +| `OTEL_METRICS_EXPORTER` | Metrics exporter to use. See https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/#exporter-selection[exporter selection] for more information. + +| `OTEL_LOGS_EXPORTER` | Logs exporter to use. See https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/#exporter-selection[exporter selection] for more information. |=== From ddf75de9d4e38b4c6948054c9e2066eab810ea40 Mon Sep 17 00:00:00 2001 From: Colleen McGinnis Date: Tue, 17 Oct 2023 18:20:03 -0500 Subject: [PATCH 4/6] use lens instead of tsvb --- docs/otel-metrics.asciidoc | 43 ++------------------------------------ 1 file changed, 2 insertions(+), 41 deletions(-) diff --git a/docs/otel-metrics.asciidoc b/docs/otel-metrics.asciidoc index a72b101a3a4..b50588a4085 100644 --- a/docs/otel-metrics.asciidoc +++ b/docs/otel-metrics.asciidoc @@ -46,45 +46,6 @@ only OpenTelemetry metrics documents. [[open-telemetry-visualize]] ==== Visualize in {kib} -TSVB within {kib} is the recommended visualization for OpenTelemetry metrics. TSVB is a time series data visualizer that allows you to use the -{es} aggregation framework's full power. With TSVB, you can combine an infinite number of aggregations to display complex data. +Use *Lens* to create visualizations for OpenTelemetry metrics. Lens enables you to build visualizations by dragging and dropping data fields. It makes smart visualization suggestions for your data, allowing you to switch between visualization types. -// lint ignore ecommerce -In this example eCommerce OpenTelemetry dashboard, there are four visualizations: sales, order count, product cache, and system load. The dashboard provides us with business -KPI metrics, along with performance-related metrics. - - -[role="screenshot"] -image::./images/ecommerce-dashboard.png[OpenTelemetry visualizations] - -Let's look at how this dashboard was created, specifically the Sales USD and System load visualizations. - -. Open the main menu, then click *Dashboard*. -. Click *Create dashboard*. -. Click *Save*, enter the name of your dashboard, and then click *Save* again. -. Let’s add a Sales USD visualization. Click *Edit*. -. Click *Create new* and then select *TSVB*. -. For the label name, enter Sales USD, and then select the following: -+ -* Aggregation: `Counter Rate`. -* Field: `order_sum`. -* Scale: `auto`. -* Group by: `Everything` -. Click *Save*, enter Sales USD as the visualization name, and then click *Save and return*. -. Now let's create a visualization of load averages on the system. Click *Create new*. -. Select *TSVB*. -. Select the following: -+ -* Aggregation: `Average`. -* Field: `system.cpu.load_average.1m`. -* Group by: `Terms`. -* By: `host.ip`. -* Top: `10`. -* Order by: `Doc Count (default)`. -* Direction: `Descending`. -. Click *Save*, enter System load per host IP as the visualization name, and then click *Save and return*. -+ -Both visualizations are now displayed on your custom dashboard. - -IMPORTANT: By default, Discover shows data for the last 15 minutes. If you have a time-based index -and no data displays, you might need to increase the time range. +For more information on using Lens, refer to the {kibana-ref}/lens.html[Lens documentation]. From 0f201673e5cd90c41527496932207775e4c87707 Mon Sep 17 00:00:00 2001 From: Colleen McGinnis Date: Tue, 17 Oct 2023 18:22:47 -0500 Subject: [PATCH 5/6] remove resource attributes link --- docs/otel-direct.asciidoc | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/otel-direct.asciidoc b/docs/otel-direct.asciidoc index 7a986240f7c..734178e0066 100644 --- a/docs/otel-direct.asciidoc +++ b/docs/otel-direct.asciidoc @@ -145,5 +145,4 @@ https://github.com/elastic/apm-server/blob/main/dev_docs/otel.md#muxing-grpc-and ==== Next steps * <> -* Add <> * Learn about the <> From 2e96a85fb9d944d4c09e86ea3231771d2961b56e Mon Sep 17 00:00:00 2001 From: Colleen McGinnis Date: Tue, 31 Oct 2023 08:44:41 -0500 Subject: [PATCH 6/6] apply suggestions from code review Co-authored-by: Alexander Wert --- docs/otel-limitations.asciidoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/otel-limitations.asciidoc b/docs/otel-limitations.asciidoc index 15791afb6a2..28d2ebeb959 100644 --- a/docs/otel-limitations.asciidoc +++ b/docs/otel-limitations.asciidoc @@ -33,11 +33,11 @@ APM Server does not yet support JSON Encoding for OTLP/HTTP. [[open-telemetry-collector-exporter]] ==== OpenTelemetry Collector exporter for Elastic -The https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/v0.57.2/exporter/elasticsearchexporter#elasticsearch-exporter[OpenTelemetry Collector exporter for Elastic] +The https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/v0.57.2/exporter/elasticexporter[OpenTelemetry Collector exporter for Elastic] has been deprecated and replaced by the native support of the OpenTelemetry Line Protocol in Elastic Observability (OTLP). // To learn more, see https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/v0.57.2/exporter/elasticsearchexporter#migration[migration]. -The https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/elasticsearchexporter#elasticsearch-exporter[OpenTelemetry Collector exporter for Elastic] +The https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/elasticsearchexporter#elasticsearch-exporter[Elasticsearch exporter for the OpenTelemetry Collector] (which is different from the legacy exporter mentioned above) is not intended to be used with Elastic APM and Elastic Observability. Use <> instead. [float]