From 2802423193fe7128fb27c684e9e630200aa9dc6a Mon Sep 17 00:00:00 2001 From: Caleb Schoepp Date: Wed, 17 Apr 2024 12:06:03 -0600 Subject: [PATCH] Update SIP Signed-off-by: Caleb Schoepp --- ...-configuring-and-emitting-observability.md | 83 ------------------- .../sips/0170-adding-otel-tracing-to-spin.md | 65 +++++++++++++++ 2 files changed, 65 insertions(+), 83 deletions(-) delete mode 100644 docs/content/sips/017-configuring-and-emitting-observability.md create mode 100644 docs/content/sips/0170-adding-otel-tracing-to-spin.md diff --git a/docs/content/sips/017-configuring-and-emitting-observability.md b/docs/content/sips/017-configuring-and-emitting-observability.md deleted file mode 100644 index df2c0a689..000000000 --- a/docs/content/sips/017-configuring-and-emitting-observability.md +++ /dev/null @@ -1,83 +0,0 @@ -title = "SIP 017 - Configuring and Emitting Observability" -template = "main" -date = "2024-02-27T12:00:00Z" - ---- - -Summary: How to configure and emit telemetry to improve the runtime and trigger observability of Spin. - -Owner: caleb.schoepp@fermyon.com - -Created: February 27, 2024 - -Updated: February 27, 2024 - -## Background - -[Observability](https://opentelemetry.io/docs/concepts/observability-primer/#what-is-observability) is critical for a great developer experience. Improving the observability of Spin can be broken down into four categories: - -1. Runtime observability: Observing the Spin runtime itself e.g. spans and metrics around important parts of the Spin runtime. -2. Trigger observability: Observing the requests made to Spin applications e.g. spans and metrics around the requests made to Spin applications. -3. Component observability: Observing the interaction between composed components e.g having Wasmtime auto-instrument Wasm component graphs to produce spans. -4. Guest observability: Observing code within the guest module e.g. building a host component for WASI Observe. - -More detail on these categories can be found [here](https://github.com/fermyon/spin/issues/2293). - -## Proposal - -This SIP aims to improve the runtime and trigger observability of Spin. First, it will suggest how observability in Spin can be configured. Second, it will present the specific runtime and trigger observability we want to emit. Finally, it will outline a plan going forward for how observability should be added to new Spin features. - -This proposal assumes that all observability data produced should be OTEL compliant. - -### Configuring observability - -The developer must be able to configure the endpoints of OTLP compliant collectors where traces and metrics can be sent. This can be expressed in the `runtime-config.toml`. - -```toml -[observability.tracing] -endpoint = "http://localhost:4317" -protocol = "http/protobuf" # "grpc" or "http/protobuf" or "http/json" and it defaults to "http/protobuf" -[observability.metrics] -endpoint = "http://localhost:4317" -protocol = "http/protobuf" # "grpc" or "http/protobuf" or "http/json" and it defaults to "http/protobuf" -``` - -**BIKESHED:** `observability.tracing`, `otel.tracing`, `telemetry.tracing`.... - -The developer must be able to configure how each component of their application handles tracing. This can be expressed in the `spin.toml` manifest. - -```toml -[component.my-component.tracing] -context_propagation = true # This is all or nothing. If you disable propagation no context will be propagated. By default this is false. -# Opportunity to add fields in the future to -# - Disable tracing for performance reasons -# - Customize span names -# - Add additional metadata -# - More complex allow-listing mechanism for what spans propagate -# - etc. -``` - -### Improving observability - -#### Runtime observability - -Currently a few spans are emitted by Spin, but there is no consistent pattern. We should have all host components emit spans when they are called at the debug level. They may emit further spans at the trace level. Each host component should also emit any relevant metrics e.g. count of times called. - -#### Trigger observability - -All existing triggers should emit spans at the info level before routing to a specific component. They should also emit an info span before executing the Wasm. They should also emit any relevant metrics e.g. count of times called. - -#### Implementation details - -The Rust [tracing](https://docs.rs/tracing/latest/tracing/) library is used throughout Spin for emitting logs. This SIP suggests that [tracing_opentelemetry](https://docs.rs/tracing-opentelemetry/latest/tracing_opentelemetry/) be used to emit the necessary spans throughout Spin. Special care will have to be taken to ensure that the spans in the tracing do not interfere with the logs Spin produces. - -### Adding observability in the future - -Improving the observability of Spin is an ongoing process. As new features are added to Spin some of them will require new observability to be added. The following is non-exhaustive list of possible new features that would require new observability to be added: - -- **New triggers native to Spin**. An info span should be emitted before the trigger routes to a specific component. An info span should be emitted before executing the Wasm. Relevant metrics should be emitted. -- **New plugin triggers**. An info span should be emitted before the trigger routes to a specific component. An info span should be emitted before executing the Wasm. Relevant metrics should be emitted. -- **New host components**. A debug span should be emitted when the host component is called. Relevant metrics should be emitted. -- **New functionality in the Spin runtime**. This should be reserved for functionality that is critical to debugging the state of the Spin runtime. Debug spans should be added where relevant. Relevant metrics should be emitted. - -Any new observability that is added to Spin should follow a consistent pattern. Defining this pattern is outside the scope of this SIP so we will defer to OTEL best practices. For guidance on naming spans see [here](https://github.com/open-telemetry/opentelemetry-specification/blob/v1.26.0/specification/trace/api.md#span). For other guidance on other semantic conventions see [here](https://opentelemetry.io/docs/concepts/semantic-conventions/). diff --git a/docs/content/sips/0170-adding-otel-tracing-to-spin.md b/docs/content/sips/0170-adding-otel-tracing-to-spin.md new file mode 100644 index 000000000..1ee4b381b --- /dev/null +++ b/docs/content/sips/0170-adding-otel-tracing-to-spin.md @@ -0,0 +1,65 @@ +title = "SIP 017 - Adding OTel tracing to Spin" +template = "main" +date = "2024-02-27T12:00:00Z" + +--- + +Summary: How to configure OTel support in Spin and add tracing. + +Owner: caleb.schoepp@fermyon.com + +Created: February 27, 2024 + +Updated: April 17, 2024 + +## Background + +[Observability](https://opentelemetry.io/docs/concepts/observability-primer/#what-is-observability) is critical for a great developer experience. Improving the observability of Spin can be broken down into four categories: + +1. Runtime observability: Observing the Spin runtime itself e.g. spans and metrics around important parts of the Spin runtime like host components. +2. Trigger observability: Observing the requests made to Spin applications e.g. spans and metrics around the requests made to Spin applications. +3. Component observability: Observing the interaction between composed components e.g having Wasmtime auto-instrument Wasm component graphs to produce spans. +4. Guest observability: Observing code within the guest module e.g. building a host component for WASI Observe. + +More detail on these categories can be found [here](https://github.com/fermyon/spin/issues/2293). + +## Proposal + +This SIP aims to improve the runtime and trigger observability of Spin by emitting tracing. First, it will suggest how OTel tracing in Spin can be configured. Second, it will present guidelines for tracing in Spin. + +### Configuring OTel + +Spin will conform to the configuration options defined by OTel [here](https://opentelemetry.io/docs/specs/otel/protocol/exporter/) and [here](https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/#general-sdk-configuration). + +A user of Spin effectively turns on observability by setting the environment variable `OTEL_EXPORTER_OTLP_ENDPOINT`. This value should be the endpoint of an OTLP compliant collector. They may explicitly turn off observability by setting `OTEL_SDK_DISABLED`. + +Under the hood Spin will use the Rust [tracing](https://docs.rs/tracing/0.1.40/tracing/) crate to handle the instrumentation. The user may set `SPIN_OTEL_TRACING_LEVEL` to configure the level of tracing they want to emit to OTel. + +In the future we will want to support per-component configuration of observability. For example this might look like the following in a `spin.toml` manifest. + +```toml +[component.my-component.tracing] +context_propagation = true # This is all or nothing. If you disable propagation no context will be propagated. By default this is false. +# Opportunity to add fields in the future to +# - Disable tracing for performance reasons +# - Customize span names +# - Add additional metadata +# - More complex allow-listing mechanism for what spans propagate +# - etc. +``` + +### Adding tracing + +As a general rule of thumb we only want to trace behavior that: + +- Is fallible. +- Is potentially slow. +- Is not tightly nested under a parent span. +- Is relevant to an end user. + +In practice this means tracing most of the triggers and host components. + +Where [OTel semantic conventions](https://opentelemetry.io/docs/concepts/semantic-conventions/) exist we should follow them as closely as possible. Where semantic conventions don't exist we should: + +- Give the span a name in the form `spin_{crate}.{operation}`. +- Track any relevant arguments as attributes.