Bring dev changes for Otel to prod

[matthewhughes-uw]

I.e. bring in equivalent changes as:

• 94b5f07
• e6749ae
• 0b607e7

Ticket: DENA-1126

[linear]

DENA-1126 otel tail sampling: update infrastructure to acomodate tail sampling

[uw-infra]

Terraform run output for

Cluster: prod-aws
Module: otel/kafka-bitnami
Path: prod-aws/otel
Commit ID: b32be3f58baa2043a885b5aeb09658808b8c0794

✅ Run Status: Ok, Run Summary: Plan: 5 to add, 0 to change, 0 to destroy.

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # kafka_topic.otlp_sampled_spans will be created
  + resource "kafka_topic" "otlp_sampled_spans" {
      + config             = {
          + "cleanup.policy"    = "delete"
          + "compression.type"  = "zstd"
          + "max.message.bytes" = "134217728"
          + "retention.bytes"   = "5368709120"
          + "retention.ms"      = "43200000"
          + "segment.bytes"     = "262144000"
          + "segment.ms"        = "10800000"
        }
      + id                 = (known after apply)
      + name               = "otel.otlp_sampled_spans"
      + partitions         = 200
      + replication_factor = 3
    }

  # module.otel_tail_sampling_collector.kafka_acl.group_acl["otel.tail-sampling-collector"] will be created
  + resource "kafka_acl" "group_acl" {
      + acl_host                     = "*"
      + acl_operation                = "Read"
      + acl_permission_type          = "Allow"
      + acl_principal                = "User:CN=otel/tail-sampling-collector"
      + id                           = (known after apply)
      + resource_name                = "otel.tail-sampling-collector"
      + resource_pattern_type_filter = "Literal"
      + resource_type                = "Group"
    }

  # module.otel_tail_sampling_collector.kafka_acl.producer_acl["otel.otlp_sampled_spans"] will be created
  + resource "kafka_acl" "producer_acl" {
      + acl_host                     = "*"
      + acl_operation                = "Write"
      + acl_permission_type          = "Allow"
      + acl_principal                = "User:CN=otel/tail-sampling-collector"
      + id                           = (known after apply)
      + resource_name                = "otel.otlp_sampled_spans"
      + resource_pattern_type_filter = "Literal"
      + resource_type                = "Topic"
    }

  # module.otel_tail_sampling_collector.kafka_acl.topic_acl["otel.otlp_spans"] will be created
  + resource "kafka_acl" "topic_acl" {
      + acl_host                     = "*"
      + acl_operation                = "Read"
      + acl_permission_type          = "Allow"
      + acl_principal                = "User:CN=otel/tail-sampling-collector"
      + id                           = (known after apply)
      + resource_name                = "otel.otlp_spans"
      + resource_pattern_type_filter = "Literal"
      + resource_type                = "Topic"
    }

  # module.tempo_distributor.kafka_acl.topic_acl["otel.otlp_sampled_spans"] will be created
  + resource "kafka_acl" "topic_acl" {
      + acl_host                     = "*"
      + acl_operation                = "Read"
      + acl_permission_type          = "Allow"
      + acl_principal                = "User:CN=otel/tempo-distributor"
      + id                           = (known after apply)
      + resource_name                = "otel.otlp_sampled_spans"
      + resource_pattern_type_filter = "Literal"
      + resource_type                = "Topic"
    }

Plan: 5 to add, 0 to change, 0 to destroy.

To manually trigger plan again please post @terraform-applier plan prod-aws/otel as comment.

[uw-infra]

Terraform run output for

Cluster: prod-aws
Module: otel/kafka-bitnami
Path: prod-aws/otel
Commit ID: 66d3d0278c6bf088813d8f006c4c83f578078443

✅ Run Status: Ok, Run Summary: Apply complete! Resources: 5 added, 0 changed, 0 destroyed

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # kafka_topic.otlp_sampled_spans will be created
  + resource "kafka_topic" "otlp_sampled_spans" {
      + config             = {
          + "cleanup.policy"    = "delete"
          + "compression.type"  = "zstd"
          + "max.message.bytes" = "134217728"
          + "retention.bytes"   = "5368709120"
          + "retention.ms"      = "43200000"
          + "segment.bytes"     = "262144000"
          + "segment.ms"        = "10800000"
        }
      + id                 = (known after apply)
      + name               = "otel.otlp_sampled_spans"
      + partitions         = 200
      + replication_factor = 3
    }

  # module.otel_tail_sampling_collector.kafka_acl.group_acl["otel.tail-sampling-collector"] will be created
  + resource "kafka_acl" "group_acl" {
      + acl_host                     = "*"
      + acl_operation                = "Read"
      + acl_permission_type          = "Allow"
      + acl_principal                = "User:CN=otel/tail-sampling-collector"
      + id                           = (known after apply)
      + resource_name                = "otel.tail-sampling-collector"
      + resource_pattern_type_filter = "Literal"
      + resource_type                = "Group"
    }

  # module.otel_tail_sampling_collector.kafka_acl.producer_acl["otel.otlp_sampled_spans"] will be created
  + resource "kafka_acl" "producer_acl" {
      + acl_host                     = "*"
      + acl_operation                = "Write"
      + acl_permission_type          = "Allow"
      + acl_principal                = "User:CN=otel/tail-sampling-collector"
      + id                           = (known after apply)
      + resource_name                = "otel.otlp_sampled_spans"
      + resource_pattern_type_filter = "Literal"
      + resource_type                = "Topic"
    }

  # module.otel_tail_sampling_collector.kafka_acl.topic_acl["otel.otlp_spans"] will be created
  + resource "kafka_acl" "topic_acl" {
      + acl_host                     = "*"
      + acl_operation                = "Read"
      + acl_permission_type          = "Allow"
      + acl_principal                = "User:CN=otel/tail-sampling-collector"
      + id                           = (known after apply)
      + resource_name                = "otel.otlp_spans"
      + resource_pattern_type_filter = "Literal"
      + resource_type                = "Topic"
    }

  # module.tempo_distributor.kafka_acl.topic_acl["otel.otlp_sampled_spans"] will be created
  + resource "kafka_acl" "topic_acl" {
      + acl_host                     = "*"
      + acl_operation                = "Read"
      + acl_permission_type          = "Allow"
      + acl_principal                = "User:CN=otel/tempo-distributor"
      + id                           = (known after apply)
      + resource_name                = "otel.otlp_sampled_spans"
      + resource_pattern_type_filter = "Literal"
      + resource_type                = "Topic"
    }

Plan: 5 to add, 0 to change, 0 to destroy.
module.otel_tail_sampling_collector.kafka_acl.topic_acl["otel.otlp_spans"]: Creating...
kafka_topic.otlp_sampled_spans: Creating...
module.otel_tail_sampling_collector.kafka_acl.group_acl["otel.tail-sampling-collector"]: Creating...
module.otel_tail_sampling_collector.kafka_acl.topic_acl["otel.otlp_spans"]: Creation complete after 0s [id=User:CN=otel/tail-sampling-collector|*|Read|Allow|Topic|otel.otlp_spans|Literal]
module.otel_tail_sampling_collector.kafka_acl.group_acl["otel.tail-sampling-collector"]: Creation complete after 0s [id=User:CN=otel/tail-sampling-collector|*|Read|Allow|Group|otel.tail-sampling-collector|Literal]
kafka_topic.otlp_sampled_spans: Creation complete after 1s [id=otel.otlp_sampled_spans]
module.otel_tail_sampling_collector.kafka_acl.producer_acl["otel.otlp_sampled_spans"]: Creating...
module.tempo_distributor.kafka_acl.topic_acl["otel.otlp_sampled_spans"]: Creating...
module.tempo_distributor.kafka_acl.topic_acl["otel.otlp_sampled_spans"]: Creation complete after 0s [id=User:CN=otel/tempo-distributor|*|Read|Allow|Topic|otel.otlp_sampled_spans|Literal]
module.otel_tail_sampling_collector.kafka_acl.producer_acl["otel.otlp_sampled_spans"]: Creation complete after 0s [id=User:CN=otel/tail-sampling-collector|*|Write|Allow|Topic|otel.otlp_sampled_spans|Literal]

Warning: Argument is deprecated

  with provider["registry.terraform.io/mongey/kafka"],
  on __env.tf line 14, in provider "kafka":
  14: provider "kafka" {

This parameter is now deprecated and will be removed in a later release,
please use `ca_cert` instead.

(and 2 more similar warnings elsewhere)

Apply complete! Resources: 5 added, 0 changed, 0 destroyed.

To manually trigger plan again please post @terraform-applier plan prod-aws/otel as comment.