Questions about Receiver using HTTP

[kuro337]

Hey all!

I was using a simple app to test and get a feel of OpenTelemetry - and I was running into an issue with the HTTP Receiver. I wanted to use an HTTP Receiver to confirm whether my Sidecar deployment is correct - but I see this error

2023/09/17 04:51:18 traces export: Post "https://0.0.0.0:4318/v1/traces": http: server gave HTTP response to HTTPS client

I have a Go application and using this - (tried both WithInsecure() and without it

func CreateOTELTracing(ctx context.Context, l *structuredlogger.CustomLogger) (cleanup func(), err error) {
	exp, err := otlptracehttp.New(context.Background(),
		//otlptracehttp.WithInsecure(),
		otlptracehttp.WithEndpoint("0.0.0.0:4318"))
	if err != nil {
		return nil, fmt.Errorf("failed to initialize stdouttrace exporter: %w", err)
	}

	bsp := sdktrace.NewBatchSpanProcessor(exp)
	tp := sdktrace.NewTracerProvider(
		sdktrace.WithSampler(sdktrace.AlwaysSample()),
		sdktrace.WithSpanProcessor(bsp),
	)
	otel.SetTracerProvider(tp)

	cleanup = func() {
		if err := tp.Shutdown(context.Background()); err != nil {
			l.Fatal(err)
		}
	}

	return cleanup, nil
}

When the insecure() is not there - I get above error log

2023/09/17 04:51:18 traces export: Post "https://0.0.0.0:4318/v1/traces": http: server gave HTTP response to HTTPS client

When I add it - there are no logs at all.

If I just have a collector setup and an app in a sidecar , is there a way to confirm whether my image is sending traces to the colleector?

I do not see any logs in the otel-collector or my go app container for successful data being sent to the collector.

I tried adding insecure , insecureSkip , etc. to my collector yaml - but have not found the correct setting yet.

Would appreciate any pointers or recommendations - I am basically trying to setup something minimal where I can have collectors running as sidecars with my app deployments - send that data to a tempo backend , and visualize the traces - and approaching it piece by piece. Thank you!

This is my otel-collector-config.yaml. (have not configured the backend yet for tempo)

apiVersion: v1
kind: ConfigMap
metadata:
  name: otel-collector-conf
  labels:
    app: opentelemetry
    component: otel-collector-conf
data:
  otel-collector-config.yaml: |
    receivers:
      otlp:
        protocols:
          grpc:
          http:
            endpoint: 0.0.0.0:4318
    processors:
    extensions:
      health_check: {}
    exporters:
      otlp:
        endpoint: localhost:4317
        tls:
          insecure: true
    service:
      extensions: [health_check]
      pipelines:
        traces:
          receivers: [otlp]
          processors: []
          exporters: [otlp]

This is my deployment with the sidecar go-otel-dep.yaml 

apiVersion: apps/v1
kind: Deployment
metadata:
  name: go-app-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: go-app
  template:
    metadata:
      labels:
        app: go-app
    spec:
      containers:
        - name: go-app
          image: ghcr.io/<user>/<image>:latest
          ports:
            - containerPort: 8080

        - name: otel-collector
          image: otel/opentelemetry-collector:latest
          args:
            - "--config=/etc/otel/otel-collector-config.yaml"
          volumeMounts:
            - name: otel-config-vol
              mountPath: /etc/otel

      imagePullSecrets:
        - name: ghcr-secret

      volumes:
        - name: otel-config-vol
          configMap:
            name: otel-collector-conf
            items:
              - key: otel-collector-config.yaml
                path: otel-collector-config.yaml