quickstart.md

Quickstart

🚨🚨 NOTE: There is currently a bug that is preventing KinD from running in GitHub codespaces. The workaround has been to manually create the docker network and exclude the --ipv6 flag before creating a the KinD cluster. This is already accounted for run you run the setup script, so no further action is required! You can track the issue here.

Pre-requisites

To get started quickly, it is recommended that you run this in a GitHub Codespace by clicking on "Use this template" and selecting the "Open in a codespace" from the dropdown in the main repo page.

Otherwise, you will need to have the following:

1. Docker - 24.0.x or greater
2. KinD - v0.20.x or greater
3. kubectl - 1.27.x or greater
4. Python - 3.11 or greater

PART 1

docker compose -f docker-compose-minimal.yml --env-file .env.uninstrumented build
docker compose -f docker-compose-minimal.yml --env-file .env.uninstrumented up

Shareable code

Set up GitHub codespace. Ref here.

Need to make sure that source GH repo is a template repo. There's a Template Repository checkbox in the repo settings page, just under the repo name.

# Install KinD
[ $(uname -m) = x86_64 ] && curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.20.0/kind-linux-amd64
chmod +x ./kind
sudo mv ./kind /usr/local/bin/kind
kind --version

# Create KinD cluster
kind create cluster --name otel-python-lab

PART 2

Installation

1. Create KinD Cluster

   kind create cluster --name otel-python-lab

2. Install the OTel Operator

   🚨 NOTE: cert-manager is a pre-requisite, so it needs to be installed first.

   # Install cert-manager
   kubectl --context kind-otel-python-lab apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.10.0/cert-manager.yaml
   
   # Install OTel Operator
   kubectl --context kind-otel-python-lab apply -f https://github.com/open-telemetry/opentelemetry-operator/releases/download/v0.81.0/opentelemetry-operator.yaml

3. Verify installation

   Check custom resource creation:

   kubectl --context kind-otel-python-lab get crds

   Sample output:

   NAME                                       CREATED AT
   certificaterequests.cert-manager.io        2023-07-25T14:44:24Z
   certificates.cert-manager.io               2023-07-25T14:44:24Z
   challenges.acme.cert-manager.io            2023-07-25T14:44:24Z
   clusterissuers.cert-manager.io             2023-07-25T14:44:24Z
   instrumentations.opentelemetry.io          2023-07-25T14:46:00Z
   issuers.cert-manager.io                    2023-07-25T14:44:24Z
   opentelemetrycollectors.opentelemetry.io   2023-07-25T14:46:00Z
   orders.acme.cert-manager.io                2023-07-25T14:44:24Z
   

   Make sure that cert-manager pods have started up:

   kubectl --context kind-otel-python-lab get pods -n cert-manager

   Sample output:

   NAME                                       READY   STATUS    RESTARTS   AGE
   cert-manager-7f6665fd8c-gp8vl              1/1     Running   0          9m32s
   cert-manager-cainjector-666564dc88-crzr9   1/1     Running   0          9m32s
   cert-manager-webhook-fd94896cd-d6s5v       1/1     Running   0          9m32s
   

   Make sure that OTel Operator pods have started up:

   kubectl --context kind-otel-python-lab get pods -n opentelemetry-operator-system

   Sample output:

   NAME                                                         READY   STATUS    RESTARTS   AGE
   opentelemetry-operator-controller-manager-7dd6b7c9c9-pxwzg   2/2     Running   0          9m11s
   

4. Deploy sample app

   Load Python Docker images locally in KinD. Reference here.

   kind load docker-image otel-python-lab:0.1.0-py-otel-server -n otel-python-lab
   
   kind load docker-image otel-python-lab:0.1.0-py-otel-client -n otel-python-lab
   
   kind load docker-image otel-python-lab-instrumented:0.1.0-py-otel-server -n otel-python-lab
   
   kind load docker-image otel-python-lab-instrumented:0.1.0-py-otel-client -n otel-python-lab
   

   Deploy the resources

   kubectl --context kind-otel-python-lab apply -f src/resources/00-namespaces.yml 
   kubectl --context kind-otel-python-lab apply -f src/resources/01-jaeger.yml 
   kubectl --context kind-otel-python-lab apply -f src/resources/02-otel-collector.yml 
   kubectl --context kind-otel-python-lab apply -f src/resources/03-python-instrumentation.yml
   kubectl --context kind-otel-python-lab apply -f src/resources/04-python-client.yml
   kubectl --context kind-otel-python-lab apply -f src/resources/05-python-server.yml

5. Port-forwarding

   Open a new terminal window for each port-forward:

   kubectl --context kind-otel-python-lab port-forward svc/jaeger 16686:16686 -n opentelemetry
   
   # Only required if you want to test the server separately
   kubectl --context kind-otel-python-lab port-forward svc/py-otel-server-svc 8082:8082 -n application

6. Logs

   Run each command in a new browser window:

   # Server logs
   kubectl --context kind-otel-python-lab logs --selector=app=py-otel-server --container py-otel-server -n application --follow
   
   # Client logs
   kubectl --context kind-otel-python-lab logs --selector=app=py-otel-client --container py-otel-client -n application --follow
   
   # Collector logs
   kubectl --context kind-otel-python-lab logs -l app=opentelemetry -n opentelemetry --follow

7. Nukify

   Nukify opentelemetry and application namespaces

   kubectl kubectl --context kind-otel-python-lab logs --selector=app=py-otel-server --container py-otel-server -n application --followdelete ns opentelemetry application

   Nukify KinD cluster

   kind delete clusters otel-python-lab

Debugging Auto-Instrumentation

# Check auto-instrumentation deployment
kubectl describe otelinst -n application

# Check events log
kubectl get events -n application

# Check operator logs
kubectl logs -l app.kubernetes.io/name=opentelemetry-operator --container manager -n opentelemetry-operator-system --follow

# Describe deployment to make sure that there's an auto-instrumentation container

# Check app logs

# Check init-containers for app. There should be a container called opentelemetry-auto-instrumentation