Proposal: New Getting Started Documentation And Reference Application

[svrnm]

This project aims to deliver a new learning experience on opentelemetry.io for OpenTelemetry end users, combined with
a "reference application" that serves as both the foundation of this experience and a demonstration tool for SIGs to showcase feature implementations.

While members of SIG Comms can create the guide, the reference application will require collaboration across SIGs. This need for cross-SIG collaboration is a key reason behind this project proposal.

This proposal is a continuation of the following existing discussions and documents:

• Restructure Language SDK & API documentation information architecture
• Nice to have: a tutorials section on opentelemetry.io where folks can easily access tutorials
• Visualizations we support in the documentation, following up on a conversation with .NET SIG in
  this issue.
• Learn OpenTelemetry: New Getting Started & Best Practices

[avillela]

Thanks for putting this together, @svrnm! I think that this captures things really well and will address a lot of the feedback that we've been getting from the community. 😁

[avillela]

@svrnm we were chatting at the End User SIG meeting today, and were wondering if it also makes sense to have a focus on infrastructure examples, and also mobile app examples?

[svrnm]

@svrnm we were chatting at the End User SIG meeting today, and were wondering if it also makes sense to have a focus on infrastructure examples, and also mobile app examples?

Eventually, yes, but at the beginning I think we need to focus to get the core right (how to add otel to an application and then get traces, metrics, logs, etc out of it). Setting up a collector should be part of that, so having a part on how infra (resources, entities, etc.) plays a roll in that, is definitely something that would come naturally. For the mobile app I see the issue that this is still very experimental, so yes we should have that (adding a "frontend" to the sample app?), but also later.

[jack-berg]

Let's promote this in the next maintainer's SIG

[mx-psi]

Should we have a language agnostic test suite for the spec? Ensuring consistency will be much easier if this is possible

[svrnm]

if this is possible, yes!

[cijothomas]

https://github.com/w3c/trace-context/tree/main/test is what W3C-TraceContext uses to test in lagn/framework agnostic way.

[julianocosta89]

Hello all 👋🏽

Wouldn't make more sense to use the OTel Demo for that?
We could have an uninstrumented fork of it, and throughout the "Learning OTel guide" we instrument each service.

We have almost all OTel supported programming languages covered, missing Erlang/Elixir that will be added back (hopefully soon).

The Demo can be run with Docker and would make life easier.

I also know that @avillela has some work on devcontainer for the Demo.
Each service could have its own devcontainer with all required tools installed, so the newcomer would just have to care about having Docker installed.

Having the SIG members to update the demo would be ideal, as of today we are mostly the Demo maintainers taking care of it. (except @pellared that keeps .NET always up-to-date 🌟 ).

[julianocosta89]

Adding another "reference application" would just be a re-work, no?
Tagging @open-telemetry/demo-maintainers to raise awareness about this issue.

[cijothomas]

Wouldn't make more sense to use the OTel Demo for that?

It maybe too complex for a helloworld/getting started doc. The main goal (I think) here is - to have the exact same app for all languages. OTel Demo app has all languages, but they all do diff. component...

[julianocosta89]

It maybe too complex for a helloworld/getting started doc. The main goal (I think) here is - to have the exact same app for all languages. OTel Demo app has all languages, but they all do diff. component...

Got it, so that would replace the rolldice app.

[svrnm]

It maybe too complex for a helloworld/getting started doc. The main goal (I think) here is - to have the exact same app for all languages. OTel Demo app has all languages, but they all do diff. component...

Yes, that was my underlying assumption. I tried to express this in the proposal as well, but I am happy to discuss this, since we should either turn the "maybe too complex" into a "too complex" or an alternative option that can reduce the potential work.

If we go down the road to use the Demo application for the getting started guide we indeed de-duplicate some work, that's a win, but here are some of my concerns, that maybe can be removed:

• each service within the demo app then needs to work standalone on a local machine, such that a developer can take the uninstrumented version of it and then follow the instructions of the specific language to get it work, i.e. a ruby developer will be presented with the "Email Service" and then work through the same steps as a JavaScript developer with the "Frontend" service.
• calling each service should be trivial. Right now I do curl localhost:8080/dice and all works, but I assume that some of the services of the demo need a payload. I don't think this is insurmountable but requires some clear considerations. I also see that most(?) services are called via gRPC, so we would need people to use grpcurl or similar.
• the base code of the service should be trivial, so finding the right spot to add code should be trivial. In the examples we have so far there is an "app file" (e.g. index.js or index.py or DiceApplication.java) and a "library file" (e.g. dice.js, dice.py, Dice.java), so if we use the demo we would need to trim that down to a very basic code and everything else are "extensions" to that. Doable but hard to manage.
• if a new feature is introduced into the getting started guide, we need to apply it across all applications, e.g. if we say we want to add exemplars, we need to figure out for each and every demo app where to put it, while with a reference app it's a single place

[svrnm]

It maybe too complex for a helloworld/getting started doc. The main goal (I think) here is - to have the exact same app for all languages. OTel Demo app has all languages, but they all do diff. component...

Got it, so that would replace the rolldice app.

yes, exactly.

[julianocosta89]

Yes, that was my underlying assumption. I tried to express this in the proposal as well, but I am happy to discuss this, since we should either turn the "maybe too complex" into a "too complex" or an alternative option that can reduce the potential work.

If we go down the road to use the Demo application for the getting started guide we indeed de-duplicate some work, that's a win, but here are some of my concerns, that maybe can be removed:

• each service within the demo app then needs to work standalone on a local machine, such that a developer can take the uninstrumented version of it and then follow the instructions of the specific language to get it work, i.e. a ruby developer will be presented with the "Email Service" and then work through the same steps as a JavaScript developer with the "Frontend" service.
• calling each service should be trivial. Right now I do curl localhost:8080/dice and all works, but I assume that some of the services of the demo need a payload. I don't think this is insurmountable but requires some clear considerations. I also see that most(?) services are called via gRPC, so we would need people to use grpcurl or similar.
• the base code of the service should be trivial, so finding the right spot to add code should be trivial. In the examples we have so far there is an "app file" (e.g. index.js or index.py or DiceApplication.java) and a "library file" (e.g. dice.js, dice.py, Dice.java), so if we use the demo we would need to trim that down to a very basic code and everything else are "extensions" to that. Doable but hard to manage.
• if a new feature is introduced into the getting started guide, we need to apply it across all applications, e.g. if we say we want to add exemplars, we need to figure out for each and every demo app where to put it, while with a reference app it's a single place

The demo would be an overkill for this.
Also, people looking for a getting started guide are not looking for a getting started in 13 different programming languages, so that would be a pain.

From the Demo, the quoteservice is the simplest service on it, but as it is the latest service called in a chain some concepts would be hard to be explained with it, like context propagation for outgoing calls or linked traces.

Either way, if we want to explain more than the basic concepts, we need at least a 2 services app, ideally 3. And if we add the Collector, that would be 4.
Then, we need to send the data somewhere, that would be Jaeger (5), Prometheus (6), and so on.
Eventually we will have a big environment with a lot of moving pieces and a lot to maintain.

Sorry for not bringing a solution, but I wanted to shed some light into the challenges.

[puckpuck]

At Honeycomb, we deliver OpenTelemetry instrumentation workshops in several languages. These workshops use a relatively simple app but cover all (nearly all?) aspects of OpenTelemetry tracing. Everything runs in a dev container, and so far, we have covered the following languages: Go, Java, JS, and Python. The workshop also has a set of public Google slides describing tracing concepts as we go through each workshop module. The slides are linked in the workshop's readme, with workshop content starting at slide 19.

Could this be used as a starting point?

[svrnm]

The demo would be an overkill for this. Also, people looking for a getting started guide are not looking for a getting started in 13 different programming languages, so that would be a pain.

Agreed, they want to start in the language that is close to them and explore concepts from there.

Either way, if we want to explain more than the basic concepts, we need at least a 2 services app, ideally 3. And if we add the Collector, that would be 4. Then, we need to send the data somewhere, that would be Jaeger (5), Prometheus (6), and so on. Eventually we will have a big environment with a lot of moving pieces and a lot to maintain.

I think that 2 services are just fine for the basic concepts, and depending on the application that we choose, we can have 1 service that can either call itself or a copied instance, think about a fibonacci service, that either does the compute itself or distributes it to peer nodes.

We need a collector for sure, but jaeger,prometheus,etc are optional/something that we will only provide via "here is how you run them in docker" in the prose, the will not be part of the reference application.

Sorry for not bringing a solution, but I wanted to shed some light into the challenges.

No worries, challenges is what we need to discuss!

At Honeycomb, we deliver OpenTelemetry instrumentation workshops in several languages. These workshops use a relatively simple app but cover all (nearly all?) aspects of OpenTelemetry tracing. Everything runs in a dev container, and so far, we have covered the following languages: Go, Java, JS, and Python. The workshop also has a set of public Google slides describing tracing concepts as we go through each workshop module. The slides are linked in the workshop's readme, with workshop content starting at slide 19.

Could this be used as a starting point?

I am more than happy to build open things that we have already, that's why I'd like to have the discussion first which reference application we choose, one point is that we should see what exists already and can be used to start from. We have the existing getting started, the new relic examples and now one more. I'll update the proposal accordingly.

[noMoreCLI]

I think that 2 services are just fine for the basic concepts, and depending on the application that we choose, we can have 1 service that can either call itself or a copied instance, think about a fibonacci service, that either does the compute itself or distributes it to peer nodes.

should these 2 services also be the foundation to demonstrate later maybe more advanced concepts such as custom context propagation or baggage? This might add complexity to the service (and maybe even confuse people just starting). However might help people on the journey to continue their learning and exposure with OTel

[svrnm]

should these 2 services also be the foundation to demonstrate later maybe more advanced concepts such as custom context propagation or baggage? This might add complexity to the service (and maybe even confuse people just starting). However might help people on the journey to continue their learning and exposure with OTel

If possible yes, but not necessarily. We should start with the basic concepts and then evolve towards more advanced topics and see how the reference application fits into that. An option I see is that there are alternate versions of the (uninstrumented) base service, e.g. if you'd like to show the concept of custom context propagation there is a variant where the 2 services communicate using a different protocol that is not easy to instrument with what is available. Same can be done for the async/message passing case.

[svrnm]

I did some prototyping on a few ideas that I had with building that out, PTAL:

https://github.com/svrnm/otel-getting-started-prototype

• It contains a sample reference specification based on the fibonacci app
• It contains a sample JS application
• It contains a sample Go application
• The application code is generated from a single source file and "evolution" of that code (e.g. add instrumentation, add OTLP) is accomplished with C macros (!). There might be better solutions, but I found it interesting to see how well this works:-)