Skip to content

Commit

Permalink
feat(EnPRO): extend OTel-shop with MongoDB, add Ansible, Terraform an…
Browse files Browse the repository at this point in the history 
…d EDMM models (#9)

* [productcatalogservice] Added MongoDB

This adds a MongoDB to the demo instead of parsing the JSON before usage. The implementation is not yet perfect but it works for now. Thus far, only docker compose is supported.

Things that still need more work are:
- The Go application still uses static MongoDB username and password and the hostname and port are also static. This should be changed to use environmental variables instead.
- I wasn't able to check whether the healthcheck of the MongoDB works or not.
- Is the resource limit too generous?
- Should we fix the port inside the MongoDB container to its default value (i.e. change using a secret here to 27017)?

* [productcatalogservice] Changed to use env vars

This changes the productcatalogservice to use environmental variables for the MongoDB username, password, hostname and port.

* [K8s] Added MongoDB

*Disclaimer:* this is not working yet!

* build(deps): bump docker/build-push-action from 6.0.1 to 6.1.0 (open-telemetry#1620)

Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 6.0.1 to 6.1.0.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](docker/build-push-action@v6.0.1...v6.1.0)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Mikko Viitanen <[email protected]>

* fix: emailservice docker compose image (open-telemetry#1630)

Co-authored-by: Pierre Tessier <[email protected]>

* add collector configs as environment variable (open-telemetry#1632)

Co-authored-by: Mikko Viitanen <[email protected]>

* fix(frontend): Use port 443 as default to connect to flagd if https is in use (open-telemetry#1609)

* fix(frontend): Use port 443 as default to connect to flagd if https is in use

* chore: Update CHANGELOG.md

---------

Co-authored-by: Pierre Tessier <[email protected]>

* Update recommendation flag to match flagd configuration (open-telemetry#1634)

Address open-telemetry#1626

* Change AccountService from Go to DotNet (auto) (open-telemetry#1538)

* Change AccountService from go to dotnet (auto)

* fix path

* fix folder name

* dockerfile and other fixes

* add copyright

* fix encoding and cleanup

* Cleanup dockerfile

* Update OTel Auto

* fix kafka processing issues and otel export

* remove eof

* Update changelog

* Use default CancellationDelayMaxMs

* update packages

* fix merge failure

* Fix tracetest 'accountingservice' is not part of the trace anymore

---------

Co-authored-by: Juliano Costa <[email protected]>
Co-authored-by: Juliano Costa <[email protected]>

* build(deps): bump docker/build-push-action from 6.1.0 to 6.2.0 (open-telemetry#1636)

Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 6.1.0 to 6.2.0.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](docker/build-push-action@v6.1.0...v6.2.0)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Pierre Tessier <[email protected]>

* [chore] clarify complete release process (open-telemetry#1638)

* clarify complete release process

* clarify complete release process

* clarify complete release process

* clarify complete release process

* clarify complete release process

* update to 1.11.0 release (open-telemetry#1639)

* update to 1.11.0 release (open-telemetry#1640)

* [chore] update demo role memberships (open-telemetry#1649)

* update demo role membership

* update demo role membership

* [otel-col] Add docker stats receiver (open-telemetry#1650)

* Add docker stats receiver

* changelog

* Nit: prevent write access to the Docker socket

Co-authored-by: Roger Coll <[email protected]>

* Add compose minimal

---------

Co-authored-by: Roger Coll <[email protected]>

* add Kafka Dockerfile env variable (open-telemetry#1652)

Co-authored-by: Pierre Tessier <[email protected]>

* build(deps): bump docker/build-push-action from 6.2.0 to 6.3.0 (open-telemetry#1656)

Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 6.2.0 to 6.3.0.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](docker/build-push-action@v6.2.0...v6.3.0)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Simplify error scenario logic (open-telemetry#1657)

* remove fractional config from rule definition

Signed-off-by: Michael Beemer <[email protected]>

* add random failure logic to ad service

Signed-off-by: Michael Beemer <[email protected]>

* add random failure logic to cart service

Signed-off-by: Michael Beemer <[email protected]>

---------

Signed-off-by: Michael Beemer <[email protected]>

* [tests] - optimize trace testing (open-telemetry#1659)

* optimize trace testing

* run trace based tests concurrently

* fix lint failures

---------

Co-authored-by: Juliano Costa <[email protected]>

* [chore] fix build-images workflow (open-telemetry#1661)

* fix build-images workflow

* use consistent naming for workflows

* Set OTLP receiver endpoint (open-telemetry#1662)

* Set OTelCol receiver endpoint

* changelog

* Apply suggestions from code review

Co-authored-by: Roger Coll <[email protected]>

* Add env vars to collector container

---------

Co-authored-by: Roger Coll <[email protected]>

* feat: [K8s] Added MongoDB

*Disclaimer:* this is not working yet!

* feat: ES-9 add EDMM Model (K8s) of otel-shop

* feat: ES-39 add Terrfaform Model of otel-shop

* feat: ES-44 add Ansible Model of otel-shop

* fixed MongoDB not working and removed duplicate MongoDB service

* feat: ES-9 add some refinement and connects_to

* Update otel_store_k8s_translated.yaml

YAML style adjusted to a uniform format.

* Update otel_store_k8s_translated.yaml

YAML style adjusted to a uniform format. Adjusted, of relation_types and components_types (underscore was missing) --> Inconsistency

* Terraform model: Deletetd testing services and fixed image declaration of the email service

* feat: ES-32 minor fix and todo

* Adds missing component types and fixes relations

* fix: ES-56 fix some missing stuff in EDMM

* fix: ES-56 fix some missing stuff in EDMM

* fix: ES-56 fix moore

* fix: ES-56 fix more

* fix: ES-56 lint

* fix: ES-56 add opensearch hosted on

* fix: ES-56 remove duplicate

* fix: ES-56 fix naming

* Added health checks where applicable

some health checks are rather dirty but they work

* fix: auto-formatted K8s file

this enables us to finally parse this example with our plugin!

* chore(ES-80): move and cleanup main.tf (#8)

* chore: ES-80 move .env to k8s folder and cleanup main.tf

* chore: ES-80 remove .env

* ES-59 Fixed main.tf and added linux support

* fix: fix merge issues

* chore(ES-86): keep models up-to-date and add target models for all technologies (#10)

* chore: update versions and memory

* chore: some more adaptions

* chore: add target edmm models

* ci: test build job for productcatalogservice

* ci: try this tag

* ci: now try this tag

* ci: try enhance tagging

* chore: make use of pc-service build, readme, switch to concrete versioning in terraform

* chore: remove branch build

* docs: finalize readme

* chore: mr adaptions

* fix: fix typos from generator

* fix: remove submodules

---------

Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: Michael Beemer <[email protected]>
Co-authored-by: Felix <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Mikko Viitanen <[email protected]>
Co-authored-by: Roger Coll <[email protected]>
Co-authored-by: Pierre Tessier <[email protected]>
Co-authored-by: Sven Kirschbaum <[email protected]>
Co-authored-by: Steve Flanders <[email protected]>
Co-authored-by: Rasmus Kuusmann <[email protected]>
Co-authored-by: Juliano Costa <[email protected]>
Co-authored-by: Juliano Costa <[email protected]>
Co-authored-by: Roger Coll <[email protected]>
Co-authored-by: Michael Beemer <[email protected]>
Co-authored-by: Arikuma97 <[email protected]>
Co-authored-by: Pascal Schur <[email protected]>
Co-authored-by: Rene Tischler <[email protected]>
  • Loading branch information
@eliasmueller @heidrifx
@dependabot @mviitane @rogercoll @puckpuck @SvenKirschbaum @flands @RassK @julianocosta89 @beeme1mr @Arikuma97 @schurpl @365Bit
16 people authored Oct 3, 2024
1 parent eb1ab39 commit a93fbb9
Show file tree
Hide file tree
Showing 100 changed files with 18,832 additions and 943 deletions.
6 changes: 6 additions & 0 deletions .env
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ OPENSEARCH_IMAGE=opensearchproject/opensearch:2.16.0
POSTGRES_IMAGE=postgres:16.4
PROMETHEUS_IMAGE=quay.io/prometheus/prometheus:v2.54.1
VALKEY_IMAGE=valkey/valkey:8.0-alpine
MONGO_IMAGE=mongo:8.0.0-rc9
# must also update the version arg in ./test/tracetesting/Dockerfile
TRACETEST_IMAGE=kubeshop/tracetest:v1.5.2

Expand Down Expand Up @@ -138,6 +139,11 @@ KAFKA_SERVICE_DOCKERFILE=./src/kafka/Dockerfile
VALKEY_PORT=6379
VALKEY_ADDR=valkey-cart:${VALKEY_PORT}

# MongoDB
MONGO_PORT=27017
MONGO_USERNAME=mongo
MONGO_PASSWORD=mongo_product_catalog

# ********************
# Telemetry Components
# ********************
Expand Down
45 changes: 45 additions & 0 deletions .github/workflows/buildAndPushImage-productcatalogservice.yml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
name: buildAndPushImage

on:
push:
branches:
- main

env:
IMAGE_URL: ghcr.io/ust-demaf
IMAGE_NAME: productcatalogservice
IMAGE_TAG: 1.11.1

jobs:
build-using-dockerfile-push-2-ghcr:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Set up QEMU
uses: docker/setup-qemu-action@v3
with:
platforms: linux/amd64, linux/arm64

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
platforms: linux/amd64, linux/arm64

- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}

# Build and push the image
- name: Build and push
uses: docker/build-push-action@v6
with:
context: .
file: ./src/productcatalogservice/Dockerfile
platforms: linux/amd64, linux/arm64
push: true
tags: ${{ env.IMAGE_URL }}/demo:${{ env.IMAGE_TAG }}-${{ env.IMAGE_NAME }}
provenance: false
184 changes: 54 additions & 130 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,140 +1,64 @@
<!-- markdownlint-disable-next-line -->
# <img src="https://opentelemetry.io/img/logos/opentelemetry-logo-nav.png" alt="OTel logo" width="45"> OpenTelemetry Demo

[![Slack](https://img.shields.io/badge/slack-@cncf/otel/demo-brightgreen.svg?logo=slack)](https://cloud-native.slack.com/archives/C03B4CWV4DA)
[![Version](https://img.shields.io/github/v/release/open-telemetry/opentelemetry-demo?color=blueviolet)](https://github.com/open-telemetry/opentelemetry-demo/releases)
[![Commits](https://img.shields.io/github/commits-since/open-telemetry/opentelemetry-demo/latest?color=ff69b4&include_prereleases)](https://github.com/open-telemetry/opentelemetry-demo/graphs/commit-activity)
[![Downloads](https://img.shields.io/docker/pulls/otel/demo)](https://hub.docker.com/r/otel/demo)
[![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg?color=red)](https://github.com/open-telemetry/opentelemetry-demo/blob/main/LICENSE)
[![Integration Tests](https://github.com/open-telemetry/opentelemetry-demo/actions/workflows/run-integration-tests.yml/badge.svg)](https://github.com/open-telemetry/opentelemetry-demo/actions/workflows/run-integration-tests.yml)
[![Artifact Hub](https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/opentelemetry-demo)](https://artifacthub.io/packages/helm/opentelemetry-helm/opentelemetry-demo)
[![OpenSSF Best Practices](https://www.bestpractices.dev/projects/9247/badge)](https://www.bestpractices.dev/en/projects/9247)

## Welcome to the OpenTelemetry Astronomy Shop Demo
## Welcome to the OpenTelemetry Astronomy Shop Demo with DeMAF extension

This repository contains the OpenTelemetry Astronomy Shop, a microservice-based
distributed system intended to illustrate the implementation of OpenTelemetry in
a near real-world environment.

Our goals are threefold:
FOR THE ORIGINAL README [CLICK HERE](./README_original.md).

The purpose of this fork is to extend the Astronomy Shop in three ways:

- Add a database to showcase that a persistence layer can also be part of the architecture without causing any problems.
- Create a deployment model to deploy the shop using [Ansible](https://docs.ansible.com/) and [Terraform](https://developer.hashicorp.com/terraform/docs) besides the given [Docker-Compose](https://opentelemetry.io/docs/demo/docker_deployment/) and [Kubernetes](https://opentelemetry.io/docs/demo/kubernetes_deployment/).
- Add translated EDMM models for [Kubernetes](./edmm/otel-store_k8s_translated.yaml), [Ansible](./edmm/otel-store_ansible_translated.yaml) and [Terraform](./edmm/otel-store_terraform_translated.yaml) to showcase the transformation result of those three DeMAF plugins in [EDMM](https://github.com/UST-EDMM).


### Database Integration
The database integration ensures that also a persistence layer is used in the provided demo-application,
making the architecture more comprehensive.

Therefore, a [MongoDB](http://mongodb.com) is added, which adds the products of the shop into the database as JSON entries.
In the default variant of the astronomy shop, only a plain file is used to provide the products.

Thus, the logic of the `productcatalogservice` is adapted, to initialize and use the additionally provided MongoDB.

The changes of the `productcatalogservice` and the addition of the MongoDB is integrated into all deployment variants in this fork.

### Translation to Ansible

To verify the [Ansible MPS Plugin](https://github.com/UST-DeMAF/ansible-mps-plugin) the Astronomy Shop is available as Ansible installation.
It is based on the [Docker-Compose](https://opentelemetry.io/docs/demo/docker_deployment/) and installs all components of the shop as docker containers on the local Docker runtime.
For each component there is a dedicated Ansible Role representing the loose coupling of the components.

A usage guide can be found in the dedicated [README in /ansible](./ansible/README.md).

### Translation to Terraform

To verify the [Terraform MPS Plugin](https://github.com/UST-DeMAF/terraform-mps-plugin) the Astronomy Shop is available as Terraform installation.
It is based on the [Docker-Compose](https://opentelemetry.io/docs/demo/docker_deployment/) and installs all components of the shop as docker containers on the local Docker runtime.

A usage guide can be found in the dedicated [README in /terraform](./terraform/README.md).

### EDMM Models

For all three evaluated technologies (Ansible, Terraform, Kubernetes) there is a transformed target model available.

The target EDMM models can be found in the [/edmm directory](./edmm)

In each model the goal is to retrieve a component for each deployed Astronomy Shop component
and as many _connects-to_ and _hosted_on_ relations as possible.

This in common, there are also differences in the target models.

Due to the imperative character of Ansible, in the [Ansible target model](./edmm/otel-store_ansible_translated.yaml), each component has operations representing the installation of the Docker containers.

- Provide a realistic example of a distributed system that can be used to
demonstrate OpenTelemetry instrumentation and observability.
- Build a base for vendors, tooling authors, and others to extend and
demonstrate their OpenTelemetry integrations.
- Create a living example for OpenTelemetry contributors to use for testing new
versions of the API, SDK, and other components or enhancements.
In the [Terraform target model](./edmm/otel-store_terraform_translated.yaml), there are common component types for all _docker_containers_,
as they all use the same terraform ressource type. This is different in the other two evaluated technologies, where each component has a dedicated type.

We've already made [huge
progress](https://github.com/open-telemetry/opentelemetry-demo/blob/main/CHANGELOG.md),
and development is ongoing. We hope to represent the full feature set of
OpenTelemetry across its languages in the future.

If you'd like to help (**which we would love**), check out our [contributing
guidance](./CONTRIBUTING.md).

If you'd like to extend this demo or maintain a fork of it, read our
[fork guidance](https://opentelemetry.io/docs/demo/forking/).

## Quick start

You can be up and running with the demo in a few minutes. Check out the docs for
your preferred deployment method:

- [Docker](https://opentelemetry.io/docs/demo/docker_deployment/)
- [Kubernetes](https://opentelemetry.io/docs/demo/kubernetes_deployment/)

## Documentation

For detailed documentation, see [Demo Documentation][docs]. If you're curious
about a specific feature, the [docs landing page][docs] can point you in the
right direction.

## Demos featuring the Astronomy Shop

We welcome any vendor to fork the project to demonstrate their services and
adding a link below. The community is committed to maintaining the project and
keeping it up to date for you.

| | | |
|-----------------------------------------|-----------------------------|----------------------------------------------------------------|
| [AlibabaCloud LogService][AlibabaCloud] | [Elastic][Elastic] | [New Relic][NewRelic] |
| [AppDynamics][AppDynamics] | [Google Cloud][GoogleCloud] | [OpenSearch][OpenSearch] |
| [Aspecto][Aspecto] | [Grafana Labs][GrafanaLabs] | [Sentry][Sentry] |
| [Axiom][Axiom] | [Guance][Guance] | [ServiceNow Cloud Observability][ServiceNowCloudObservability] |
| [Axoflow][Axoflow] | [Helios][Helios] | [Splunk][Splunk] |
| [Azure Data Explorer][Azure] | [Honeycomb.io][Honeycombio] | [Sumo Logic][SumoLogic] |
| [Coralogix][Coralogix] | [Instana][Instana] | [TelemetryHub][TelemetryHub] |
| [Dash0][Dash0] | [Kloudfuse][Kloudfuse] | [Teletrace][Teletrace] |
| [Datadog][Datadog] | [Liatrio][Liatrio] | [Tracetest][Tracetest] |
| [Dynatrace][Dynatrace] | [Logz.io][Logzio] | [Uptrace][Uptrace] |

## Contributing

To get involved with the project see our [CONTRIBUTING](CONTRIBUTING.md)
documentation. Our [SIG Calls](CONTRIBUTING.md#join-a-sig-call) are every other
Monday at 8:30 AM PST and anyone is welcome.

## Project leadership

[Maintainers](https://github.com/open-telemetry/community/blob/main/guides/contributor/membership.md#maintainer)
([@open-telemetry/demo-maintainers](https://github.com/orgs/open-telemetry/teams/demo-maintainers)):

- [Juliano Costa](https://github.com/julianocosta89), Datadog
- [Mikko Viitanen](https://github.com/mviitane), Dynatrace
- [Pierre Tessier](https://github.com/puckpuck), Honeycomb

[Approvers](https://github.com/open-telemetry/community/blob/main/guides/contributor/membership.md#approver)
([@open-telemetry/demo-approvers](https://github.com/orgs/open-telemetry/teams/demo-approvers)):

- [Cedric Ziel](https://github.com/cedricziel) Grafana Labs
- [Penghan Wang](https://github.com/wph95), AppDynamics
- [Reiley Yang](https://github.com/reyang), Microsoft
- [Roger Coll](https://github.com/rogercoll), Elastic
- [Ziqi Zhao](https://github.com/fatsheep9146), Alibaba

Emeritus:

- [Austin Parker](https://github.com/austinlparker)
- [Carter Socha](https://github.com/cartersocha)
- [Michael Maxwell](https://github.com/mic-max)
- [Morgan McLean](https://github.com/mtwo)

### Thanks to all the people who have contributed

[![contributors](https://contributors-img.web.app/image?repo=open-telemetry/opentelemetry-demo)](https://github.com/open-telemetry/opentelemetry-demo/graphs/contributors)

[docs]: https://opentelemetry.io/docs/demo/

<!-- Links for Demos featuring the Astronomy Shop section -->

[AlibabaCloud]: https://github.com/aliyun-sls/opentelemetry-demo
[AppDynamics]: https://www.appdynamics.com/blog/cloud/how-to-observe-opentelemetry-demo-app-in-appdynamics-cloud/
[Aspecto]: https://github.com/aspecto-io/opentelemetry-demo
[Axiom]: https://play.axiom.co/axiom-play-qf1k/dashboards/otel.traces.otel-demo-traces
[Axoflow]: https://axoflow.com/opentelemetry-support-in-more-detail-in-axosyslog-and-syslog-ng/
[Azure]: https://github.com/Azure/Azure-kusto-opentelemetry-demo
[Coralogix]: https://coralogix.com/blog/configure-otel-demo-send-telemetry-data-coralogix
[Dash0]: https://github.com/dash0hq/opentelemetry-demo
[Datadog]: https://docs.datadoghq.com/opentelemetry/guide/otel_demo_to_datadog
[Dynatrace]: https://www.dynatrace.com/news/blog/opentelemetry-demo-application-with-dynatrace/
[Elastic]: https://github.com/elastic/opentelemetry-demo
[GoogleCloud]: https://github.com/GoogleCloudPlatform/opentelemetry-demo
[GrafanaLabs]: https://github.com/grafana/opentelemetry-demo
[Guance]: https://github.com/GuanceCloud/opentelemetry-demo
[Helios]: https://otelsandbox.gethelios.dev
[Honeycombio]: https://github.com/honeycombio/opentelemetry-demo
[Instana]: https://github.com/instana/opentelemetry-demo
[Kloudfuse]: https://github.com/kloudfuse/opentelemetry-demo
[Liatrio]: https://github.com/liatrio/opentelemetry-demo
[Logzio]: https://logz.io/learn/how-to-run-opentelemetry-demo-with-logz-io/
[NewRelic]: https://github.com/newrelic/opentelemetry-demo
[OpenSearch]: https://github.com/opensearch-project/opentelemetry-demo
[Sentry]: https://github.com/getsentry/opentelemetry-demo
[ServiceNowCloudObservability]: https://docs.lightstep.com/otel/quick-start-operator#send-data-from-the-opentelemetry-demo
[Splunk]: https://github.com/signalfx/opentelemetry-demo
[SumoLogic]: https://www.sumologic.com/blog/common-opentelemetry-demo-application/
[TelemetryHub]: https://github.com/TelemetryHub/opentelemetry-demo/tree/telemetryhub-backend
[Teletrace]: https://github.com/teletrace/opentelemetry-demo
[Tracetest]: https://github.com/kubeshop/opentelemetry-demo
[Uptrace]: https://github.com/uptrace/uptrace/tree/master/example/opentelemetry-demo
For the Kubernetes model there are two variants. The [first](./edmm/otel-store_k8s_translated.yaml) is the actual result of the translation, lacking _hosted_on_ relations.
That is, as the Kubernetes plugin only creates _hosted_on_ relations when a component named _container_runtime_ with a matching type is present in the TADM.
Therefore, there is a [second target model](./edmm/otel-store_k8s_translated_with_container_runtime.yaml) where this _container_runtime_ has been manually added, before creating the relations.
Thus, all components obtain a _hosted_on_ relation.
Loading

0 comments on commit a93fbb9

Please sign in to comment.