Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: case study from HDI Global SE #3010

Merged
merged 19 commits into from
Aug 1, 2024
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
162 changes: 162 additions & 0 deletions config/casestudies/hdi.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
id: hdiglobal
company:
name: HDI Global SE
description: The HDI brand operates in Germany and internationally, offering life and property/casualty insurance services. They cater to both private individuals and corporate clients, and have been providing industrial insurance since 2016.
customers: 5000
industry: Insurance
revenue: 9.1 Mrd. EUR
website: https://www.hdi.global/
logo: /img/casestudies/hdi/hdi_logo.svg
contact:
- name: Vladislav Zwenihorodski
link: https://www.linkedin.com/in/vladislav-zwenihorodski-9680a1209/
challenges: |
The HDI has various platform teams, among them the Integration Platform Team, which offers three products: Azure API Management, Azure Event Hub, and the Azure Service Bus.
For synchronous communication, we already use OpenAPI, as such we want to ensure the same level of transparency and discoverability for asynchronous scenarios as well.
Our central platform team offers the Azure Service Bus with self-service capabilities.
Customers are able to manage their own topics and subscriptions by simply modifying the according values in their repositories.
This means that most of the data required to document asynchronous communication is already available, although distributed between different repositories.
This necessitated the creation of a comprehensive catalog of existing topics, allowing potential subscribers easy access to messages from any topics of their choosing.


solution: |
The solution was to create an an AsyncAPI document where each topic owned by the customer is represented as a channel.
This document is automatically generated by a pipeline whenever there's a change in the topic configuration.
The necessary information is read from the customer repositories and then passed to a bash script as input.
After successful creation, this file, along with a generated markdown for it, is saved within a documentation repository.
This ensures that the information is readily accessible to every developer, allowing easy access to messages from any topics of their choosing.
This approach makes our asynchronous communication is as transparent and discoverable as our synchronous communication.
technical: #We need some more technical information related to case study.
languages:
- bash
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

languages are not only for the language used to provide your solution to a problem. It is more about entire EDA solution, what programming languages you use. I guess it is Java, right? then please change that

frameworks:
- # cant be removed is expected to exist
protocols:
- AMQP
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so AMQP is main protocol, and HTTPS is just a transport?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes exactly the primary protocol for Service Bus is AMQP

testing: |
The AsyncAPI document is validated using the AsyncAPI CLI to confirm that the document is correctly formatted and adheres to the AsyncAPI specification.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is more related to overall testing strategy for EDA, how do you approach it.

info about validating AsyncAPI documents with CLI, please move it to Solution section, where you can specify if the check happens in a pipeline or where exactly?

architecture: |
The following [enterprise integration patterns](https://www.enterpriseintegrationpatterns.com/patterns/messaging) are applied :
- [Message Channel](https://www.enterpriseintegrationpatterns.com/patterns/messaging/MessageChannel.html)
Each channel in the AsyncAPI document corresponds to an existing Service Bus topic that can be subscribed to.
```
channels:
helloworld-emea:
servers:
- $ref: '#/servers/box-emea'
address: https://namespace.namespace/topic/example/helloworld
```
- [Message](https://www.enterpriseintegrationpatterns.com/patterns/messaging/Message.html)
The 'messages' section under each channel in the AsyncAPI document adheres to this pattern.
Each message is identified by a name and contains a payload, which represents the data transferred between applications.
```
messages:
helloworld:
name: helloworld-Message
contentType: application/json
payload:
$ref: "#/components/schemas/helloworld-topic-v1.json"
```
- [Document Message](https://www.enterpriseintegrationpatterns.com/patterns/messaging/DocumentMessage.html)
The 'payload' under each message in the AsyncAPI document adheres to this pattern.
The payload is a self-contained document that describes the message and is comprehensible to the message receiver.
```
payload:
$ref: "#/components/schemas/helloworld-topic-v1.json"
```
In our repository structure, the message-schema file is stored separately and just referenced in the Topic configs.
To make sure that all information is accessible in one place, the schema is directly copied to the AsyncAPI document.
```
components:
schemas:
helloworld-topic-v1.json: {
"$schema": "http://json-schema.org/draft-06/schema#",
"type": "object",
"properties": {
"eventType": {
"type": "object",
"enum": [
"create",
"update",
"delete"
]
},
"hotel": {
"type": "object",
"properties": {
"id": {
"type": "string"
},
"name": {
"type": "string"
},
"slogan": {
"type": "string"
},
"location": {
"type": "string"
},
"total_number_of_rooms": {
"type": "integer"
},
"currently_available_rooms": {
"type": "integer"
},
"currently_occupied_rooms": {
"type": "integer"
}
},
"required": [
"id",
"name"
]
}
},
"required": [
"eventType",
"hotel"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hotel? just curious, the example document you share, does it correspond to HDI a bit?

]
}
```
codegen: |
Our team does not use code generation tools.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it should be more about in general, not your team or customers, but basically HDI approach. If you know teams generate DTOs or even something more, them please share, and if some AsyncAPI tool is used, please share which

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

more important is that you as a team do not have it as a service, like you for example do not provide a maven plugin or pipelines, or anything that would make it easier for teams to get DTOs

However, our customers are free to use any tool they prefer to generate DTOs from the AsyncAPI document.
schemas: #It is useful and interesting to learn how teams handle messages schemas.
description: JSON Schema
storage: A Git repository functions as a self-service portal where customers can manage their own configurations for the Azure Service Bus.
This includes defining their own message schemas for any topics they own.
registry: none
versioning: The customer has the freedom to choose the versioning for their own message-schema files.
validation: Validation using [Ajv](https://ajv.js.org/), is used solely for the purpose of validating that the JSON data adheres to the expected schema.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you specify if validation takes place in runtime, and also if it is done by consumers only or producers as well?

It does not perform any other form of validation or processing of the data.
asyncapi: #More specific details about AsyncAPI itself and how is it used.
usecase: |
- The AsyncAPI file is used for documentation purposes by the platform team.
vlazw marked this conversation as resolved.
Show resolved Hide resolved
It provides a comprehensive overview of the asynchronous communication in our system, including the available topics and the structure of the messages.
- The AsyncAPI document serves as a source for generating data transfer objects (DTOs) in Java, for example, by using the AsyncAPI Java generator.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you specify which java generator, maybe a link?

and also what is more or less the workflow. So team that wants to consume messages downloads the file in a pipeline and generate DTO? would be nice to know details

These DTOs are used in the code to ensure type safety and to provide a clear understanding of the data structure.
versions:
- 3.0
storage: |
A seperate Git repository functions as a "customer Wiki", serving as a central location for all relevant documents about the existing infrastructure.
This includes the AsyncAPI document, which provides a comprehensive overview of the asynchronous communication in our system.
editing: |
If changes for any topic are applied by a customer, the documentation pipeline is triggered.
This pipeline uses a bash script to read the updated configuration from the customer's repository.
It then generates a new AsyncAPI file reflecting these changes.
vlazw marked this conversation as resolved.
Show resolved Hide resolved
maintainers: |
No specific maintainers are assigned to the AsyncAPI document. The document is automatically generated by a pipeline.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you also elaborate more, here or in Solution section about the custom format you have, for specifying topics. To explain where AsyncAPI document is generated from, what is the source of information, what comes from custom format, what from code - and who maintains these sources

audience: #Specify if AsyncAPI audience is internal or external or both.
internal: true
external: false
extensions: |
none
documentation: |
Documentation is generated via AsyncAPI CLI and published to the wiki right after the AsyncAPI file is generated.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when you write wiki,

the central repo with all AsyncAPI docs and generated markdown - do you use GitHub Wiki solution to render these markdowns? or? why you call it wiki?

bindings: |
none
tools: |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you add CLI to the list please? and also code generators that you use, which ones

* [AsyncAPI Generator](https://github.com/asyncapi/generator):
* [markdown-template](https://github.com/asyncapi/markdown-template).
fullExample: resources/casestudies/hdi/asyncapi.yaml
additionalResources: #Provide some additional resources if there are such. Just block of text with links to articles or videos about the case study.
19 changes: 19 additions & 0 deletions public/img/casestudies/hdi/hdi_logo.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
151 changes: 151 additions & 0 deletions public/resources/casestudies/hdi/asyncapi.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
asyncapi: 3.0.0
info:
title: customer-example
version: 1.0.0
description: |
This is an AsyncAPI document for customer-example.
It contains every Topic owned by the customer in form of channel.
Dowload the coresponding schema files from the following link:
servers:
box-apac:
host: namespace.servicebus.windows.net
protocol: amqp
description: Azure Service Bus namespace endpoint for box.
box-emea:
host: namespace.servicebus.windows.net
protocol: amqp
description: Azure Service Bus namespace endpoint for box.
channels:
helloworld-emea:
servers:
- $ref: '#/servers/box-emea'
address: https://namespace.servicebus.windows.net/topic/example/helloworld
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you elaborate a bit more about the address please, so I understand how things work on your end - I don't have much experience with Azure Service Bus

so in servers, you specify host as namespace.servicebus.windows.net and protocol is amqp but in channel address you have a link using HTTPS scheme. Can you share why it works this way?

messages:
helloworld:
name: helloworld-Message
contentType: application/json
payload:
$ref: '#/components/schemas/helloworld-topic-v1.json'
test-topic-emea:
servers:
- $ref: '#/servers/box-emea'
address: https://namespace.servicebus.windows.net/topic/example/test-topic
messages:
test-topic:
name: test-topic-Message
contentType: application/json
payload:
$ref: '#/components/schemas/test-topic-v1.json'
test-pipeline-emea:
servers:
- $ref: '#/servers/box-emea'
address: https://namespace.servicebus.windows.net/topic/example/test-pipeline
messages:
test-pipeline:
name: test-pipeline-Message
contentType: application/json
payload:
$ref: '#/components/schemas/test-topic-v1.json'
source1-emea:
servers:
- $ref: '#/servers/box-emea'
address: https://namespace.servicebus.windows.net/topic/example/source1
messages:
source1:
name: source1-Message
contentType: application/json
payload:
$ref: '#/components/schemas/test-topic-v1.json'
operations:
derberg marked this conversation as resolved.
Show resolved Hide resolved
helloworld-emea:
action: send
channel:
$ref: '#/channels/helloworld-emea'
messages:
- $ref: '#/channels/helloworld-emea/messages/helloworld'
test-topic-emea:
action: send
channel:
$ref: '#/channels/test-topic-emea'
messages:
- $ref: '#/channels/test-topic-emea/messages/test-topic'
test-pipeline-emea:
action: send
channel:
$ref: '#/channels/test-pipeline-emea'
messages:
- $ref: '#/channels/test-pipeline-emea/messages/test-pipeline'
source1-emea:
action: send
channel:
$ref: '#/channels/source1-emea'
messages:
- $ref: '#/channels/source1-emea/messages/source1'
components:
schemas:
test-topic-v1.json:
$schema: http://json-schema.org/draft-06/schema#
type: object
properties:
eventType:
type: object
enum:
- create
- update
- delete
hotel:
type: object
properties:
id:
type: string
name:
type: string
slogan:
type: string
location:
type: string
total_number_of_rooms:
type: integer
currently_available_rooms:
type: integer
currently_occupied_rooms:
type: integer
required:
- id
- name
required:
- eventType
- hotel
helloworld-topic-v1.json:
$schema: http://json-schema.org/draft-06/schema#
type: object
properties:
eventType:
type: object
enum:
- create
- update
- delete
hotel:
type: object
properties:
id:
type: string
name:
type: string
slogan:
type: string
location:
type: string
total_number_of_rooms:
type: integer
currently_available_rooms:
type: integer
currently_occupied_rooms:
type: integer
required:
- id
- name
required:
- eventType
- hotel
Loading