-
-
Notifications
You must be signed in to change notification settings - Fork 681
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
docs: case study from HDI Global SE #3010
Changes from 7 commits
80d8006
5208a9b
c8960fa
42a220b
d7c5475
6b4d9fb
43c9d45
0616344
d9aa032
153fe35
12a383f
43f217b
295cb4d
2c5d9e7
f9dd9be
edac993
ef00623
9350f09
1f8a794
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,162 @@ | ||
id: hdiglobal | ||
company: | ||
name: HDI Global SE | ||
description: The HDI brand operates in Germany and internationally, offering life and property/casualty insurance services. They cater to both private individuals and corporate clients, and have been providing industrial insurance since 2016. | ||
customers: 5000 | ||
industry: Insurance | ||
revenue: 9.1 Mrd. EUR | ||
website: https://www.hdi.global/ | ||
logo: /img/casestudies/hdi/hdi_logo.svg | ||
contact: | ||
- name: Vladislav Zwenihorodski | ||
link: https://www.linkedin.com/in/vladislav-zwenihorodski-9680a1209/ | ||
challenges: | | ||
The HDI has various platform teams, among them the Integration Platform Team, which offers three products: Azure API Management, Azure Event Hub, and the Azure Service Bus. | ||
For synchronous communication, we already use OpenAPI, as such we want to ensure the same level of transparency and discoverability for asynchronous scenarios as well. | ||
Our central platform team offers the Azure Service Bus with self-service capabilities. | ||
Customers are able to manage their own topics and subscriptions by simply modifying the according values in their repositories. | ||
This means that most of the data required to document asynchronous communication is already available, although distributed between different repositories. | ||
This necessitated the creation of a comprehensive catalog of existing topics, allowing potential subscribers easy access to messages from any topics of their choosing. | ||
|
||
|
||
solution: | | ||
The solution was to create an an AsyncAPI document where each topic owned by the customer is represented as a channel. | ||
This document is automatically generated by a pipeline whenever there's a change in the topic configuration. | ||
The necessary information is read from the customer repositories and then passed to a bash script as input. | ||
After successful creation, this file, along with a generated markdown for it, is saved within a documentation repository. | ||
This ensures that the information is readily accessible to every developer, allowing easy access to messages from any topics of their choosing. | ||
This approach makes our asynchronous communication is as transparent and discoverable as our synchronous communication. | ||
technical: #We need some more technical information related to case study. | ||
languages: | ||
- bash | ||
frameworks: | ||
- # cant be removed is expected to exist | ||
protocols: | ||
- AMQP | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. so AMQP is main protocol, and HTTPS is just a transport? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes exactly the primary protocol for Service Bus is AMQP |
||
testing: | | ||
The AsyncAPI document is validated using the AsyncAPI CLI to confirm that the document is correctly formatted and adheres to the AsyncAPI specification. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. it is more related to overall testing strategy for EDA, how do you approach it. info about validating AsyncAPI documents with CLI, please move it to |
||
architecture: | | ||
The following [enterprise integration patterns](https://www.enterpriseintegrationpatterns.com/patterns/messaging) are applied : | ||
- [Message Channel](https://www.enterpriseintegrationpatterns.com/patterns/messaging/MessageChannel.html) | ||
Each channel in the AsyncAPI document corresponds to an existing Service Bus topic that can be subscribed to. | ||
``` | ||
channels: | ||
helloworld-emea: | ||
servers: | ||
- $ref: '#/servers/box-emea' | ||
address: https://namespace.namespace/topic/example/helloworld | ||
``` | ||
- [Message](https://www.enterpriseintegrationpatterns.com/patterns/messaging/Message.html) | ||
The 'messages' section under each channel in the AsyncAPI document adheres to this pattern. | ||
Each message is identified by a name and contains a payload, which represents the data transferred between applications. | ||
``` | ||
messages: | ||
helloworld: | ||
name: helloworld-Message | ||
contentType: application/json | ||
payload: | ||
$ref: "#/components/schemas/helloworld-topic-v1.json" | ||
``` | ||
- [Document Message](https://www.enterpriseintegrationpatterns.com/patterns/messaging/DocumentMessage.html) | ||
The 'payload' under each message in the AsyncAPI document adheres to this pattern. | ||
The payload is a self-contained document that describes the message and is comprehensible to the message receiver. | ||
``` | ||
payload: | ||
$ref: "#/components/schemas/helloworld-topic-v1.json" | ||
``` | ||
In our repository structure, the message-schema file is stored separately and just referenced in the Topic configs. | ||
To make sure that all information is accessible in one place, the schema is directly copied to the AsyncAPI document. | ||
``` | ||
components: | ||
schemas: | ||
helloworld-topic-v1.json: { | ||
"$schema": "http://json-schema.org/draft-06/schema#", | ||
"type": "object", | ||
"properties": { | ||
"eventType": { | ||
"type": "object", | ||
"enum": [ | ||
"create", | ||
"update", | ||
"delete" | ||
] | ||
}, | ||
"hotel": { | ||
"type": "object", | ||
"properties": { | ||
"id": { | ||
"type": "string" | ||
}, | ||
"name": { | ||
"type": "string" | ||
}, | ||
"slogan": { | ||
"type": "string" | ||
}, | ||
"location": { | ||
"type": "string" | ||
}, | ||
"total_number_of_rooms": { | ||
"type": "integer" | ||
}, | ||
"currently_available_rooms": { | ||
"type": "integer" | ||
}, | ||
"currently_occupied_rooms": { | ||
"type": "integer" | ||
} | ||
}, | ||
"required": [ | ||
"id", | ||
"name" | ||
] | ||
} | ||
}, | ||
"required": [ | ||
"eventType", | ||
"hotel" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. hotel? just curious, the example document you share, does it correspond to HDI a bit? |
||
] | ||
} | ||
``` | ||
codegen: | | ||
Our team does not use code generation tools. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. it should be more about in general, not your team or customers, but basically HDI approach. If you know teams generate DTOs or even something more, them please share, and if some AsyncAPI tool is used, please share which There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. more important is that you as a team do not have it as a service, like you for example do not provide a maven plugin or pipelines, or anything that would make it easier for teams to get DTOs |
||
However, our customers are free to use any tool they prefer to generate DTOs from the AsyncAPI document. | ||
schemas: #It is useful and interesting to learn how teams handle messages schemas. | ||
description: JSON Schema | ||
storage: A Git repository functions as a self-service portal where customers can manage their own configurations for the Azure Service Bus. | ||
This includes defining their own message schemas for any topics they own. | ||
registry: none | ||
versioning: The customer has the freedom to choose the versioning for their own message-schema files. | ||
validation: Validation using [Ajv](https://ajv.js.org/), is used solely for the purpose of validating that the JSON data adheres to the expected schema. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. can you specify if validation takes place in runtime, and also if it is done by consumers only or producers as well? |
||
It does not perform any other form of validation or processing of the data. | ||
asyncapi: #More specific details about AsyncAPI itself and how is it used. | ||
usecase: | | ||
- The AsyncAPI file is used for documentation purposes by the platform team. | ||
vlazw marked this conversation as resolved.
Show resolved
Hide resolved
|
||
It provides a comprehensive overview of the asynchronous communication in our system, including the available topics and the structure of the messages. | ||
- The AsyncAPI document serves as a source for generating data transfer objects (DTOs) in Java, for example, by using the AsyncAPI Java generator. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. can you specify which java generator, maybe a link? and also what is more or less the workflow. So team that wants to consume messages downloads the file in a pipeline and generate DTO? would be nice to know details |
||
These DTOs are used in the code to ensure type safety and to provide a clear understanding of the data structure. | ||
versions: | ||
- 3.0 | ||
storage: | | ||
A seperate Git repository functions as a "customer Wiki", serving as a central location for all relevant documents about the existing infrastructure. | ||
This includes the AsyncAPI document, which provides a comprehensive overview of the asynchronous communication in our system. | ||
editing: | | ||
If changes for any topic are applied by a customer, the documentation pipeline is triggered. | ||
This pipeline uses a bash script to read the updated configuration from the customer's repository. | ||
It then generates a new AsyncAPI file reflecting these changes. | ||
vlazw marked this conversation as resolved.
Show resolved
Hide resolved
|
||
maintainers: | | ||
No specific maintainers are assigned to the AsyncAPI document. The document is automatically generated by a pipeline. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. can you also elaborate more, here or in |
||
audience: #Specify if AsyncAPI audience is internal or external or both. | ||
internal: true | ||
external: false | ||
extensions: | | ||
none | ||
documentation: | | ||
Documentation is generated via AsyncAPI CLI and published to the wiki right after the AsyncAPI file is generated. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. when you write wiki, the central repo with all AsyncAPI docs and generated markdown - do you use GitHub Wiki solution to render these markdowns? or? why you call it wiki? |
||
bindings: | | ||
none | ||
tools: | | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. can you add CLI to the list please? and also code generators that you use, which ones |
||
* [AsyncAPI Generator](https://github.com/asyncapi/generator): | ||
* [markdown-template](https://github.com/asyncapi/markdown-template). | ||
fullExample: resources/casestudies/hdi/asyncapi.yaml | ||
additionalResources: #Provide some additional resources if there are such. Just block of text with links to articles or videos about the case study. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,151 @@ | ||
asyncapi: 3.0.0 | ||
info: | ||
title: customer-example | ||
version: 1.0.0 | ||
description: | | ||
This is an AsyncAPI document for customer-example. | ||
It contains every Topic owned by the customer in form of channel. | ||
Dowload the coresponding schema files from the following link: | ||
servers: | ||
box-apac: | ||
host: namespace.servicebus.windows.net | ||
protocol: amqp | ||
description: Azure Service Bus namespace endpoint for box. | ||
box-emea: | ||
host: namespace.servicebus.windows.net | ||
protocol: amqp | ||
description: Azure Service Bus namespace endpoint for box. | ||
channels: | ||
helloworld-emea: | ||
servers: | ||
- $ref: '#/servers/box-emea' | ||
address: https://namespace.servicebus.windows.net/topic/example/helloworld | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. can you elaborate a bit more about the address please, so I understand how things work on your end - I don't have much experience with Azure Service Bus so in servers, you specify host as |
||
messages: | ||
helloworld: | ||
name: helloworld-Message | ||
contentType: application/json | ||
payload: | ||
$ref: '#/components/schemas/helloworld-topic-v1.json' | ||
test-topic-emea: | ||
servers: | ||
- $ref: '#/servers/box-emea' | ||
address: https://namespace.servicebus.windows.net/topic/example/test-topic | ||
messages: | ||
test-topic: | ||
name: test-topic-Message | ||
contentType: application/json | ||
payload: | ||
$ref: '#/components/schemas/test-topic-v1.json' | ||
test-pipeline-emea: | ||
servers: | ||
- $ref: '#/servers/box-emea' | ||
address: https://namespace.servicebus.windows.net/topic/example/test-pipeline | ||
messages: | ||
test-pipeline: | ||
name: test-pipeline-Message | ||
contentType: application/json | ||
payload: | ||
$ref: '#/components/schemas/test-topic-v1.json' | ||
source1-emea: | ||
servers: | ||
- $ref: '#/servers/box-emea' | ||
address: https://namespace.servicebus.windows.net/topic/example/source1 | ||
messages: | ||
source1: | ||
name: source1-Message | ||
contentType: application/json | ||
payload: | ||
$ref: '#/components/schemas/test-topic-v1.json' | ||
operations: | ||
derberg marked this conversation as resolved.
Show resolved
Hide resolved
|
||
helloworld-emea: | ||
action: send | ||
channel: | ||
$ref: '#/channels/helloworld-emea' | ||
messages: | ||
- $ref: '#/channels/helloworld-emea/messages/helloworld' | ||
test-topic-emea: | ||
action: send | ||
channel: | ||
$ref: '#/channels/test-topic-emea' | ||
messages: | ||
- $ref: '#/channels/test-topic-emea/messages/test-topic' | ||
test-pipeline-emea: | ||
action: send | ||
channel: | ||
$ref: '#/channels/test-pipeline-emea' | ||
messages: | ||
- $ref: '#/channels/test-pipeline-emea/messages/test-pipeline' | ||
source1-emea: | ||
action: send | ||
channel: | ||
$ref: '#/channels/source1-emea' | ||
messages: | ||
- $ref: '#/channels/source1-emea/messages/source1' | ||
components: | ||
schemas: | ||
test-topic-v1.json: | ||
$schema: http://json-schema.org/draft-06/schema# | ||
type: object | ||
properties: | ||
eventType: | ||
type: object | ||
enum: | ||
- create | ||
- update | ||
- delete | ||
hotel: | ||
type: object | ||
properties: | ||
id: | ||
type: string | ||
name: | ||
type: string | ||
slogan: | ||
type: string | ||
location: | ||
type: string | ||
total_number_of_rooms: | ||
type: integer | ||
currently_available_rooms: | ||
type: integer | ||
currently_occupied_rooms: | ||
type: integer | ||
required: | ||
- id | ||
- name | ||
required: | ||
- eventType | ||
- hotel | ||
helloworld-topic-v1.json: | ||
$schema: http://json-schema.org/draft-06/schema# | ||
type: object | ||
properties: | ||
eventType: | ||
type: object | ||
enum: | ||
- create | ||
- update | ||
- delete | ||
hotel: | ||
type: object | ||
properties: | ||
id: | ||
type: string | ||
name: | ||
type: string | ||
slogan: | ||
type: string | ||
location: | ||
type: string | ||
total_number_of_rooms: | ||
type: integer | ||
currently_available_rooms: | ||
type: integer | ||
currently_occupied_rooms: | ||
type: integer | ||
required: | ||
- id | ||
- name | ||
required: | ||
- eventType | ||
- hotel |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
languages are not only for the language used to provide your solution to a problem. It is more about entire EDA solution, what programming languages you use. I guess it is Java, right? then please change that