Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Apache Kafka Event Interface For Repositories #523

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

geso02
Copy link
Contributor

@geso02 geso02 commented Nov 11, 2024

#522

Description:

This pull request introduces an Apache Kafka event interface, enhancing message processing reliability and scalability, and making the system more suitable for larger and more demanding environments.

Key Features

  • Kafka Integration: Adds Apache Kafka as a message broker, allowing both repositories and registries to utilize Kafka for event processing.
  • Commit-Based Acknowledgments: Implements Kafka’s commit-based message acknowledgment to ensure reliable and confirmed message processing.
  • Documentation: Updated README files with detailed instructions on setting up and configuring the Kafka event interface.
  • Unit Tests: Comprehensive unit tests are included to verify the functionality and reliability of the Kafka integration, ensuring robustness under various conditions.

Thank you for reviewing this addition!

@aaronzi
Copy link
Member

aaronzi commented Nov 12, 2024

Hi @geso02,

thank you very much for your PR and the nice addition to BaSyx. Can you please check the AAS Environment. There is a test that seems to be failing.

- Corrected automatic upload of files by ensuring paths are annotated with @value, so the correct files are uploaded as expected.
- Removed the `basyx.environment` property from tests, as it is unrelated to the test scope and streamlines the configuration.
@geso02
Copy link
Contributor Author

geso02 commented Nov 14, 2024

I found some incorrect configurations in the upload within the AASEnvironment. The pathsToLoad used during bean creation was missing the @value annotation in all places. I've now added it. In the authorization test case, we then encountered an issue because basyx.environment was now being referenced, but this test was not concerned with it and did not provide mocks for it, causing it to fail. I've now pushed the changes again and will check if there are issues in other areas as well.

This commit resolves an issue where a Submodel, which should only be used in submodel-repository tests, was mistakenly registered in environment-related tests.

The Submodel is now conditionally provided via profiles, ensuring it is only available in the appropriate tests and no longer causes failures in the environment tests.
@geso02
Copy link
Contributor Author

geso02 commented Nov 14, 2024

I’ve fixed the issue with the build by adjusting the registration of a Submodel that was intended for submodel-related tests but was mistakenly included in environment tests due to test dependencies. I’ve now ensured that the Submodel is only registered in the submodel tests using profiles, and it’s no longer active in the environment tests, which resolves the issue and allows the build to pass again.

@geso02 geso02 changed the title Add Apache Kafka Event Interface For Repositories [DRAFT] Add Apache Kafka Event Interface For Repositories Nov 15, 2024
@geso02 geso02 marked this pull request as draft November 15, 2024 10:43
basyx environment is not assigned in auth preloader otherwise
@geso02 geso02 changed the title [DRAFT] Add Apache Kafka Event Interface For Repositories Add Apache Kafka Event Interface For Repositories Nov 15, 2024
@geso02 geso02 marked this pull request as ready for review November 15, 2024 12:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants