Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix: need to speed up testing for confluent #1638

Closed
Flosckow opened this issue Aug 4, 2024 · 1 comment · Fixed by #1635
Closed

Fix: need to speed up testing for confluent #1638

Flosckow opened this issue Aug 4, 2024 · 1 comment · Fixed by #1635
Assignees
Labels
Confluent Issues related to `faststream.confluent` module enhancement New feature or request

Comments

@Flosckow
Copy link
Contributor

Flosckow commented Aug 4, 2024

To suggest an idea or inquire about a new Message Broker supporting feature or any other enhancement, please follow this template:

Is your feature request related to a problem? Please describe.
Slow running tests for confluent

Describe the solution you'd like
Significantly speed up test execution.

@Flosckow Flosckow added the enhancement New feature or request label Aug 4, 2024
@Lancetnik Lancetnik added the Confluent Issues related to `faststream.confluent` module label Aug 4, 2024
@Lancetnik Lancetnik moved this to In Progress in FastStream Aug 4, 2024
@Lancetnik
Copy link
Member

Finally, I found the problem - confluent-kafka takes a lot of time at broker starting dur automatic partition assignment:

So, for this code the first message will be consumed after about 3 seconds after startup

consumer.subscribe(["test-in",])

while True:
    msg = consumer.poll(timeout=0.1)

But, in the next case the message will be delivired immideately

consumer.assign([TopicPartition(topic="test-in", partition=0, offset=0)])

while True:
    msg = consumer.poll(timeout=0.1)

I think, we should use manual partition assignment for aiokafka and confluent kafka both in tests this reason

@kumaranvpl what do you think? Have you any other solutions?

  • refactor tests to provide custom args, kwargs to broker.subscriber test
  • impl manual TopicPartition assignment for confluent

@Lancetnik Lancetnik linked a pull request Aug 7, 2024 that will close this issue
13 tasks
@Lancetnik Lancetnik mentioned this issue Aug 7, 2024
13 tasks
@github-project-automation github-project-automation bot moved this from In Progress to Done in FastStream Aug 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Confluent Issues related to `faststream.confluent` module enhancement New feature or request
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

2 participants