Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FLINK-21373] Add RabbitMQ SinkV2 Implementation, Port Flink version to Flink 1.19 #29

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

[FLINK-21373] Add RabbitMQ SinkV2 Implementation, Port Flink version to Flink 1.19 #29

wants to merge 1 commit into from

Conversation

vahmed-hamdy
Copy link

@vahmed-hamdy vahmed-hamdy commented May 2, 2024 •
edited
Loading

Purpose of the change

Add SinkV2 implemetation for RabbitMQ.

Verifying this change

  • Added unit tests for respective new implementations
  • Added IT tests (moved to follow up PR to reduce PR size).
  • Manually verified against local RabbitMQ cluster

Significant changes

(Please check any boxes [x] if the answer is "yes". You can first publish the PR and check them afterwards, for convenience.)

  • Dependencies have been added or upgraded
  • Public API has been changed (Public API is any class annotated with @Public(Evolving))
  • Serializers have been changed
  • New feature has been introduced
    • If yes, how is this documented? Follow up with another JIRA to update docs

@boring-cyborg boring-cyborg
Copy link

boring-cyborg bot commented May 2, 2024

Thanks for opening this pull request! Please check out our contributing guidelines. (https://flink.apache.org/contributing/how-to-contribute.html)

@vahmed-hamdy vahmed-hamdy force-pushed the flink-21373-rabbitmq-sink branch 6 times, most recently from b519ef2 to 0b3fd4b Compare May 3, 2024 08:21
@vahmed-hamdy
Copy link
Author

@MartijnVisser @zentol could you please review?

This was referenced May 4, 2024
Open
Closed
morazow
morazow suggested changes May 15, 2024
View reviewed changes
Copy link

@morazow morazow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @vahmed-hamdy, great work!

Added some suggestions

...k-connector-rabbitmq/src/main/java/org/apache/flink/connector/rabbitmq/common/Constants.java Show resolved Hide resolved
...bitmq/src/main/java/org/apache/flink/connector/rabbitmq/common/RabbitMQConnectionConfig.java Show resolved Hide resolved
...bitmq/src/main/java/org/apache/flink/connector/rabbitmq/common/RabbitMQConnectionConfig.java Show resolved Hide resolved
...bitmq/src/main/java/org/apache/flink/connector/rabbitmq/common/RabbitMQConnectionConfig.java Outdated Show resolved Hide resolved
...c/main/java/org/apache/flink/connector/rabbitmq/common/RabbitMQConnectionSetupException.java Show resolved Hide resolved
...ctor-rabbitmq/src/main/java/org/apache/flink/connector/rabbitmq/sink/RabbitMQSinkWriter.java Outdated Show resolved Hide resolved
...ctor-rabbitmq/src/main/java/org/apache/flink/connector/rabbitmq/sink/RabbitMQSinkWriter.java Show resolved Hide resolved
...ctor-rabbitmq/src/main/java/org/apache/flink/connector/rabbitmq/sink/RabbitMQSinkWriter.java Outdated Show resolved Hide resolved
...ctor-rabbitmq/src/main/java/org/apache/flink/connector/rabbitmq/sink/RabbitMQSinkWriter.java Show resolved Hide resolved
...connector-rabbitmq/src/main/java/org/apache/flink/streaming/connectors/rabbitmq/RMQSink.java Show resolved Hide resolved
@vahmed-hamdy vahmed-hamdy requested a review from morazow May 21, 2024 16:05
morazow
morazow approved these changes May 22, 2024
View reviewed changes
Copy link

@morazow morazow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @vahmed-hamdy, looks good 👍

...ctor-rabbitmq/src/main/java/org/apache/flink/connector/rabbitmq/sink/RabbitMQSinkWriter.java Show resolved Hide resolved
@paulh86
Copy link

paulh86 commented Jul 3, 2024

Any updates?

@vahmed-hamdy
Copy link
Author

@paulh86 Blocked on review from the community, will reach out on dev slack channel for a reviewer

@rjbaucells
Copy link

Hello,

What is the status of this PR?

Thanks

@vahmed-hamdy
Copy link
Author

@rjbaucells Hello, It is pending approval from @MartijnVisser

@amritesh007
Copy link

Hi, are there any plans to merge this PR. Since, the SourceFunction is deprecated in the FLINK and is being replaced with SOURCE API this PR is having the latest code to migrate the connector to use Source and Sink API.

@grahambunce
Copy link

grahambunce commented Oct 27, 2024 •
edited
Loading

Any further update?

From what I understand, the 1.17 version of the Flink connector does not work with 1.18, 1.19 or 1.20 so anybody wanting to connect Flink to Rabbit is stuck on 1.17.

According to the Flink docs anyway here

https://nightlies.apache.org/flink/flink-docs-release-1.18/docs/connectors/datastream/rabbitmq/

lvyanquan
lvyanquan reviewed Oct 28, 2024
View reviewed changes
pom.xml
@@ -48,7 +48,7 @@ under the License.
</modules>

<properties>
<flink.version>1.16.0</flink.version>
<flink.version>1.19.0</flink.version>
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see that we have release 3.0.1-1.17 & 1.16 in maven repo, This PR will bump the version to 1.19. What is our release plan for 1.18 and will we skip this version

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lvyanquan lvyanquan lvyanquan left review comments

@morazow morazow morazow approved these changes

Assignees
No one assigned
Labels
component=BuildSystem
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@vahmed-hamdy @paulh86 @rjbaucells @amritesh007 @grahambunce @morazow @lvyanquan