Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fixes #537: Kafka sink connector does not work with CUD ingestion strategy and Avro format data #582

Merged
merged 2 commits into from
Sep 1, 2023

Conversation

conker84
Copy link
Contributor

Fixes #537

The bug was related to the fact that the strategy expects a Map<String, Any?> (with the raw values) instead it gets a Map<String, Value>. I changed the converter in order to return Map<String, Any?> (with the raw values)

Proposed Changes (Mandatory)

A brief list of proposed changes in order to fix the issue:

  • fixed the bug
  • implemented test

Copy link
Contributor

@ali-ince ali-ince left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just a couple of minor things.

@@ -1351,6 +1353,54 @@ class Neo4jSinkTaskTest {
}
}

@Test
fun `should successfully parse nested timestamps`() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this just timestamps, or more general sub-maps?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is general sub-maps

}

@Test
fun `should be able to process a CUD format AVRO structure`() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we also include a test for relationship CUD format?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Contributor

@ali-ince ali-ince left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@conker84 conker84 merged commit 60b1851 into neo4j-contrib:5.0 Sep 1, 2023
3 checks passed
fbiville pushed a commit to neo4j/neo4j-kafka-connector that referenced this pull request Nov 3, 2023
Fix Kafka sink connector to work with CUD ingestion strategy and Avro format data.

See neo4j-contrib/neo4j-streams#582
fbiville pushed a commit to neo4j/neo4j-kafka-connector that referenced this pull request Nov 3, 2023
Fix Kafka sink connector to work with CUD ingestion strategy and Avro format data.

See neo4j-contrib/neo4j-streams#582
fbiville pushed a commit to neo4j/neo4j-kafka-connector that referenced this pull request Nov 3, 2023
Fix Kafka sink connector to work with CUD ingestion strategy and Avro format data.

See neo4j-contrib/neo4j-streams#582
fbiville pushed a commit to neo4j/neo4j-kafka-connector that referenced this pull request Nov 3, 2023
Fix Kafka sink connector to work with CUD ingestion strategy and Avro format data.

See neo4j-contrib/neo4j-streams#582
ali-ince pushed a commit to neo4j/neo4j-kafka-connector that referenced this pull request Nov 3, 2023
Fix Kafka sink connector to work with CUD ingestion strategy and Avro format data.

See neo4j-contrib/neo4j-streams#582

Co-authored-by: Andrea Santurbano <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Kafka sink connector does not work with CUD ingestion strategy and Avro format data
2 participants