Skip to content

Commit

Permalink
fix: check if Kafka columns in writer are *exactly* {key, value}
Browse files Browse the repository at this point in the history
Co-authored-by: Reza Khanipour <[email protected]>
  • Loading branch information
wtfzambo and shahinism authored Sep 27, 2024
1 parent df76a64 commit bb87018
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion src/sparkle/writer/kafka_writer.py
Original file line number Diff line number Diff line change
Expand Up @@ -183,7 +183,7 @@ def write(self, df: DataFrame) -> None:
# Convert the DataFrame to a Kafka-friendly format
kafka_df = to_kafka_dataframe(self.unique_identifier_column_name, df)

if "key" not in kafka_df.columns or "value" not in kafka_df.columns:
if set(kafka_df.columns) != {"key", "value"}
raise KeyError(
"The DataFrame must contain 'key' and 'value' columns. "
"Ensure that `to_kafka_dataframe` transformation is correctly applied."
Expand Down

0 comments on commit bb87018

Please sign in to comment.