0.6.4

v0.6.4 (2024-11-07)

Refactor

• use Cachix
• use Cachix
• add cache to CI

[main 259c251] bump: version 0.6.3 → 0.6.4
3 files changed, 11 insertions(+), 3 deletions(-)

0.6.3

v0.6.3 (2024-10-18)

Fix

• include utils as a package

[main aa59f23] bump: version 0.6.2 → 0.6.3
3 files changed, 9 insertions(+), 3 deletions(-)

0.6.2

v0.6.2 (2024-10-16)

Refactor

• apply ruff fix and format so CI tests pass
• use ruff for lint and formatting (remove black)

[main 811841f] bump: version 0.6.1 → 0.6.2
3 files changed, 10 insertions(+), 3 deletions(-)

0.6.1

v0.6.1 (2024-10-03)

Fix

• KafkaReader and TableReader should inherit from base Reader class

Refactor

• accept type[Writer] for main Sparkle interface

[main 7c59a62] bump: version 0.6.0 → 0.6.1
3 files changed, 13 insertions(+), 3 deletions(-)

0.6.0

v0.6.0 (2024-10-03)

Feat

• kafkaWriter: add batch publisher
• add chispa pkg for easy spark DF comparisons
• add kafka-ui service so I can see what goes on

Fix

• add missing colon
• check if Kafka columns in writer are exactly {key, value}

[main 65be59d] bump: version 0.5.1 → 0.6.0
3 files changed, 16 insertions(+), 3 deletions(-)

0.5.1

v0.5.1 (2024-10-02)

Fix

• regression on #21.
• package version in the package.

[main 36a2c91] bump: version 0.5.0 → 0.5.1
3 files changed, 10 insertions(+), 3 deletions(-)

0.5.0

v0.5.0 (2024-09-30)

Fix

• not supported environment exception

Refactor

• rename kafka_options to kafka_spark_options
• rename local to generic
• accept readers in Sparkle class
• delete local spark session
• Spark configuration
• let Sparke create Spark session

[main df02179] bump: version 0.4.0 → 0.5.0
2 files changed, 16 insertions(+), 1 deletion(-)

0.4.0

v0.4.0 (2024-09-16)

Feat

• wait for schema registry to become available
• add Kafka Avro reader
• add Kafka Avro reader
• add avro parser
• add schema registry class

Fix

• start from earliest checkpoint
• use colima on mac
• wait for schema registry to be ready before running tests

Refactor

• schema type to config
• Switch to Poetry from PDM

[main 2cb5e1c] bump: version 0.3.1 → 0.4.0
2 files changed, 22 insertions(+), 1 deletion(-)

0.3.1

v0.3.1 (2024-09-11)

Fix

• spark: lock package version, to ensure they match plugins.
• typing: error "AnalysisException" has no attribute "desc"

Refactor

• test: run development services using devenv.
• switch to poetry.

[main 48627c8] bump: version 0.3.0 → 0.3.1
2 files changed, 13 insertions(+), 1 deletion(-)

0.3.0

v0.3.0 (2024-09-10)

Feat

• Add Kafka stream publisher
• add dataframe to kafka compatible dataframe transforner

[main 5ec6089] bump: version 0.2.0 → 0.3.0
2 files changed, 8 insertions(+), 1 deletion(-)