0.1.7

What's Changed

• Improve drop and whitelist performance by @BrendBraeckmans in #11

Full Changelog: 0.1.6...0.1.7

0.1.6

What's Changed

• Minor documentation improvements by @BrendBraeckmans in #10
• Fix/add field when ancestor is null by @BrendBraeckmans in #9

Full Changelog: 0.1.5...0.1.6

0.1.5

What's Changed

• Fix badge in README pypi by @BrendBraeckmans

Full Changelog: 0.1.4...0.1.5

0.1.4

What's Changed

• Fix authors on pypi by @BrendBraeckmans in #8

Full Changelog: 0.1.3...0.1.4

0.1.3

What's Changed

• Improve whitelist performance by @BrendBraeckmans in #6
• Make it easier to add new nested fields with expr by not needing to specify an existing but unrelated field by @BrendBraeckmans in #7
• Rename apply_add_operation to add_nested_field as this is more clear from a user perspective by @BrendBraeckmans in #7

Full Changelog: 0.1.2...0.1.3

0.1.2

What's Changed

• Use a trusted publisher to deploy to pypi as username / password is no longer supported by @BrendBraeckmans in #4
• Fix misalignment by @BrendBraeckmans in #5

Full Changelog: 0.1.1...0.1.2

0.1.1

Add some new functionalities which can be applied to nested fields.

Changes in this release include:

• Fix in TerminalOperationProcessor which causes terminal operations on arrays to work.
• Implement fillna for nested types based on the pure pyspark fillna signature
• Added duplicate processor to duplicate nested fields
• Add AddOperationProcessor which makes it possible to add nested fields
• Make it possible to add nested fields based on an expression

What's Changed

• Feature/fillna duplicate and adding nested columns by @BrendBraeckmans in #3

Full Changelog: 0.1.0...0.1.1

0.1.0

Support all Spark versions from 3.1.1 onwards

Changes inlcude:

• Extended support for all Spark versions greater than 3.1.1 and less than 4.0.
• Limited Python version support to 3.7 and onwards due to the use of annotations.
• Allowed all pytest versions greater than or equal to 7.1.0.
• Upgraded flake8 to at least version 4.0 to address the issue described python/importlib_metadata#406.
• Run UT on all relevant python / spark / hadoop combinations

What's Changed

• Allow all spark version > 3.1.1 and < 4.0. by @BrendBraeckmans in #1
• Fix and improve CI/CD after previous PR by @BrendBraeckmans in #2

New Contributors

• @BrendBraeckmans made their first contribution in #1

Full Changelog: 0.0.9...0.1.0

0.0.9

format date respects nulls, but throw exception in case input format …

0.0.8

redact function preserve nulls