Pandas21 compat #196
Merged
Pandas21 compat #196
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Additional code synchronizations (and the addition of a dtype-preserving map method). These changes were initially developed to support uncertainties, but the uncertainty changes have all been stripped out to simplify merging of underlying code. Once these changes are fully synced with a release version of Pandas 2.1, we can look at adding back uncertainties. These changes also tolerate complex128 as a base type for magnitudes, with one except (under discussion as pandas-dev/pandas#54445). Signed-off-by: Michael Tiemann <[email protected]>
Signed-off-by: Michael Tiemann <[email protected]>
The PintArray map function should return a PintArray if the mapper returns PintQuantities, but otherwise it should just return whatever was the result of the mapper. We follow the examples from pandas in the definition and use of _get_expected_exception in the test suite. Signed-off-by: Michael Tiemann <[email protected]>
|
Pandas 2.1rc0 has been released: https://github.com/pandas-dev/pandas/releases/tag/v2.1.0rc0 |
Fix canonicalization of NaNs to use na_value (with units) rather than pure np.nan. While np.nan is correctly handled as a value in PintArrays, and work as expected with addition (adding a quantity to a NaN produces a NaN, which remains value), it doesn't work for multiplication (where a quantity times a NaN produces not a NaN but new quantity of NaN magnitude the units of the non-NaN value). Canonicalizing to na_value provides consistent unit handling with NaNs through all arithmetic operations. Signed-off-by: Michael Tiemann <[email protected]>
Make black happy. Signed-off-by: Michael Tiemann <[email protected]>
This was referenced Aug 12, 2023
|
you can add the 2.1rc to https://github.com/hgrecco/pint-pandas/blob/bf84e37ed13d032a13edec948c1990909d920dec/.github/workflows/ci.yml#L11C10-L11C10 to get CI to test against it |
Fix errors that crept into value_counts concerning na_type. And canonicalize na_value in quantities a bit more generally. As stated in a previous commit comment, if we leave naked NA/NaNs in our Quantity array, the code gracefully handles unit addition/subtraction, but unit multiplication/division loses. Signed-off-by: Michael Tiemann <[email protected]>
Simplify type annotations for _get_expected_exception to make Python 3.9 happy. Signed-off-by: Michael Tiemann <[email protected]>
|
I see that while these changes run happily with later versions of Pint and Pandas, there are many errors in the CI/CD system for Pandas 2.0.2 and Pint 0.21.3. I'll look at those and see how much of a gap there is to bridge, or whether these changes really are only for 2.1 and beyond. |
Delete the last vestiges of some old code from a different PR. Also use Pandas public APIs for `infer_dtype` (used in `_from_factorized`). Signed-off-by: Michael Tiemann <[email protected]>
Remove unrelated comment about UFloats. Signed-off-by: Michael Tiemann <[email protected]>
|
Looking at Pandas 2.0.3 failures:
I'll update this list with other problems I find, or rewrite text to describe how the fixes were implemented (as a start for the WhatsNew). |
Restore some Pandas 2.0.x interfaces and use pandas_version_info from pint_array to condition logic. Also change some more `self.assert_series_equal` to `tm.assert_series_equal` in the restored code to accommodate Pandas 2.1 BaseExtensionTests behavior. Signed-off-by: Michael Tiemann <[email protected]>
|
this is looking great, thanks for putting this together. Just needs adding the 2.1rc to the CI and a changes note |
Simplify PintArray.map to let `pandas.core.algorithms.map_array` do the mapping and we just clean up the PintArray details if necessary. Pandas 2.1.0rc0 added to CI Updated CHANGES Signed-off-by: Michael Tiemann <[email protected]>
|
Ugh..the map changes worked for Pandas 2.1, but need fixing for 2.0.2. I'm on it... |
Implement `map` for PintArrays also only with Pandas 2.0.2 interfaces. Signed-off-by: Michael Tiemann <[email protected]>
Until Pandas resolves pandas-dev/pandas#54445 we cannot feed complex128 types to the test_setitem_2d_values test case. Signed-off-by: Michael Tiemann <[email protected]>
|
bors r+ |
|
Build succeeded! The publicly hosted instance of bors-ng is deprecated and will go away soon. If you want to self-host your own instance, instructions are here. If you want to switch to GitHub's built-in merge queue, visit their help page. |
MichaelTiemannOSC
added a commit
to MichaelTiemannOSC/pint-pandas
that referenced
this pull request
Aug 15, 2023
First draft of 100% working test cases after (re)merging with changes extracted from these changes as part of PR hgrecco#196. Signed-off-by: Michael Tiemann <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
pre-commit run --all-fileswith no errorsCI/CD doesn't quite work yet because we don't have a Pandas 2.1 rc to point to. But comments are welcome!