Bug fix for float precision calculation using categorical data with trailing 0s #1125
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
|
SchadtJ
changed the title
Comment on lines
+215
to
+219
| categorical_series = pd.Series( | ||
| [202209, 202210, 202211], dtype="category" | ||
| ).apply(str) | ||
| float_profiler = FloatColumn("Name") | ||
| float_profiler.update(categorical_series) |
There was a problem hiding this comment.
testing this locally so may have some additional comments, but initial thought is that there should definitely be some assert statements here to validate this is actually working as intended post-change @SchadtJ @scottiegarcia
There was a problem hiding this comment.
thoughts @scottiegarcia @SchadtJ?
|
reverting this accidental merge -- @SchadtJ please reopen into |
micdavis
pushed a commit
that referenced
this pull request
May 20, 2024
* Replace snappy with cramjam (#1091) * add downloads tile (#1085) * Replace snappy with cramjam * Delete test_no_snappy --------- Co-authored-by: Taylor Turner <[email protected]> * Quick fix for dependency max pins (#1120) * Fix dask_expr * Keras and Tensorflow version fix * Keras and Tensorflow version fix * Fix keras bug * pre-commit fix (#1122) * docs: update test link to latest version (#1114) * docs: add contributor notes on where to find documentation branches (#1113) * docs: add contributor notes on where to find documentation branches * docs: update documentation wording to spell out why `dev-gh-pages` and `gh-pages` branches exist for staging content * docs: add note on fork Co-authored-by: Taylor Turner <[email protected]> * Update .github/CONTRIBUTING.md Co-authored-by: Taylor Turner <[email protected]> --------- Co-authored-by: Taylor Turner <[email protected]> * update black version (#1131) * Add memray max version (#1132) * Bug fix for float precision calculation using categorical data with trailing zeros. (#1125) * Revert "Bug fix for float precision calculation using categorical data with t…" (#1133) This reverts commit d3159bd. * fix * make up to date * yep, shouldn't change * bump version --------- Co-authored-by: Gábor Lipták <[email protected]> Co-authored-by: abajpai15 <[email protected]> Co-authored-by: Patrick Carlson <[email protected]> Co-authored-by: James Schadt <[email protected]>
JGSweets
pushed a commit
to JGSweets/data-profiler
that referenced
this pull request
Jun 5, 2024
taylorfturner
added a commit
that referenced
this pull request
Jun 6, 2024
* Replace snappy with cramjam (#1091) * add downloads tile (#1085) * Replace snappy with cramjam * Delete test_no_snappy --------- Co-authored-by: Taylor Turner <[email protected]> * pre-commit fix (#1122) * Bug fix for float precision calculation using categorical data with trailing zeros. (#1125) * Revert "Bug fix for float precision calculation using categorical data with t…" (#1133) This reverts commit d3159bd. * refactor: move layers outside of class * refactor: update model to keras 3.0 * fix: manifest * fix: bugs in compile and train * fix: bug in load_from_library * fix: bugs in CharCNN * refactor: loading tf model labeler * fix: bug in data_labeler identification * fix: update model to use proper softmax layer names * fix: formatting * fix: remove unused line * refactor: drop support for 3.8 * fix: comments * fix: comment --------- Co-authored-by: Gábor Lipták <[email protected]> Co-authored-by: Taylor Turner <[email protected]> Co-authored-by: James Schadt <[email protected]>
micdavis
pushed a commit
that referenced
this pull request
Jun 14, 2024
* refactor: Upgrade the models to use keras 3.0 (#1138) * Replace snappy with cramjam (#1091) * add downloads tile (#1085) * Replace snappy with cramjam * Delete test_no_snappy --------- Co-authored-by: Taylor Turner <[email protected]> * pre-commit fix (#1122) * Bug fix for float precision calculation using categorical data with trailing zeros. (#1125) * Revert "Bug fix for float precision calculation using categorical data with t…" (#1133) This reverts commit d3159bd. * refactor: move layers outside of class * refactor: update model to keras 3.0 * fix: manifest * fix: bugs in compile and train * fix: bug in load_from_library * fix: bugs in CharCNN * refactor: loading tf model labeler * fix: bug in data_labeler identification * fix: update model to use proper softmax layer names * fix: formatting * fix: remove unused line * refactor: drop support for 3.8 * fix: comments * fix: comment --------- Co-authored-by: Gábor Lipták <[email protected]> Co-authored-by: Taylor Turner <[email protected]> Co-authored-by: James Schadt <[email protected]> * Fix Tox (#1143) * tox new * update * update * update * update * update * update * update * update tox.ini * update * update * remove docs * empty retrigger * update (#1146) * bump version * update 3.11 * remove dist/ --------- Co-authored-by: JGSweets <[email protected]> Co-authored-by: Gábor Lipták <[email protected]> Co-authored-by: James Schadt <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Bug fix for #1048 (comment).
The float precision calculation errors out for categorical data when one of the values has leading/ trailing zeros. This is due to the regex operation stripping these zeros and the resulting value being outside the list of possible values.
Passing tests:
