Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

read_commandline supports polars engine #1356

Merged
merged 58 commits into from
Jun 9, 2024
Merged

Conversation

samukweku
Copy link
Collaborator

@samukweku samukweku commented May 1, 2024

PR Description

Please describe the changes proposed in the pull request:

  • read_commandline now supports polars

**This PR relates to #1352 **

import janitor as jn
jn.read_commandline('unzip -p ../Downloads/iris.zip', engine='polars')
Out[12]: 
shape: (150, 5)
┌──────────────┬─────────────┬──────────────┬─────────────┬───────────┐
│ sepal_lengthsepal_widthpetal_lengthpetal_widthspecies   │
│ ---------------       │
│ f64f64f64f64str       │
╞══════════════╪═════════════╪══════════════╪═════════════╪═══════════╡
│ 5.13.51.40.2setosa    │
│ 4.93.01.40.2setosa    │
│ 4.73.21.30.2setosa    │
│ 4.63.11.50.2setosa    │
│ 5.03.61.40.2setosa    │
│ …            ┆ …           ┆ …            ┆ …           ┆ …         │
│ 6.73.05.22.3virginica │
│ 6.32.55.01.9virginica │
│ 6.53.05.22.0virginica │
│ 6.23.45.42.3virginica │
│ 5.93.05.11.8virginica │
└──────────────┴─────────────┴──────────────┴─────────────┴───────────┘

PR Checklist

Please ensure that you have done the following:

  1. PR in from a fork off your branch. Do not PR from <your_username>:dev, but rather from <your_username>:<feature-branch_name>.
  1. If you're not on the contributors list, add yourself to AUTHORS.md.
  1. Add a line to CHANGELOG.md under the latest version header (i.e. the one that is "on deck") describing the contribution.
    • Do use some discretion here; if there are multiple PRs that are related, keep them in a single line.

Automatic checks

There will be automatic checks run on the PR. These include:

  • Building a preview of the docs on Netlify
  • Automatically linting the code
  • Making sure the code is documented
  • Making sure that all tests are passed
  • Making sure that code coverage doesn't go down.

Relevant Reviewers

Please tag maintainers to review.

@ericmjl
Copy link
Member

ericmjl commented May 1, 2024

Copy link
Member

@ericmjl ericmjl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking great! I am going to see if I can push up a test for read_commandline().

This commit modifies the test_read_commandline function in the io module to accept an additional parameter, "engine". This allows the function to be tested with different engines, specifically "pandas" and "polars". The function calls within the test have been updated to pass this new parameter. The change is intended to improve the comprehensiveness of the test coverage.
This commit removes the default value for the 'engine' parameter in the test_read_commandline function. This change ensures that the function's behavior is explicitly defined by the test parameters.
@ericmjl
Copy link
Member

ericmjl commented Jun 2, 2024

@samukweku we're close with this one -- I think the line that is failing in test_readcommandline() is this one under the polars engine:

assert df.equals(dataframe)

Would you be kind enough to take a look at this particular error? I am quite sure it's got to do with the polars internal API, but that's one I'm not familiar enough with to know.

ericmjl and others added 4 commits June 2, 2024 20:24
…ne parameter

- Removed the `engine` parameter from `test_read_commandline` function to simplify test logic.
- Updated calls to `janitor.io.read_commandline` within the test to not pass the `engine` argument, aligning with the function's updated signature.

This change does not affect the external API, hence no breaking change is introduced.
@ericmjl
Copy link
Member

ericmjl commented Jun 9, 2024

Also LGTM, thanks @samukweku!

@ericmjl ericmjl merged commit 2dff4f6 into dev Jun 9, 2024
4 checks passed
@ericmjl ericmjl deleted the samukweku/polars_read_commandline branch June 9, 2024 14:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants