Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: JSON: refactor IO for JSON; make input implementations column optimizable #94

Merged
merged 54 commits into from
Sep 15, 2023

Conversation

douglasdavis
Copy link
Collaborator

@douglasdavis douglasdavis commented Oct 31, 2022

More JSON development to enable schema <--> layout/form round trips (motivated by column projection optimizations)

@douglasdavis douglasdavis force-pushed the more-json-dev branch 2 times, most recently from bd2dc50 to 2a77f3c Compare November 1, 2022 19:50
@codecov-commenter
Copy link

codecov-commenter commented Nov 1, 2022

Codecov Report

Merging #94 (a9c6031) into main (3d7d2c5) will decrease coverage by 1.50%.
The diff coverage is 79.22%.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

@@            Coverage Diff             @@
##             main      #94      +/-   ##
==========================================
- Coverage   94.92%   93.43%   -1.50%     
==========================================
  Files          23       23              
  Lines        2780     2923     +143     
==========================================
+ Hits         2639     2731      +92     
- Misses        141      192      +51     
Files Changed Coverage Δ
src/dask_awkward/lib/_utils.py 88.23% <33.33%> (-11.77%) ⬇️
src/dask_awkward/lib/io/io.py 95.93% <50.00%> (+0.01%) ⬆️
src/dask_awkward/lib/io/parquet.py 92.39% <71.42%> (-0.90%) ⬇️
src/dask_awkward/lib/io/json.py 83.51% <79.71%> (-15.80%) ⬇️
src/dask_awkward/__init__.py 100.00% <100.00%> (ø)
src/dask_awkward/lib/core.py 92.60% <100.00%> (-0.08%) ⬇️
src/dask_awkward/lib/testutils.py 100.00% <100.00%> (ø)

... and 1 file with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@martindurant
Copy link
Collaborator

Ah, this is where the code we were talking about lives. We should be in a position to enable this with the new optimization?

@douglasdavis
Copy link
Collaborator Author

Yep! Once #141 is in this should fall into place without too much difficulty (knock on wood)

@douglasdavis douglasdavis changed the title JSON dev continued feat: JSON dev continued Apr 5, 2023
@douglasdavis douglasdavis changed the title feat: JSON dev continued feat: JSON input project_columns implementation to generate minimal JSON Schema Jul 19, 2023
@douglasdavis douglasdavis marked this pull request as ready for review August 31, 2023 20:44
@douglasdavis douglasdavis changed the title feat: JSON input project_columns implementation to generate minimal JSON Schema feat: JSON: refactor IO for JSON; make input implementations column optimizable Aug 31, 2023
@douglasdavis douglasdavis merged commit b60a25e into dask-contrib:main Sep 15, 2023
19 checks passed
@douglasdavis douglasdavis deleted the more-json-dev branch September 15, 2023 18:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants