Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Explicit-comms: preserve partition IDs #1240

Merged
merged 4 commits into from
Sep 26, 2023
Merged

Explicit-comms: preserve partition IDs #1240

merged 4 commits into from
Sep 26, 2023

Conversation

madsbk
Copy link
Member

@madsbk madsbk commented Sep 26, 2023

shuffle_task() now returns a dict mapping partition IDs to dataframes`

Fixes #1239

@madsbk madsbk added bug Something isn't working non-breaking Non-breaking change labels Sep 26, 2023
@github-actions github-actions bot added the python python code needed label Sep 26, 2023
@pentschev
Copy link
Member

Could we add the sample from #1239 or some variation of it as a test?

@madsbk madsbk marked this pull request as ready for review September 26, 2023 12:47
@madsbk madsbk requested a review from a team as a code owner September 26, 2023 12:47
rjzamora
rjzamora approved these changes Sep 26, 2023
View reviewed changes
Copy link
Member

@rjzamora rjzamora left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @madsbk !

pentschev
pentschev reviewed Sep 26, 2023
View reviewed changes
Copy link
Member

@pentschev pentschev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just a minor question/suggestion on one of your comments.

dask_cuda/tests/test_explicit_comms.py Outdated Show resolved Hide resolved
@madsbk @rjzamora
Typo
95070d2 
Co-authored-by: Richard (Rick) Zamora <[email protected]>
pentschev
pentschev approved these changes Sep 26, 2023
View reviewed changes
Copy link
Member

@pentschev pentschev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @madsbk and @rjzamora .

@pentschev
Copy link
Member

/merge

@rapids-bot rapids-bot bot merged commit 6bd4ba4 into rapidsai:branch-23.10 Sep 26, 2023
19 checks passed
@madsbk madsbk deleted the excomm_preserve_partition_ids branch September 27, 2023 13:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@rjzamora rjzamora rjzamora approved these changes

@pentschev pentschev pentschev approved these changes

Assignees
No one assigned
Labels
bug Something isn't working non-breaking Non-breaking change python python code needed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[BUG] Explicit-comms shuffle does not obey "_partitions" column
3 participants
@madsbk @pentschev @rjzamora