Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

v23.09.00 #1053

Merged
merged 33 commits into from
Sep 30, 2023
Merged

v23.09.00 #1053

merged 33 commits into from
Sep 30, 2023

Conversation

marcinz
Copy link
Collaborator

@marcinz marcinz commented Sep 29, 2023

v23.09.00

manopapad and others added 30 commits July 19, 2023 17:11
@manopapad
Fix Wundefined-var-template clang warning (#992)
1d62db3
@yimoj
improve random bitgenerator code coverage (#990)
46862c4 
* improve random bitgenerator code coverage

* add error description to zipf failure

* Replace misused assert in generator with appropriate exceptions
@manopapad
Missing alignment on histogram call (#999)
5abc82e
@ipdemes @pre-commit-ci
Updating version for docs (#994)
388cd3e 
* adding new version for documentation + fixing warning

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
@marcinz @m3vaz
pin cuda-version (#998) (#1002)
fa33ee5 
Co-authored-by: Mark Vaz <[email protected]>
@marcinz
Bump up cunumeric version and legate.core version (#1004)
1883b71 
Bump up cunumeric version to 23.09.00
@marcinz
Fix file name (#1008)
c08bacf
@rohany @pre-commit-ci @manopapad
src/cunumeric: add missing openmp variants to BitGenerator and Unique…
7d882df 
…Reduce (#1010)

* src/cunumeric: add missing openmp variants to BitGenerator and UniqueReduce

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Full OpenMP implementation for unique_reduce

* Leave a note to do a full OpenMP implementation

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Manolis Papadakis <[email protected]>
@yimoj
Improve fft code coverage (#1013)
89dd7a5
@aschaffer
Histogram refactor (#1003)
04ff6d1 
* Unification GPU/CPU/OMP refactoring.

* Unification GPU/CPU/OMP refactoring. Clean-up.

* SFINAE on decltype(exe_pol_obj).

* Added synchronization policy to remove some possible artificial code dependencies.
@ipdemes
Fix for control replication violation in test (#1005)
d54a3dd 
* removing the use of python's random

* seed python's random to avoid violation of control replication

* removing python's random seed from the BOILERPLATE
@bryevdv
fix build instrutions link (#1014)
f89dd2a
@manopapad
Various test fixes (#1015)
e6a33ce 
* Skip advanced RNG tests on any setup that's missing cuRand

* Don't compare arrays created with np.empty

They contain uninitialized data, that may be equivalent to NaN when
interpreted as a floating-point number, and np.array_equal considers
that NaN != NaN.
@manopapad
Add back None as an accepted value for axis on some type sigs (#1017)
b31e999
@manopapad
If a scalar ufunc arg is cn.ndarray use its type directly (#1011)
daf31c2 
* If a scalar ufunc arg is cn.ndarray use its type directly

* Fix pedantic type compatibility tests
@manopapad
More RNG test fixes for MacOS (#1019)
c41a47e
@marcinz @manopapad
Update to the latest Legion (#1007)
67cd9f9 
Update legate.core SHA

---------

Co-authored-by: Manolis Papadakis <[email protected]>
Co-authored-by: Manolis Papadakis <[email protected]>
@manopapad
Don't seed np.random in the tests, since the core will do it (#1020)
e8abe21
@sandeepd-nv
GH docker based reusable CI workflows. (#993)
f2a87a7
@manopapad
Skip the docstrings for functions pulled from cloned modules (#1024)
9017514 
* Skip the docstrings for functions pulled from cloned modules

The docstring for numpy.binomial has an unused reference, which
causes our doc build to panic, in cases where curand is not
installed, and thus we end up falling back to numpy for binomial.

* Add a stub docstring instead of dropping it completely

* Fix unit test
@manopapad
Fix random test failures in CPU-only runs (#1025)
06cd534 
* Anticipate some test failures when cuRand is not available

* Ensure input to unstable sort algorithms contains no duplicates

* Add mergesort to stable sort methods
@aschaffer @pre-commit-ci @manopapad
Quantile Implementation (#664)
780fd40 
* Quantile Python implementation and test.

* Added boilerplate integration code for quantile.

* fixed ceil() call

* Added 1st quantile pytest.

* Fixed 1st quantile pytest: added missing main.

* Fixed global dtype to numpy.dtype.

* Replaced product() with numpy.product().

* Debugging 'nearest'. Failed attempts, but some useful ways to simplify the repro.

* Simplified 'nearest' implementation.

* Fix for 'nearest' implementation.

* Added more quantile tests.

* Clean-up.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* FIxed module.py formatting issues.

* FIxed test_quantiles.py formatting issues.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Adrressed comments on re-using 'out' if given.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fixed 'out' management for scalar input.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix for keepdims with multiple axes.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Added tests to check for the supplied output case.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Added the keepdims = True case for out is not None. Caveat: there's a numpy bug associated with it: numpy/numpy#22544

* Addressed comment on scalar input asarray().

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Addressed comment on scalar input np.asarray().

* Addressed comment on add_boilerplate.

* Fixed LEGATE_MAX_DIM issues in tests.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Attempts to fix ndenumerate() approach.

* Attempts to fix ndenumerate() approach. More debugging.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fixed ndenumerate, non-flattening approach.

* Clean-up (flake8).

* Addressed comment on directly using qs_all for result.

* Addressed comment on input_is_scalar.

* Addressed comment on replacing take([pos],...) by just take(pos,...).

* Addressed comment on always using asarray(q).

* Addressed comment on removing is_sorted() check.

* Addressed comment on more pythonic list comprehension in any().

* Addressed comment on using math.floor().

* Addressed comment on using sorted() instead of sort().

* Addressed comment on removing last 2 args, to be deprecated.

* Removed docstring on deprecated 'interpolation' arg.

* Addressed comment on qs_all[...].

* Addressed comment on in-place update of qs_all (avoids copies).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Addressed comment on overwrite_input test. Added.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Addressed comment on using allclose().

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fixed keepdims=True failures.

* Addressed comment on adding multiple consecutive axes test, (0, 1).

* Added test to debug axis=None case.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Partial fix for axis=None failures.

* Fix for axis=None failures.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Addressed review on not doing all(shapes).

* Addressed review on not returning empty tuple from in_place_interp().

* Added tests for random input.

* Addressed review on throwing on complex input.

* Addressed review on float128.

* Addressed reviews on better replacement for in_place_interp() and using TypeError.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Addressed reviews on axes_set confusing apparent dependency on arr.shape and remaining_shape as tuple of 1s.

* Completely removed the artificial dependency of axes_set to arr.shape.

* Addressed comments on replacing np with num; fixed Weibull issue.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Addressed comments on mypy function signatures: floor_i().

* Addressed comments on mypy function signatures: method functions.

* Addressed comments on mypy function signatures: is_diff().

* Addressed comments on mypy function signatures: reshuffle_reshape().

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Addressed comments on mypy function signatures: quantile_impl().

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Addressed comments on mypy function signatures: quantile(). Tentative 1.

* Addressed comments on mypy function signatures: quantile(). Tentative 2 (temporary fix, until better axis type can be provided).

* Addressed comments on mypy function signatures: quantile(). Tentative 3: adjust quantile_impl() signature.

* Addressed comments on mypy function signatures: quantile().

* Addressed reviews on corner cases: quantile limits {0.0, 1.0}.

* Possible solution for the case of a <- scalar, list, or tuple. Attempt 1.

* Better solution for the case of a <- scalar, list, or tuple.

* Addressed review on testing output cast, but functionality awaits bug-fixes in numpy.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Addressed review on annotating a: ndarray and removing the asarray() coercion.

* Re-activated skipped test to, at least, run the output cast, w/o checking, for now, due to numpy issue 22766.

* Re-activated skipped output downcast test from float64 to float32 (rather than int, which fails in numpy, due to numpy issue 22766).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Cosmetic changes

* Use np instead of num

* Use num instead of cu

* Make sure all test input arrays are vanilla NumPy arrays

Because cuNumeric supports the __array_function__ protocol, if you pass a
cuNumeric array to a vanilla NumPy method, it will dispatch through to the
cuNumeric implementation, meaning that np.foo(arr) and cn.foo(arr) end up
doing the same thing, and we're not testing anything by comparing them.

* Fix typing of axis

* Minor cleanups

* Clean up some comments

* No need for helper method for tuple comparison

* Clean up docs

* Typo in docs

* Fix for upper-bound quantile on averaged_interpolated_cdf.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Added seeding to quantile tests.

* Added percentile() main function.

* Percentile. Fix. Test failing.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fixed args fwd-ing.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Added profiling test for quantiles.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Increased dimension of input array.

* Fixed cupy failure with q.ndim > 1.

* Refactored for performance.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Replaced deprecated np.product().

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fixed typo in str_method.

* Fix pre-commit complaints

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Manolis Papadakis <[email protected]>
Co-authored-by: Andrei Schaffer <[email protected]>
Co-authored-by: Andrei Schaffer <[email protected]>
Co-authored-by: Andrei Schaffer <[email protected]>
@bryevdv
ignore last-failed file (#1026)
821d1c2
@sandeepd-nv
Updated versions.json SHA. (#1033)
0060dc7
@ajschmidt8
Use copy-pr-bot (#1035)
a7afb95 
This PR replaces the `copy_prs` functionality from the `ops-bot` with the new dedicated `copy-pr-bot` GitHub application.

Thorough documentation for the new `copy-pr-bot` application can be viewed below.

- https://docs.gha-runners.nvidia.com/apps/copy-pr-bot/

**Important**: `copy-pr-bot` enforces signed commits. If an organization member opens a PR that contains unsigned commits, it will be deemed untrusted and therefore require an `/ok to test` comment. See the GitHub docs [here](https://docs.github.com/en/authentication/managing-commit-signature-verification/about-commit-signature-verification) for information on how to set up commit signing.

Any time a PR is deemed untrusted, it will receive a comment that looks like this: rapidsai/ci-imgs#63 (comment).

Every subsequent commit on an untrusted PR will require an additional `/ok to test` comment.

Any existing PRs that have unsigned commits after this change is merged will require an `/ok to test` comment for each subsequent commit _or_ the PR can be rebased to include signed commits as mentioned in the docs below:
https://docs.gha-runners.nvidia.com/cpr/contributors.

This information is all included on the documentation page linked above.

_I've skipped CI on this PR since it's not a change that is tested._

[skip ci]
@bryevdv
Update pre-commit mypy (#1037)
2c13393 
* update pre-commit mypy

* do not warn unused ignores
@XiaLuNV @pre-commit-ci
enhance test_matrix_power.py/cholesky.py/norm.py (#1038)
8dddd18 
* enhance test_matrix_power.py

* enhance test_ch

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
@manopapad
Don't cast histogram to int64 when density=True (#1042)
f014105
@manopapad
Explicitly cast result of shift binary operators (#1046)
3a6ca9e
@manopapad
Reject types that DeferredArray doesn't support (#1044)
892799f 
* Reject types that DeferredArray doesn't support

IMHO there is little value in pretending that we support these types.
We currently accept them by falling back to EagerArray, and that
works for a little bit, but it is possible that later we will need to
convert to a truly Legion-backed implementation, in which case the
failure will pop up there.

Instead we should just reject them from the start.

* Fix unit test
manopapad and others added 3 commits September 22, 2023 09:32
@manopapad
Remove use of deprecated np.find_common_type (#1045)
b76a58a
@bryevdv @manopapad
Add best practices info to sphinx docs (#1048)
0b11958 
* Add best practices info to sphinx docs

* Apply suggestions from code review

Co-authored-by: Manolis Papadakis <[email protected]>

* Apply suggestions from code review

---------

Co-authored-by: Manolis Papadakis <[email protected]>
@ipdemes
adding 23.09 version to docs (#1049)
443f751
@copy-pr-bot
Copy link

copy-pr-bot bot commented Sep 29, 2023

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@marcinz marcinz added the category:task PR is a simple task and will not be included in release notes label Sep 29, 2023
@marcinz
Copy link
Collaborator Author

marcinz commented Sep 29, 2023

This must be merged with a merge commit.

@marcinz
Copy link
Collaborator Author

marcinz commented Sep 29, 2023

/ok to test

@marcinz
Copy link
Collaborator Author

marcinz commented Sep 29, 2023 •
edited
Loading

@sandeepd-nv The GPU build fails, but it also failed on the tip of branch-23.09. Do we treat this as a non issue? I guess CI still fails on branch-23.11.

@manopapad
Copy link
Contributor

It doesn't seem possible to allow merge commits only on specific branches, so I just disabled the requirement for pull requests on the main branch. You should be able to just merge branch-23.09 into main and push the update without going through a PR.

@marcinz
Copy link
Collaborator Author

marcinz commented Sep 29, 2023 •
edited
Loading

I usually enable merge commits for a moment when I actually hit the button so that someone does not use it accidentally. :) The problem here is that the CI is not passing, but I think we can ignore this, right?

Edit: Ah, I see. You are suggesting that I push without a PR. I prefer to use PR and just enable merge commits for a moment. It has not caused an issue until now, although there is always the possibility of the first time. I like to have a PR for that merge if possible.

@manopapad
Copy link
Contributor

The problem here is that the CI is not passing, but I think we can ignore this, right?

Yeah, CI doesn't matter for this

@marcinz marcinz merged commit e66a063 into main Sep 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@manopapad manopapad Awaiting requested review from manopapad

@magnatelee magnatelee Awaiting requested review from magnatelee

Assignees
No one assigned
Labels
category:task PR is a simple task and will not be included in release notes
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
10 participants
@marcinz @manopapad @yimoj @ipdemes @rohany @aschaffer @bryevdv @sandeepd-nv @ajschmidt8 @XiaLuNV