Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add some basic test coverage for cuda.core.experimental #153

Merged
merged 13 commits into from
Oct 29, 2024

Conversation

ksimpson-work
Copy link
Contributor

@ksimpson-work ksimpson-work commented Oct 10, 2024

Close #78.

This change adds a minimal testsuite to the repo so we can establish pipeline, and develop the tests alongside the cuda.core object model. It uses a pytest tree structure.

Example:

python -m pytest tests/
===================================================== test session starts ======================================================
platform linux -- Python 3.12.7, pytest-8.3.3, pluggy-1.5.0
benchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/ksimpson/cuda-python/cuda-python/cuda_core
configfile: pyproject.toml
plugins: benchmark-4.0.0
collected 8 items

tests/test_context.py .                                                                                                  [ 12%]
tests/test_device.py .                                                                                                   [ 25%]
tests/test_event.py .                                                                                                    [ 37%]
tests/test_launcher.py .                                                                                                 [ 50%]
tests/test_memory.py .                                                                                                   [ 62%]
tests/test_module.py .                                                                                                   [ 75%]
tests/test_program.py .                                                                                                  [ 87%]
tests/test_stream.py .                                                                                                   [100%]

====================================================== 8 passed in 2.11s =======================================================

Copy link

copy-pr-bot bot commented Oct 10, 2024

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@leofang leofang added P0 High priority - Must do! test Addition or improved tests cuda.core Everything related to the cuda.core module labels Oct 11, 2024
@leofang leofang added this to the cuda.core beta 1 milestone Oct 11, 2024
@m3vaz
Copy link
Collaborator

m3vaz commented Oct 15, 2024

/ok to test

m3vaz
m3vaz previously approved these changes Oct 15, 2024
@leofang
Copy link
Member

leofang commented Oct 15, 2024

Let's also hook up the code samples with the test system. We can do something like this: Collect the test files in-process, and loop over them.

cuda_core/tests/test_compiler.py Outdated Show resolved Hide resolved
cuda_core/tests/test_context.py Outdated Show resolved Hide resolved
@leofang
Copy link
Member

leofang commented Oct 15, 2024

@ksimpson-work heads up, we've done the move to the experimental submodule (#171), so you might need to merge with the latest main and make some changes 😅

@ksimpson-work ksimpson-work changed the title Add barebones tests to repo to start a pipeline Add some basic test coverage for cuda.core.experimental Oct 25, 2024
@leofang leofang requested review from vzhurba01 and removed request for jollylili October 25, 2024 18:20
cuda_core/tests/conftest.py Show resolved Hide resolved
cuda_core/tests/example_tests/utils.py Show resolved Hide resolved
cuda_core/tests/test_device.py Show resolved Hide resolved
cuda_core/tests/test_launcher.py Show resolved Hide resolved
cuda_core/tests/test_memory.py Show resolved Hide resolved
cuda_core/tests/test_program.py Show resolved Hide resolved
cuda_core/tests/test_stream.py Outdated Show resolved Hide resolved
cuda_core/tests/example_tests/utils.py Outdated Show resolved Hide resolved
cuda_core/tests/example_tests/utils.py Outdated Show resolved Hide resolved
cuda_core/tests/example_tests/utils.py Outdated Show resolved Hide resolved
cuda_core/tests/example_tests/utils.py Outdated Show resolved Hide resolved
Copy link
Collaborator

@vzhurba01 vzhurba01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With this review I'm done looking at all the files. I'm sorry for all the nits, and thank you for handling the ones I've already shared! 🙏

cuda_core/tests/test_memory.py Outdated Show resolved Hide resolved
cuda_core/tests/test_memory.py Outdated Show resolved Hide resolved
cuda_core/tests/test_memory.py Outdated Show resolved Hide resolved
cuda_core/tests/conftest.py Outdated Show resolved Hide resolved
cuda_core/tests/test_memory.py Outdated Show resolved Hide resolved
cuda_core/tests/example_tests/utils.py Show resolved Hide resolved
cuda_core/tests/test_program.py Show resolved Hide resolved
cuda_core/tests/test_device.py Show resolved Hide resolved
cuda_core/tests/test_event.py Show resolved Hide resolved
cuda_core/tests/conftest.py Outdated Show resolved Hide resolved
@leofang
Copy link
Member

leofang commented Oct 28, 2024

Let's get the formatting feedbacks addressed quickly and move on. I've opened #195 to track an automated solution so that we don't need to spend time on reviewing formatting issues, which could sometimes be distracting/frustrating 🙂

@ksimpson-work
Copy link
Contributor Author

Thankyou @vzhurba01 for your review. I have addressed all the comments and implemented all your nits as I agreed with all. Thanks @leofang for your review as well. I believe all comments have been addressed as of the most recent push

vzhurba01
vzhurba01 previously approved these changes Oct 28, 2024
Copy link
Collaborator

@vzhurba01 vzhurba01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@leofang leofang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, Keenan/Vlad, for all the hard work! Unless I am missing something I don't think we have "relational tests" added yet (ex: Device.create_stream() returns a Stream; Stream.record() returns an Event, etc), but let's add them in the next PR.

Copy link
Member

@leofang leofang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ksimpson-work do you see these warnings locally? This is concerning and making me wonder if we need to fix #141 as part of beta 1:

============================================================== warnings summary ===============================================================
<frozen importlib._bootstrap_external>:1297
  <frozen importlib._bootstrap_external>:1297: DeprecationWarning: The cuda.cuda module is deprecated and will be removed in a future release, please switch to use the cuda.bindings.driver module instead.

<frozen importlib._bootstrap_external>:1297
  <frozen importlib._bootstrap_external>:1297: DeprecationWarning: The cuda.cudart module is deprecated and will be removed in a future release, please switch to use the cuda.bindings.runtime module instead.

<frozen importlib._bootstrap_external>:1297
  <frozen importlib._bootstrap_external>:1297: DeprecationWarning: The cuda.nvrtc module is deprecated and will be removed in a future release, please switch to use the cuda.bindings.nvrtc module instead.

tests/test_memory.py::test_buffer_initialization
tests/test_memory.py::test_buffer_copy_to
tests/test_memory.py::test_buffer_copy_from
  /local/home/leof/miniforge3/envs/cuda_py_cuda126/lib/python3.12/site-packages/_pytest/unraisableexception.py:85: PytestUnraisableExceptionWarning: Exception ignored in: <function Buffer.__del__ at 0x7f3fb7ce1ee0>
  
  Traceback (most recent call last):
    File "/local/home/leof/dev/cuda-python/cuda_core/cuda/core/experimental/_memory.py", line 35, in __del__
      self.close(default_stream())
    File "/local/home/leof/dev/cuda-python/cuda_core/cuda/core/experimental/_memory.py", line 41, in close
      self._mr.deallocate(self._ptr, self._size, stream)
    File "/local/home/leof/dev/cuda-python/cuda_core/tests/test_memory.py", line 96, in deallocate
      handle_return(cuda.cuMemFreeHost(ptr))
    File "/local/home/leof/dev/cuda-python/cuda_core/cuda/core/experimental/_utils.py", line 57, in handle_return
      _check_error(result[0], handle=handle)
    File "/local/home/leof/dev/cuda-python/cuda_core/cuda/core/experimental/_utils.py", line 29, in _check_error
      raise CUDAError(f"{name.decode()}: {desc.decode()}")
  cuda.core.experimental._utils.CUDAError: CUDA_ERROR_INVALID_VALUE: invalid argument
  
    warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))

...

Could you check if explicitly calling Buffer.close() by the end of each test would make it go away? If so we should add it for now and circle back right after beta 1 (#141 needs a long-term solution).

@leofang
Copy link
Member

leofang commented Oct 28, 2024

Could you check if explicitly calling Buffer.close() by the end of each test would make it go away?

Confirmed locally that this is the way to go. mr.deallocate was meant to be called by buf.close, not by end users.

@leofang
Copy link
Member

leofang commented Oct 28, 2024

but let's add them in the next PR.

Tracked in #196. @vzhurba01 @ksimpson-work merge?

@leofang
Copy link
Member

leofang commented Oct 28, 2024

@ksimpson-work feel free to merge!

@ksimpson-work ksimpson-work merged commit 213baf4 into main Oct 29, 2024
@leofang leofang deleted the ksimpson/cuda_core_tests branch October 29, 2024 00:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cuda.core Everything related to the cuda.core module P0 High priority - Must do! test Addition or improved tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add basic functionality tests for cuda.core
4 participants