-
Notifications
You must be signed in to change notification settings - Fork 208
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Test failures on PyPy3.10: mostly ValueError: ex_uses_vml parameter is required
#463
Comments
Sorry T_T I forgot to call test(). I'll remember to do this afterwards. My testing will output
I found out that this is caused by importing I don't know why numexpr tries to import |
I'm completely new to |
I think you're testing PyPy3.9; I don't know how well that works with NumPy. We've only tried PyPy3.10. |
I'm using CPython 3.9 and I'm figuring out how to install pypy T_T |
If your distribution doesn't include packages for PyPy, I suppose you can download and run the precompiled version from https://www.pypy.org/download.html. Or possibly you could try a nightly from https://buildbot.pypy.org/nightly/py3.10/. They all can be run straight after unpacking, without having to "install" them in any way. |
Thanks, I'm using mac m1 and now it's working using the precompiled version. Another stupid question is why can't I import |
Yes. |
It's continuously saying |
I'm sorry but I really have no clue about Apple. |
np, I have another windows machine😀
Michał Górny ***@***.***> 于2023年12月2日周六 14:20写道:
… I'm sorry but I really have no clue about Apple.
—
Reply to this email directly, view it on GitHub
<#463 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/A33BDH3XHRNFPGGZFP5LS7LYHMTKLAVCNFSM6AAAAABADLN5FKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMZXGE2DONRQGU>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
I've finally replicated this error on Fedora with pypy 3.9, numexpr 2.8, numpy 1.26, I will try to fix this. |
Ah, I seem to find the problem. When we run in Python:
It will actually invoke In CPython, if we don't provide any python dictionary to I don't quite understand why this will occur because I don't have a comprehensive understanding of how Python binds to C++, but I guess this is something wrong while calling C++ function, it's like the default argument is not 0 in pypy, but it is 0 in CPython. But this could be fixed in |
It is probably a bug in PyPy. Could you help me replicate it so I can fix it? Is there a smaller reproducer than a complete Numexpr build? |
It don't need to build numexpr to replicate it, using pypy and running Since this involves how python calls C/C++ extension, I will learn how to use By the way, is this behaviour considered a bug? I found that although |
So sorry, I don't know how replicate this error with a simple C extension. I tried to write a very simple python extension Anyways, I should fix it in the And may I ask another question, there is still an error says "AttributeError: module 'sys' has no attribute 'getrefcount'", and I tried this snippet: import sys
a = [1]
print(sys.getrefcount(a)) And it gets the same error, but according to the Python standard this should be a valid attribute within module Is this a bug or intended not to implement, or I missed something? |
It is intentional. PyPy does not use a refcounting garbage collector. You are correct that it is hard to find information about that difference, I will add it to the garbage collection strategies section about differences between PyPy and CPython.
|
It's fixed by https://foss.heptapod.net/pypy/pypy/-/commit/402204e196ac7a86975d8d38f6abf2c384f1dd9d, should be closed now. |
Thanks to you both! |
This slight incompatibility with CPython was unconvered in pydata/numexpr#463
I can still reproduce the problem with the newest PyPy3.10 nightly. |
@mgorny you are still seeing |
Yes: $ curl https://buildbot.pypy.org/nightly/py3.10/pypy-c-jit-184208-0c87577e022f-linux64.tar.bz2 | tar -xj
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 31.3M 100 31.3M 0 0 3305k 0 0:00:09 0:00:09 --:--:-- 3358k
$ git clone https://github.com/pydata/numexpr
Cloning into 'numexpr'...
remote: Enumerating objects: 4682, done.
remote: Counting objects: 100% (731/731), done.
remote: Compressing objects: 100% (287/287), done.
remote: Total 4682 (delta 402), reused 660 (delta 363), pack-reused 3951
Receiving objects: 100% (4682/4682), 2.72 MiB | 13.12 MiB/s, done.
Resolving deltas: 100% (3085/3085), done.
$ /tmp/pypy-c-jit-184208-0c87577e022f-linux64/bin/pypy3 -m venv venv
$ . venv/bin/activate
$ pip install ./numexpr/
Processing ./numexpr
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Collecting numpy>=1.13.3
Using cached numpy-1.26.3-pp310-pypy310_pp73-linux_x86_64.whl
Building wheels for collected packages: numexpr
Building wheel for numexpr (pyproject.toml) ... done
Created wheel for numexpr: filename=numexpr-2.8.9.dev1-pp310-pypy310_pp73-linux_x86_64.whl size=100692 sha256=9959c020ee42b83e0eab8e0
30f4a157fc3bb7cfedff70b18b10707e7a7125913
Stored in directory: /tmp/pip-ephem-wheel-cache-0ce9li2k/wheels/6a/90/d2/38a94db313dad36daf3e688b0f11f9019e3d64d3709d562cac
Successfully built numexpr
Installing collected packages: numpy, numexpr
Successfully installed numexpr-2.8.9.dev1 numpy-1.26.3
WARNING: There was an error checking the latest version of pip.
(venv) mgorny@pomiot /tmp $ pypy
Python 3.10.13 (0c87577e022f, Jan 04 2024, 10:42:56)
[PyPy 7.3.15-alpha0 with GCC 10.2.1 20210130 (Red Hat 10.2.1-11)] on linux
Type "help", "copyright", "credits" or "license" for more information.
And now for something completely different: ``snow, snow, snow, snow''
>>>> import numexpr
>>>> numexpr.test()
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Numexpr version: 2.8.9.dev1
NumPy version: 1.26.3
Python version: 3.10.13 (0c87577e022f, Jan 04 2024, 10:42:56)
[PyPy 7.3.15-alpha0 with GCC 10.2.1 20210130 (Red Hat 10.2.1-11)]
Platform: linux-x86_64-#1 SMP PREEMPT_DYNAMIC Mon Jan 1 16:05:17 -00 2024
CPU vendor:
CPU model:
CPU clock speed: MHz
VML available? False
Number of threads used by default: 8 (out of 12 detected cores)
Maximum number of threads: 64
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
....E.EEEE...............E.EEEE...........E.E...................E......................................................................
.......................................................................................................................................
.......................................................................................................................................
.......................................................................................................................................
.......................................................................................................................................
.......................................................................................................................................
.......................................................................................................................................
.......................................................................................................................................
.......................................................................................................................................
.......................................................................................................................................
.......................................................................................................................................
.......................................................................................................................................
.......................................................................................................................................
.......................................................................................................................................
.......................................................................................................................................
.......................................................................................................................................
.......................................................................................................................................
.......................................................................................................................................
.......................................................................................................................................
.......................................................................................................................................
.......................................................................................................................................
.......................................................................................................................................
.......................................................................................................................................
.......................................................................................................................................
.......................................................................................................................................
.......................................................................................................................................
.......................................................................................................................................
.......................................................................................................................................
.......................................................................................................................................
.......................................................................................................................................
.......................................................................................................................................
.......................................................................................................................................
.......................................................................................................................................
.......................................................................................................................................
.......................................................................................................................................
.......................................................................................................................................
.......................................................................................................................................
.......................................................................................................................................
.......................................................................................................................................
.......................................................................................................................................
.......................................................................................................................................
.
======================================================================
ERROR: test_rational_expr (numexpr.tests.test_numexpr.test_numexpr)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/tmp/venv/lib/pypy3.10/site-packages/numexpr/tests/test_numexpr.py", line 76, in test_rational_expr
y = func(a, b)
ValueError: ex_uses_vml parameter is required
======================================================================
ERROR: test_refcount (numexpr.tests.test_numexpr.test_numexpr)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/tmp/venv/lib/pypy3.10/site-packages/numexpr/tests/test_numexpr.py", line 314, in test_refcount
assert sys.getrefcount(a) == 2
AttributeError: module 'sys' has no attribute 'getrefcount'
======================================================================
ERROR: test_simple (numexpr.tests.test_numexpr.test_numexpr)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/tmp/venv/lib/pypy3.10/site-packages/numexpr/tests/test_numexpr.py", line 56, in test_simple
x = func(array([1., 2, 3]), array([4., 5, 6]), array([7., 8, 9]))
ValueError: ex_uses_vml parameter is required
======================================================================
ERROR: test_simple_expr (numexpr.tests.test_numexpr.test_numexpr)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/tmp/venv/lib/pypy3.10/site-packages/numexpr/tests/test_numexpr.py", line 68, in test_simple_expr
y = func(x)
ValueError: ex_uses_vml parameter is required
======================================================================
ERROR: test_simple_expr_small_array (numexpr.tests.test_numexpr.test_numexpr)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/tmp/venv/lib/pypy3.10/site-packages/numexpr/tests/test_numexpr.py", line 62, in test_simple_expr_small_array
y = func(x)
ValueError: ex_uses_vml parameter is required
======================================================================
ERROR: test_rational_expr (numexpr.tests.test_numexpr.test_numexpr2)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/tmp/venv/lib/pypy3.10/site-packages/numexpr/tests/test_numexpr.py", line 76, in test_rational_expr
y = func(a, b)
ValueError: ex_uses_vml parameter is required
======================================================================
ERROR: test_refcount (numexpr.tests.test_numexpr.test_numexpr2)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/tmp/venv/lib/pypy3.10/site-packages/numexpr/tests/test_numexpr.py", line 314, in test_refcount
assert sys.getrefcount(a) == 2
AttributeError: module 'sys' has no attribute 'getrefcount'
======================================================================
ERROR: test_simple (numexpr.tests.test_numexpr.test_numexpr2)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/tmp/venv/lib/pypy3.10/site-packages/numexpr/tests/test_numexpr.py", line 56, in test_simple
x = func(array([1., 2, 3]), array([4., 5, 6]), array([7., 8, 9]))
ValueError: ex_uses_vml parameter is required
======================================================================
ERROR: test_simple_expr (numexpr.tests.test_numexpr.test_numexpr2)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/tmp/venv/lib/pypy3.10/site-packages/numexpr/tests/test_numexpr.py", line 68, in test_simple_expr
y = func(x)
ValueError: ex_uses_vml parameter is required
======================================================================
ERROR: test_simple_expr_small_array (numexpr.tests.test_numexpr.test_numexpr2)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/tmp/venv/lib/pypy3.10/site-packages/numexpr/tests/test_numexpr.py", line 62, in test_simple_expr_small_array
y = func(x)
ValueError: ex_uses_vml parameter is required
======================================================================
ERROR: test_all_scalar (numexpr.tests.test_numexpr.test_evaluate)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/tmp/venv/lib/pypy3.10/site-packages/numexpr/tests/test_numexpr.py", line 495, in test_all_scalar
assert_equal(expr(a, b), 2 * a + 3 * b)
ValueError: ex_uses_vml parameter is required
======================================================================
ERROR: test_broadcasting (numexpr.tests.test_numexpr.test_evaluate)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/tmp/venv/lib/pypy3.10/site-packages/numexpr/tests/test_numexpr.py", line 488, in test_broadcasting
assert_array_equal(expr(a, c), 2.0 * a + 3.0 * c)
ValueError: ex_uses_vml parameter is required
======================================================================
ERROR: test_run (numexpr.tests.test_numexpr.test_evaluate)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/tmp/venv/lib/pypy3.10/site-packages/numexpr/tests/test_numexpr.py", line 501, in test_run
assert_array_equal(expr(a, b), expr.run(a, b))
ValueError: ex_uses_vml parameter is required
----------------------------------------------------------------------
Ran 5536 tests in 11.023s
FAILED (errors=13)
<unittest.runner.TextTestResult run=5536 errors=13 failures=0>
>>>> |
So maybe PyPy still needs the fix from #467? |
Yes, that would help. That still leaves failures from |
I see your reproducer is using |
Yes, though I've used 7.3.14 and not nightly. Do you need me to retry with nightly? |
No, there is no real difference between 7.3.14 and nightly. |
Could this be closed with latests merged PRs? |
Yes, I've just confirmed that all tests pass now. |
When running the test suite on PyPy3.10 7.3.13, I'm getting the following failures:
This is on 05bb401 but I've seen roughly the same result on 2.8.7.
CC @mattip
The text was updated successfully, but these errors were encountered: