ci: use cibuildwheel for pyodide test

[henryiii]

Using cibuildwheel for pyodide. Adds testing.

[jpivarski]

Since "Build C++ WASM" still fails, we'll keep this in pr-inactive.

It looks like it has a significant merge conflict, too.

[henryiii]

Okay, now getting a Pyodide fatal error:

Error: Dynamic linking error: cannot resolve symbol _ZN7awkward12ArrayBuilderC1ERKNS_7OptionsIJxdEEE

[jpivarski]

_ZN7awkward12ArrayBuilderC1ERKNS_7OptionsIJxdEEE

is

awkward::ArrayBuilder::ArrayBuilder(awkward::Options<long long, double> const&)

which is core libawkward.

[ariostas]

Okay, now getting a Pyodide fatal error:

Error: Dynamic linking error: cannot resolve symbol _ZN7awkward12ArrayBuilderC1ERKNS_7OptionsIJxdEEE

This issue is solved by bumping CIBW to 2.22. However, the tests are skipped or fail because none of the dependencies are installed and Pyodide doesn't have multiprocessing. Maybe it would make sense to add a tests-wasm directory with a few simple tests that don't have any dependencies.

[agriyakhetarpal]

I've just merged pyodide/pyodide#5247, so awkward-cpp will be fully up to date in the Pyodide 0.27 release which we'll get out any day now. If the fatal error is indeed resolved with cibuildwheel 2.22 here as mentioned by @ariostas, it would be great to revisit this. :)

That said, I can't fully tell what the error was caused by and what resolved it, there weren't many changes between Pyodide 0.26.1 in cibuildwheel < 2.22 and cibuildwheel >=2.22's Pyodide 0.26.4, besides a few more JSPI-related bug fixes for stack switching and a bump from Python 3.12.1 to Python 3.12.7.

[agriyakhetarpal]

I tried it on my fork via agriyakhetarpal#1 and, after temporarily skipping seven tests where ForthMachine does not seem to work with 32-bit environments and emits conversion errors, the rest of the tests all pass now:
https://github.com/agriyakhetarpal/awkward/actions/runs/12483892163/job/34840440332

I'd like to reinstate the WASM job that was disabled as an effect of #3329 so that I can work on some further improvements around interactive docs for awkward via jupyterlite-sphinx, as an extension to the "Try it" button on the homepage. If this would be alright with those here and with the rest of the awkward-cpp developers, could I go ahead and perform those improvements in a new PR that builds on this one?

[jpivarski]

Fixing awkward-cpp's WASM issues is very welcome! (I had been under the impression that it was working. We do have a Windows 32-bit suite of tests to ensure that all integer/pointer handling correctly distinguishes between size_t and uint632_t/uint64_t, so I don't think the WASM 32-bit issue could be only because it's 32-bit...)

awkward-cpp has two main pieces: one is exposed through pybind11 in the ordinary way for a Python extension, and 3 subsystems are accessed this way:

• ArrayBuilder (with all of its component classes and subclasses, including GrowableBuffer)
• ForthMachine (which has ForthInputBuffer and ForthOutputBuffer and that's it—the rest is internal)
• JSON I/O (a few functions for different cases: from string, from file, one big JSON, newline-delimited JSON)

The other main piece is accessed through ctypes, by compiling the C++ functions with an extern C API:

• ArrayBuilder again, so that it can be used in Numba (which can call C ABI function pointers)
• a few hundred "kernels," which are given raw buffers from NumPy arrays for most Awkward Array functionality (so that the CPU-based interface would look as much as possible like a CUDA one)

I think all of these things are loaded on startup, when import awkward is called, so a dynamic link failure in any one of them probably has more to do with the linking itself (visibility of symbols? name clashes? namespaces?) than the implementation. Some explicit C macros were added to ensure visibility, but they might have been needed for reasons that are no longer relevant and (who knows?) might be interfering with the WASM build. I'm referring to EXPORT_SYMBOL defined here:

awkward/awkward-cpp/include/awkward/common.h

Lines 6 to 7 in 0cc66e8

	#ifdef _MSC_VER
	#define EXPORT_SYMBOL __declspec(dllexport)

awkward/awkward-cpp/include/awkward/common.h

Lines 24 to 25 in 0cc66e8

	#else
	#define EXPORT_SYMBOL __attribute__((visibility("default")))

awkward/awkward-cpp/include/awkward/common.h

Lines 36 to 41 in 0cc66e8

	#ifdef __GNUC__
	// Silence a gcc warning: type attributes ignored after type is already defined
	#define EXPORT_TEMPLATE_INST
	#else
	#define EXPORT_TEMPLATE_INST EXPORT_SYMBOL
	#endif

This is one possible culprit, but anything like this is suspect and can be investigated. We have enough tests for all the other builds that if you need to change something to make WASM work and all other tests still pass, it's a good fix. (To be super-cautious, we can also manually check the CUDA tests—which don't run in GitHub Actions—but I think they're very separate from the rest of the build procedure because they use CuPy for runtime compilation.)

[agriyakhetarpal]

Thanks for the extended information! awkward in WASM is working and no longer fails on import, fortunately – it is just the tests in these files:

• test_3209_awkwardforth_read_negative_number_of_items.py, and
• test_1345_avro_reader.py

that are failing at the moment. I am not an experienced C++ programmer in any fashion, but I can try to see where I can debug it while also getting the WASM wheels deployed with the docs again.

[jpivarski]

The Avro reader is a "canary in a coal mine" in the sense that it's one of the few extended uses of AwkwardForth in the Awkward codebase. AwkwardForth is primarily needed for Uproot, but Awkward Array can't load Uproot in its tests without a circular dependence.

Even though these two AwkwardForth-based tests fail, do the Uproot tests run (when using this compiled version of awkward-cpp)? That would give us an indication of how serious the failure is. For instance, what if it's failing in these two tests for a reason that's irrelevant to Uproot? If that's the case, then these two tests are using functionality we may not need and perhaps could remove. AwkwardForth is not a user-facing API (technically public so that a library like Uproot can use it, but not publicized); we can consider removing features.

Of course, it would be even better to understand the failure and fix it. I've been assuming that the issue has something to do with compiling and linking—an area in which I'm not familiar with the details—but if it's a feature that's not used by Uproot, then it's possible that there's a segfault or other undefined behavior in the C++. So, checking the Uproot tests would be informative, either way.

The Uproot tests that use AwkwardForth most intensively are:

• tests/test_0637_setup_tests_for_AwkwardForth.py: this one more than any other—this was used to develop Uproot's use of AwkwardForth, and it probably calls every command in the AwkwardForth language that Uproot ever calls
• tests/test_0816_separate_AwkwardForth_machines_by_TBranch.py: some more aspects of AwkwardForth, outside of the commands themselves
• tests/test_1221_AwkwardForth_bug.py: at one point, we found and fixed a bug
• tests/test_1282_add_known_forth_for_atlas.py: @nikoladze added explicit AwkwardForth (as opposed to the code that Uproot generates on the fly) for some ATLAS data types

Uproot does have a Pyodide test. I think it only tests local files (to avoid unimplemented features in Pyodide, many of which involve threading), but the above tests are based on local files. (scikit-hep-testdata downloads the file locally before using it.)