Building "collapse" package for use with webr in jupyterlite webr kernel

[psychemedia]

I've been stuck on this for a couple of hours now, and don't see what I'm doing wrong?

I'm trying to use the plm package, which has a dependency collapse which pulls in an item in its NAMESPACE file from parallel, which is understandably not in the core webr package. The function that is loaded from parallel is guarded by a parallel= flag with a default FALSE, so I assume that commenting out the NAMESPACE item in the package shouldn't have a side effect if R doesn't statically check that all functions are declared.

I tried downloading the collapse package source, commenting the unwanted import out, tarring the package and then attempting to compile it using the webr package builder action, but couldn't find a way of naming the local file that didn't cause an error when using the package builder action.

I (mistakenly) noticed an issue on the collapse repo (SebKrantz/collapse#550) pointing to what I mistook for a non-parallel branch, and thought to try to import src file using the package string github::SebKrantz/collapse@no_parallel_simd. The build job runs but the collapse package is not built.

So what am I doing wrong, in the two respects of:

• missing something in the naming convention and/or packaging of local package files;
• not being able to build from a source file imported from a github repo (irrespective of this not being the solution I originally thought it might be; purely: how do I specify a GitHub repo branch to build from?)

[SebKrantz]

The no_parallel_simd branch is something else, having to do with OpenMP. The parallel package is part of base R now and should be included in any R distribution. I only conditionally use the mclapply() function from it.

Please try the following: remove the import line in the NAMESPACE and replace all instances of mclappy with parallel::mclapply. If that works I can change the code towards this end.

[psychemedia]

Thanks, will give that a try.

Updates made, but I'm obviously doing something elementarily wrong in trying to build the package! I need to take a break from this for a while because I'm just not seeing it?!

https://github.com/ouseful-testing/webr-package-repo-demo/actions/runs/10917893917/job/30302138793#step:5:332

https://github.com/ouseful-testing/webr-package-repo-demo/blob/74b2f532796bc6fd277e97e07108b17c2e85c198/.github/workflows/test-package_build.yaml#L13

[georgestagg]

I am not sure why collapse_1.9.6.tar.gz has not been found in the GitHub Actions run that you have linked, I can see the file exists in the repo. Perhaps something has gone wrong and the archive is not a valid R package anymore.

In any case there are some bigger issues here.

@SebKrantz is correct, the parallel package should be available as part of the webR distribution. You should be able to run library(parallel) to load it from the network. It succeeds on https://webr.r-wasm.org/v0.3.3/, for example.

This indicates to me that your base webR distribution is corrupt in some way. IIRC you are using your own baseUrl? On your server there should be a directory vfs served as part of the base webR distribution. Is that directory available? In there the parallel package is served as under the path .../vfs/usr/lib/R/library/parallel.data.

Once that is working, you should be able to load the collapse package without modifying its source code at all. Again, as an example, it can be loaded at https://webr.r-wasm.org/v0.3.3/.

---

A bigger problem is that I notice that separately to this you are trying to compile the curl package. Unfortunately, that isn't going to work at all.

The curl package relies on the system dependency libcurl and we haven't compiled a version of libcurl for webR/Wasm yet. In fact, this is a very hard problem because WebAssembly in a browser does not have direct access to the network sockets on the host machine.

The only real way forward is to create a "shim" curl package or system library that replaces HTTP curl requests with JavaScript browser API calls. This is something we're planning on looking at at some point in the future, but the project will be a large undertaking. Curl has a wide API and supports many network protocols, and so it is not entirely clear which subset of curl APIs can be translated into browser APIs, not to speak about actually implementing them.

[psychemedia]

@georgestagg yes, the curl was in there as a leftover (trying to get all the dependencies / imports for all the packages that are used in notebooks in the distribution (TTR, tseries, qauntmod and quantprod are also broken in the distribution. But they don't break the build).

Re: the tar.gz: I downloaded it from https://cran.r-project.org/src/contrib/Archive/collapse/, made the edit, then tar -czvf collapse_2.0.15.tar.gz collapse (on a Mac).

Re: the custom path: ah, parallel is missing from ./vfs/usr/lib/R/library/.... and .. doh! There was .gitignore inmy repo that had the webr files from https://github.com/r-wasm/webr/releases/tag/v0.3.2 that knocked some of the files out. Bah... Apols for causing the distraction.

[psychemedia]

parallel should be available - my mistake in the webr distribution

[georgestagg]

Re: the tar.gz: I downloaded it from https://cran.r-project.org/src/contrib/Archive/collapse/, made the edit, then tar -czvf collapse_2.0.15.tar.gz collapse (on a Mac).

Interesting, I believe that should work even if the work is not actually required here. It would be interesting to know if the original .tar.gz worked. I can see you're compiling other source files using local:: and they seem to work OK?

[psychemedia]

@georgestagg Yes, the others work fine (MASS was from the same CRAN source but untouched).