Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Option "Using a custom Pyodide distribution" (specified in jupyter-lite.json) not working #59

Closed
michaelweinold opened this issue Sep 14, 2023 · 11 comments · Fixed by #62
Labels
bug Something isn't working

Comments

@michaelweinold
Copy link

michaelweinold commented Sep 14, 2023

Description

Even though I specify the most recent version of pyodide==0.24.0 in the jupyter-lite.json file, as per the JupyterLite Documentation: "Using a custom Pyodide distribution":

{
  "jupyter-lite-schema-version": 0,
  "jupyter-config-data": {
    "disabledExtensions": [
      "@jupyterlab/drawio-extension",
      "jupyterlab-kernel-spy",
      "jupyterlab-tour"
    ],
    "@jupyterlite/pyodide-kernel-extension:kernel": {
      "pyodideUrl": "https://cdn.jsdelivr.net/pyodide/v0.24.0/full/pyodide.js"
    },
  }
}

and the site build from the GH actions workflow

- name: Build the JupyterLite site
        run: |
          cp README.md content
          jupyter lite build --contents content --output-dir dist

completes successfully, the pyodide version does not change from the default 0.23.4 shipped with jupyterlite-pyodide-kernel==0.1.1:

A WebAssembly-powered Python kernel backed by Pyodide
import pyodide
pyodide.__version__
'0.23.4'
import pyodide_kernel
pyodide_kernel.__version__
'0.1.1'

Is there another way to specify the exact version of Pyodide during the build?

Reproduce

Specify pyodide=0.24.0 in the

  1. Use the above .json config file in a fork of the JupyterLite demo.

Expected behavior

The most recent version of pyodide is used.

Context

  • JupyterLite version: jupyterlite-core==0.1.2
  • Operating System and version: GH actions platform
  • Browser and version: various (Safari Version 16.6 (18615.3.12.11.2), Chrome Version 116.0.5845.110 (Official Build) (arm64))
Browser Output
remoteEntry.a37e37c87d212fe85e13.js:1 Unsatisfied version 1.37.1 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @lumino/widgets (required ^2.0.1)
p @ remoteEntry.a37e37c87d212fe85e13.js:1
remoteEntry.a37e37c87d212fe85e13.js:1 Unsatisfied version 1.10.3 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @lumino/messaging (required ^2.0.0)
p @ remoteEntry.a37e37c87d212fe85e13.js:1
remoteEntry.a37e37c87d212fe85e13.js:1 Unsatisfied version 1.12.1 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @lumino/coreutils (required ^2.0.0)
p @ remoteEntry.a37e37c87d212fe85e13.js:1
remoteEntry.a37e37c87d212fe85e13.js:1 Unsatisfied version 1.11.1 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @lumino/signaling (required ^2.0.0)
p @ remoteEntry.a37e37c87d212fe85e13.js:1
remoteEntry.a37e37c87d212fe85e13.js:1 Unsatisfied version 1.9.2 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @lumino/algorithm (required ^2.0.0)
p @ remoteEntry.a37e37c87d212fe85e13.js:1
remoteEntry.a37e37c87d212fe85e13.js:1 Unsatisfied version 3.5.3 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @jupyterlab/translation (required ^4.0.5)
p @ remoteEntry.a37e37c87d212fe85e13.js:1
remoteEntry.a37e37c87d212fe85e13.js:1 Unsatisfied version 3.5.3 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @jupyterlab/settingregistry (required ^4.0.5)
p @ remoteEntry.a37e37c87d212fe85e13.js:1
remoteEntry.a37e37c87d212fe85e13.js:1 Unsatisfied version 3.5.3 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @jupyterlab/notebook (required ^4.0.5)
p @ remoteEntry.a37e37c87d212fe85e13.js:1
remoteEntry.a37e37c87d212fe85e13.js:1 Unsatisfied version 6.5.3 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @jupyterlab/services (required ^7.0.5)
p @ remoteEntry.a37e37c87d212fe85e13.js:1
remoteEntry.a37e37c87d212fe85e13.js:1 Unsatisfied version 1.8.2 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @lumino/properties (required ^2.0.0)
p @ remoteEntry.a37e37c87d212fe85e13.js:1
remoteEntry.a37e37c87d212fe85e13.js:1 No satisfying version (^4.0.5) of shared module @jupyterlab/outputarea found in shared scope default.
Available versions: 3.5.3 from _JUPYTERLAB.CORE_OUTPUT
p @ remoteEntry.a37e37c87d212fe85e13.js:1
remoteEntry.a37e37c87d212fe85e13.js:1 Unsatisfied version 3.5.3 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @jupyterlab/logconsole (required ^4.0.5)
p @ remoteEntry.a37e37c87d212fe85e13.js:1
remoteEntry.a37e37c87d212fe85e13.js:1 Unsatisfied version 3.5.3 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @jupyterlab/rendermime (required ^4.0.5)
p @ remoteEntry.a37e37c87d212fe85e13.js:1
remoteEntry.a37e37c87d212fe85e13.js:1 Unsatisfied version 1.10.4 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @lumino/disposable (required ^2.0.0)
p @ remoteEntry.a37e37c87d212fe85e13.js:1
remoteEntry.a37e37c87d212fe85e13.js:1 Unsatisfied version 3.5.3 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @jupyterlab/mainmenu (required ^4.0.5)
p @ remoteEntry.a37e37c87d212fe85e13.js:1
remoteEntry.b15a25cb741a6c7381f8.js:1 Unsatisfied version 1.37.1 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @lumino/widgets (required ^2.0.1)
s @ remoteEntry.b15a25cb741a6c7381f8.js:1
remoteEntry.6a76d3e37f02d3977b44.js:1 Unsatisfied version 3.5.3 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @jupyterlab/apputils (required ^4.0.0)
d @ remoteEntry.6a76d3e37f02d3977b44.js:1
remoteEntry.6a76d3e37f02d3977b44.js:1 Unsatisfied version 1.9.2 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @lumino/algorithm (required ^2.0.0)
d @ remoteEntry.6a76d3e37f02d3977b44.js:1
remoteEntry.6a76d3e37f02d3977b44.js:1 Unsatisfied version 1.37.1 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @lumino/widgets (required ^2.0.1)
d @ remoteEntry.6a76d3e37f02d3977b44.js:1
remoteEntry.6a76d3e37f02d3977b44.js:1 Unsatisfied version 3.5.3 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @jupyterlab/ui-components (required ^4.0.0)
d @ remoteEntry.6a76d3e37f02d3977b44.js:1
remoteEntry.ade722230aea9b7793b2.js:1 Unsatisfied version 1.37.1 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @lumino/widgets (required ^1.37.2)
u @ remoteEntry.ade722230aea9b7793b2.js:1
remoteEntry.edc72ee71d4ec67598f6.js:1 Unsatisfied version 1.37.1 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @lumino/widgets (required ^1.37.2)
s @ remoteEntry.edc72ee71d4ec67598f6.js:1
remoteEntry.d50eb71a2b1d4b9ee0b9.js:1 Unsatisfied version 1.37.1 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @lumino/widgets (required ^1.37.2)
s @ remoteEntry.d50eb71a2b1d4b9ee0b9.js:1
service-manager.ts:50 Registering new JupyterLite ServiceWorker https://michaelweinold.github.io/jl_pyodide_test/service-worker-f7de73d.js
service-manager.ts:52 JupyterLite ServiceWorker was sucessfully registered
index.ts:235 Kernel filesystem and JupyterLite contents will be synced
index.js:51 Pyodide contents will be synced with Jupyter Contents
pyodide.asm.js:9 Loading micropip, packaging
pyodide.asm.js:9 Loaded packaging, micropip
pyodide.asm.js:9 Loading sqlite3
pyodide.asm.js:9 Loaded sqlite3
pyodide.asm.js:9 Loading decorator, jedi, parso, pygments, six
pyodide.asm.js:9 Loaded decorator, six, parso, jedi, pygments
@michaelweinold michaelweinold added the bug Something isn't working label Sep 14, 2023
@michaelweinold
Copy link
Author

Hey, @jtpio - sorry for the quick ping. I would greatly appreciate it if you could have a quick look. Is this something that needs to be fixed in either the pyodide-kernel or jupyterlite?

I added a package to pyodide back in June, which was only now included in the 0.24.0 release - I'd very much like to demo that at our conference next week. If would be very helpful for me to find out if I will be able to use the latest version of pyodide within a JupyterLite environment by then.

@jtpio
Copy link
Member

jtpio commented Sep 14, 2023

@michaelweinold normally the config option as specified in the top comment should do the trick.

But we could also update the Pyodide version in this repo and make a new release. Would you like to open a PR? The diff would be similar to #50.

Thanks!

@michaelweinold
Copy link
Author

...ok, since the config option seems to be correct as-is, I opened the PR you suggested: #61

@jtpio
Copy link
Member

jtpio commented Sep 14, 2023

Maybe there is still an issue with the config option that would have to be fixed anyway (would need to check).

@bollwyvl
Copy link
Contributor

There are more than a few breaking changes in the new release. The first one is the renaming of repodata.json to pyodide-lock.json, which we check in a couple places, but there are probably others.

@bollwyvl
Copy link
Contributor

micropip also changed a lot. #62 is open as an alternative, exploring the changes. Mostly seems to work locally, we'll see what CI says.

@michaelweinold
Copy link
Author

Maybe there is still an issue with the config option that would have to be fixed anyway (would need to check).

It would seem so, @jtpio - I tried again with:

{
    "jupyter-lite-schema-version": 0,
    "jupyter-config-data": {
        "settingsStorageDrivers": ["asyncStorage"],
        "contentsStorageDrivers": ["asyncStorage"],
        "litePluginSettings": {
            "@jupyterlite/pyodide-kernel-extension:kernel": {
                "pyodideUrl": "https://cdn.jsdelivr.net/pyodide/v0.26.2/full/pyodide.js"
            },
        },
    },
}

But I still get:

Screenshot 2024-07-30 at 08 34 06

The GH Actions build log also does not tell me which version was ultimately used.

@jtpio
Copy link
Member

jtpio commented Jul 30, 2024

@michaelweinold could it be because this JSON snippet contains trailing commas ?

I just tried in a clean folder with jupyterlite-core==0.4.0 and jupyterlite-pyodide-kernel==0.4.0 and it seems to be working fine:

{
    "jupyter-lite-schema-version": 0,
    "jupyter-config-data": {
        "settingsStorageDrivers": ["asyncStorage"],
        "contentsStorageDrivers": ["asyncStorage"],
        "litePluginSettings": {
            "@jupyterlite/pyodide-kernel-extension:kernel": {
                "pyodideUrl": "https://cdn.jsdelivr.net/pyodide/v0.26.2/full/pyodide.js"
            }
        }
    }
}

image

@jtpio
Copy link
Member

jtpio commented Jul 30, 2024

could it be because this JSON snippet contains trailing commas ?

If that's the case and it fixes your issue, maybe the jupyter lite build process should validate the jupyter-lite.json if it exists, and error out if it can't be parsed correctly.

@michaelweinold
Copy link
Author

...it was the trailing commas. Having removed them:

{
    "jupyter-lite-schema-version": 0,
    "jupyter-config-data": {
        "settingsStorageDrivers": ["asyncStorage"],
        "contentsStorageDrivers": ["asyncStorage"],
        "litePluginSettings": {
            "@jupyterlite/pyodide-kernel-extension:kernel": {
                "pyodideUrl": "https://cdn.jsdelivr.net/pyodide/v0.26.2/full/pyodide.js"
            }
        }
    }
}

Gives me the correct Pyodide version:

Screenshot 2024-07-30 at 11 04 38

@jtpio
Copy link
Member

jtpio commented Jul 30, 2024

Thanks @michaelweinold for checking 👍

I opened jupyterlite/jupyterlite#1447 to improve this in the jupyter lite build process.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants