Issues with ipykernel and ipython in VS Code using Rye

[Fei-Wang]

Hi everyone,

VS Code mentions that Jupyter Notebook functionality only requires installing ipython and ipykernel. However, I prefer not to install these packages separately for each project. I've tried using rye tools to install ipykernel and ipython, but they don't seem to work properly.

Has anyone else encountered this issue or knows how to get it working correctly in VS Code?

Any help would be appreciated!

Thanks!

https://github.com/microsoft/vscode-jupyter/wiki/Jupyter-Kernels-and-the-Jupyter-Extension#python-extension-and-ipykernel

rye tools list --include-version
ipykernel 6.29.5 ([email protected])
ipython 8.26.0 ([email protected])

[bluss]

I don't know any way to not install ipykernel into each notebook environment, except maybe the include system site packages option (ipykernel must then be installed like with pip install --user or similar.).

What's the problem you want to solve?

If it's disk space, then Uv by default copies installed packages from cache in a smart way and identical environments and packages don't use more space (see example from uv help). The default option saves disk space for you already.

      --link-mode <LINK_MODE>
          The method to use when installing packages from the global cache.
          
          Defaults to `clone` (also known as Copy-on-Write) on macOS, and `hardlink` on Linux and
          Windows.
          
          [env: UV_LINK_MODE=]

          Possible values:
          - clone:    Clone (i.e., copy-on-write) packages from the wheel into the site packages
          - copy:     Copy packages from the wheel into the site packages
          - hardlink: Hard link packages from the wheel into the site packages

[Fei-Wang]

@bluss
I understand that installing ipykernel into each notebook environment is generally required. However, my main concern is to avoid manually installing the same Python packages across multiple projects. I noticed in the Rye tools documentation that examples are provided for installing tools like black and ruff. Therefore, I’m wondering if it’s possible to install some global utility packages using tools.

If tools don't support this functionality, what specific tools are supported? Is there a way to know in advance which packages can be supported by tools?

Thank you for your assistance!

[bluss]

Each tool installs into its own venv. Rye does not in general have features for using multiple venvs at a time (that's what you require, right, if you want to use packages from multiple venvs at the same time?). Apart from the system site packages as mentioned, I don't have other ideas.

The tools feature is based on running as executables, where the venv isolation works well (usually).

I would say in my experience, I work with jupyter notebooks: the only mandatory dependency to add is ipykernel, and it's easy to add. (A tool like pyproject-local-kernel even reminds you if ipykernel is not present.) You can have many notebooks in the same directory, notebooks that have common dependencies or belong to the same project. Using Rye this way has been a joy (and a lifesaver to be honest, to be able to check it all into git, using jupytext to have plain .py notebooks.)

[Fei-Wang]

@bluss

I'm not looking to use packages from multiple virtual environments simultaneously. What I want is to have ipykernel set up via Rye tools so that VS Code can directly use it. From your explanation, I understand that tools installed via Rye are command-line tools, while VS Code needs them as libraries, not command-line tools. So, if I use rye tools install ipykernel, VS Code wouldn't recognize it. Is that correct?

By the way, pyproject-local-kernel looks interesting; I plan to try it out.

[philippemiron]

I think a workaround would be to use rye add ipykernel --dev.

[bluss]

@Fei-Wang What about other dependencies that the notebook wants to use, like for example numpy (just an example). Do you have such dependencies, and where do you want to install them?

This is what I mean by using packages from multiple environments. You have dependencies and you have ipykernel. You're asking how to install them into multiple separate environments? But the solution that works well is to install the notebook dependencies and ipykernel into one environment. You can pick any location for that environment and just use it from vscode, as long as it's one environment.

[Fei-Wang]

@philippemiron @bluss

Yes, that is also my current solution. However, my ideal approach is to use ipykernel as a tool for rye, without needing to install it in the venv of each Python project, similar to how ruff and black are used. In each Python project venv, only the other dependency packages required by the project, such as numpy, torch, etc., are installed. At present, this does not seem feasible. The possible reason is that VS Code needs to use ipykernel as a package, while rye tools can only be used as command line tools after installation?

[bluss]

That's not the main reason. The main reason is that ipykernel expects to be able to import the dependencies your notebook is using. If ipykernel is not installed in the same environment as those dependencies, it doesn't work. If it's possible to coerce it into doing this somehow, I don't know. Good luck with the further search. 🙂

[bluss]

Ok, so I've learned something new. You can chain virtualenvs to some extent.

• We knew about the virtualenv -> system interpreter chaining using --system-site-packages, that's an option
• using sitecustomize or .pth files to include packages from an other environment

Given an installation of ipykernel in for example ~/global_ipykernel, you can use it from another virtualenv using the command:

echo $HOME/global_ipykernel/.venv/lib/python3.12/site-packages > .venv/lib/python3.12/site-packages/_ipykernel.pth

Uv or Rye don't in general support just adding .pth files in their configuration, it ruins their dependency tracking. But you could add it by proxy using a specially crafted local project.

The ipykernel is installed not really as "a tool" but if wanted you could use it's tool installation environment as the base.