Intellisense for notebooks
Concatentation of the notebook cells
Pylance can now understand notebooks
Intellisense in VS code works by sending LSP requests to a separate process (well in most cases, see this for more info)
Something like so:
Intellisense for notebooks works pretty much the same way but with each cell of a notebook being a text document:
This poses a problem for the language server (Pylance) because code from one cell can be referenced in another.
Example:
In that example, the pandas import crosses the cell boundary.
This means pylance cannot just analyze each cell individually.
The solution was changes to LSP:
| Message | When it is sent | What it's for |
|---|---|---|
| DidOpenNotebookDocument | On notebook open | Sends the cells and their contents on open |
| DidChangeNotebookDocument | On moving, deleting or adding cells | For cell changes, moves, deletes, and adds |
| DidSaveNotebookDocument | On notebook save | When the notebook is saved its relative path is established. This is necessary to determine location with respect to imports. |
| DidCloseNotebookDocument | On notebook close | Server can clean up in memory state about the notebook. |
These 4 new messages means that pylance knows the relative order of all of the cells in a notebook. Here's generally the sequence of events:
From this pylance knows that:
- There are 3 cells
- They belong to the notebook
foo.ipynb - Their order is 3, 1, 2
So if the notebook looked like so:
Internally pylance could have a concatenated document like so (pylance may or may not concat the cells):
# Cell 1
import pandas as pd
# Cell 2
df = pd.read_csv('./data/test.csv')
# Cell 3
df.head()There's still one missing part in this design. How does pylance know what interpreter to use?
When Pylance is started, the Python extension tells it what the pythonPath is through the workspace/configuration message:
But a notebook uses a kernel. The kernel isn't tied to the global pythonPath.
To workaround this problem, pylance sends the workspace/configuration for every notebook opened.
The Python extension handles this message and then asks if it knows the pythonPath for a URI.
If Jupyter has a kernel active for a notebook, it returns the known pythonPath for that kernel.
Sometimes things go wrong with intellisense. And one of the most difficult parts of diagnosing these failures is reproing the problem.
Pylance logs a lot of data about