Is it possible to obtain detailed information about a CellExecutionError in jupyterbook?

[CalMacCQ]

I use jupyterbook fairly often in my work. I find it very useful to display notebooks as built html pages.

Here is my book -> https://tket.quantinuum.com/examples/contextual_optimization.html

I usually have a CI workflow that executes the notebooks and builds the html pages. I use the "-W" flag in jupyter-book build to treat warnings as errors so the workflow will fail if a cell fails to execute.

I've recently been having issues with certain notebooks rasing a CellExecutionError which I'm having a hard time reproducing locally.

The output logs in github actions show that a particular notebook was the cause of the issue.

sphinx.errors.SphinxWarning: /Users/callum/work_projects/pytket/examples/circuit_generation_example.ipynb:Executing notebook failed: CellExecutionError [mystnb.exec]

he above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/callum/work_projects/pytket/.venv/bin/jupyter-book", line 8, in <module>
    sys.exit(main())
             ^^^^^^
             
 .................

  File "/Users/callum/work_projects/pytket/.venv/lib/python3.11/site-packages/jupyter_book/cli/main.py", line 317, in build
    builder_specific_actions(
  File "/Users/callum/work_projects/pytket/.venv/lib/python3.11/site-packages/jupyter_book/cli/main.py", line 525, in builder_specific_actions
    raise RuntimeError(_message_box(msg, color="red", doprint=False)) from result
RuntimeError: 

However it'd be nice to get some information about the specific lines of code that caused the CellExecutionError in the logs.

For instance if my circuit_generation_example.ipynb had a line of code like "hello" + 14 I'd like to get the following output in the logs.

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: can only concatenate str (not "int") to str`

Is this possible within jupuyterbook at the moment? I'd appreciate any help with this.

Thanks in advance :)

==============================

What I've tried so far...

I thought I could achieve this by using the -- verbose flag with jupyter-book build but this doesn't seem to work.

I've also tried the folllowing in _config.yml but no luck.

execute:
  stderr_output: show

[agahkarakuzu]

@CalMacCQ the book you linked looks quite fascinating!

One option in GitHub actions would be to look for reports directory, where an error log will appear when execution fails. Leaving here a function that I use to fetch it:

def get_reports_dir(root_dir):
    """
    Depending on the format of the Jupyter Book (single or multipage),
    the location of the reports vary. This helper function
    checks both possible options.
    """
    multi_page_path = f"{root_dir}/_build/html/reports" 
    single_page_path = f"{root_dir}/_build/_page/index/singlehtml/reports"
    if os.path.exists(multi_page_path) and os.path.isdir(multi_page_path):
        return multi_page_path
    elif os.path.exists(single_page_path) and os.path.isdir(single_page_path):
        return multi_page_path
    else: 
        return None

You can either create a step that looks for error logs in the reports directory which posts the directory as a variable, then use conditional artifact upload to have logs files uploaded at the end of a run.

This get_reports_dir approach would be redundant if you are always working with multi-page book builds, you can simply raise an exception in a step if the /_build/html/reports is not empty, then upload the whole folder as an artifact in the step that follows.

[CalMacCQ]

Thank you :)

I will give the fetching reports idea a try when I can.

[agoose77]

Please see this thread — I think it should prove useful :)