Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ensure that errors due to env instantiation are printed #215

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

Ensure that errors due to env instantiation are printed #215

wants to merge 1 commit into from

Conversation

MichaelHatherly
Copy link
Collaborator

Addresses #210 by making sure that the underlying error is printed out in quarto logs when it happens.

MichaelHatherly
MichaelHatherly commented Nov 26, 2024
View reviewed changes
src/worker.jl Outdated Show resolved Hide resolved
@MichaelHatherly
Copy link
Collaborator Author

The errors now show up when run via quarto as follows:

QuartoNotebookRunner on  mh/surface-env-errors [!?] is 📦 v0.11.7 via ஃ v1.11.1
❯ QUARTO_JULIA_PROJECT=/Users/mike/pumas/QuartoNotebookRunner quarto render debug/file.qmd
Starting julia control server process. This might take a while...
Precompiling QuartoNotebookRunner...
  1 dependency successfully precompiled in 12 seconds. 55 already precompiled.
Julia server process started.
ERROR: Julia server returned error after receiving "run" command:
Failed to run notebook: /Users/mike/pumas/QuartoNotebookRunner/debug/file.qmd
ERROR: Remote exception from Malt.Worker on port 9009 with PID 48009:

The following 1 dependency failed to precompile:

REPL

Failed to precompile REPL [3fa0cd96-eef1-5676-8a61-b3b8758bbffb] to "/Users/mike/.julia/compiled/v1.11/REPL/jl_i2hD96".
ERROR: LoadError: ArgumentError: Package REPL does not have StyledStrings in its dependencies:
- Note that the following manifests in the load path were resolved with a different
  julia version, which may be the cause of the error. Try to re-resolve them in the
  current version, or consider deleting them if that fails:
    /Users/mike/pumas/QuartoNotebookRunner/debug/Manifest.toml (v1.10.6)
- You may have a partially installed environment. Try `Pkg.instantiate()`
  to ensure all packages in the environment are installed.
- Or, if you have REPL checked out for development and have
  added StyledStrings as a dependency but haven't updated your primary
  environment's manifest file, try `Pkg.resolve()`.
- Otherwise you may need to report an issue with REPL
Stacktrace:
 [1] macro expansion
   @ ./loading.jl:2226 [inlined]
 [2] macro expansion
   @ ./lock.jl:273 [inlined]
 [3] __require(into::Module, mod::Symbol)
   @ Base ./loading.jl:2198
 [4] #invoke_in_world#3
   @ ./essentials.jl:1089 [inlined]
 [5] invoke_in_world
   @ ./essentials.jl:1086 [inlined]
 [6] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:2191
 [7] include
   @ ./Base.jl:557 [inlined]
 [8] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_
   @ Base ./loading.jl:2790
 [9] top-level scope
   @ stdin:5
in expression starting at /Users/mike/.julia/juliaup/julia-1.11.1+0.aarch64.apple.darwin14/share/julia/stdlib/v1.11/REPL/src/RE
in expression starting at stdin:
Stacktrace:
  [1] unwrap_worker_result(worker::Malt.Worker, result::Malt.WorkerResult)
    @ Malt ~/.julia/packages/Malt/YJ2Ml/src/Malt.jl:50
  [2] _wait_for_response(worker::Malt.Worker, msg_id::UInt64)
    @ Malt ~/.julia/packages/Malt/YJ2Ml/src/Malt.jl:325
  [3] _send_receive
    @ ~/.julia/packages/Malt/YJ2Ml/src/Malt.jl:336 [inlined]
  [4] #remote_call_fetch#41
    @ ~/.julia/packages/Malt/YJ2Ml/src/Malt.jl:406 [inlined]
  [5] remote_call_fetch
    @ ~/.julia/packages/Malt/YJ2Ml/src/Malt.jl:405 [inlined]
  [6] remote_eval_fetch
    @ ~/.julia/packages/Malt/YJ2Ml/src/Malt.jl:484 [inlined]
  [7] remote_eval_fetch
    @ ~/.julia/packages/Malt/YJ2Ml/src/Malt.jl:485 [inlined]
  [8] init!
    @ ~/pumas/QuartoNotebookRunner/src/server.jl:112 [inlined]
  [9] QuartoNotebookRunner.File(path::String, options::Dict{String, Any})
    @ QuartoNotebookRunner ~/pumas/QuartoNotebookRunner/src/server.jl:26
 [10] (::QuartoNotebookRunner.var"#41#44"{Bool, Dict{String, Any}, Server, String})()
    @ QuartoNotebookRunner ~/pumas/QuartoNotebookRunner/src/server.jl:1155
 [11] lock(f::QuartoNotebookRunner.var"#41#44"{Bool, Dict{String, Any}, Server, String}, l::ReentrantLock)
    @ Base ./lock.jl:232
 [12] borrow_file!(f::QuartoNotebookRunner.var"#32#36"{Nothing, String, Bool, Dict{String, Any}, QuartoNotebookRunner.var"#chunk_callback#70"{TCPSocket}, Server}, server::Server, path::String; optionally_create::Bool, options::Dict{String, Any})
    @ QuartoNotebookRunner ~/pumas/QuartoNotebookRunner/src/server.jl:1148
 [13] borrow_file!
    @ ~/pumas/QuartoNotebookRunner/src/server.jl:1139 [inlined]
 [14] #run!#31
    @ ~/pumas/QuartoNotebookRunner/src/server.jl:1108 [inlined]
 [15] _handle_response(socket::TCPSocket, notebooks::Server, request::@NamedTuple{type::String, content::Union{Dict{String, Any}, String}}, showprogress::Bool)
    @ QuartoNotebookRunner ~/pumas/QuartoNotebookRunner/src/socket.jl:293
 [16] (::QuartoNotebookRunner.var"#60#68"{@NamedTuple{type::String, content::Union{Dict{String, Any}, String}}, Bool, Server})()
    @ QuartoNotebookRunner ~/pumas/QuartoNotebookRunner/src/socket.jl:202
 [17] #invokelatest#2
    @ ./essentials.jl:1055 [inlined]
 [18] invokelatest
    @ ./essentials.jl:1052 [inlined]
 [19] _try_revise(f::QuartoNotebookRunner.var"#60#68"{@NamedTuple{type::String, content::Union{Dict{String, Any}, String}}, Bool, Server})
    @ QuartoNotebookRunner ~/pumas/QuartoNotebookRunner/src/socket.jl:231
 [20] (::QuartoNotebookRunner.var"#59#67"{Bool, Base.RefValue{Bool}, Server, Base.UUID})()
    @ QuartoNotebookRunner ~/pumas/QuartoNotebookRunner/src/socket.jl:201
ERROR: Internal julia server error

Stack trace:
    at writeJuliaCommand (file:///Applications/quarto/bin/quarto.js:41721:19)
    at eventLoopTick (ext:core/01_core.js:153:7)
    at async executeJulia (file:///Applications/quarto/bin/quarto.js:41615:22)
    at async Object.execute (file:///Applications/quarto/bin/quarto.js:41352:20)
    at async renderExecute (file:///Applications/quarto/bin/quarto.js:78033:27)
    at async renderFileInternal (file:///Applications/quarto/bin/quarto.js:78201:43)
    at async renderFiles (file:///Applications/quarto/bin/quarto.js:78069:17)
    at async render (file:///Applications/quarto/bin/quarto.js:82929:21)
    at async Command.actionHandler (file:///Applications/quarto/bin/quarto.js:83077:32)
    at async Command.execute (file:///Applications/quarto/bin/quarto.js:8017:13)

@felixcremer, @penelopeysm give this branch a try if you're able to and see whether it's showing up the errors correctly for you as well?

@MichaelHatherly MichaelHatherly linked an issue Nov 26, 2024 that may be closed by this pull request
QuartoNotebookWorker precompile error when trying quarto render #210
Open
@MichaelHatherly MichaelHatherly force-pushed the mh/surface-env-errors branch 2 times, most recently from 1c35e3e to 81b19d6 Compare November 26, 2024 14:49
@penelopeysm
Copy link

Works for me 😄

@jkrumbiegel
Copy link
Collaborator

Maybe we should put this behind an env variable flag because I've read once on Slack how people were annoyed not being able to run certain code when a dep wouldn't precompile that they didn't actually need. So there should maybe be an escape hatch to avoid that in case the user cannot fix the precompile failure.

@codecov Codecov
Copy link

codecov bot commented Nov 26, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 51.63%. Comparing base (6c3920e) to head (74643fc).
Report is 1 commits behind head on main.

Additional details and impacted files 
@@           Coverage Diff           @@
##             main     #215   +/-   ##
=======================================
  Coverage   51.63%   51.63%           
=======================================
  Files          30       30           
  Lines        1286     1286           
=======================================
  Hits          664      664           
  Misses        622      622

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@MichaelHatherly
Copy link
Collaborator Author

So there should maybe be an escape hatch to avoid that in case the user cannot fix the precompile failure.

Preferable we'd not need to add extra escape hatches for this. Perhaps there's a more straightforward approach to this check, maybe just checking if the manifest has the right julia_version would be sufficient.

@felixcremer
Copy link

I can also see the error message, but I don't like that it is drowned out by the whole stack trace and if I wouldn't know what to look for I might have missed the important part of the error.

There is Base.collect_manifest_warnings() which gives the text of the warning that is shown when the manifest file was resolved with a different julia version and there is also Base.find_unsuitable_manifests_versions() which gives the list of unsuitable manifest files. Maybe we could use these to throw a warning but not give a full error, so that if the manifest would still work a user can decide for themselves whether to resolve or not. I don't know the QNR code good enough to think about the best place to insert such a warning so that it is visible to the user.
Our current example is really bad, because it brings the QuartoNotebookWorker itself to a crash and then it is not clear as a user what to do about it.

@MichaelHatherly
Copy link
Collaborator Author

drowned out by the whole stack trace

Right, agreed that there's a lot there.

Pretty sure this part is not able to be changed:

Stack trace:
    at writeJuliaCommand (file:///Applications/quarto/bin/quarto.js:41721:19)
    at eventLoopTick (ext:core/01_core.js:153:7)
    at async executeJulia (file:///Applications/quarto/bin/quarto.js:41615:22)
    at async Object.execute (file:///Applications/quarto/bin/quarto.js:41352:20)
    at async renderExecute (file:///Applications/quarto/bin/quarto.js:78033:27)
    at async renderFileInternal (file:///Applications/quarto/bin/quarto.js:78201:43)
    at async renderFiles (file:///Applications/quarto/bin/quarto.js:78069:17)
    at async render (file:///Applications/quarto/bin/quarto.js:82929:21)
    at async Command.actionHandler (file:///Applications/quarto/bin/quarto.js:83077:32)
    at async Command.execute (file:///Applications/quarto/bin/quarto.js:8017:13)

I'm sure the Malt part can also be dropped:

Stacktrace:
  [1] unwrap_worker_result(worker::Malt.Worker, result::Malt.WorkerResult)
    @ Malt ~/.julia/packages/Malt/YJ2Ml/src/Malt.jl:50
  [2] _wait_for_response(worker::Malt.Worker, msg_id::UInt64)
    @ Malt ~/.julia/packages/Malt/YJ2Ml/src/Malt.jl:325
  [3] _send_receive
    @ ~/.julia/packages/Malt/YJ2Ml/src/Malt.jl:336 [inlined]
  [4] #remote_call_fetch#41
    @ ~/.julia/packages/Malt/YJ2Ml/src/Malt.jl:406 [inlined]
  [5] remote_call_fetch
    @ ~/.julia/packages/Malt/YJ2Ml/src/Malt.jl:405 [inlined]
  [6] remote_eval_fetch
    @ ~/.julia/packages/Malt/YJ2Ml/src/Malt.jl:484 [inlined]
  [7] remote_eval_fetch
    @ ~/.julia/packages/Malt/YJ2Ml/src/Malt.jl:485 [inlined]
  [8] init!
    @ ~/pumas/QuartoNotebookRunner/src/server.jl:112 [inlined]
  [9] QuartoNotebookRunner.File(path::String, options::Dict{String, Any})
    @ QuartoNotebookRunner ~/pumas/QuartoNotebookRunner/src/server.jl:26
 [10] (::QuartoNotebookRunner.var"#41#44"{Bool, Dict{String, Any}, Server, String})()
    @ QuartoNotebookRunner ~/pumas/QuartoNotebookRunner/src/server.jl:1155
 [11] lock(f::QuartoNotebookRunner.var"#41#44"{Bool, Dict{String, Any}, Server, String}, l::ReentrantLock)
    @ Base ./lock.jl:232
 [12] borrow_file!(f::QuartoNotebookRunner.var"#32#36"{Nothing, String, Bool, Dict{String, Any}, QuartoNotebookRunner.var"#chunk_callback#70"{TCPSocket}, Server}, server::Server, path::String; optionally_create::Bool, options::Dict{String, Any})
    @ QuartoNotebookRunner ~/pumas/QuartoNotebookRunner/src/server.jl:1148
 [13] borrow_file!
    @ ~/pumas/QuartoNotebookRunner/src/server.jl:1139 [inlined]
 [14] #run!#31
    @ ~/pumas/QuartoNotebookRunner/src/server.jl:1108 [inlined]
 [15] _handle_response(socket::TCPSocket, notebooks::Server, request::@NamedTuple{type::String, content::Union{Dict{String, Any}, String}}, showprogress::Bool)
    @ QuartoNotebookRunner ~/pumas/QuartoNotebookRunner/src/socket.jl:293
 [16] (::QuartoNotebookRunner.var"#60#68"{@NamedTuple{type::String, content::Union{Dict{String, Any}, String}}, Bool, Server})()
    @ QuartoNotebookRunner ~/pumas/QuartoNotebookRunner/src/socket.jl:202
 [17] #invokelatest#2
    @ ./essentials.jl:1055 [inlined]
 [18] invokelatest
    @ ./essentials.jl:1052 [inlined]
 [19] _try_revise(f::QuartoNotebookRunner.var"#60#68"{@NamedTuple{type::String, content::Union{Dict{String, Any}, String}}, Bool, Server})
    @ QuartoNotebookRunner ~/pumas/QuartoNotebookRunner/src/socket.jl:231
 [20] (::QuartoNotebookRunner.var"#59#67"{Bool, Base.RefValue{Bool}, Server, Base.UUID})()
    @ QuartoNotebookRunner ~/pumas/QuartoNotebookRunner/src/socket.jl:201

The following needs to stay as far as I'm concerned, if the Pkg.precompile approach is kept:


The following 1 dependency failed to precompile:

REPL

Failed to precompile REPL [3fa0cd96-eef1-5676-8a61-b3b8758bbffb] to "/Users/mike/.julia/compiled/v1.11/REPL/jl_i2hD96".
ERROR: LoadError: ArgumentError: Package REPL does not have StyledStrings in its dependencies:
- Note that the following manifests in the load path were resolved with a different
  julia version, which may be the cause of the error. Try to re-resolve them in the
  current version, or consider deleting them if that fails:
    /Users/mike/pumas/QuartoNotebookRunner/debug/Manifest.toml (v1.10.6)
- You may have a partially installed environment. Try `Pkg.instantiate()`
  to ensure all packages in the environment are installed.
- Or, if you have REPL checked out for development and have
  added StyledStrings as a dependency but haven't updated your primary
  environment's manifest file, try `Pkg.resolve()`.
- Otherwise you may need to report an issue with REPL
Stacktrace:
 [1] macro expansion
   @ ./loading.jl:2226 [inlined]
 [2] macro expansion
   @ ./lock.jl:273 [inlined]
 [3] __require(into::Module, mod::Symbol)
   @ Base ./loading.jl:2198
 [4] #invoke_in_world#3
   @ ./essentials.jl:1089 [inlined]
 [5] invoke_in_world
   @ ./essentials.jl:1086 [inlined]
 [6] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:2191
 [7] include
   @ ./Base.jl:557 [inlined]
 [8] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_
   @ Base ./loading.jl:2790
 [9] top-level scope
   @ stdin:5
in expression starting at /Users/mike/.julia/juliaup/julia-1.11.1+0.aarch64.apple.darwin14/share/julia/stdlib/v1.11/REPL/src/RE
in expression starting at stdin:

@felixcremer
Copy link

Yes dropping the Malt part and maybe removing the stacktrace of the precompile error would make it much more digestible.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jkrumbiegel jkrumbiegel jkrumbiegel left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

QuartoNotebookWorker precompile error when trying quarto render
4 participants
@MichaelHatherly @penelopeysm @jkrumbiegel @felixcremer