Cache inference #2141
Cache inference #2141
Conversation
Benchmark Results
Benchmark PlotsA plot of the benchmark results have been uploaded as an artifact to the workflow run for this PR. |
| @@ -55,7 +74,8 @@ function EnzymeInterpreter( | |||
| reverse_rules::Bool, | |||
| deferred_lower::Bool = true, | |||
| broadcast_rewrite::Bool = true, | |||
| handler = nothing | |||
| handler = nothing, | |||
| local_cache = get_or_create_inference_cache(world, forward_rules, reverse_rules, deferred_lower, broadcast_rewrite, handler) | |||
There was a problem hiding this comment.
If I understand correctly, the local_cache ought to be ephermal and can't be shared across multiple abstract interpreter instances.
cc: @aviatesk
There was a problem hiding this comment.
Yeah but in this case we actually are effectively caching the absint itself (since all data about it is in the key)
There was a problem hiding this comment.
All caches that ought to survive the absint go into the global cache?
| sv::AbsIntState, | ||
| max_methods::Int, | ||
| ) | ||
| ret = Core.invoke(Core.Compiler.abstract_call_gf_by_type, Tuple{AbstractInterpreter, Any, ArgInfo, StmtInfo, Any, AbsIntState, Int}, interp, f, arginfo, si, atype, sv, max_methods) |
There was a problem hiding this comment.
We were spending a huge amount of runtime on that function (which appears to compile as jl_invoke). Tried playing around with to fix to no avail
There was a problem hiding this comment.
Yeah this change doesn't do anything.
|
I spoke with @aviatesk and I think this is fine, but also sadly this did not resolve the issue”we spend like 50 mins in absint” problems |
No description provided.