Interactive profiling

[maleadt]

VTune is great, but it can currently only profile a whole application or attach and start/stop manually. It'd be great to be able to denote profile ranges a la CUDA.@profile, and in addition to be able to do so with an attached program for interactive profiling.

@vchuravy pointed me to Intel's Instrumentation and Tracing Technology (ITT), https://github.com/intel/ittapi, as an equivalent for NVIDIA's NVTX, which has similar APIs:

• ranges: https://software.intel.com/content/www/us/en/develop/documentation/vtune-help/top/api-support/instrumentation-and-tracing-technology-apis/instrumentation-tracing-technology-api-reference/domain-api.html
• markers: https://software.intel.com/content/www/us/en/develop/documentation/vtune-help/top/api-support/instrumentation-and-tracing-technology-apis/instrumentation-tracing-technology-api-reference/event-api.html

Apparently ITT is already being built as part of LLVM for the purpose of JIT registration, so we could take it from there.

[maleadt]

Looks like we should pick up ITT from VTune using an env var:

$ ~/intel/oneapi/vtune/2021.7.1/bin64/vtune -collect gpu-offload /home/tim/.cache/jl/installs/bin/linux/x64/1.7/julia-1.7-latest-linux-x86_64/bin/julia                                                        
vtune: Collection started. To stop the collection, either press CTRL-C or enter from another console window: vtune -r /home/tim/.julia/dev/r000go -command stop.

julia> filter(item->contains(item[1], "INTEL"), ENV)
Dict{String, String} with 8 entries:
  "INTEL_LIBITTNOTIFY64"   => "/home/tim/intel/oneapi/vtune/2021.7.1/lib64/runtime/libittnotify_collector.so"
  "INTEL_LIBITTNOTIFY32"   => "/home/tim/intel/oneapi/vtune/2021.7.1/lib32/runtime/libittnotify_collector.so"
  "INTEL_JIT_PROFILER64"   => "/home/tim/intel/oneapi/vtune/2021.7.1/lib64/runtime/libittnotify_collector.so"
  "INTEL_JIT_PROFILER32"   => "/home/tim/intel/oneapi/vtune/2021.7.1/lib32/runtime/libittnotify_collector.so"

There doesn't seem to be an API to force a write of the current profile, so I don't think we can do interactive profiling. Also, the user needs to set these env vars manually when attaching: https://software.intel.com/content/www/us/en/develop/documentation/vtune-help/top/api-support/instrumentation-and-tracing-technology-apis/basic-usage-and-configuration/attaching-itt-apis-to-a-launched-application.html (although oneAPI.jl could try to detect them). Maybe we can do something very hacky and launch VTune to attach to ourselves... 🤔