I attempted to use https://spec.oneapi.io/level-zero/latest/tools/api.html#zetkernelgetprofileinfo to store register use when compiling the kernel, but I am not having success getting zeModuleCreate to respect the profile flag (the register count always returns 0). I looked at SYCL APIs for getting registers and was not successful there either (I don't think privateMemory is what we need, but I could not get that to work anyway). But, IGC is able to get the number of registers allocated as that info shows up in debug logs if the compilation fails due to register spills. We likely need to raise this with the IGC team and/or the level zero team to see if there is a known config that allows retrieving allocated registers for a kernel, or if we need to create a feature request.

@EikanWang maybe we should create a feature reqeust to IGC team and/or the level zero team for this feature?

Hi, @vlad-penkin , we need this feature for performance optimization, will Triton team raise this request to the IGC team ?

Hi, @vlad-penkin , we need this feature for performance optimization, will Triton team raise this request to the IGC team ?

Yes

Conceptually this is similar to what Triton (for the XPU target) already does to get the spill size from IGC. The code for that is (see driver.c in the intel third_party directory):

We use L0 APIs to retrieve the structure ze_kernel_properties_t . That structure declaration is:

So it doesn't contain a field for the number of registers used by the kernel. We need the L0 API team to add the n_regs field to that structure. Once the L0 API is updated, the work in Triton to expose the n_regs field to users (e.g. PyTorch) is pretty straightforward.

@vlad-penkin can you please open a feature request for this ?

@vlad-penkin can you please open a feature request for this ?
Done.

@etaf you should receive an email notification on this feature request creation

@vlad-penkin can you please open a feature request for this ?
Done.

@etaf you should receive an email notification on this feature request creation

Thanks.

After investigating further this issue is not applicable to us - our GPUs have a fixed set of registers per thread and we do not scale occupancy of a work group based on register pressure (unlike CUDA, which dynamically adjust registers allocated per thread and adjust parallelism down for kernels with high memory pressure). Therefore, returning 0 is appropriate here because we do not want to tell the PyTorch Inductor auto-tuner to tune based on a parameter that has no effect.

Note that with #1654 we now auto-tune the GRF mode when compiling the Triton kernel to recompile with large GRF mode in the presence of register spills. I suppose we could consider doing this tuning in PyTorch Inductor instead, but that would require quite a bit more work and it is not clear what the benefit would be over what we are doing now.

@vlad-penkin I think we can close this ticket.

After investigating further this issue is not applicable to us - our GPUs have a fixed set of registers per thread and we do not scale occupancy of a work group based on register pressure (unlike CUDA, which dynamically adjust registers allocated per thread and adjust parallelism down for kernels with high memory pressure). Therefore, returning 0 is appropriate here because we do not want to tell the PyTorch Inductor auto-tuner to tune based on a parameter that has no effect.

Note that with #1654 we now auto-tune the GRF mode when compiling the Triton kernel to recompile with large GRF mode in the presence of register spills. I suppose we could consider doing this tuning in PyTorch Inductor instead, but that would require quite a bit more work and it is not clear what the benefit would be over what we are doing now.

@vlad-penkin I think we can close this ticket.

Thanks @alexbaden , agree.
And In Inductor side, besides tune GRF mode, we can reduce RBLOCK in in the presence of register spills.