Enable QNN HTP spill fill buffer setting to save RAM usage.

[HectorSVC]

Description

Enable QNN HTP spill fill buffer setting to save RAM usage.
This feature is available after QNN 2.28. Need to re-generate QNN context binary.
https://docs.qualcomm.com/bundle/publicresource/topics/80-63442-50/htp_backend.html#qnn-htp-backend-api

Requirements:

1. Linux x86_64 platform + QNN QNN 2.28.2.241116
2. Need weight sharing enabled
3. Need to re-generate the Onnx model with QNN context binary by set the EP option enable_htp_spill_fill_buffer = 1.
4. Works for a model with multiple Context binaries. Need manually merge multiple Onnx models with context binary into 1 Onnx model.

command example to generate context model:
./onnxruntime_qnn_ctx_gen -i "soc_model|60 htp_graph_finalization_optimization_mode|3 enable_htp_spill_fill_buffer|1" /mnt/model/share1_part_1.onnx,/mnt/model/share2_part_1.onnx

No need to do extra thing while running the model inference.

The generated EPContext node will have a max_size attribute with the maximum spill fill buffer size for the context binary

[github-actions]

You can commit the suggested changes from lintrunner.

[HectorSVC]

@chiwwang, could you help to take a look?

[chiwwang]

Hi Hector,
This looks good for me but let me ping others and see if they can also take a look.

[HectorSVC]

Comments from QC: The approach has the limitation that it always gets the max spill fill buffer size form the 1st QNN context. The max spill file buffer size should be across all QNN contexts. To fill the gap, we need to go through all QNN context to:

1. Load the QNN context binary buffer and extract the max spill fille buffer size for each QNN context
2. Compare the max spill fille buffer size across all QNN context and track the index of the QNN context
3. Load and deserialize the QNN context (to get the graph info for future execute) which has the max spill fille buffer size first, also set the max spill fill buffer, set the group handle to 0.
4. Load and deserialize other QNN contexts, set the max spill buffer size, and set the group handle to the context in step3.

Considering this feature is mostly target for large models which has large context binary size, so there will be big overhead for step 1 & 2. Another approach is we dump the max spill fill buffer size for each Qnn context in EPContext node when we generate the model to make this information ready ahead of time instead of get it during normal session creation time. We can get the information from all EPContext nodes to get the max size and load that one first.

[quic-ashigarg]

Based on my understanding, the following is occurring:

• Determine the maximum size of spill-fill around the context (there could be multiple graphs).
• Identify the maximum size among all the contexts associated with a group. The first index will always be the context with the maximum size.
• For each context in the list, if it is the first one in the group (i.e., with the maximum spill-fill value), use the context ID of 0x0; otherwise, use the context ID of the first one in the group.

If so, this seems good.