Experimental Flag for Preserve All Tensors changes model output

[ShardulNalegave]

I am running the YOLOv8n model (Full Integer Quantized) using the LiteRT interpreter. I want to check the outputs of intermediate node tensors for debugging purposes but doing so directly isn't possible as intermediate tensors are freed up by LiteRT (as they should be).

To prevent this I am using the experimental flag "Preserve All tensors". The expected behavior would be execution stays the same but I can read data of intermediate tensors after complete execution. This is not what happens, instead the value of my output layer itself changes.

Is there something that I am doing wrong or missing?

The upper part shows part of my output layer when the flag is disabled and the bottom one shows the same part when the flag is enabled.

[pkgoogle]

Hi @ShardulNalegave, can you please share your main.py file? I think that will help us a lot to figure out what's going on.

[ShardulNalegave]

Hey! Really sorry for the late reply.
Sure sending the file here.

The change in values looks minimal but the reason behind why it does this is not obvious to me.
Thanks for replying!

import numpy as np
import tensorflow as tf
from PIL import Image

def load_image(image_path, input_shape):
  """Preprocess image for TFLite model inference."""
  image = Image.open(image_path).convert("RGB")
  image = image.resize((input_shape[1], input_shape[2]))
  image_np = np.array(image, dtype=np.float32)
  return image_np

# interpreter = tf.lite.Interpreter("yolov8n_full_integer_quant.tflite") # YOLO v8
interpreter = tf.lite.Interpreter("yolov5nu_full_integer_quant.tflite", experimental_preserve_all_tensors=True) # YOLO v5
interpreter.allocate_tensors()

input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()

input_shape = input_details[0]['shape']
input_data = load_image('bus.jpg', input_shape)
input_data = np.expand_dims(input_data, axis=0)

input_data = input_data - 128.0 # Preprocess - Quantisation
interpreter.set_tensor(input_details[0]['index'], input_data.astype(np.int8))

interpreter.invoke()

output_data: np.ndarray = interpreter.get_tensor(output_details[0]['index'])[0].astype(np.float32)
print(output_data.transpose()[0])

[pkgoogle]

Hi @ShardulNalegave, Thanks can you share your process to get yolov5nu_full_integer_quant.tflite or share that file directly?