Updating AxesImage objects (.imshow(), .pcolormesh()) of matplotlib.

[Potescito]

Description

Hi 😊, I recently stumbled upon something interesting when I was attempting to update the data of an AxesImage instance, which differs from updating a standard 2D matplotlib plot. I will describe what I have done in case someone faces the same issues.

*The basic idea: I want to continuously show image frames coming from a sensor device, using matplotlib, not opencv.

Initial example (snippet):

from nicegui import ui
import asyncio
import numpy as np

class refs(object):
 # This is just a helper to hold references
 def __init__(self):
  self.frame = np.zeros((40,40)) # Just a default 0-image
  self.img = None # To save the AxesImage instance
  self.pyplot_element = None # To store the pyplot element

@ui.refreshable 
def _img_plot(refs):
 # Refreshable function for displaying the image frames
 with ui.card():
  with ui.row():
   with ui.pyplot(figsize=(8, 8), close=False) as element:
    ax  = element.fig.gca()
    refs.img = ax.imshow(refs.frame) # Here the AxesImage object is instantiated and the initial plot created
    refs.pyplot_element = element

Now, let's assume I have an asynchronous function that retrieves the frames:

async def continuous_acquisition(refs):
 refs.frame = await asyncio.to_thread(some_function_that_fetches_the_frames()) # Assume I get the frame as an array
 if refs.frame is not None:
  with refs.pyplot_element:
   refs.img.set_data(refs.frame)
   ui.update(refs.pyplot_element)

Guided by the nicegui documentation and this discussion post: #55 (comment), I attempted to use ui.update() to refresh the image plot with new data, but this approach did not work and, in fact, there is no need of ui.update(). Interestingly, even after setting the new data successfully, the plot failed to refresh. The only way in which I could make it work was by either updating the plot's color limits along with the new frames: refs.img.set_clim((refs.frame.min(), refs.frame.max())), or by defining the limits upon the AxesImage object instantiation, if they are known: refs.img = ax.imshow(refs.data, vmin=0, vmax=255). After that, the plot updates every time there is a new frame, without the need of recalling _img_plot.refresh(refs).

I hope this will be helpful to someone! 😊