GPU Acceleration

Current State

PhotonVision currently supports GPU acceleration on one niche but popular platform - Pi3(B+)/CM3(+)/PiZ2W, with Pi Cameras (V1/V2).
The acceleration takes care of capturing the image via MMAL, thresholding it with an OpenGL shader, and then preparing it for the journey to the CPU. (More detail/links/references needed)

This only works on Pi3 based platforms as the OpenGL drivers on Pi4 do not support the features used that enable the zero-copy to the CPU.

Pi3 Details

Declan pls fill in here

Summer 2023 Goal

To make more platforms as performant as the Pi3+PiCam. The current targets are Pi4 (via libcamera) and Nvidia Jetson (via v4l + CUDA shared memory).

Pi4

@carbotaniuman on the FRC discord had begun work on using libcamera to accelerate image capture and thresholding on the Pi4 here: libcamera_meme.

Jetson (CUDA)

@bankst