USB UVC support for device_next

[josuah]

I would be nice to have UVC support for Zephyr USB stack.

• usb: device_next: new USB Video Class (UVC) implementation #76798

Also I need it soon, so I will gather ideas and try get a PoC going.
Nothing presentable yet, only enough to get video frames in/out as an early goal.

There seems to be an existing effort merged into Zephyr, which lead to a Zephyr sample:
#17194 - Video 4 Zephyr - Video API, Drivers and Samples

[josuah]

I am almost ready with this: now preparing the final integration with the Video API.

The descriptors can be encoded on the devicetree like this:

&udc0 {
        /* IN and OUT counting together as one endpoint */
        num-bidir-endpoints = <2>;

        /* full-speed: 12 Mhz, high-speed: 480 Mhz, super-speed: 5000 Mhz */
        maximum-speed = "super-speed";

        uvc0: zephyr_uvc0 {
                compatible = "zephyr,uvc";
                status = "okay";

                source = <&video_app>;

                port {
                        uvc0_in: endpoint {
                                remote-endpoint = <&app_out>;
                        };
                };

                format_mjpeg {
                        compatible = "zephyr,uvc-format-mjpeg", "zephyr,uvc-format";
                        fourcc = "JPEG";

                        640x400 {
                                size = <640 400>;
                                fps = <15>;
                        };

                        64x64 {
                                size = <64 64>;
                                fps = <15>;
                        };
                };

                format_uncompressed {
                        compatible = "zephyr,uvc-format-uncompressed", "zephyr,uvc-format";
                        fourcc = "YUYV";
                        guid = [UVC_GUID_YUY2];
                        bits-per-pixel = <16>;

                        640x400 {
                                size = <640 400>;
                                fps = <15>;
                        };

                        64x64 {
                                size = <64 64>;
                                fps = <30>;
                        };
                };
        };
};

This gets exposed to the host as USB descriptors, and to Zephyr as struct video_caps.

I will make a draft pull request with a sample for any device_next supported device, and the sw_generator.c as source.

[uLipe]

@josuah , when this PR comes out, could you please tag me into, I'm doing some experiments with the Arduino Nicla Vision, and UVC would save me from writing custom transport over USB (I'm using cdc-acm and a protocol on top of it).

UVC on Nicla Vision would be great since it has high-speed USB support present on Zephyr.

[josuah]

@ehughes I had some issue with NXP LinkServer to get the image on the MCXN947, but will try again once I figure out what went wrong. I can also try with JLink also supported. SWD alone does not permit flashing... Maybe without XIP though it'll work.

I had it enumerate with the STM32H743 board from WeAct Studio though.
It is recognized as an /dev/video2 interface (assuming your laptop webcam is /dev/video0).
So likely it is not too far away from being mergeable.

[uLipe]

@josuah do you have the draft branch where I can try out with my Nicla Visio?

[josuah]

Here it is! #76798
I am currently bringing-up the UART to get logs instead of default USB: I will need to log the transfers to make sure I did not misunderstand the USB API.

[ehughes]

@ehughes I had some issue with NXP LinkServer to get the image on the MCXN947, but will try again once I figure out what went wrong. I can also try with JLink also supported. SWD alone does not permit flashing... Maybe without XIP though it'll work.

I had it enumerate with the STM32H743 board from WeAct Studio though. It is recognized as an /dev/video2 interface (assuming your laptop webcam is /dev/video0). So likely it is not too far away from being mergeable.

FYI:

I use segger/jlink on MCXN947 and it works well. I typically use segger ozone to debug as you can just point it towards /build/zephyr/zephyr.elf. It detects when a rebuild happens and can auto reload.