H264: Error decoding image from local MCAP

[robotville]

Hello, I am having issues visualizing H264 encoded video from an MCAP file. I get a red exclamation mark and the error decoding image message.

I have verified that the video is not corrupted by decoding with ROS image transport.

[snosenzo]

Hi, thanks for filing the issue! Would you be able to provide a snippet of some sample data of this happening? If not publically feel free to email it to me at sam[at]foxglove[dot]dev.
One thing you can check yourself is the error code being reported to chrome devtools, which might be more informative than what the decoder gives you. Here's how you can do that:

1. Open Foxglove and the source you wish to debug and open chrome devtools and go to the Media tab,
2. Create a layout in foxglove with only a single problematic video topic and playback until you get the error.
3. in the Media tab's list of Players there may be multiple players, click through and select the Messages tab for each until you see something like this:

This will give you more information about the error including an error code which might give you more specifics. Send it here when you have it and I can take a look too.

[snosenzo]

Another thing you can try for the web version of Foxglove is going to chrome://flags in your address bar and disabling "Hardware-accelerated video decode". Sometimes decode errors can show up using one or the other unfortunately.

[robotville]

Thanks very much for your help. For some reason my Players tab is empty, but I do see messages in the console that point to the error:
3952.8430f264489acaba9488.js:598 DOMException: The source image could not be decoded. handleDecodeError @ 3952.8430f264489acaba9488.js:598

I believe the most likely culprit given this error message is that the video data is somehow corrupted.

This evidence pointed me to some reference documentation where I realized that I had forgotten to run this command before viewing the MCAP
ros2 run isaac_ros_data_replayer foxglove_converter.py <input> <output>

By doing this conversion, everything works perfectly. Therefore this was not a problem with Foxglove at all, but rather an issue with my own data processing.

Thanks very much for your help it is greatly appreciated.

[snosenzo]

This might be because the original schema uses CompressedImage, whereas Foxglove requires the foxglove.CompressedVideo schema for video topics. They have the same structure but different values are supported for the format field, and the messages go through different pipelines to be visualized. I believe that converter likely just converts those messages to the proper schema. Another good hint that this is the case is the fact that you saw nothing in the Players tab, which means that no video decoders were configured for your topic. This needs to happen before video frames can even begin to be decoded, so the fact that you were getting decode errors without that happening means it was likely not being processed as a video frame.
Hope that clears things up a bit better :)