[CM4] BCM-USB-MSD seems broken

[dirkdimehead]

Describe the bug

In order to be able to easily do updates to an embedded device in the field, I would like to use BCM-USB-MSD (0x5) before eMMC.
Unfortunately I'm failing at that with two problems. I tried it with pieeprom-2023-01-11.bin and pieeprom-2024-01-22.bin.
My boot.conf:

[all]
BOOT_UART=1
WAKE_ON_GPIO=1
POWER_OFF_ON_HALT=0

BOOT_ORDER=0xf15

ENABLE_SELF_UPDATE=1

If I plug in a keyboard everything looks good:

XHCI-STOP
xHC ver: 272 HCS: 01000140 0c0000f1 07ff000a HCC: 0220fe65
xHC ports 1 slots 64 intrs 1
USB2[1] 00020ae1 connected
USB2[1] 00200a03 connected enabled
USB2 root HUB port 1 init
DEV [01:00] 1.16 000000:01 class 0 VID 0dc6 PID 2013
HID [01:00] 1.16 000000:01 register HID

But 11 of 12 different USB sticks are not recognized:

Boot mode: BCM-USB-MSD (05) order f1
XHCI-STOP
xHC ver: 272 HCS: 01000140 0c0000f1 07ff000a HCC: 0220fe65
xHC ports 1 slots 64 intrs 1
USB2[1] 000206e1 connected
USB2[1] 00200603 connected enabled
USB2 root HUB port 1 init
xHC-CMD err: 4 type: 11 [01:00] 0.00 000000:01
   EVT (33   1) 10 f0 f6 3f 00 00 00 00 00 00 00 04 01 84 00 01
   CMD (11   1) 00 20 f7 3f 00 00 00 00 00 00 00 00 01 2c 00 01
SLOT IN
00 00 10 08 00 00 01 00
00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00
SLOT OUT
00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00
EP0 CTX
00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00
xHC-CMD err: 4 type: 11 [01:00] 0.00 000000:01
   EVT (33   1) 40 f0 f6 3f 00 00 00 00 00 00 00 04 01 84 00 01
   CMD (11   1) 00 20 f7 3f 00 00 00 00 00 00 00 00 01 2c 00 01
SLOT IN

The error messages are identical and repeat continuously until the timeout. All sticks are recognized without any problems under Linux.

second problem
If I activate BCM-USB-MSD and have xhci (regardless of otg_mode=1 in config.txt or overlay) active in Linux at the same time, the eMMC boot process hangs until I insert a USB stick.

MESS:00:00:32.786805:0: brfs: File read: /mfs/sd/Image.bin
MESS:00:00:32.789185:0: Loaded 'Image.bin' to 0x200000 size 0x15dea00
MESS:00:00:32.795365:0: Device tree loaded to 0x2eff2c00 (size 0xd3a0)
MESS:00:00:32.804018:0: uart: Set PL011 baud rate to 103448.300000 Hz
MESS:00:00:32.810664:0: uart: Baud rate change done...
MESS:00:00:32.812686:0:

I would be grateful for any help.

Greetings
Dirk

Steps to reproduce the behaviour

see above

Device (s)

Raspberry Pi CM4

Bootloader configuration.

[all]
BOOT_UART=1
WAKE_ON_GPIO=1
POWER_OFF_ON_HALT=0

BOOT_ORDER=0xf15

ENABLE_SELF_UPDATE=1

System

USB is connected directly to a USB-A socket without another HUB-IC etc!

Bootloader logs

No response

USB boot

No response

NVMe boot

No response

Network (TFTP boot)

No response

[dirkdimehead]

New insight:
if I insert a "passive" USB HUB in between, neither problem occurs. BCM-USB-MSD seems to expect a USB hub (IC).
Unfortunately, the hardware is already finished and partially produced :-(

[timg236]

It's probably worth comparing results on a CM4IO board (or other boards). Booting from a USB drive with the BCM XHCI controller is pretty common so it sounds like there is another variable here

For interop questions, it's worth asking on the Compute Module forums
https://forums.raspberrypi.com/viewforum.php?f=98&sid=baba68dc73ed3677f253cea03298b0c1

[timg236]

Merging this as duplicate of #588

There's some test firmware which might resolve the problem for you here
#527 (comment)