Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TQMa8MxML: Problem boot from some micro SD cards #4

Open
wico-frad opened this issue Sep 9, 2024 · 12 comments
Open

TQMa8MxML: Problem boot from some micro SD cards #4

wico-frad opened this issue Sep 9, 2024 · 12 comments
Assignees

Comments

@wico-frad
Copy link

Hi everyone,

We are experiencing boot issues with our embedded module TQMa8MxML when using certain micro SD cards, specifically the SanDisk MAX ENDURANCE microSD™ Card - 32GB Class 10.

The u-boot fails to load the Device Tree Blob from the micro SD card, resulting in the error message: 'WARN: Cannot load the DT!'. The boot log below illustrates the failure after enabling #define CONFIG_MMC_TRACE.

Does anyone have any insights into why this might be occurring?

 BuildInfo:
  - ATF cfccd00
  - U-Boot 2020.04-tq-ga04fdffac7

Normal Boot
Hit any key to stop autoboot:  3 ��� 2 ��� 1 ��� 0 
Booting from mmc ...
CMD_SEND:0
		ARG			 0x00000000
		MMC_RSP_NONE
CMD_SEND:8
		ARG			 0x000001aa
		MMC_RSP_R1,5,6,7 	 0x000001aa 
CMD_SEND:55
		ARG			 0x00000000
		MMC_RSP_R1,5,6,7 	 0x00000120 
CMD_SEND:41
		ARG			 0x41300000
		MMC_RSP_R3,4		 0x41ff8000 
CMD_SEND:55
		ARG			 0x00000000
		MMC_RSP_R1,5,6,7 	 0x00000120 
CMD_SEND:41
		ARG			 0x41300000
		MMC_RSP_R3,4		 0x40ff8000 
CMD_SEND:55
		ARG			 0x00000000
		MMC_RSP_R1,5,6,7 	 0x00000120 
CMD_SEND:41
		ARG			 0x41300000
		MMC_RSP_R3,4		 0xc0ff8000 
CMD_SEND:2
		ARG			 0x00000000
		MMC_RSP_R2		 0x03534453 
		          		 0x48333247 
		          		 0x8097e54d 
		          		 0x40017b00 
					DUMPING DATA
					000 - 03 53 44 53 
					004 - 48 33 32 47 
					008 - 80 97 e5 4d 
					012 - 40 01 7b 00 
CMD_SEND:3
		ARG			 0x00000000
		MMC_RSP_R1,5,6,7 	 0xaaaa0520 
CMD_SEND:9
		ARG			 0xaaaa0000
		MMC_RSP_R2		 0x400e0032 
		          		 0xdb790000 
		          		 0xedc87f80 
		          		 0x0a404000 
					DUMPING DATA
					000 - 40 0e 00 32 
					004 - db 79 00 00 
					008 - ed c8 7f 80 
					012 - 0a 40 40 00 
CMD_SEND:7
		ARG			 0xaaaa0000
		MMC_RSP_R1,5,6,7 	 0x00000700 
CMD_SEND:55
		ARG			 0xaaaa0000
		MMC_RSP_R1,5,6,7 	 0x00000920 
CMD_SEND:51
		ARG			 0x00000000
		MMC_RSP_R1,5,6,7 	 0x00000920 
CMD_SEND:6
		ARG			 0x00fffff1
		MMC_RSP_R1,5,6,7 	 0x00000900 
CMD_SEND:55
		ARG			 0xaaaa0000
		MMC_RSP_R1,5,6,7 	 0x00000920 
CMD_SEND:6
		ARG			 0x00000002
		MMC_RSP_R1,5,6,7 	 0x00000920 
CMD_SEND:6
		ARG			 0x80fffff1
		MMC_RSP_R1,5,6,7 	 0x00000900 
CMD_SEND:55
		ARG			 0xaaaa0000
		MMC_RSP_R1,5,6,7 	 0x00000920 
CMD_SEND:13
		ARG			 0x00000000
		MMC_RSP_R1,5,6,7 	 0x00000920 
switch to partitions #0, OK
mmc1 is current device
CMD_SEND:0
		ARG			 0x00000000
		MMC_RSP_NONE
CMD_SEND:8
		ARG			 0x000001aa
		MMC_RSP_R1,5,6,7 	 0x000001aa 
CMD_SEND:55
		ARG			 0x00000000
		MMC_RSP_R1,5,6,7 	 0x00000120 
CMD_SEND:41
		ARG			 0x41300000
		MMC_RSP_R3,4		 0x40ff8000 
CMD_SEND:55
		ARG			 0x00000000
		MMC_RSP_R1,5,6,7 	 0x00000120 
CMD_SEND:41
		ARG			 0x41300000
		MMC_RSP_R3,4		 0x40ff8000 
CMD_SEND:55
		ARG			 0x00000000
		MMC_RSP_R1,5,6,7 	 0x00000120 
CMD_SEND:41
		ARG			 0x41300000
		MMC_RSP_R3,4		 0xc0ff8000 
CMD_SEND:2
		ARG			 0x00000000
		MMC_RSP_R2		 0x03534453 
		          		 0x48333247 
		          		 0x8097e54d 
		          		 0x40017b00 
					DUMPING DATA
					000 - 03 53 44 53 
					004 - 48 33 32 47 
					008 - 80 97 e5 4d 
					012 - 40 01 7b 00 
CMD_SEND:3
		ARG			 0x00000000
		MMC_RSP_R1,5,6,7 	 0xaaaa0520 
CMD_SEND:9
		ARG			 0xaaaa0000
		MMC_RSP_R2		 0x400e0032 
		          		 0xdb790000 
		          		 0xedc87f80 
		          		 0x0a404000 
					DUMPING DATA
					000 - 40 0e 00 32 
					004 - db 79 00 00 
					008 - ed c8 7f 80 
					012 - 0a 40 40 00 
CMD_SEND:7
		ARG			 0xaaaa0000
		MMC_RSP_R1,5,6,7 	 0x00000700 
CMD_SEND:55
		ARG			 0xaaaa0000
		MMC_RSP_R1,5,6,7 	 0x00000920 
CMD_SEND:51
		ARG			 0x00000000
		MMC_RSP_R1,5,6,7 	 0x00000920 
CMD_SEND:6
		ARG			 0x00fffff1
		MMC_RSP_R1,5,6,7 	 0x00000900 
CMD_SEND:55
		ARG			 0xaaaa0000
		MMC_RSP_R1,5,6,7 	 0x00000920 
CMD_SEND:6
		ARG			 0x00000002
		MMC_RSP_R1,5,6,7 	 0x00000920 
CMD_SEND:6
		ARG			 0x80fffff1
		MMC_RSP_R1,5,6,7 	 0x00000900 
CMD_SEND:55
		ARG			 0xaaaa0000
		MMC_RSP_R1,5,6,7 	 0x00000920 
CMD_SEND:13
		ARG			 0x00000000
		MMC_RSP_R1,5,6,7 	 0x00000920 
CMD_SEND:16
		ARG			 0x00000200
		MMC_RSP_R1,5,6,7 	 0x00000900 
CMD_SEND:17
		ARG			 0x00000000
		MMC_RSP_R1,5,6,7 	 0x00000900 
CMD_SEND:16
		ARG			 0x00000200
		MMC_RSP_R1,5,6,7 	 0x00000900 
CMD_SEND:17
		ARG			 0x00004000
		MMC_RSP_R1,5,6,7 	 0x00000900 
CMD_SEND:16
		ARG			 0x00000200
		MMC_RSP_R1,5,6,7 	 0x00000900 
CMD_SEND:17
		ARG			 0x00000000
		MMC_RSP_R1,5,6,7 	 0x00000900 
CMD_SEND:16
		ARG			 0x00000200
		MMC_RSP_R1,5,6,7 	 0x00000900 
CMD_SEND:17
		ARG			 0x00004000
		MMC_RSP_R1,5,6,7 	 0x00000900 
CMD_SEND:16
		ARG			 0x00000200
		MMC_RSP_R1,5,6,7 	 0x00000900 
CMD_SEND:18
		ARG			 0x00004154
		RET			 -70
CMD_SEND:16
		ARG			 0x00000200
		RET			 -110
CMD_SEND:16
		ARG			 0x00000200
		RET			 -110
CMD_SEND:16
		ARG			 0x00000200
		RET			 -110
CMD_SEND:16
		ARG			 0x00000200
		RET			 -110
CMD_SEND:16
		ARG			 0x00000200
		RET			 -110
CMD_SEND:16
		ARG			 0x00000200
		RET			 -110
CMD_SEND:0
		ARG			 0x00000000
		MMC_RSP_NONE
CMD_SEND:8
		ARG			 0x000001aa
		MMC_RSP_R1,5,6,7 	 0x000001aa 
CMD_SEND:55
		ARG			 0x00000000
		MMC_RSP_R1,5,6,7 	 0x00000120 
CMD_SEND:41
		ARG			 0x41300000
		MMC_RSP_R3,4		 0x40ff8000 
CMD_SEND:55
		ARG			 0x00000000
		MMC_RSP_R1,5,6,7 	 0x00000120 
CMD_SEND:41
		ARG			 0x41300000
		MMC_RSP_R3,4		 0x40ff8000 
CMD_SEND:55
		ARG			 0x00000000
		MMC_RSP_R1,5,6,7 	 0x00000120 
CMD_SEND:41
		ARG			 0x41300000
		MMC_RSP_R3,4		 0xc0ff8000 
CMD_SEND:2
		ARG			 0x00000000
		MMC_RSP_R2		 0x03534453 
		          		 0x48333247 
		          		 0x8097e54d 
		          		 0x40017b00 
					DUMPING DATA
					000 - 03 53 44 53 
					004 - 48 33 32 47 
					008 - 80 97 e5 4d 
					012 - 40 01 7b 00 
CMD_SEND:3
		ARG			 0x00000000
		MMC_RSP_R1,5,6,7 	 0xaaaa0520 
CMD_SEND:9
		ARG			 0xaaaa0000
		MMC_RSP_R2		 0x400e0032 
		          		 0xdb790000 
		          		 0xedc87f80 
		          		 0x0a404000 
					DUMPING DATA
					000 - 40 0e 00 32 
					004 - db 79 00 00 
					008 - ed c8 7f 80 
					012 - 0a 40 40 00 
CMD_SEND:7
		ARG			 0xaaaa0000
		MMC_RSP_R1,5,6,7 	 0x00000700 
CMD_SEND:55
		ARG			 0xaaaa0000
		MMC_RSP_R1,5,6,7 	 0x00000920 
CMD_SEND:51
		ARG			 0x00000000
		MMC_RSP_R1,5,6,7 	 0x00000920 
CMD_SEND:6
		ARG			 0x00fffff1
		MMC_RSP_R1,5,6,7 	 0x00000900 
CMD_SEND:55
		ARG			 0xaaaa0000
		MMC_RSP_R1,5,6,7 	 0x00000920 
CMD_SEND:6
		ARG			 0x00000002
		MMC_RSP_R1,5,6,7 	 0x00000920 
CMD_SEND:6
		ARG			 0x80fffff1
		MMC_RSP_R1,5,6,7 	 0x00000900 
CMD_SEND:55
		ARG			 0xaaaa0000
		MMC_RSP_R1,5,6,7 	 0x00000920 
CMD_SEND:13
		ARG			 0x00000000
		MMC_RSP_R1,5,6,7 	 0x00000920 
switch to partitions #0, OK
mmc1 is current device
CMD_SEND:0
		ARG			 0x00000000
		MMC_RSP_NONE
CMD_SEND:8
		ARG			 0x000001aa
		MMC_RSP_R1,5,6,7 	 0x000001aa 
CMD_SEND:55
		ARG			 0x00000000
		MMC_RSP_R1,5,6,7 	 0x00000120 
CMD_SEND:41
		ARG			 0x41300000
		MMC_RSP_R3,4		 0x40ff8000 
CMD_SEND:55
		ARG			 0x00000000
		MMC_RSP_R1,5,6,7 	 0x00000120 
CMD_SEND:41
		ARG			 0x41300000
		MMC_RSP_R3,4		 0x40ff8000 
CMD_SEND:55
		ARG			 0x00000000
		MMC_RSP_R1,5,6,7 	 0x00000120 
CMD_SEND:41
		ARG			 0x41300000
		MMC_RSP_R3,4		 0xc0ff8000 
CMD_SEND:2
		ARG			 0x00000000
		MMC_RSP_R2		 0x03534453 
		          		 0x48333247 
		          		 0x8097e54d 
		          		 0x40017b00 
					DUMPING DATA
					000 - 03 53 44 53 
					004 - 48 33 32 47 
					008 - 80 97 e5 4d 
					012 - 40 01 7b 00 
CMD_SEND:3
		ARG			 0x00000000
		MMC_RSP_R1,5,6,7 	 0xaaaa0520 
CMD_SEND:9
		ARG			 0xaaaa0000
		MMC_RSP_R2		 0x400e0032 
		          		 0xdb790000 
		          		 0xedc87f80 
		          		 0x0a404000 
					DUMPING DATA
					000 - 40 0e 00 32 
					004 - db 79 00 00 
					008 - ed c8 7f 80 
					012 - 0a 40 40 00 
CMD_SEND:7
		ARG			 0xaaaa0000
		MMC_RSP_R1,5,6,7 	 0x00000700 
CMD_SEND:55
		ARG			 0xaaaa0000
		MMC_RSP_R1,5,6,7 	 0x00000920 
CMD_SEND:51
		ARG			 0x00000000
		MMC_RSP_R1,5,6,7 	 0x00000920 
CMD_SEND:6
		ARG			 0x00fffff1
		MMC_RSP_R1,5,6,7 	 0x00000900 
CMD_SEND:55
		ARG			 0xaaaa0000
		MMC_RSP_R1,5,6,7 	 0x00000920 
CMD_SEND:6
		ARG			 0x00000002
		MMC_RSP_R1,5,6,7 	 0x00000920 
CMD_SEND:6
		ARG			 0x80fffff1
		MMC_RSP_R1,5,6,7 	 0x00000900 
CMD_SEND:55
		ARG			 0xaaaa0000
		MMC_RSP_R1,5,6,7 	 0x00000920 
CMD_SEND:13
		ARG			 0x00000000
		MMC_RSP_R1,5,6,7 	 0x00000920 
CMD_SEND:16
		ARG			 0x00000200
		MMC_RSP_R1,5,6,7 	 0x00000900 
CMD_SEND:17
		ARG			 0x00000000
		MMC_RSP_R1,5,6,7 	 0x00000900 
CMD_SEND:16
		ARG			 0x00000200
		MMC_RSP_R1,5,6,7 	 0x00000900 
CMD_SEND:17
		ARG			 0x00004000
		MMC_RSP_R1,5,6,7 	 0x00000900 
CMD_SEND:16
		ARG			 0x00000200
		MMC_RSP_R1,5,6,7 	 0x00000900 
CMD_SEND:17
		ARG			 0x00000000
		MMC_RSP_R1,5,6,7 	 0x00000900 
CMD_SEND:16
		ARG			 0x00000200
		MMC_RSP_R1,5,6,7 	 0x00000900 
CMD_SEND:17
		ARG			 0x00004000
		MMC_RSP_R1,5,6,7 	 0x00000900 
CMD_SEND:16
		ARG			 0x00000200
		MMC_RSP_R1,5,6,7 	 0x00000900 
CMD_SEND:18
		ARG			 0x00004154
		RET			 -70
CMD_SEND:16
		ARG			 0x00000200
		RET			 -110
CMD_SEND:16
		ARG			 0x00000200
		RET			 -110
CMD_SEND:16
		ARG			 0x00000200
		RET			 -110
CMD_SEND:16
		ARG			 0x00000200
		RET			 -110
CMD_SEND:16
		ARG			 0x00000200
		RET			 -110
CMD_SEND:16
		ARG			 0x00000200
		RET			 -110
WARN: Cannot load the DT ```
@tq-schifferm
Copy link
Member

tq-schifferm commented Sep 9, 2024

Hi,
the issue was fixed in 19e90b5, so updating to the kirkstone.TQMa8.BSP.SW.0092 release (or the current version of the TQMa8-v2020.04_imx_5.4.70_2.3.0 U-Boot branch if you're not using our Yocto BSP) should help.

(Edit: fixed commit reference, accidentally got the TQMa8MPxL instead of TQMa8MxML version of the fix at first)

@wico-frad
Copy link
Author

Hi, the issue was fixed in 19e90b5, so updating to the kirkstone.TQMa8.BSP.SW.0092 release (or the current version of the TQMa8-v2020.04_imx_5.4.70_2.3.0 U-Boot branch if you're not using our Yocto BSP) should help.

(Edit: fixed commit reference, accidentally got the TQMa8MPxL instead of TQMa8MxML version of the fix at first)

Thank you for your prompt reply.

Currently, we are working with the kirkstone.TQMa8MxML.BSP.SW.0091 release (current branch: TQMa8-v2020.04_imx_5.4.70_2.3.0) . We’ve tried cherry-picking the commit 19e90b5, but the issue persists when testing on the hardware.

Any suggestions would be appreciated.

@tq-schifferm
Copy link
Member

Hmm, that is unfortunate. The errors we were seeing (also with certain SanDisk SD cards) looked extremely similar, with the card stopping to respond to certain commands after an incomplete reset, resulting in error 110 (ETIMEDOUT) - although I don't remember the error 70 (ECOMM) happening here. However, I think that our issue happened earlier in the boot - as the reset in the SPL was to short, loading the U-Boot proper failed, instead of U-Boot failing to load the OS.

Some things to try:

  • Interrupt the boot in the U-Boot command line, remove and reinsert the SD card to power cycle it, continue boot
  • U-Boot proper will handle the SD card reset in the real MMC driver, which is the udelay(2000) in mmc_power_cycle in drivers/mmc/mmc.c. Try increasing the delay, maybe to 10000 as as test.
  • Check mmc dev 1; mmc info for the used mode. The card will likely show a UHS mode like SDR104. You can try disabling CONFIG_MMC_UHS_SUPPORT in the U-Boot config to check if the issue disappears in slower modes.

@tq-niebelm
Copy link
Member

We had a similar sounding problem - U-Boot came up but U-Boot proper could nor read from SD-Card. But the error was seen only on one board for exactly one card type. It was blamed on this board. At this time we picked some patches that correct several issues in the USDHC controller driver and as far as I can tell the problem was solved on this board, too.

I rebased and pushed the patches to branch `testing/v2020.04-imx-sd-card-fixes.

@wico-frad
Copy link
Author

Hmm, that is unfortunate. The errors we were seeing (also with certain SanDisk SD cards) looked extremely similar, with the card stopping to respond to certain commands after an incomplete reset, resulting in error 110 (ETIMEDOUT) - although I don't remember the error 70 (ECOMM) happening here. However, I think that our issue happened earlier in the boot - as the reset in the SPL was to short, loading the U-Boot proper failed, instead of U-Boot failing to load the OS.

Some things to try:

* Interrupt the boot in the U-Boot command line, remove and reinsert the SD card to power cycle it, continue boot

* U-Boot proper will handle the SD card reset in the real MMC driver, which is the `udelay(2000)` in `mmc_power_cycle` in `drivers/mmc/mmc.c`. Try increasing the delay, maybe to 10000 as as test.

* Check `mmc dev 1; mmc info` for the used mode. The card will likely show a UHS mode like SDR104. You can try disabling `CONFIG_MMC_UHS_SUPPORT` in the U-Boot config to check if the issue disappears in slower modes.

Thank you for the suggestions. Here's what we've tried so far:

  • Interrupting the Boot: We interrupted the boot process in the U-Boot command line, removed and reinserted the SD card to power cycle it, and then continued the boot. Unfortunately, this did not resolve the issue.

  • Increasing the Delay: We modified the udelay in mmc_power_cycle within drivers/mmc/mmc.c from 2000 to 10000 to see if a longer delay would help the SD card reset. This change did not resolve the problem either.

The i.MX8M Mini can support high-speed grades in the USDHC controllers. Turning off the corresponding configuration option for the TQMa8MxML variant in Kconfig before booting the kernel seems to resolve the issue.

Additionally, we tried capping the USDHC controller's capabilities by disabling the sd-uhs-sdr104 flag in imx8mm-mba8mx-u-boot.dtsi. Surprisingly, this also resolves the issue. I'm still curious why limiting the USDHC controller's capabilities in this way helps. From what I understand, the generic MMC driver uses these flags to set the host capabilities. If anyone has insights into why reducing the speed grades seems to improve stability, I would appreciate your input.

@tq-niebelm
Copy link
Member

Hello,

disabling sd-uhs-sdr104 will cause some changes. To switch from / to this mode (supported by SD 3.0 cards) additional code paths in the complete SD-Card stack are involved. The above mentioned branch testing/v2020.04-imx-sd-card-fixes contains some fixes for these code paths backported from newer U-Boot versions. Did this solve the issue?

@wico-frad
Copy link
Author

disabling sd-uhs-sdr104 will cause some changes. To switch from / to this mode (supported by SD 3.0 cards) additional code paths in the complete SD-Card stack are involved. The above mentioned branch testing/v2020.04-imx-sd-card-fixes contains some fixes for these code paths backported from newer U-Boot versions. Did this solve the issue?

Hello,

disabling sd-uhs-sdr104 will cause some changes. To switch from / to this mode (supported by SD 3.0 cards) additional code paths in the complete SD-Card stack are involved. The above mentioned branch testing/v2020.04-imx-sd-card-fixes contains some fixes for these code paths backported from newer U-Boot versions. Did this solve the issue?

Hi,

Compiling U-Boot based on the testing/v2020.04-imx-sd-card-fixes branch did not resolve the issue.
When executing the command mmc dev 1, the response was:

=> mmc dev 1
Card did not respond to voltage select!

However, disabling either MMC_UHS_SUPPORT (preventing the SD Host controller to try to move to UHS modes) or sd-uhs-sdr104 resolved the issue.

=> mmc dev 1
switch to partitions #0, OK
mmc1 is current device
=> mmc info
Device: FSL_SDHC
Manufacturer ID: 3
OEM: 5344
Name: SH32G 
Bus Speed: 50000000
Mode: SD High Speed (50MHz)
Rd Block Len: 512
SD version 3.0
High Capacity: Yes
Capacity: 29.7 GiB
Bus Width: 4-bit
Erase Group Size: 512 Bytes

@tq-niebelm
Copy link
Member

Hello,

this means, that after successful detecting the card and reading card data switching to the lower 1.8 V IO Voltage which is needed for UHS modes did not succeed. Does this happen on an MBa8Mx (EVK) or on a custom mainboard?

@wico-frad
Copy link
Author

Hello,

this means, that after successful detecting the card and reading card data switching to the lower 1.8 V IO Voltage which is needed for UHS modes did not succeed. Does this happen on an MBa8Mx (EVK) or on a custom mainboard?

Hi,

We are using a custom main board based on the TQMa8MxML SOM revision 0200.
If I'm not mistaken, the System Boot section of the i.MX 8M Mini Applications Processor Reference Manual (Rev. 3, 11/2020) states that CMD11 is sent to switch the voltage to 1.8 V.

I reviewed the previous U-Boot traces and the CMD11 appears as follows:

CMD_SEND:11
		ARG			 0x00000000
Run CMD11 1.8V switch
		MMC_RSP_R1,5,6,7 	 0x00000320 

@embsupport
Copy link

Hello,

since the SD Card issue occurs on a custom specific baseboard, can you please contact us at [email protected]

@embsupport
Copy link

Hello,

since the SD Card issue occurs on a custom specific baseboard, can you please contact us at [email protected]

We have not yet received an email from you, has the issue been resolved in the meantime?

@wico-frad
Copy link
Author

TQMa8MxML SOM revision 0200

Thank you for following up. I haven't sent an email yet as I'm still in the process of running some tests. Although the issue occurs on a custom baseboard, it is based on the TQMa8MxML SOM variant and involves the Yocto-based BSP (specifically the U-Boot component) maintained by TQM. Therefore, I believe it's still relevant to continue the technical discussion here. I also appreciate the valuable input from @tq-niebelm and @tq-schifferm. I'll reach out via email if direct support is needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants