Something broken on Caroline (skylake)

[seppiola]

Describe the bug
Some audio devices, namely speakers and one other I'm not sure what is used for, seem to be inaccessible and create a weird situation in which it is not even created a dummy output and every software trying for audio simply crashes.

I don't know if this issue is relatively new because I only just updated from a working configuration from several months ago, since some big changes in in the driver broke my config and I just reverted to a previous linux version for the time being since I had no time to debug that and I needed audio. Still have very little time but I think this might need to be addressed before too much time passes.

Boardname
Caroline (skylake)

Logs
Multiple things hang when trying to access some sound cards, including pipewire, external software requiring sound access and alsa tools used by your script to get logs. Trying to make the script to work properly, given the hangs and segfaults and my lack of familiarity with audio devices inner workings, proved to be a waste of time. I can provide specific logs if needed.

I'm on archlinux and I downgraded alsa-ucm-conf to 1.2.12 prior to debugging to prevent new syntax issues and avoid possible errors in new ucm configurations, since I read about it in #185.

Also, the driver module snd_soc_avs throws a bad error at loading time, I'll include the kernel call trace though probably useless:

[   22.447051] #PF: supervisor read access in kernel mode
[   22.447055] #PF: error_code(0x0000) - not-present page
[   22.447058] PGD 0 P4D 0 
[   22.447062] Oops: Oops: 0000 [#1] PREEMPT SMP PTI
[   22.447067] CPU: 1 UID: 1000 PID: 951 Comm: wireplumber Tainted: G         C         6.12.1-arch1-1 #1 33f4a68ee85c59cb5d6edb747af0349869779b24
[   22.447074] Tainted: [C]=CRAP
[   22.447077] Hardware name: Google Caroline/Caroline, BIOS SeppiaTech 02/20/2018
[   22.447081] RIP: 0010:avs_dai_fe_hw_params+0x1f/0x100 [snd_soc_avs]
[   22.447111] Code: 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 0f 1f 44 00 00 41 55 49 89 f1 41 54 55 53 48 63 47 3c 48 8d 04 40 4c 8b 6c c2 30 <49> 8b 75 08 48 85 f6 74 0f 31 ed 5b 89 e8 5d 41 5c 41 5d c3 cc cc
[   22.447115] RSP: 0018:ffffb41e0f4e3890 EFLAGS: 00010286
[   22.447119] RAX: 0000000000000003 RBX: ffff970d8ccb1028 RCX: 0000000000000000
[   22.447122] RDX: ffff970d8ccb1028 RSI: ffffb41e0f4e38f0 RDI: ffff970d8449d000
[   22.447125] RBP: ffff970d8449d000 R08: ffff970d8449d228 R09: ffffb41e0f4e38f0
[   22.447128] R10: 0000000000000000 R11: ffff970da798afb8 R12: ffff970da744b828
[   22.447130] R13: 0000000000000000 R14: 0000000000000000 R15: ffff970db0321800
[   22.447133] FS:  0000734726a91400(0000) GS:ffff970dfbc80000(0000) knlGS:0000000000000000
[   22.447137] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   22.447140] CR2: 0000000000000008 CR3: 000000012ea80002 CR4: 00000000003726f0
[   22.447143] Call Trace:
[   22.447147]  <TASK>
[   22.447150]  ? __die_body.cold+0x19/0x27
[   22.447157]  ? page_fault_oops+0x15a/0x2d0
[   22.447166]  ? exc_page_fault+0x81/0x190
[   22.447172]  ? asm_exc_page_fault+0x26/0x30
[   22.447181]  ? avs_dai_fe_hw_params+0x1f/0x100 [snd_soc_avs 6a3d03215b20085d7cf889e3cd6a64426f6dfd08]
[   22.447200]  snd_soc_dai_hw_params+0x39/0x80 [snd_soc_core 2ed91f7b7ecc741332d7b8494a2e0d5b0c029be8]
[   22.447258]  __soc_pcm_hw_params+0x547/0x680 [snd_soc_core 2ed91f7b7ecc741332d7b8494a2e0d5b0c029be8]
[   22.447306]  dpcm_fe_dai_hw_params+0xde/0x2a0 [snd_soc_core 2ed91f7b7ecc741332d7b8494a2e0d5b0c029be8]
[   22.447340]  snd_pcm_hw_params+0x2e1/0x6f0 [snd_pcm 9b700b3a2034e224048acc8b817daa4ae522e7f5]
[   22.447360]  snd_pcm_common_ioctl+0xaf1/0x13a0 [snd_pcm 9b700b3a2034e224048acc8b817daa4ae522e7f5]
[   22.447374]  ? __seccomp_filter+0x303/0x520
[   22.447383]  snd_pcm_ioctl+0x2b/0x40 [snd_pcm 9b700b3a2034e224048acc8b817daa4ae522e7f5]
[   22.447402]  __x64_sys_ioctl+0x91/0xd0
[   22.447411]  do_syscall_64+0x82/0x190
[   22.447422]  ? syscall_exit_to_user_mode+0x37/0x1c0
[   22.447431]  ? do_syscall_64+0x8e/0x190
[   22.447437]  ? __seccomp_filter+0x303/0x520
[   22.447447]  ? syscall_exit_to_user_mode+0x37/0x1c0
[   22.447453]  ? do_syscall_64+0x8e/0x190
[   22.447457]  ? do_syscall_64+0x8e/0x190
[   22.447463]  entry_SYSCALL_64_after_hwframe+0x76/0x7e
[   22.447468] RIP: 0033:0x734726ecbced
[   22.447498] Code: 04 25 28 00 00 00 48 89 45 c8 31 c0 48 8d 45 10 c7 45 b0 10 00 00 00 48 89 45 b8 48 8d 45 d0 48 89 45 c0 b8 10 00 00 00 0f 05 <89> c2 3d 00 f0 ff ff 77 1a 48 8b 45 c8 64 48 2b 04 25 28 00 00 00
[   22.447502] RSP: 002b:00007fff740d4370 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
[   22.447507] RAX: ffffffffffffffda RBX: 00007fff740d44e0 RCX: 0000734726ecbced
[   22.447510] RDX: 00007fff740d44e0 RSI: 00000000c2604111 RDI: 000000000000001a
[   22.447512] RBP: 00007fff740d43c0 R08: 0000000000000000 R09: 0000000000000000
[   22.447515] R10: 0000000000000004 R11: 0000000000000246 R12: 00006244b5ad9910
[   22.447518] R13: 00006244b5add540 R14: 00007fff740d4af0 R15: 00007fff740d44e0
[   22.447526]  </TASK>
[   22.447529] Modules linked in: xt_hl ip6t_rt ip6table_filter ip6table_nat ip6_tables xt_TCPMSS iptable_mangle xt_comment xt_addrtype xt_tcpudp xt_conntrack xt_LOG nf_log_syslog xt_limit iptable_filter iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c cros_ec_sensors cros_ec_sensors_core industrialio_triggered_buffer kfifo_buf industrialio snd_soc_avs_hdaudio intel_pmc_core_pltdrv snd_soc_dmic snd_soc_avs_nau8825 snd_soc_avs_ssm4567 snd_soc_avs_dmic snd_soc_avs_probe intel_pmc_core intel_vsec leds_cros_ec pmt_telemetry led_class_multicolor pmt_class cros_ec_sysfs joydev cros_usbpd_charger cros_charge_control cros_ec_sensorhub gpio_cros_ec intel_tcc_cooling mousedev cros_ec_chardev cros_ec_hwmon cros_ec_debugfs cros_usbpd_logger iwlmvm iTCO_wdt intel_pmc_bxt x86_pkg_temp_thermal iTCO_vendor_support hid_generic snd_hda_codec_hdmi intel_powerclamp mac80211 snd_soc_avs 8250_dw snd_soc_hda_codec coretemp snd_hda_ext_core intel_rapl_msr libarc4 snd_hda_intel ptp pps_core cros_ec_dev kvm_intel btusb
[   22.447646]  memconsole_coreboot btrtl snd_intel_dspcfg cbmem framebuffer_coreboot memconsole btintel chromeos_laptop snd_intel_sdw_acpi iwlwifi btbcm kvm snd_hda_codec processor_thermal_device_pci_legacy btmtk ipu3_imgu(C) i2c_i801 processor_thermal_device snd_soc_nau8825 uvcvideo atmel_mxt_ts spi_nor nls_iso8859_1 snd_soc_ssm4567 processor_thermal_wt_hint rapl snd_hda_core intel_cstate vfat i2c_smbus fat intel_uncore mtd pcspkr uvc bluetooth cfg80211 snd_soc_core i2c_mux snd_hwdep videobuf2_dma_sg videobuf2_vmalloc processor_thermal_rfim snd_compress videobuf2_memops processor_thermal_rapl rfkill videobuf2_v4l2 ac97_bus intel_rapl_common intel_lpss_pci intel_xhci_usb_role_switch snd_pcm_dmaengine videobuf2_common processor_thermal_wt_req intel_lpss snd_pcm processor_thermal_power_floor idma64 roles intel_pch_thermal processor_thermal_mbox videodev snd_timer intel_soc_dts_iosf int3403_thermal i2c_hid_acpi snd mc soundcore int340x_thermal_zone i2c_hid cros_ec_lpcs cros_usbpd_notify cros_ec coreboot_table
[   22.447772]  int3400_thermal pinctrl_sunrisepoint acpi_thermal_rel mac_hid zram 842_decompress 842_compress lz4hc_compress lz4_compress sg crypto_user loop nfnetlink ip_tables x_tables ext4 crc32c_generic crc16 mbcache jbd2 dm_crypt mmc_block rpmb_core cbc encrypted_keys trusted asn1_encoder tee dm_mod crct10dif_pclmul crc32_pclmul crc32c_intel polyval_clmulni polyval_generic ghash_clmulni_intel sha512_ssse3 sha256_ssse3 serio_raw sha1_ssse3 atkbd aesni_intel libps2 vivaldi_fmap gf128mul sdhci_pci crypto_simd cqhci cryptd sdhci spi_intel_pci mmc_core spi_intel gpio_keys i8042 serio i915 i2c_algo_bit drm_buddy video wmi ttm intel_gtt drm_display_helper cec
[   22.447873] CR2: 0000000000000008
[   22.447878] ---[ end trace 0000000000000000 ]---
[   22.447882] RIP: 0010:avs_dai_fe_hw_params+0x1f/0x100 [snd_soc_avs]
[   22.447914] Code: 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 0f 1f 44 00 00 41 55 49 89 f1 41 54 55 53 48 63 47 3c 48 8d 04 40 4c 8b 6c c2 30 <49> 8b 75 08 48 85 f6 74 0f 31 ed 5b 89 e8 5d 41 5c 41 5d c3 cc cc
[   22.447919] RSP: 0018:ffffb41e0f4e3890 EFLAGS: 00010286
[   22.447925] RAX: 0000000000000003 RBX: ffff970d8ccb1028 RCX: 0000000000000000
[   22.447928] RDX: ffff970d8ccb1028 RSI: ffffb41e0f4e38f0 RDI: ffff970d8449d000
[   22.447932] RBP: ffff970d8449d000 R08: ffff970d8449d228 R09: ffffb41e0f4e38f0
[   22.447936] R10: 0000000000000000 R11: ffff970da798afb8 R12: ffff970da744b828
[   22.447940] R13: 0000000000000000 R14: 0000000000000000 R15: ffff970db0321800
[   22.447944] FS:  0000734726a91400(0000) GS:ffff970dfbc80000(0000) knlGS:0000000000000000
[   22.447949] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   22.447953] CR2: 0000000000000008 CR3: 000000012ea80002 CR4: 00000000003726f0
[   22.447957] note: wireplumber[951] exited with irqs disabled

These might also be indicative of the problem:

[   13.594579] avs_nau8825 avs_nau8825.2: ASoC: Parent card not yet available, widget card binding deferred
[   13.609260] Bluetooth: hci0: Intel BT fw patch 0x43 completed & activated
[   13.613635] input: avs_nau8825 Headset Jack as /devices/platform/avs_nau8825.2/sound/card1/input17
[   13.622600] avs_ssm4567 avs_ssm4567.1: ASoC: Parent card not yet available, widget card binding deferred
[   13.653980] iwlwifi 0000:01:00.0 wlp1s0: renamed from wlan0
[   13.659202] snd_hda_codec_hdmi hdaudioB0D2: creating for HDMI 0 0
[   13.659213] snd_hda_codec_hdmi hdaudioB0D2: skipping capture dai for HDMI 0
[   13.659217] snd_hda_codec_hdmi hdaudioB0D2: creating for HDMI 1 1
[   13.659221] snd_hda_codec_hdmi hdaudioB0D2: skipping capture dai for HDMI 1
[   13.659224] snd_hda_codec_hdmi hdaudioB0D2: creating for HDMI 2 2
[   13.659228] snd_hda_codec_hdmi hdaudioB0D2: skipping capture dai for HDMI 2
[   13.660102] avs_hdaudio avs_hdaudio.2: ASoC: Parent card not yet available, widget card binding deferred
[   13.660234] avs_hdaudio avs_hdaudio.2: avs_card_late_probe: mapping HDMI converter 1 to PCM 1 (00000000c3d737ff)
[   13.660241] avs_hdaudio avs_hdaudio.2: avs_card_late_probe: mapping HDMI converter 2 to PCM 2 (00000000f7c3085c)
[   13.660246] avs_hdaudio avs_hdaudio.2: avs_card_late_probe: mapping HDMI converter 3 to PCM 3 (00000000d979681d)
[   13.669987] input: hdaudioB0D2 HDMI/DP,pcm=1 as /devices/platform/avs_hdaudio.2/sound/card3/input18
[   13.680540] input: hdaudioB0D2 HDMI/DP,pcm=2 as /devices/platform/avs_hdaudio.2/sound/card3/input19
[   13.680880] input: hdaudioB0D2 HDMI/DP,pcm=3 as /devices/platform/avs_hdaudio.2/sound/card3/input20
[   13.686661] avs_dmic avs_dmic: ASoC: Parent card not yet available, widget card binding deferred

ALSA lib main.c:1554:(snd_use_case_mgr_open) error: failed to import hw:0 use case configuration -2
ALSA lib main.c:1554:(snd_use_case_mgr_open) error: failed to import hw:2 use case configuration -2
  0: hw:1
    Google-Caroline-1.0
  1: hw:3
    Google-Caroline-1.0
  2: hw:4
    Google-Caroline-1.0

 0 [avsprobemb     ]: avs_probe_mb - avs_probe_mb
                      Google-Caroline-1.0
 1 [avsnau8825     ]: avs_nau8825 - avs_nau8825
                      Google-Caroline-1.0
 2 [avsssm4567     ]: avs_ssm4567 - avs_ssm4567
                      Google-Caroline-1.0
 3 [hdaudioB0D2    ]: hdaudioB0D2 - hdaudioB0D2
                      Google-Caroline-1.0
 4 [avsdmic        ]: avs_dmic - avs_dmic
                      Google-Caroline-1.0

card 1: avsnau8825 [avs_nau8825], device 1: Headset (*) []
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 2: avsssm4567 [avs_ssm4567], device 1: Built-in Speakers (*) []
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 3: hdaudioB0D2 [hdaudioB0D2], device 1: HDMI1 (*) []
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 3: hdaudioB0D2 [hdaudioB0D2], device 2: HDMI2 (*) []
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 3: hdaudioB0D2 [hdaudioB0D2], device 3: HDMI3 (*) []
  Subdevices: 1/1
  Subdevice #0: subdevice #0

[djfergus]

FWIW I have the same issue on an Asus C302:

Hardware name: Google Cave/Cave, BIOS MrChromebox-2405.0 06/15/2024

Running Arch, all working yesterday, I ran a "yay -Syu" and now sound is broken with similar symptoms (and dmesg) as above. Rerunning scripts didn't change anything.

Please let me know if there is any more info, logs or troubleshooting I can provide to help.

[seppiola]

@djfergus that's skylake as well. Do you happen to know which linux version you upgraded from yesterday? Could you try to revert back just the kernel to that last working version?
Note that you should downgrade alsa-ucm-conf anyway to 1.2.12 since 1.2.13 needs new syntax for ucm confs and won't work otherwise even on previously working kernels.

I checked various kernel versions and all present issues from 6.9 onward. The last version on which audio works is 6.8. From there on seems to get progressively worse, up to 6.12 which is completely broken, so it seems that after this even more important changes were made.

Anyway even on 6.8 there are some issues since not all cards are detected but at least speakers do work:

ALSA lib main.c:1554:(snd_use_case_mgr_open) error: failed to import hw:0 use case configuration -2
  0: hw:1
    Google-Caroline-1.0
  1: hw:2
    Google-Caroline-1.0
  2: hw:3
    Google-Caroline-1.0
  3: hw:4
    Google-Caroline-1.0

But at least this seems mostly a missing firmware issue

[   23.162696] snd_soc_avs 0000:00:1f.3: no DMIC ep_blob found
[   23.162702] snd_soc_avs 0000:00:1f.3: module-type create failed: -2
[   23.162705] snd_soc_avs 0000:00:1f.3: error creating module -2
[   23.162759] snd_soc_avs 0000:00:1f.3: create path failed: -2
[   23.162762] snd_soc_avs 0000:00:1f.3: ASoC: error at snd_soc_dai_hw_params on DMIC Pin: -2
[   23.162768]  DMIC: ASoC: error at __soc_pcm_hw_params on DMIC: -2
[   23.162771]  Digital Microphone: ASoC: error at dpcm_fe_dai_hw_params on Digital Microphone: -2

[seppiola]

I took the time to remove all ucm modifications, configurations for linux audio modules, userspace software configurations for alsa, pipewire and wireplumber and firmware additions on my system that accumulated during the years to start from scratch.

I manually reinstalled ucm2 configurations provided in this repo with updated syntax version 7, modules options and wireplumber conf provided in this repository, all according to the setup-audio script. No need to install any firmware on archlinux since all the needed ones for my board are already packaged in linux-firmware.

The mic that wasn't working lately now works. There was probably some configuration that got messed up during upgrades.
So, using linux 6.8.9 speakers and mic work like used to previously and I confirm that updated configurations provided here do in fact work (syntax 7 actually seems to be mostly ignored by alsa since it even worked before updating it) on Skylake Caroline (with system provided and packaged firmware blobs that match the ones provided here).

Everything falls apart when I try to upgrade the kernel as I described in the previous comments.
On linux 6.9.x and 6.10.x I get the dummy output with various errors and on 6.12.x I got the broken module errors from the previous comments and the system actually hangs when trying to use audio.
I reverted this change and rebuilt linux 6.9.1 and I confirm I got audio working again there.
I didn't try to revert even further changes in subsequent versions of linux.

So, as far as I can tell, the issue seems to be in the kernel driver, that from 6.9 got progressively broken, at least on skylake boards.
I don't know if this is due to misconfigurations of kernel modules or userspace tools, or if it needs to be addressed in the updated driver.

[djfergus]

I just did a full system upgrade (yay -Syu on Arch) and audio (speaker and mic) works again (after a reboot).

df@asus302:~$ yay -Qi alsa-ucm-conf Name : alsa-ucm-conf Version : 1.2.13-2

df@asus302:~$ uname -a Linux asus302 6.12.8-arch1-1 #1 SMP PREEMPT_DYNAMIC Thu, 02 Jan 2025 22:52:26 +0000 x86_64 GNU/Linux

However headphones do not work and after plugging headphones audio stops working until reboot.

[djfergus]

Update: closing lid suspend/resume gets audio back after I plug headphones.