Add conformance reports

google · Sep 29, 2023 · 7814743 · 7814743
1 parent 1a5938e
commit 7814743
Show file tree

Hide file tree

Showing 14 changed files with 752 additions and 2 deletions.
diff --git a/README.md b/README.md
@@ -110,8 +110,7 @@ $ make fuzz -j  # Run encoder and decoder fuzzers in parallel
 
 The implementation is qualified under the [_QDID 194161_](https://launchstudio.bluetooth.com/ListingDetails/160904) as part of Google Fluoride 1.5.
 
-For more detail on conformance, refer to [_Bluetooth Conformance
-Documents and scripts_](https://www.bluetooth.com/specifications/specs/low-complexity-communication-codec-1-0/)
+The conformance reports can be found [here](conformance/README.md)
 
 ## Listening Test
 

diff --git a/conformance/README.md b/conformance/README.md
@@ -0,0 +1,245 @@
+# Conformance test reports
+
+For more detail on conformance, refer to [_Bluetooth Conformance
+Documents and scripts_](https://www.bluetooth.com/specifications/specs/low-complexity-communication-codec-1-0/)
+
+Exposed reports were obtained by exercising the release [1.0.4](https://github.com/google/liblc3/releases/tag/v1.0.4), using the [_Test Software 1.0.6_](https://www.bluetooth.com/specifications/specs/low-complexity-communication-codec-1-0/)
+
+## Speech
+
+* Encoding 10ms
+
+```
+[speech_encode_10m]
+
+# test modes
+test_sqam           = 1
+test_band_limiting  = 1
+test_low_pass       = 0
+test_rate_switching = 0
+
+#         Mode, Samplingrate, Bitrate
+configs = encode,  8000, 24000
+          encode, 16000, 32000
+          encode, 24000, 48000
+          encode, 32000, 64000
+```
+
+[Report](speech_encode_10m.html)
+
+* Decoding 10ms
+
+```
+[speech_decode_10m]
+
+# test modes
+test_sqam           = 1
+test_band_limiting  = 0
+test_low_pass       = 0
+test_rate_switching = 0
+
+#         Mode, Samplingrate, Bitrate
+configs = decode,  8000, 24000
+          decode, 16000, 32000
+          decode, 24000, 48000
+          decode, 32000, 64000
+```
+
+[Report](speech_decode_10m.html)
+
+* Encoding + Decoding 10ms
+
+```
+[speech_encdec_10m]
+
+# test modes
+test_sqam           = 1
+test_band_limiting  = 1
+test_low_pass       = 0
+test_rate_switching = 0
+
+#         Mode, Samplingrate, Bitrate
+configs = encdec,  8000, 24000
+          encdec, 16000, 32000
+          encdec, 24000, 48000
+          encdec, 32000, 64000
+```
+
+[Report](speech_encdec_10m.html)
+
+* Encoding 7.5ms
+
+```
+[speech_encode_7m5]
+
+# test modes
+test_sqam           = 1
+test_band_limiting  = 1
+test_low_pass       = 0
+test_rate_switching = 0
+
+#         Mode, Samplingrate, Bitrate
+configs = encode,  8000, 27734
+          encode, 16000, 32000
+          encode, 24000, 48000
+          encode, 32000, 64000
+```
+
+[Report](speech_encode_7m5.html)
+
+* Decoding 7.5ms
+
+```
+[speech_decode_7m5]
+
+# test modes
+test_sqam           = 1
+test_band_limiting  = 0
+test_low_pass       = 0
+test_rate_switching = 0
+
+#         Mode, Samplingrate, Bitrate
+configs = decode,  8000, 27734
+          decode, 16000, 32000
+          decode, 24000, 48000
+          decode, 32000, 64000
+```
+
+[Report](speech_decode_7m5.html)
+
+* Encoding + Decoding 7.5ms
+
+```
+[speech_encdec_7m5]
+
+# test modes
+test_sqam           = 1
+test_band_limiting  = 1
+test_low_pass       = 0
+test_rate_switching = 0
+
+#         Mode, Samplingrate, Bitrate
+configs = encdec,  8000, 27734
+          encdec, 16000, 32000
+          encdec, 24000, 48000
+          encdec, 32000, 64000
+```
+
+[Report](speech_encdec_7m5.html)
+
+## Music
+
+* Encoding 10ms
+
+```
+[music_encode_10m]
+
+# test modes
+test_sqam           = 1
+test_band_limiting  = 1
+test_low_pass       = 1
+test_rate_switching = 0
+
+#         Mode, Samplingrate, Bitrate
+configs = encode, 48000,  80000
+          encode, 48000,  96000
+          encode, 48000, 124000
+```
+
+[Report](music_encode_10m.html)
+
+* Decoding 10ms
+
+```
+[music_decode_10m]
+
+# test modes
+test_sqam           = 1
+test_band_limiting  = 0
+test_low_pass       = 1
+test_rate_switching = 0
+
+#         Mode, Samplingrate, Bitrate
+configs = decode, 48000,  80000
+          decode, 48000,  96000
+          decode, 48000, 124000
+```
+
+[Report](music_decode_10m.html)
+
+* Encoding + Decoding 10ms
+
+```
+[music_encdec_10m]
+
+# test modes
+test_sqam           = 1
+test_band_limiting  = 1
+test_low_pass       = 1
+test_rate_switching = 0
+
+#         Mode, Samplingrate, Bitrate
+configs = encdec, 48000,  80000
+          encdec, 48000,  96000
+          encdec, 48000, 124000
+```
+
+[Report](music_encdec_10m.html)
+
+* Encoding 7.5ms
+
+```
+[music_encode_7m5]
+
+# test modes
+test_sqam           = 1
+test_band_limiting  = 1
+test_low_pass       = 1
+test_rate_switching = 0
+
+#         Mode, Samplingrate, Bitrate
+configs = encode, 48000,  80000
+          encode, 48000,  96000
+          encode, 48000, 124800
+```
+
+[Report](music_encode_7m5.html)
+
+
+* Decoding 7.5ms
+
+```
+[music_decode_7m5]
+
+# test modes
+test_sqam           = 1
+test_band_limiting  = 0
+test_low_pass       = 1
+test_rate_switching = 0
+
+#         Mode, Samplingrate, Bitrate
+configs = decode, 48000,  80000
+          decode, 48000,  96000
+          decode, 48000, 124800
+```
+
+[Report](music_decode_7m5.html)
+
+* Encoding + Decoding 7.5ms
+
+```
+[music_encdec_7m5]
+
+# test modes
+test_sqam           = 1
+test_band_limiting  = 1
+test_low_pass       = 1
+test_rate_switching = 0
+
+#         Mode, Samplingrate, Bitrate
+configs = encdec, 48000,  80000
+          encdec, 48000,  96000
+          encdec, 48000, 124800
+```
+
+[Report](music_encdec_7m5.html)
diff --git a/conformance/music_decode_10m.html b/conformance/music_decode_10m.html
@@ -0,0 +1,31 @@
+<!DOCTYPE html><head><title>music_decode_10m Report</title><style>body {font-family:sans-serif; color:#f8f8f2; background-color:#272822; font-size:80%} div {border:1px solid #8f908a; border-radius:4px; overflow:hidden; display:table; margin-left:30px; margin-bottom:30px} h2 {text-align:left; margin-left:30px} h3 {text-align:left; margin:4px} table {border-spacing:0px} th {padding:4px} td {padding:4px} tr:nth-child(even) {background-color:rgba(255,255,255,0.1)} td.pass {background-color:rgba(0,192,255,0.4)} td.fail {background-color:rgba(255,0,0,0.4)} td.warn {background-color:rgba(214,137,16,0.4)}</style></head><body><h2>Conformance test for "music_decode_10m" (Frame Size 10 ms) passed!</h2><div><table><tr><h3>SQAM items - 100%</h3></tr>
+<tr><th>Mode</th><th>Item</th><th>Samplingrate</th><th>Bitrate</th><th>Max. Abs. Diff (threshold)</th><th>RMS (threshold) [dB]</th><th>RMS reached (threshold) [bits]</th></tr>
+<tr><td>worst value</td><td></td><td></td><td></td><td>0.000458</td><td>-91.079</td><td>14</td></tr>
+<tr><td>decode</td><td>ABBA</td><td>48000</td><td>80000</td><td class=pass>0.000214 (0.00148)</td><td class=pass>-95.242 (-89.060)</td><td class=none>15 (14)</td></tr>
+<tr><td>decode</td><td>ABBA</td><td>48000</td><td>96000</td><td class=pass>0.000244 (0.00148)</td><td class=pass>-95.226 (-89.060)</td><td class=none>15 (14)</td></tr>
+<tr><td>decode</td><td>ABBA</td><td>48000</td><td>124000</td><td class=pass>0.000214 (0.00148)</td><td class=pass>-95.311 (-89.060)</td><td class=none>15 (14)</td></tr>
+<tr><td>decode</td><td>Castanets</td><td>48000</td><td>80000</td><td class=pass>0.000427 (0.00148)</td><td class=pass>-96.479 (-89.060)</td><td class=none>15 (14)</td></tr>
+<tr><td>decode</td><td>Castanets</td><td>48000</td><td>96000</td><td class=pass>0.000458 (0.00148)</td><td class=pass>-96.631 (-89.060)</td><td class=none>15 (14)</td></tr>
+<tr><td>decode</td><td>Castanets</td><td>48000</td><td>124000</td><td class=pass>0.000427 (0.00148)</td><td class=pass>-96.478 (-89.060)</td><td class=none>15 (14)</td></tr>
+<tr><td>decode</td><td>Eddie_Rabbitt</td><td>48000</td><td>80000</td><td class=pass>0.000244 (0.00148)</td><td class=pass>-91.875 (-89.060)</td><td class=none>14 (14)</td></tr>
+<tr><td>decode</td><td>Eddie_Rabbitt</td><td>48000</td><td>96000</td><td class=pass>0.000214 (0.00148)</td><td class=pass>-91.868 (-89.060)</td><td class=none>14 (14)</td></tr>
+<tr><td>decode</td><td>Eddie_Rabbitt</td><td>48000</td><td>124000</td><td class=pass>0.000244 (0.00148)</td><td class=pass>-91.951 (-89.060)</td><td class=none>14 (14)</td></tr>
+<tr><td>decode</td><td>Female_Speech_German</td><td>48000</td><td>80000</td><td class=pass>0.000244 (0.00148)</td><td class=pass>-92.549 (-89.060)</td><td class=none>14 (14)</td></tr>
+<tr><td>decode</td><td>Female_Speech_German</td><td>48000</td><td>96000</td><td class=pass>0.000214 (0.00148)</td><td class=pass>-92.647 (-89.060)</td><td class=none>14 (14)</td></tr>
+<tr><td>decode</td><td>Female_Speech_German</td><td>48000</td><td>124000</td><td class=pass>0.000244 (0.00148)</td><td class=pass>-92.672 (-89.060)</td><td class=none>14 (14)</td></tr>
+<tr><td>decode</td><td>Glockenspiel</td><td>48000</td><td>80000</td><td class=pass>0.000244 (0.00148)</td><td class=pass>-94.290 (-89.060)</td><td class=none>14 (14)</td></tr>
+<tr><td>decode</td><td>Glockenspiel</td><td>48000</td><td>96000</td><td class=pass>0.000244 (0.00148)</td><td class=pass>-94.435 (-89.060)</td><td class=none>14 (14)</td></tr>
+<tr><td>decode</td><td>Glockenspiel</td><td>48000</td><td>124000</td><td class=pass>0.000244 (0.00148)</td><td class=pass>-94.411 (-89.060)</td><td class=none>14 (14)</td></tr>
+<tr><td>decode</td><td>Harpsichord</td><td>48000</td><td>80000</td><td class=pass>0.000061 (0.00148)</td><td class=pass>-99.405 (-89.060)</td><td class=none>15 (14)</td></tr>
+<tr><td>decode</td><td>Harpsichord</td><td>48000</td><td>96000</td><td class=pass>0.000061 (0.00148)</td><td class=pass>-99.129 (-89.060)</td><td class=none>15 (14)</td></tr>
+<tr><td>decode</td><td>Harpsichord</td><td>48000</td><td>124000</td><td class=pass>0.000061 (0.00148)</td><td class=pass>-99.152 (-89.060)</td><td class=none>15 (14)</td></tr>
+<tr><td>decode</td><td>Male_Speech_English</td><td>48000</td><td>80000</td><td class=pass>0.000244 (0.00148)</td><td class=pass>-91.220 (-89.060)</td><td class=none>14 (14)</td></tr>
+<tr><td>decode</td><td>Male_Speech_English</td><td>48000</td><td>96000</td><td class=pass>0.000275 (0.00148)</td><td class=pass>-91.115 (-89.060)</td><td class=none>14 (14)</td></tr>
+<tr><td>decode</td><td>Male_Speech_English</td><td>48000</td><td>124000</td><td class=pass>0.000275 (0.00148)</td><td class=pass>-91.079 (-89.060)</td><td class=none>14 (14)</td></tr>
+<tr><td>decode</td><td>Piano_Schubert</td><td>48000</td><td>80000</td><td class=pass>0.000153 (0.00148)</td><td class=pass>-94.611 (-89.060)</td><td class=none>14 (14)</td></tr>
+<tr><td>decode</td><td>Piano_Schubert</td><td>48000</td><td>96000</td><td class=pass>0.000153 (0.00148)</td><td class=pass>-94.638 (-89.060)</td><td class=none>14 (14)</td></tr>
+<tr><td>decode</td><td>Piano_Schubert</td><td>48000</td><td>124000</td><td class=pass>0.000183 (0.00148)</td><td class=pass>-94.625 (-89.060)</td><td class=none>14 (14)</td></tr>
+<tr><td>decode</td><td>Violoncello</td><td>48000</td><td>80000</td><td class=pass>0.000244 (0.00148)</td><td class=pass>-91.510 (-89.060)</td><td class=none>14 (14)</td></tr>
+<tr><td>decode</td><td>Violoncello</td><td>48000</td><td>96000</td><td class=pass>0.000244 (0.00148)</td><td class=pass>-91.493 (-89.060)</td><td class=none>14 (14)</td></tr>
+<tr><td>decode</td><td>Violoncello</td><td>48000</td><td>124000</td><td class=pass>0.000244 (0.00148)</td><td class=pass>-91.547 (-89.060)</td><td class=none>14 (14)</td></tr>
+</table></div></body>
diff --git a/conformance/music_decode_7m5.html b/conformance/music_decode_7m5.html
@@ -0,0 +1,31 @@
+<!DOCTYPE html><head><title>music_decode_7m5 Report</title><style>body {font-family:sans-serif; color:#f8f8f2; background-color:#272822; font-size:80%} div {border:1px solid #8f908a; border-radius:4px; overflow:hidden; display:table; margin-left:30px; margin-bottom:30px} h2 {text-align:left; margin-left:30px} h3 {text-align:left; margin:4px} table {border-spacing:0px} th {padding:4px} td {padding:4px} tr:nth-child(even) {background-color:rgba(255,255,255,0.1)} td.pass {background-color:rgba(0,192,255,0.4)} td.fail {background-color:rgba(255,0,0,0.4)} td.warn {background-color:rgba(214,137,16,0.4)}</style></head><body><h2>Conformance test for "music_decode_7m5" (Frame Size 7.5 ms) passed!</h2><div><table><tr><h3>SQAM items - 100%</h3></tr>
+<tr><th>Mode</th><th>Item</th><th>Samplingrate</th><th>Bitrate</th><th>Max. Abs. Diff (threshold)</th><th>RMS (threshold) [dB]</th><th>RMS reached (threshold) [bits]</th></tr>
+<tr><td>worst value</td><td></td><td></td><td></td><td>0.000397</td><td>-91.571</td><td>14</td></tr>
+<tr><td>decode</td><td>ABBA</td><td>48000</td><td>80000</td><td class=pass>0.000214 (0.00148)</td><td class=pass>-95.409 (-89.060)</td><td class=none>15 (14)</td></tr>
+<tr><td>decode</td><td>ABBA</td><td>48000</td><td>96000</td><td class=pass>0.000214 (0.00148)</td><td class=pass>-95.367 (-89.060)</td><td class=none>15 (14)</td></tr>
+<tr><td>decode</td><td>ABBA</td><td>48000</td><td>124800</td><td class=pass>0.000244 (0.00148)</td><td class=pass>-95.442 (-89.060)</td><td class=none>15 (14)</td></tr>
+<tr><td>decode</td><td>Castanets</td><td>48000</td><td>80000</td><td class=pass>0.000305 (0.00148)</td><td class=pass>-97.500 (-89.060)</td><td class=none>15 (14)</td></tr>
+<tr><td>decode</td><td>Castanets</td><td>48000</td><td>96000</td><td class=pass>0.000336 (0.00148)</td><td class=pass>-97.559 (-89.060)</td><td class=none>15 (14)</td></tr>
+<tr><td>decode</td><td>Castanets</td><td>48000</td><td>124800</td><td class=pass>0.000336 (0.00148)</td><td class=pass>-97.472 (-89.060)</td><td class=none>15 (14)</td></tr>
+<tr><td>decode</td><td>Eddie_Rabbitt</td><td>48000</td><td>80000</td><td class=pass>0.000183 (0.00148)</td><td class=pass>-92.158 (-89.060)</td><td class=none>14 (14)</td></tr>
+<tr><td>decode</td><td>Eddie_Rabbitt</td><td>48000</td><td>96000</td><td class=pass>0.000214 (0.00148)</td><td class=pass>-92.181 (-89.060)</td><td class=none>14 (14)</td></tr>
+<tr><td>decode</td><td>Eddie_Rabbitt</td><td>48000</td><td>124800</td><td class=pass>0.000214 (0.00148)</td><td class=pass>-92.207 (-89.060)</td><td class=none>14 (14)</td></tr>
+<tr><td>decode</td><td>Female_Speech_German</td><td>48000</td><td>80000</td><td class=pass>0.000275 (0.00148)</td><td class=pass>-92.727 (-89.060)</td><td class=none>14 (14)</td></tr>
+<tr><td>decode</td><td>Female_Speech_German</td><td>48000</td><td>96000</td><td class=pass>0.000305 (0.00148)</td><td class=pass>-92.894 (-89.060)</td><td class=none>14 (14)</td></tr>
+<tr><td>decode</td><td>Female_Speech_German</td><td>48000</td><td>124800</td><td class=pass>0.000305 (0.00148)</td><td class=pass>-92.843 (-89.060)</td><td class=none>14 (14)</td></tr>
+<tr><td>decode</td><td>Glockenspiel</td><td>48000</td><td>80000</td><td class=pass>0.000153 (0.00148)</td><td class=pass>-95.027 (-89.060)</td><td class=none>14 (14)</td></tr>
+<tr><td>decode</td><td>Glockenspiel</td><td>48000</td><td>96000</td><td class=pass>0.000183 (0.00148)</td><td class=pass>-95.014 (-89.060)</td><td class=none>14 (14)</td></tr>
+<tr><td>decode</td><td>Glockenspiel</td><td>48000</td><td>124800</td><td class=pass>0.000183 (0.00148)</td><td class=pass>-95.000 (-89.060)</td><td class=none>14 (14)</td></tr>
+<tr><td>decode</td><td>Harpsichord</td><td>48000</td><td>80000</td><td class=pass>0.000061 (0.00148)</td><td class=pass>-99.279 (-89.060)</td><td class=none>15 (14)</td></tr>
+<tr><td>decode</td><td>Harpsichord</td><td>48000</td><td>96000</td><td class=pass>0.000061 (0.00148)</td><td class=pass>-99.164 (-89.060)</td><td class=none>15 (14)</td></tr>
+<tr><td>decode</td><td>Harpsichord</td><td>48000</td><td>124800</td><td class=pass>0.000061 (0.00148)</td><td class=pass>-99.154 (-89.060)</td><td class=none>15 (14)</td></tr>
+<tr><td>decode</td><td>Male_Speech_English</td><td>48000</td><td>80000</td><td class=pass>0.000366 (0.00148)</td><td class=pass>-91.764 (-89.060)</td><td class=none>14 (14)</td></tr>
+<tr><td>decode</td><td>Male_Speech_English</td><td>48000</td><td>96000</td><td class=pass>0.000366 (0.00148)</td><td class=pass>-91.615 (-89.060)</td><td class=none>14 (14)</td></tr>
+<tr><td>decode</td><td>Male_Speech_English</td><td>48000</td><td>124800</td><td class=pass>0.000397 (0.00148)</td><td class=pass>-91.593 (-89.060)</td><td class=none>14 (14)</td></tr>
+<tr><td>decode</td><td>Piano_Schubert</td><td>48000</td><td>80000</td><td class=pass>0.000153 (0.00148)</td><td class=pass>-94.766 (-89.060)</td><td class=none>14 (14)</td></tr>
+<tr><td>decode</td><td>Piano_Schubert</td><td>48000</td><td>96000</td><td class=pass>0.000153 (0.00148)</td><td class=pass>-94.827 (-89.060)</td><td class=none>14 (14)</td></tr>
+<tr><td>decode</td><td>Piano_Schubert</td><td>48000</td><td>124800</td><td class=pass>0.000153 (0.00148)</td><td class=pass>-94.831 (-89.060)</td><td class=none>14 (14)</td></tr>
+<tr><td>decode</td><td>Violoncello</td><td>48000</td><td>80000</td><td class=pass>0.000214 (0.00148)</td><td class=pass>-91.649 (-89.060)</td><td class=none>14 (14)</td></tr>
+<tr><td>decode</td><td>Violoncello</td><td>48000</td><td>96000</td><td class=pass>0.000275 (0.00148)</td><td class=pass>-91.571 (-89.060)</td><td class=none>14 (14)</td></tr>
+<tr><td>decode</td><td>Violoncello</td><td>48000</td><td>124800</td><td class=pass>0.000244 (0.00148)</td><td class=pass>-91.640 (-89.060)</td><td class=none>14 (14)</td></tr>
+</table></div></body>