EC tests failing

[sharkcz]

EC-related tests fail with recent libica in our guests (zEC12, z13), I suppose it's because the required HW is not available there. It my assumption is true, then the tests should use a check first like the SHA3 tests do and be "SKIP"-ed.

make  check-TESTS
make[2]: Entering directory '/home/sharkcz/projects/libica/test'
make[3]: Entering directory '/home/sharkcz/projects/libica/test'
PASS: fips_test
PASS: icastats_test
PASS: get_functionlist_test
PASS: get_version_test
PASS: rng_test
PASS: drbg_test
PASS: drbg_birthdays_test.pl
PASS: des_test
PASS: des_ecb_test
PASS: des_cbc_test
PASS: des_ctr_test
PASS: des_cfb_test
PASS: des_ofb_test
PASS: tdes_test
PASS: tdes_ecb_test
PASS: tdes_cbc_test
PASS: tdes_ctr_test
PASS: tdes_cfb_test
PASS: tdes_ofb_test
PASS: aes_128_test
PASS: aes_192_test
PASS: aes_256_test
PASS: aes_ecb_test
PASS: aes_cbc_test
PASS: aes_ctr_test
PASS: aes_cfb_test
PASS: aes_ofb_test
PASS: aes_xts_test
PASS: aes_gcm_test
PASS: aes_gcm_kma_test
PASS: cbccs_test
PASS: ccm_test
PASS: cmac_test
PASS: sha2_test.sh
SKIP: sha3_test.sh
PASS: sha1_test
PASS: sha256_test
SKIP: sha3_224_test
SKIP: sha3_256_test
SKIP: sha3_384_test
SKIP: sha3_512_test
SKIP: shake_128_test
SKIP: shake_256_test
PASS: rsa_keygen1024_test.sh
PASS: rsa_keygen2048_test.sh
PASS: rsa_keygen3072_test.sh
PASS: rsa_keygen4096_test.sh
PASS: rsa_key_check_test
PASS: rsa_test
FAIL: ec_keygen1_test.sh
FAIL: ecdh1_test.sh
FAIL: ecdsa1_test.sh
FAIL: ec_keygen2_test.sh
FAIL: ecdh2_test.sh
FAIL: ecdsa2_test.sh
SKIP: mp_test
============================================================================
Testsuite summary for libica 3.3.0
============================================================================
# TOTAL: 56
# PASS:  42
# SKIP:  8
# XFAIL: 0
# FAIL:  6
# XPASS: 0
# ERROR: 0
============================================================================
See test/test-suite.log
Please report to [email protected]
============================================================================

The failure messages are like

12 of 12 EC keygen tests failed.
FAIL ec_keygen1_test.sh (exit status: 1)

[p-steuer]

Thanks for problem report. PR #16 addresses the issue.

For the test cases to work, you would need to have a CEX5C(*) adapter online:

# lszcrypt
CARD.DOMAIN TYPE  MODE        STATUS  REQUEST_CNT
-------------------------------------------------
00          CEX5C CCA-Coproc  online         1069
[...]

I tested the fixed testcases on my systems. Does it work for you? If so, would you like to have it available via a 3.3.2 release ?

(*) edit: (or newer)

[sharkcz]

Tests are properly skipped now, thanks. Yes, a new release will be appreciated.

BTW won't make sense to add a doc explaining what features what tests expect? I'm sure it would help our QA team when planning testing resources.

[p-steuer]

The problem is fixed with the 3.3.2 release.

We maintain a doc which lists the HW requirements of libica's interfaces here:
https://www.ibm.com/support/knowledgecenter/en/linuxonibm/com.ibm.linux.z.lxci/lxci_linuxonz.html

However, for the testcases the situation is different, because most functions implement fallbacks to older HW or to SW (libcrypto), so a test might be expected to pass even if HW is not available.

I ll keep this idea in mind for future releases: #17

[sharkcz]

Sounds good, and thanks for all updates :-)