PKCS11.C_SignInit() related problem

[WilburZjh]

Grinder

<<javax/crypto/Mac/ByteBuffers.java.ByteBuffers>>
<<sun/security/pkcs11/Mac/MacKAT.java.MacKAT>>
<<sun/security/pkcs11/Mac/MacSameTest.java.MacSameTest>>
<<sun/security/pkcs11/Mac/ReinitMac.java>>

TraceStack

java.security.InvalidKeyException: init() failed
	at sun.security.pkcs11.P11Mac.engineInit(P11Mac.java:208)
	at javax.crypto.Mac.init(Mac.java:413)
	at MacKAT$MacTest.run(MacKAT.java:247)
	at MacKAT.main(MacKAT.java:194)
	at PKCS11Test.premain(PKCS11Test.java:125)
	at PKCS11Test.testDefault(PKCS11Test.java:207)
	at PKCS11Test.main(PKCS11Test.java:155)
	at MacKAT.main(MacKAT.java:182)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127)
	at java.lang.Thread.run(Thread.java:826)
Caused by: sun.security.pkcs11.wrapper.PKCS11Exception: CKR_KEY_TYPE_INCONSISTENT
	at sun.security.pkcs11.wrapper.PKCS11.C_SignInit(Native Method)
	at sun.security.pkcs11.P11Mac.initialize(P11Mac.java:177)
	at sun.security.pkcs11.P11Mac.engineInit(P11Mac.java:206)

[WilburZjh]

<<sun/security/pkcs11/tls/TestPRF.java.TestPRF>>

TraceStack

java.lang.Exception: Error on line: 5
	at TestPRF.main(TestPRF.java:121)
	at PKCS11Test.premain(PKCS11Test.java:125)
	at PKCS11Test.testDefault(PKCS11Test.java:207)
	at PKCS11Test.main(PKCS11Test.java:155)
	at TestPRF.main(TestPRF.java:49)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127)
	at java.lang.Thread.run(Thread.java:826)
Caused by: java.security.ProviderException: Could not calculate PRF
	at sun.security.pkcs11.P11TlsPrfGenerator.engineGenerateKey(P11TlsPrfGenerator.java:182)
	at javax.crypto.KeyGenerator.generateKey(KeyGenerator.java:540)
	at TestPRF.main(TestPRF.java:110)
	... 10 more
Caused by: sun.security.pkcs11.wrapper.PKCS11Exception: CKR_KEY_TYPE_INCONSISTENT
	at sun.security.pkcs11.wrapper.PKCS11.C_SignInit(Native Method)
	at sun.security.pkcs11.P11TlsPrfGenerator.engineGenerateKey(P11TlsPrfGenerator.java:175)
	... 12 more

[WilburZjh]

The common exception is:

Caused by: sun.security.pkcs11.wrapper.PKCS11Exception: CKR_KEY_TYPE_INCONSISTENT
	at sun.security.pkcs11.wrapper.PKCS11.C_SignInit(Native Method)

CKR_KEY_TYPE_INCONSISTENT: The specified key is not the correct type of key to use with the specified mechanism.

This issue is more likely related to the bug report 8278640. One of the solutions could be RedHat-2102431.

[WilburZjh]

After adding attributes(*,CKO_SECRET_KEY,CKK_GENERIC_SECRET)={ CKA_SIGN=true } in the nss.fips.cfg file.

Grinder

<<javax/crypto/Mac/ByteBuffers.java.ByteBuffers>>
<<sun/security/pkcs11/Mac/MacKAT.java.MacKAT>>
<<sun/security/pkcs11/tls/TestPRF.java.TestPRF>>

---

The exception of these two has been changed from CKR_KEY_TYPE_INCONSISTENT

<<sun/security/pkcs11/Mac/MacSameTest.java.MacSameTest>>
<<sun/security/pkcs11/Mac/ReinitMac.java>>

to Initialization failed PKCS11Exception: CKR_SLOT_ID_INVALID.

Both of the two below tests will call PKCS11Test.getSunPKCS11(PKCS11Test.java:105) to get the SunPKCS11 provider. When testing in the FIPS mode, the SunPKCS11 will first be initialized as a FIPS provider SunPKCS11-NSS-FIPS. And then in the test code PKCS11Test, line 480 and 486. It will try to configure the SunPKCS11 using the p11-nss.txt to the NSS mode. But in the FIPS mode, there can only be a single PKCS11 provider. So configure the SunPKCS11 to the NSS mode will failed.

The exception caused by CKR_KEY_TYPE_INCONSISTENT seems to be solved by adding attributes(*,CKO_SECRET_KEY,CKK_GENERIC_SECRET)={ CKA_SIGN=true } in the nss.fips.cfg file..