voms-proxy-init with OpenSSL3 does not work for specific VOMS server

[hmiyake]

Hello,

When we use DIRACOS2 2.36, we have an error with voms-proxy-init as follows

$ voms-proxy-init -voms belle:/belle -bits 2048 -vomses ./diracos/etc/grid-security/vomses -r -timeout 12
Enter GRID pass phrase:
Your identity: /C=JP/O=KEK/OU=CRC/CN=Hideki Miyake
Creating temporary proxy ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. Done
Contacting  voms.cc.kek.jp:15020 [/C=JP/O=KEK/OU=CRC/CN=host/voms.cc.kek.jp] "belle" Done
Creating proxy .......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... Done

Your proxy is valid until Sun Dec 10 07:12:51 2023
Error: verification failed.
Cannot verify AC signature!

Curiously it works with our the other VOMS server at DESY

Contacting  grid-voms.desy.de:15020 [/DC=org/DC=terena/DC=tcs/C=DE/ST=Hamburg/O=Deutsches Elektronen-Synchrotron DESY/CN=grid-voms.desy.de] "belle" Done
Creating proxy ..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... Done

Your proxy is valid until Sun Dec 10 07:14:25 2023 

With previous DIRACOS2 using OpenSSL 1.1.1s, both voms-proxy-init work.
I suspect this is related to CA certificate since KEK issues theirs with SHA-1 while DESY with SHA-384, though I haven't been able to confirm that point since there is no clue from the command output.
Indeed we can get proxy itself (FYI: KEK issues user certificate with SHA-512) but the command fails at the last AC verification.

I wonder if we are able to configure OpenSSL3 to accept SHA-1 certificate assuming the cause, but my attempts are failing... (perhaps it should be done during the DIRACOS2 build?)

Following Chris's comment during BiLD, I gave various bit lengths, 512/1024/2048/4096, then the first failed with segmentation fault, and the rest failed with the AC verification, unfortunately (1024-4096 succeed with previous DIRACOS2).

We'd appreciate it if you could kindly advise us anything to resolve the issue, and/or your experience with VOMS server issuing SHA-1 CA certificate, if you have.

Best Regards,
Hideki

[chaen]

Maybe running the voms-proxy-init command with -debug will give you useful information ?

[hmiyake]

Thanks, tried that, but verbosity for AC verification is not changed...

$ voms-proxy-init -voms belle:/belle -bits 2048 -vomses ./diracos/etc/grid-security/vomses -r -timeout 12 -debug
PCI extension info: 
 Path length: -1
 Policy language not specified.
 Policy file not specified.
Number of bits in key :2048
Files being used:
 CA certificate file: none
 Trusted certificates directory : /etc/grid-security/certificates
 Proxy certificate file : /tmp/x509up_u21313
 User certificate file: /home/belle2/hideki/.globus/usercert.pem
 User key file: /home/belle2/hideki/.globus/userkey.pem
Output to /tmp/x509up_u21313
Enter GRID pass phrase:
Your identity: /C=JP/O=KEK/OU=CRC/CN=Hideki Miyake
Using configuration file ./diracos/etc/grid-security/vomses
Creating temporary proxy to /tmp/tmp_x509up_u21313_148824 ..........+...+++++++++++++++++++++++++++++++++++++++*.....+..............+++++++++++++++++++++++++++++++++++++++*.......+.....+.......+...+........+.........+...+....+.....+.+..+......+............+...+.......+...+......+.....+.............+.................+................+.........+.....+...+................+..+...............+.......+..+...+.+..+.........................+..+.......+........+.+.....+....+.....+...+..........+...+...........+.+...+.........+......+.........+.....+.+..+...+.........................+..+....+...+............+...+...............+..+...+.......+...+.....+.+..............+................+.....+.+......+.....+...+.......+..+..........+..+.+...+............+..+.+......+........+................+.....+....+..+.........+............+...............+.......+..+.+...+.....+....+..+...............+...+...++++++
..+..+....+......+..............+....+..+...+......+............+.............+..+.......+..+.........+....+++++++++++++++++++++++++++++++++++++++*.....+......+....+..+.........+...+...+...+.+...+......+..+....+...+++++++++++++++++++++++++++++++++++++++*....+...+..+...+....+...+............+.....+...+.+......+.....+.+........+....+...+........+.........+...++++++
 Done
Contacting  voms.cc.kek.jp:15020 [/C=JP/O=KEK/OU=CRC/CN=host/voms.cc.kek.jp] "belle" Done
Creating proxy to /tmp/x509up_u21313 ...+....+........+.+.........+..+.+++++++++++++++++++++++++++++++++++++++*.+.....+...+...................+........+...............+....+...+++++++++++++++++++++++++++++++++++++++*......+......+.+..+.+.....+...................+...+........+....+..+...............+......+...+...................+......+...........+....+..+...+....+...+..+...+....+...+..+.............+.....+...+............+.+......+......+...+.....+......+.........+......+.............+...+..............+.+...+...........+.+...+..+.......+..+......+...+.+........+.+..+...+.+.....+...+..................+.......+..+.+.........+.....+................+..+...............+.+......+........+......+.+.....+...+.+......+......+.....+....+........+............+......+...+....+............+...+............+...+.....+...+.......+...+...+.....+.......+...+.....+......++++++
..........+.....+......+.........+.+...+......+..............+....+........+.........+.+........+.+......+..+.+...........+.........+.+...+...+...+.........+...+.....+.........+....+.....+....+.................+...+.+++++++++++++++++++++++++++++++++++++++*..+.........+.....+.+.....+...+....+.....+......+++++++++++++++++++++++++++++++++++++++*................+...+.....+......+................+.....+...+.+..+.......+.....+..........+..+.+...+..+.......+...+..+..........+...............+......+..+...+.+.....+.+...+..............+.+.....+......+...+......+....+...........................+...+...........+.........+.+..+.........+.+..+.+......+......+.................+..................++++++

No policy language specified, Gsi impersonation proxy assumed. Done

Your proxy is valid until Fri Dec 15 11:45:52 2023
Error: verification failed.
Cannot verify AC signature!

[chaen]

I suspect the vomsdir location is wrong, or does not contain what it needs for the KEK server. Can you check that the value of X509_VOMS_DIR is what you need it to be ?

[hmiyake]

Thank you for the advise! I tried several conditions. To avoid possible confusion from existing directories, I changed my environment.

Here are our variables relevant to X509

X509_VOMSES=/ext/home/hideki/voms_test/diracos/etc/grid-security/vomses
X509_VOMS_DIR=/ext/home/hideki/voms_test/diracos/etc/grid-security/vomsdir

They are given by ./diracos/diracosrc.
This configuration works with our another VOMS server at DESY.
Furthermore, both VOMS servers work with OpenSSL1.1.1 version of DIRACOS2.

$ ls -l /ext/home/hideki/voms_test/diracos/etc/grid-security/vomsdir/belle
total 8
-rw-r--r-- 1 hideki b2_belle2 154 Dec 18 12:10 grid-voms.desy.de.lsc
-rw-r--r-- 1 hideki b2_belle2  95 Dec 18 12:10 voms.cc.kek.jp.lsc
$ cat /ext/home/hideki/voms_test/diracos/etc/grid-security/vomsdir/belle/voms.cc.kek.jp.lsc 
/C=JP/O=KEK/OU=CRC/CN=host/voms.cc.kek.jp
/C=JP/O=KEK/OU=CRC/CN=KEK GRID Certificate Authority
$ cat /ext/home/hideki/voms_test/diracos/etc/grid-security/vomsdir/belle/grid-voms.desy.de.lsc 
/DC=org/DC=terena/DC=tcs/C=DE/ST=Hamburg/O=Deutsches Elektronen-Synchrotron DESY/CN=grid-voms.desy.de
/C=NL/O=GEANT Vereniging/CN=GEANT eScience SSL CA 4

If I unset X509_VOMS_DIR or point wrong dir, it doesn't work at DESY too. KEK does not work always regardless of X509_VOMS_DIR. But again, it works with OpenSSL 1.1.1 and correct X509_VOMS_DIR...

[hmiyake]

Follow-up

Our colleague at KEK computing center found a solution.
voms-clients-cpp doesn't work while voms-clients-java works at ALMA Linux 9.

[voms-clients-cpp]

$ voms-proxy-init --voms belle:/belle -bits 2048 --vomses ./files/vomses_kek
Enter GRID pass phrase:
Your identity: /C=JP/O=KEK/OU=CRC/CN=Hideki Miyake
Cannot open fileFilename=/home/miyake/.rnd
Function:
Creating temporary proxy ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. Done
Contacting  voms.cc.kek.jp:15020 [/C=JP/O=KEK/OU=CRC/CN=host/voms.cc.kek.jp] "belle" Done
Creating proxy .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. Done

Your proxy is valid until Mon Dec 25 22:34:09 2023
Error: verification failed.
Cannot verify AC signature!  Underlying error: Unable to match certificate chain against file: /etc/grid-security/vomsdir/belle/voms.cc.kek.jp.lsc

[voms-clients-java]

$ voms-proxy-init --voms belle:/belle -bits 2048 --vomses ./files/vomses_kek
Enter GRID pass phrase for this identity:
Contacting voms.cc.kek.jp:15020 [/C=JP/O=KEK/OU=CRC/CN=host/voms.cc.kek.jp] "belle"...
Remote VOMS server contacted succesfully.


Created proxy in /tmp/x509up_u1000.

Your proxy is valid until Mon Dec 25 22:31:50 JST 2023

So still I'm not sure what prevents from AC verification at cpp client, it might be a clue that java client can avoid the issue...

[adriansev]

could be related to the discussion from xrootd/xrootd#2150
AFAIU there is a difference between java and openssl usage of CA chains...

[chaen]

I had missed your answer and the same thing just occurred to me ! It indeed seems likely