Wrong '.der' file-check after creating Certificates via RFC 2136 and Inhouse FreeIPA #1164

tuxmi · 2024-10-13T20:41:20Z

tuxmi
Oct 13, 2024

Hi ZoeyVid

I tried to implement the FreeIPA ACME over your custom ACME solution. This sadly did not work, because it seems, that FreeIPA needs additional Hooks, to work correctly and Issue Certificates with NPMplus.

So I tried over DNS-Challange and the RFC 2136 Provider. This is not the best option, because I need everytime to manually select this provider again and fill out all needed config values.

But.. it seems to work - nearly :)

Inside the below log output, you see that it gets correctly certificates with the following command:

certbot --logs-dir /tmp/certbot-log --work-dir /tmp/certbot-work --config-dir /data/tls/certbot certonly --config "/data/tls/certbot/config.ini" --cert-name "npm-4" --domains "proxyint.michu-it.corp" --authenticator dns-rfc2136 --dns-rfc2136-credentials "/data/tls/certbot/credentials/credentials-4" --email "[email protected]"

The certificates are valid and correctly stored, but the config-check is failing then see:

nginx: [emerg] BIO_new_file("/data/tls/certbot/live/npm-4.der") failed (SSL: error:80000002:system library::No such file or directory:calling fopen(/data/tls/certbot/live/npm-4.der, rb) error:10000080:BIO routines::no such file) nginx: configuration file /usr/local/nginx/conf/nginx.conf test failed

The cause seems to sit in the file "_certificates.conf" on line 7: ssl_stapling_file /data/tls/certbot/live/npm-{{ certificate_id }}.der;

Full Log:

nginx: [emerg] BIO_new_file("/data/tls/certbot/live/npm-3.der") failed (SSL: error:80000002:system library::No such file or directory:calling fopen(/data/tls/certbot/live/npm-3.der, rb) error:10000080:BIO routines::no such file)
nginx: configuration file /usr/local/nginx/conf/nginx.conf test failed
[Nginx    ] › ℹ  info      Reloading Nginx
[SSL      ] › ℹ  info      Revoking Certbot certificates for Cert #3: proxyint.swissmakers.corp
[SSL      ] › ℹ  info      Deleted all files relating to certificate npm-3.
Congratulations! You have successfully revoked the certificate that was located at /data/tls/certbot/live/npm-3/fullchain.pem.

[SSL      ] › ℹ  info      Credentials file deleted successfully
[Nginx    ] › ℹ  info      Reloading Nginx
[Nginx    ] › ℹ  info      Reloading Nginx
[Certbot  ] › ▶  start     Installing rfc2136...
[Certbot  ] › ☒  complete  Installed rfc2136
[SSL      ] › ℹ  info      Requesting Certbot certificates via RFC 2136 for Cert #4: proxyint.swissmakers.corp
[SSL      ] › ℹ  info      Command: certbot --logs-dir /tmp/certbot-log --work-dir /tmp/certbot-work --config-dir /data/tls/certbot certonly --config "/data/tls/certbot/config.ini" --cert-name "npm-4" --domains "proxyint.michu-it.corp" --authenticator dns-rfc2136 --dns-rfc2136-credentials "/data/tls/certbot/credentials/credentials-4" --email "[email protected]"
[SSL      ] › ℹ  info      Requesting a certificate for proxyint.michu-it.corp
Waiting 60 seconds for DNS changes to propagate

Successfully received certificate.
Certificate is saved at: /data/tls/certbot/live/npm-4/fullchain.pem
Key is saved at:         /data/tls/certbot/live/npm-4/privkey.pem
This certificate expires on 2025-01-11.
These files will be updated when the certificate renews.
NEXT STEPS:
- The certificate will need to be renewed before it expires. Certbot can automatically renew the certificate in the background, but you may need to take steps to enable that functionality. See https://certbot.org/renewal-setup for instructions.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
If you like Certbot, please consider supporting our work by:
 * Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
 * Donating to EFF:                    https://eff.org/donate-le
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

nginx: [emerg] BIO_new_file("/data/tls/certbot/live/npm-4.der") failed (SSL: error:80000002:system library::No such file or directory:calling fopen(/data/tls/certbot/live/npm-4.der, rb) error:10000080:BIO routines::no such file)
nginx: configuration file /usr/local/nginx/conf/nginx.conf test failed
[Nginx    ] › ℹ  info      Reloading Nginx
2024/10/13 19:02:00 [warn] 574#574: deleting socket /run/nginx-1.sock
[SSL      ] › ⬤  debug     /data/tls/certbot/archive/npm-4/cert1.pem added to certificate zip
[SSL      ] › ⬤  debug     /data/tls/certbot/archive/npm-4/chain1.pem added to certificate zip
[SSL      ] › ⬤  debug     /data/tls/certbot/archive/npm-4/fullchain1.pem added to certificate zip
[SSL      ] › ⬤  debug     /data/tls/certbot/archive/npm-4/privkey1.pem added to certificate zip
[SSL      ] › ⬤  debug     zip completed :  /tmp/npm-4-1728838938244.zip
[Nginx    ] › ℹ  info      Reloading Nginx
nginx: [emerg] BIO_new_file("/data/tls/certbot/live/npm-4.der") failed (SSL: error:80000002:system library::No such file or directory:calling fopen(/data/tls/certbot/live/npm-4.der, rb) error:10000080:BIO routines::no such file)
nginx: configuration file /usr/local/nginx/conf/nginx.conf test failed
[Nginx    ] › ℹ  info      Reloading Nginx

Cheers from switzerland

Answered by tuxmi

Oct 20, 2024

I only had to add this three environments variables to my systemd-file, which starts my NPMplus proxy:

  -e "REQUESTS_CA_BUNDLE=/data/etc/npm/ca.crt" \
  -e "SSL_CERT_FILE=/data/etc/npm/ca.crt" \
  -e "SSL_CERT_DIR=/data/etc/npm" \

Now it seems, that only a liddle hook is missing, that will right execute the needed command to create the .der file, before running the configuration file /usr/local/nginx/conf/nginx.conf test, which will fail without the .der-file

View full answer

Zoey2936 · 2024-10-13T20:45:01Z

Zoey2936
Oct 13, 2024
Maintainer

can you please run ls /opt/npm/tls/certbot/live on the host system?

0 replies

tuxmi · 2024-10-13T21:24:56Z

tuxmi
Oct 13, 2024
Author

I think you did mean the path /data/tls/certbot/live/

[root@rlinfp01 opt]# podman exec -it proxy_INT ls /data/tls/certbot/live/
README  npm-4

[root@rlinfp01 opt]# podman exec -it proxy_INT ls /data/tls/certbot/live/npm-4/
README  cert.pem  chain.pem  fullchain.pem  privkey.pem

14 replies

tuxmi Oct 20, 2024
Author

Yes but it's trusted via Linux-Trust-store and the CA-certificate of my CA is also copied to: /data/etc/npm/ca.crt inside the NPMplus container and referenced with REQUESTS_CA_BUNDLE from Certbot to use this CA-Cert to Issue new Certificates to the now trusted CA :)

tuxmi Oct 20, 2024
Author

The folder, that your script seems to use are also missing inside the container, see:

rlinfp01:/app# ls -la /usr/local/openssl/.openssl/
total 16
drwxr-xr-x 5 root root    62 Oct  2 06:52 .
drwxr-xr-x 3 root root    22 Oct  2 08:26 ..
drwxr-xr-x 2 root root    37 Oct  2 06:52 bin
drwxr-xr-x 3 root root    21 Oct  2 06:46 include
drwxr-xr-x 5 root root    98 Oct  2 06:46 lib
-rw-r--r-- 1 root root 12401 Oct  2 06:52 openssl.cnf

maybe this is a bug?

00000000:error:80000002:system library:file_open:No such file or directory:providers/implementations/storemgmt/file_store.c:267:calling stat(/usr/local/openssl/.openssl/ssl/certs)
00000000:error:80000002:system library:file_open:No such file or directory:providers/implementations/storemgmt/file_store.c:267:calling stat(/usr/local/openssl/.openssl/ssl/certs)
00000000:error:80000002:system library:file_open:No such file or directory:providers/implementations/storemgmt/file_store.c:267:calling stat(/usr/local/openssl/.openssl/ssl/certs)

Zoey2936 Oct 20, 2024
Maintainer

no the problem is that your ca uses self signed certificates, which don't provied ocsp

Zoey2936 Oct 20, 2024
Maintainer

I will try to fix this

tuxmi Oct 20, 2024
Author

With my CA-file added to the command, the freeipa ocsp responder works.

podman exec -it proxy_INT openssl ocsp -issuer /data/tls/certbot/live/npm-11/chain.pem -cert /data/tls/certbot/live/npm-11/cert.pem -url https://freeipa-server.corp/ca/ocsp -CAfile /etc/ssl/certs/ca-certificates.crt
Response verify OK
/data/tls/certbot/live/npm-11/cert.pem: good
        This Update: Oct 20 11:17:10 2024 GMT

Maybe a CUSTOM_CA environment variable inside your container-image could be a good thing :)

tuxmi · 2024-10-20T11:52:12Z

tuxmi
Oct 20, 2024
Author

I fixed the validation via environment variables, after reading the OpenSSL-doc. Now your script and the validation seems to work:

[root@rlinfp01 ~]# podman exec -it proxy_INT certbot-ocsp-fetcher.sh -c /data/tls/certbot -o /data/tls/certbot/live --no-reload-webserver -v
Running in stand-alone mode...

Response verify OK
Response verify OK
Response verify OK
Response verify OK
Response verify OK
Response verify OK
Response verify OK
Response verify OK
Response verify OK
Response verify OK
Response verify OK
Response verify OK
Response verify OK
Response verify OK
Response verify OK
Response verify OK
Response verify OK
Response verify OK
LINEAGE RESULT  REASON
npm-11  updated
npm-12  updated
npm-16  updated
npm-4   updated
npm-5   updated
npm-6   updated
npm-7   updated
npm-8   updated
npm-9   updated


Install the BSD utility `column` for properly formatted output.
If the version of `column` supports the `--output-separator` flag,
the output will be formatted as TSV.

But there seems still a error directly after creating a new proxy entry with a new certificate, the vhost went directly offline with the error:

nginx: [emerg] BIO_new_file("/data/tls/certbot/live/npm-17.der") failed (SSL: error:80000002:system library::No such file or directory:calling fopen(/data/tls/certbot/live/npm-17.der, rb) error:10000080:BIO routines::no such file)

Successfully received certificate.
Certificate is saved at: /data/tls/certbot/live/npm-17/fullchain.pem
Key is saved at:         /data/tls/certbot/live/npm-17/privkey.pem
This certificate expires on 2025-01-18.
These files will be updated when the certificate renews.
NEXT STEPS:
- The certificate will need to be renewed before it expires. Certbot can automatically renew the certificate in the background, but you may need to take steps to enable that functionality. See https://certbot.org/renewal-setup for instructions.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
If you like Certbot, please consider supporting our work by:
 * Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
 * Donating to EFF:                    https://eff.org/donate-le
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

nginx: [emerg] BIO_new_file("/data/tls/certbot/live/npm-17.der") failed (SSL: error:80000002:system library::No such file or directory:calling fopen(/data/tls/certbot/live/npm-17.der, rb) error:10000080:BIO routines::no such file)
nginx: configuration file /usr/local/nginx/conf/nginx.conf test failed
[Nginx    ] › ℹ  info      Reloading Nginx
2024/10/20 13:47:19 [warn] 942#942: deleting socket /run/nginx-9.sock
[Nginx    ] › ℹ  info      Reloading Nginx
[Nginx    ] › ℹ  info      Reloading Nginx

But after manual "disable" and "enable" it again via NPMplus web-UI it seems to work. Also the *.der files are now created.

[root@rlinfp01 ~]#
[root@rlinfp01 ~]#
[root@rlinfp01 ~]# podman exec -it proxy_INT ls -la /data/tls/certbot/live/
total 48
drwxrwx--- 12 root root 4096 Oct 20 13:47 .
drwxrwx---  8 root root  148 Oct 20 13:47 ..
-rwxrwx---  1 root root  740 Oct 13 18:56 README
drwxrwx---  2 root root   93 Oct 16 22:23 npm-11
-rw-r--r--  1 root root 3554 Oct 20 13:47 npm-11.der
drwxrwx---  2 root root   93 Oct 19 15:58 npm-12
-rw-r--r--  1 root root 3554 Oct 20 13:47 npm-12.der
drwxrwx---  2 root root   93 Oct 20 11:18 npm-16
-rw-r--r--  1 root root 3553 Oct 20 13:47 npm-16.der
drwxr-xr-x  2 root root   93 Oct 20 13:47 npm-17
-rw-r--r--  1 root root 3554 Oct 20 13:47 npm-17.der
drwxrwx---  2 root root   93 Oct 13 19:01 npm-4
-rw-r--r--  1 root root 3554 Oct 20 13:47 npm-4.der
drwxrwx---  2 root root   93 Oct 13 23:41 npm-5
-rw-r--r--  1 root root 3554 Oct 20 13:47 npm-5.der
drwxrwx---  2 root root   93 Oct 14 17:35 npm-6
-rw-r--r--  1 root root 3554 Oct 20 13:47 npm-6.der
drwxrwx---  2 root root   93 Oct 14 20:02 npm-7
-rw-r--r--  1 root root 3554 Oct 20 13:47 npm-7.der
drwxrwx---  2 root root   93 Oct 16 13:19 npm-8
-rw-r--r--  1 root root 3553 Oct 20 13:47 npm-8.der
drwxrwx---  2 root root   93 Oct 16 13:46 npm-9
-rw-r--r--  1 root root 3554 Oct 20 13:47 npm-9.der

9 replies

Zoey2936 Oct 20, 2024
Maintainer

what did you changed exactly to make it work?

tuxmi Oct 20, 2024
Author

I only had to add this three environments variables to my systemd-file, which starts my NPMplus proxy:

  -e "REQUESTS_CA_BUNDLE=/data/etc/npm/ca.crt" \
  -e "SSL_CERT_FILE=/data/etc/npm/ca.crt" \
  -e "SSL_CERT_DIR=/data/etc/npm" \

Now it seems, that only a liddle hook is missing, that will right execute the needed command to create the .der file, before running the configuration file /usr/local/nginx/conf/nginx.conf test, which will fail without the .der-file

Answer selected by Zoey2936

Zoey2936 Oct 20, 2024
Maintainer

the hook should not be needed, because if you create a new certs ocsp fetcher is run before reloading

tuxmi Oct 20, 2024
Author

Hmm ok maybe the delay is to short, until reload? or is the reload only performed after successful run? - I would say not, because before the correct ENV-variables, the script was never run without errors, so it should not reload, but it did.

And now after manual disable & enable it works, so it looks async to me. What do you think?

Zoey2936 Oct 20, 2024
Maintainer

the scripts runs before reloading

Zoey2936 Oct 20, 2024
Maintainer

at least it should

AngelAsukii · 2024-11-26T02:20:17Z

AngelAsukii
Nov 26, 2024

hi i got the same issue, but the answer didnt fix my problem

3 replies

Zoey2936 Nov 26, 2024
Maintainer

Do you also use a custom acme server?

AngelAsukii Nov 26, 2024

i think no, i get this error after some updates

Zoey2936 Nov 26, 2024
Maintainer

then please open a new discussion this is related when using your own ca with self signed certs

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Wrong '.der' file-check after creating Certificates via RFC 2136 and Inhouse FreeIPA #1164

{{title}}

Replies: 4 comments 26 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Wrong '.der' file-check after creating Certificates via RFC 2136 and Inhouse FreeIPA #1164

tuxmi Oct 13, 2024

Replies: 4 comments · 26 replies

Zoey2936 Oct 13, 2024 Maintainer

tuxmi Oct 13, 2024 Author

tuxmi Oct 20, 2024 Author

tuxmi Oct 20, 2024 Author

Zoey2936 Oct 20, 2024 Maintainer

Zoey2936 Oct 20, 2024 Maintainer

tuxmi Oct 20, 2024 Author

tuxmi Oct 20, 2024 Author

Zoey2936 Oct 20, 2024 Maintainer

tuxmi Oct 20, 2024 Author

Zoey2936 Oct 20, 2024 Maintainer

tuxmi Oct 20, 2024 Author

Zoey2936 Oct 20, 2024 Maintainer

Zoey2936 Oct 20, 2024 Maintainer

AngelAsukii Nov 26, 2024

Zoey2936 Nov 26, 2024 Maintainer

AngelAsukii Nov 26, 2024

Zoey2936 Nov 26, 2024 Maintainer

tuxmi
Oct 13, 2024

Replies: 4 comments 26 replies

Zoey2936
Oct 13, 2024
Maintainer

tuxmi
Oct 13, 2024
Author

tuxmi Oct 20, 2024
Author

tuxmi Oct 20, 2024
Author

Zoey2936 Oct 20, 2024
Maintainer

Zoey2936 Oct 20, 2024
Maintainer

tuxmi Oct 20, 2024
Author

tuxmi
Oct 20, 2024
Author

Zoey2936 Oct 20, 2024
Maintainer

tuxmi Oct 20, 2024
Author

Zoey2936 Oct 20, 2024
Maintainer

tuxmi Oct 20, 2024
Author

Zoey2936 Oct 20, 2024
Maintainer

Zoey2936 Oct 20, 2024
Maintainer

AngelAsukii
Nov 26, 2024

Zoey2936 Nov 26, 2024
Maintainer

Zoey2936 Nov 26, 2024
Maintainer