Added support for validating HMC server certificates

docs/appendix.rst

@@ -75,5 +75,8 @@ Bibliography
    HMC Operations Guide 2.13.1
        `IBM z Systems Hardware Management Console Operations Guide (Version 2.13.1) <https://www-01.ibm.com/support/docview.wss?uid=isg20351070eb1b67cd985257f7000487d13>`_
 
+   HMC Security
+       `Hardware Management Console Security <https://www.ibm.com/support/pages/node/6017320>`_
+
    KVM for IBM z Systems V1.1.2 System Administration
        `IBM SC27-8237, KVM for IBM z Systems V1.1.2 System Administration <https://www.ibm.com/support/knowledgecenter/SSNW54_1.1.2/com.ibm.kvm.v112.kvmlp/KVM.htm>`_

docs/changes.rst

@@ -28,12 +28,24 @@ Released: not yet
 
 **Incompatible changes:**
 
+* The zhmc command now verifies HMC server certificates by default, using the
+  CA certificates in the 'certifi' Python package. This verification will reject
+  the self-signed certificates the HMC is set up with initially. To deal with
+  this, install a CA-verifiable certificate in the HMC and specify the correct
+  CA certificates with the new '-c / --ca-certs' option. As a temporary quick
+  fix, you can disable the verification with the new '-n / --no-verify'
+  option.
+
 **Deprecations:**
 
 **Bug fixes:**
 
 **Enhancements:**
 
+* The zhmc command now supports verification of the HMC server certificate.
+  There are two new command line options '-n / --no-verify' and '-c / --ca-certs'
+  that control the verification behavior.
+
 **Cleanup:**
 
 **Known issues:**

docs/cli.rst

@@ -135,6 +135,18 @@ commands:
                                       environment variable).
       -p, --password TEXT             Password for the HMC (Default: ZHMC_PASSWORD
                                       environment variable).
+      -n, --no-verify                 Do not verify the HMC certificate. (Default:
+                                      ZHMC_NO_VERIFY environment variable, or
+                                      verify the HMC certificate).
+      -c, --ca-certs TEXT             Path name of certificate file or directory
+                                      with CA certificates to be used for
+                                      verifying the HMC certificate. (Default:
+                                      Path name in ZHMC_CA_CERTS environment
+                                      variable, or path name in REQUESTS_CA_BUNDLE
+                                      environment variable, or path name in
+                                      CURL_CA_BUNDLE environment variable, or the
+                                      'certifi' Python package which provides the
+                                      Mozilla CA Certificate List).
       -o, --output-format [[table|plain|simple|psql|rst|mediawiki|html|latex|
                           json]]
                                       Output format (Default: table).
@@ -205,6 +217,8 @@ examples, an underscore ``_`` is shown as the cursor:
         --host            Hostname or IP address of the HMC (Default: ZHMC_HOST environment variable).
         --userid          Username for the HMC (Default: ZHMC_USERID environment variable).
         --password        Password for the HMC (Default: ZHMC_PASSWORD environment variable).
+        --no-verify       Do not verify the HMC certificate. (Default: ZHMC_NO_VERIFY ...
+        --ca-certs        Path name of certificate file or directory with CA certificates ...
         --output-format   Output format (Default: table).
         --transpose       Transpose the output table for metrics.
         --error-format    Error message format (Default: msg).
@@ -306,25 +320,33 @@ command line. This can be done in either of two ways:
 
   .. code-block:: text
 
-      $ zhmc -h zhmc.example.com -u hmcuser session create
+      $ zhmc -n -h zhmc.example.com -u hmcuser session create
       Enter password: <password>
       export ZHMC_HOST=zhmc.example.com
       export ZHMC_USERID=hmcuser
       export ZHMC_SESSION_ID=<session-id>
+      export ZHMC_NO_VERIFY=TRUE
+      unset ZHMC_CA_CERTS
+
+  Note that the ``-n`` option is used to make this command work for
+  demonstration purposes regardless of the actual HMC certificate setup. It is
+  not recommended to use this option in production environments.
+  See :ref:`HMC certificate` for details.
 
   This ability can be used to set those environment variables and thus to
   persist the session-id in the shell environment, from where it will be used
   in any subsequent zhmc commands:
 
   .. code-block:: text
 
-      $ eval $(zhmc -h zhmc.example.com -u hmcuser session create)
+      $ eval $(zhmc -n -h zhmc.example.com -u hmcuser session create)
       Enter password: <password>
 
       $ env |grep ZHMC
       ZHMC_HOST=zhmc.example.com
       ZHMC_USERID=hmcuser
       ZHMC_SESSION_ID=<session-id>
+      ZHMC_NO_VERIFY=TRUE
 
       $ zhmc cpc list
       . . . <list of CPCs managed by this HMC>
@@ -336,10 +358,80 @@ command line. This can be done in either of two ways:
   Using the session-id from the environment is also a performance improvement,
   because it avoids the HMC Logon operation that otherwise would take place.
 
-* by storing the HMC password in the ZHMC_PASSWORD environment variable.
+* by storing the HMC password in the ``ZHMC_PASSWORD`` environment variable.
+
+The ``ZHMC_HOST``, ``ZHMC_USERID``, ``ZHMC_PASSWORD``, ``ZHMC_NO_VERIFY``, and
+``ZHMC_CA_CERTS`` environment variables act as defaults for the corresponding
+command line options.
+
+
+.. _`HMC certificate`:
+
+HMC certificate
+---------------
+
+By default, the HMC is configured with a self-signed certificate. That is the
+X.509 certificate presented by the HMC as the server certificate during SSL/TLS
+handshake at its Web Services API.
+
+Starting with version 0.22, the 'zhmc' command will reject self-signed
+certificates by default.
+
+The HMC should be configured to use a CA-verifiable certificate. This can be
+done in the HMC task "Certificate Management". See also the :term:`HMC Security`
+book and Chapter 3 "Invoking API operations" in the :term:`HMC API` book.
+
+Starting with version 0.22, the zhmc command provides control knobs for the
+verification of the HMC certificate via the ``-c`` / ``--ca-certs`` and
+``-n`` / ``--no-verify`` command line options, as follows:
+
+* None of the two options specified (default): Verify the HMC certificate using
+  the CA certificates from the first of these locations:
+
+  - The certificate file or directory in the ``ZHMC_CA_CERTS`` environment
+    variable, if set
+  - The certificate file or directory in the ``REQUESTS_CA_BUNDLE`` environment
+    variable, if set
+  - The certificate file or directory in the ``CURL_CA_BUNDLE`` environment
+    variable, if set
+  - The `Python 'certifi' package <https://pypi.org/project/certifi/>`_
+    (which contains the
+    `Mozilla Included CA Certificate List <https://wiki.mozilla.org/CA/Included_Certificates>`_).
+
+* ``-c`` / ``--ca-certs`` specified: Verify the HMC certificate using the CA
+  certificates in the specified certificate file or directory.
+
+* ``-n`` / ``--no-verify`` specified or ``ZHMC_NO_VERIFY`` environment variable
+  set: Do not verify the HMC certificate. Not verifying the HMC certificate
+  means that hostname mismatches, expired certificates, revoked certificates,
+  or otherwise invalid certificates will not be detected. Since this mode makes
+  the connection vulnerable to man-in-the-middle attacks, it is insecure and
+  should not be used in production environments.
+
+If a certificate file is specified (using any of the ways listed above), that
+file must be in PEM format and must contain all CA certificates that are
+supposed to be used. Usually they are in the order from leaf to root, but
+that is not a hard requirement. The single certificates are concatenated
+in the file.
+
+If a certificate directory is specified (using any of the ways listed above),
+it must contain PEM files with all CA certificates that are supposed to be used,
+and copies of the PEM files or symbolic links to them in the hashed format
+created by the OpenSSL command ``c_rehash``.
+
+An X.509 certificate in PEM format is base64-encoded, begins with the line
+``-----BEGIN CERTIFICATE-----``, and ends with the line
+``-----END CERTIFICATE-----``.
+More information about the PEM format is for example on this
+`www.ssl.com page <https://www.ssl.com/guide/pem-der-crt-and-cer-x-509-encodings-and-conversions>`_
+or in this `serverfault.com answer <https://serverfault.com/a/9717/330351>`_.
+
+Note that setting the ``REQUESTS_CA_BUNDLE`` or ``CURL_CA_BUNDLE`` environment
+variables influences other programs that use these variables, too.
 
-The ZHMC_HOST, ZHMC_USERID, and ZHMC_PASSWORD environment variables act as
-defaults for the corresponding command line options.
+For more information, see the
+`Security <https://python-zhmcclient.readthedocs.io/en/latest/security.html>`_
+section in the documentation of the 'zhmcclient' package.
 
 
 .. _`CLI commands`:

minimum-constraints.txt

@@ -59,7 +59,8 @@ wheel==0.29.0
 
 # Direct dependencies for runtime (must be consistent with requirements.txt)
 
-zhmcclient==0.30.0
+# TODO: Enable 0.31.0 again before releasing
+# zhmcclient==0.31.0
 
 click==7.0
 click-repl==0.1.0

requirements.txt

@@ -9,8 +9,9 @@
 
 # Direct dependencies (except pip, setuptools, wheel):
 
-# git+https://github.com/zhmcclient/python-zhmcclient.git@master#egg=zhmcclient
-zhmcclient>=0.30.0 # Apache
+# TODO: Enable 0.31.0 again before releasing
+git+https://github.com/zhmcclient/python-zhmcclient.git@master#egg=zhmcclient
+# zhmcclient>=0.31.0 # Apache
 
 # click <7.0 did not properly declare supported Python versions
 # click 7.0 dropped support for Python 3.4, but still installs on it and works fine

zhmccli/_cmd_session.py

@@ -75,9 +75,28 @@ def cmd_session_create(cmd_ctx):
         raise click_exception(exc, cmd_ctx.error_format)
 
     cmd_ctx.spinner.stop()
+
+    if session.verify_cert is False:
+        no_verify = 'TRUE'
+        ca_certs = None
+    elif session.verify_cert is True:
+        no_verify = None
+        ca_certs = None
+    else:
+        no_verify = None
+        ca_certs = session.verify_cert
+
     click.echo("export ZHMC_HOST={h}".format(h=session.host))
     click.echo("export ZHMC_USERID={u}".format(u=session.userid))
     click.echo("export ZHMC_SESSION_ID={s}".format(s=session.session_id))
+    if no_verify is None:
+        click.echo("unset ZHMC_NO_VERIFY")
+    else:
+        click.echo("export ZHMC_NO_VERIFY={nv}".format(nv=no_verify))
+    if ca_certs is None:
+        click.echo("unset ZHMC_CA_CERTS")
+    else:
+        click.echo("export ZHMC_CA_CERTS={cc}".format(cc=ca_certs))
 
 
 def cmd_session_delete(cmd_ctx):

zhmccli/_helper.py

@@ -165,11 +165,14 @@ class CmdContext(object):
     data.
     """
 
-    def __init__(self, host, userid, password, output_format, transpose,
-                 error_format, timestats, session_id, get_password):
+    def __init__(self, host, userid, password, no_verify, ca_certs,
+                 output_format, transpose, error_format, timestats, session_id,
+                 get_password):
         self._host = host
         self._userid = userid
         self._password = password
+        self._no_verify = no_verify
+        self._ca_certs = ca_certs
         self._output_format = output_format
         self._transpose = transpose
         self._error_format = error_format
@@ -182,6 +185,7 @@ def __init__(self, host, userid, password, output_format, transpose,
     def __repr__(self):
         ret = "CmdContext(at 0x{ctx:08x}, host={s._host!r}, " \
             "userid={s._userid!r}, password={pw!r}, " \
+            "no_verify={s._no_verify!r}, ca_certs={s._ca_certs!r}, " \
             "output_format={s._output_format!r}, transpose={s._transpose!r}, " \
             "error_format={s._error_format!r}, session_id={s._session_id!r}, " \
             "session={s._session!r}, ...)". \
@@ -202,6 +206,23 @@ def userid(self):
         """
         return self._userid
 
+    @property
+    def no_verify(self):
+        """
+        bool: Do not verify the server certificate presented by the HMC
+        during SSL/TLS handshake.
+        """
+        return self._no_verify
+
+    @property
+    def ca_certs(self):
+        """
+        :term:`string`: Path name of certificate file or directory with CA
+        certificates for verifying the HMC certificate. If `None`, the
+        zhmcclient will be set up to use the 'certifi' package.
+        """
+        return self._ca_certs
+
     @property
     def output_format(self):
         """
@@ -278,10 +299,17 @@ def execute_cmd(self, cmd):
                 if self._host is None:
                     raise click_exception("No HMC host provided",
                                           self._error_format)
+                if self._no_verify:
+                    verify_cert = False
+                elif self._ca_certs is None:
+                    verify_cert = True  # Use 'certifi' package
+                else:
+                    verify_cert = self._ca_certs
                 self._session = zhmcclient.Session(
                     self._host, self._userid, self._password,
                     session_id=self._session_id,
-                    get_password=self._get_password)
+                    get_password=self._get_password,
+                    verify_cert=verify_cert)
         if self.timestats:
             self._session.time_stats_keeper.enable()
         self.spinner.start()

zhmccli/zhmccli.py

@@ -88,6 +88,18 @@
 @click.option('-p', '--password', type=str, envvar='ZHMC_PASSWORD',
               help="Password for the HMC "
                    "(Default: ZHMC_PASSWORD environment variable).")
+@click.option('-n', '--no-verify', is_flag=True, envvar='ZHMC_NO_VERIFY',
+              help="Do not verify the HMC certificate. "
+                   "(Default: ZHMC_NO_VERIFY environment variable, or verify "
+                   "the HMC certificate).")
+@click.option('-c', '--ca-certs', type=str, envvar='ZHMC_CA_CERTS',
+              help="Path name of certificate file or directory with CA "
+                   "certificates to be used for verifying the HMC certificate. "
+                   "(Default: Path name in ZHMC_CA_CERTS environment variable, "
+                   "or path name in REQUESTS_CA_BUNDLE environment variable, "
+                   "or path name in CURL_CA_BUNDLE environment variable, "
+                   "or the 'certifi' Python package which provides the "
+                   "Mozilla CA Certificate List).")
 @click.option('-o', '--output-format',
               type=click.Choice(TABLE_FORMATS + ['json']),
               help='Output format (Default: {def_of}).'.
@@ -117,8 +129,8 @@
     help="Show the versions of this command and of the zhmcclient package and "
     "exit.")
 @click.pass_context
-def cli(ctx, host, userid, password, output_format, transpose, error_format,
-        timestats, log, log_dest, syslog_facility):
+def cli(ctx, host, userid, password, no_verify, ca_certs, output_format,
+        transpose, error_format, timestats, log, log_dest, syslog_facility):
     """
     Command line interface for the IBM Z HMC.
 
@@ -155,6 +167,10 @@ def cli(ctx, host, userid, password, output_format, transpose, error_format,
         if password is None:
             # pylint: disable=protected-access
             password = ctx.obj._password
+        if no_verify is None:
+            no_verify = ctx.obj.no_verify
+        if ca_certs is None:
+            ca_certs = ctx.obj.ca_certs
         if output_format is None:
             output_format = ctx.obj.output_format
         if transpose is None:
@@ -171,6 +187,13 @@ def cli(ctx, host, userid, password, output_format, transpose, error_format,
             format(of=output_format),
             error_format)
 
+    if no_verify and ca_certs:
+        raise click_exception(
+            "Disabling HMC certificate verification (-n / --no-verify / "
+            "ZHMC_NO_VERIFY) conflicts with specifying a CA certificate path "
+            "(-c / --ca-certs / ZHMC_CA_CERTS)",
+            error_format)
+
     # TODO: Add context support for the following options:
     if log is None:
         log = DEFAULT_LOG
@@ -291,9 +314,9 @@ def get_password_via_prompt(host, userid):
     # We create a command context for each command: An interactive command has
     # its own command context different from the command context for the
     # command line.
-    ctx.obj = CmdContext(host, userid, password, output_format, transpose,
-                         error_format, timestats, session_id,
-                         get_password_via_prompt)
+    ctx.obj = CmdContext(host, userid, password, no_verify, ca_certs,
+                         output_format, transpose, error_format, timestats,
+                         session_id, get_password_via_prompt)
 
     # Invoke default command
     if ctx.invoked_subcommand is None: