Merge pull request #181 in ESSOL-BB/beegfs-csi-driver from webere-mis…

…cellaneous-1.2.1-doc-updates to master

* commit '102a5429895de788ce60ae43e4d551db37d960d5':
  Update the changelog for v1.2.1
  Add note for BeeGFS admins regarding .csi directories
  Fix CHANGELOG link typo
  Move anchors in markdown files
  Clarify orphan mounts troubleshooting

CHANGELOG.md

@@ -1,6 +1,28 @@
 # Changelog
 Notable changes to the BeeGFS CSI driver will be documented in this file. 
 
+[1.2.1] - 2021-12-20
+--------------------
+### Added
+- Support for BeeGFS v7.2.5, Kubernetes v1.22, and RedHat OpenShift v4.9. 
+- The ability to persist state in BeeGFS using a .csi/ directory structure that
+ exists alongside dynamically provisioned volumes in their `volDirBasePath`.
+ This is automatically enabled by default but can be [optionally
+ disabled](docs/usage.md#notes-for-beegfs-administrators).
+
+### Fixed
+- Common causes of [orphaned BeeGFS
+ mounts](docs/troubleshooting.md#orphan-mounts) being left on Kubernetes nodes
+ (listed as a known issue in v1.2.0) by maintaining a record of nodes with
+ active BeeGFS mounts for each volume in the new .csi/ directory and falling
+ back on a newly added timeout (`--node-unstage-timeout`) when needed.
+
+### Security
+Note: The BeeGFS CSI driver is written in Golang and does not import or
+implement any functionality that makes it susceptible to the recent Log4j
+vulnerability threat. For more details please refer to [NetApp's official
+response](https://www.netapp.com/newsroom/netapp-apache-log4j-response/).
+
 [1.2.0] - 2021-10-11
 --------------------
 ### Added
@@ -14,7 +36,7 @@ Notable changes to the BeeGFS CSI driver will be documented in this file.
  These are being provided as an example for others who might want to
  experiment with using BeeGFS and Nomad, in particular anyone interested in
  [contributing](CONTRIBUTING.md) to any future efforts around Nomad.
-- [Documentation](docs/deployment.md##mixed-kubernetes-deployment) on how to
+- [Documentation](docs/deployment.md#mixed-kubernetes-deployment) on how to
  deploy the driver to Kubernetes clusters where some nodes can access BeeGFS
  volumes, and some cannot.
 - Support for BeeGFS v7.2.4, Kubernetes v1.21, and RedHat OpenShift v4.8. 

README.md

@@ -4,8 +4,8 @@
 [![Docker pulls](https://img.shields.io/docker/pulls/netapp/beegfs-csi-driver)](https://hub.docker.com/r/netapp/beegfs-csi-driver)
 [![Go report card](https://goreportcard.com/badge/github.com/netapp/beegfs-csi-driver)](https://goreportcard.com/report/github.com/netapp/beegfs-csi-driver)
 
-## Contents
 <a name="contents"></a>
+## Contents
 
 * [Overview](#overview)
 * [Getting Started](#getting-started)
@@ -15,8 +15,9 @@
 * [License](#license)
 * [Maintainers](#maintainers)
 
-## Overview 
 <a name="overview"></a>
+## Overview 
+
 The BeeGFS Container Storage Interface (CSI) driver provides high performing and
 scalable storage for workloads running in container orchestrators like
 Kubernetes. This driver allows containers to access existing datasets or request
@@ -29,8 +30,9 @@ configure) and day-2 (reconfigure/update) tasks for the driver. This especially
 discovery and installation from Operator Lifecycle Manger (OLM) enabled clusters like Red Hat 
 OpenShift. 
 
-### Notable Features
 <a name="notable-features"></a>
+### Notable Features
+
 * Integration of [Storage Classes in Kubernetes](docs/usage.md#create-a-storage-class) with [storage
  pools](https://doc.beegfs.io/latest/advanced_topics/storage_pools.html) in
  BeeGFS, allowing different tiers of storage within the same file system to be
@@ -51,8 +53,8 @@ OpenShift.
  share access to the same working directories and enable multi-user/application
  access to common datasets.
 
-### Interoperability and CSI Feature Matrix
 <a name="interoperability-and-csi-feature-matrix"></a>
+### Interoperability and CSI Feature Matrix
 
 | beegfs.csi.netapp.com | K8s Versions | Red Hat OpenShift Versions | BeeGFS Versions | CSI Version |
 | ---------------------- | -------------------------------- | -------------------------- | --------------- | ------------ |
@@ -147,11 +149,12 @@ the use of the BeeGFS CSI driver with HashiCorp Nomad, we may not test with
 Nomad for every driver release and do not currently consider Nomad to be a 
 supported container orchestrator.
 
-## Getting Started 
 <a name="getting-started"></a>
+## Getting Started
 
-### Prerequisite(s) 
 <a name="prerequisites"></a>
+### Prerequisite(s)
+
 * Deploying the driver requires access to a terminal with kubectl. 
 * The [BeeGFS DKMS
  client](https://doc.beegfs.io/latest/advanced_topics/client_dkms.html) must be
@@ -169,8 +172,9 @@ supported container orchestrator.
  nodes over a TCP/IP and/or RDMA (InfiniBand/RoCE) capable network (not
  required to deploy the driver).
 
-### Quick Start
 <a name="quick-start"></a>
+### Quick Start
+
 The steps in this section allow you to get the driver up and running quickly.
 For production use cases or air-gapped environments it is recommended to read
 through the full [kubectl deployment guide](docs/deployment.md) or [operator 
@@ -193,8 +197,9 @@ pods -n kube-system | grep csi-beegfs`
 Provided all Pods are running the driver is now ready for use. See the following
 sections for how to get started using the driver.
 
-## Basic Use
 <a name="basic-use"></a>
+## Basic Use
+
  This section provides a quick summary of basic driver use and functionality.
  Please see the full [usage documentation](docs/usage.md) for a complete
  overview of all available functionality. The driver was designed to support
@@ -206,28 +211,32 @@ sections for how to get started using the driver.
  isolation between multiple applications and users using the same BeeGFS file
  system when desired. 
 
-### Dynamic Storage Provisioning:
 <a name="dynamic-storage-provisioning"></a>
+### Dynamic Storage Provisioning:
+
 Administrators create a Storage Class in Kubernetes referencing at minimum a
 specific BeeGFS file system and parent directory within that file system. Users
 can then submit PVCs against the Storage Class, and are provided isolated access
 to new directories under the parent specified in the Storage Class. 
 
-### Static Provisioning:
 <a name="static-provisioning"></a>
+### Static Provisioning:
+
 Administrators create a PV and PVC representing an existing directory in a
 BeeGFS file system. This is useful for exposing some existing dataset or shared
 directory to Kubernetes users and applications.
 
-### Examples
 <a name="examples"></a>
+### Examples
+
 [Example Kubernetes manifests](examples/k8s/README.md) of how to use the driver are
 provided. These are meant to be repurposed to simplify creating objects related
 to the driver including Storage Classes, Persistent Volumes, and Persistent
 Volume Claims in your environment.
 
-## Requesting Enhancements and Reporting Issues 
 <a name="requesting-enhancements-and-reporting-issues"></a>
+## Requesting Enhancements and Reporting Issues 
+
 If you have any questions, feature requests, or would like to report an issue
 please submit them at https://github.com/NetApp/beegfs-csi-driver/issues. 
 
@@ -236,12 +245,14 @@ The BeeGFS CSI Driver maintainers welcome improvements from the BeeGFS and
 open source community! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for how 
 to get started.
 
-## License 
 <a name="license"></a>
+## License 
+
 Apache License 2.0
 
-## Maintainers 
 <a name="maintainers"></a>
+## Maintainers 
+
 * Eric Weber (@ejweber).
 * Jason Eastburn
 * Joe McCormick (@iamjoemccormick).

deploy/k8s/README.md

@@ -1,13 +1,13 @@
 # Kustomize Specific Deployment Details
 
-## Contents
 <a name="contents"></a>
+## Contents
 
 * [Basics](#basics)
 * [Upgrading to v1.2.0](#upgrade-1.2.0-kubernetes-deployment)
 
-## Basics
 <a name="basics"></a>
+## Basics
 
 The BeeGFS CSI driver uses [Kustomize](https://kustomize.io/) as its default 
 deployment mechanism. While Kustomize CAN be downloaded, installed, and run as 
@@ -45,8 +45,8 @@ by the development team to the base manifests or version patches will be picked
 up when you pull a new version of the project and your custom modifications will 
 continue to work unless otherwise noted.
 
-### Upgrading to v1.2.0
 <a name="upgrade-1.2.0-kubernetes-deployment"></a>
+### Upgrading to v1.2.0
 
 v1.2.0 includes changes to the structure of the deployment manifests. To upgrade
 from v1.1.0, follow these steps:

docs/deployment.md

@@ -1,7 +1,7 @@
 # BeeGFS CSI Driver Deployment
 
-## Contents
 <a name="contents"></a>
+## Contents
 
 * [Deploying to Kubernetes](#deploying-to-kubernetes)
  * [Kubernetes Node Preparation](#kubernetes-node-preparation)
@@ -18,11 +18,12 @@
  * [Security and Networking Considerations](#security-considerations)
 * [Removing the Driver from Kubernetes](#removing-the-driver-from-kubernetes)
 
-## Deploying to Kubernetes
 <a name="deploying-to-kubernetes"></a>
+## Deploying to Kubernetes
 
-### Kubernetes Node Preparation
 <a name="kubernetes-node-preparation"></a>
+### Kubernetes Node Preparation
+
 The following MUST be completed on any Kubernetes node (master OR worker) that
 runs a component of the driver:
 * Download the [BeeGFS repository
@@ -40,8 +41,9 @@ IMPORTANT: By default the driver uses the beegfs-client.conf file at
 */etc/beegfs/beegfs-client.conf* for base configuration. Modifying the location
 of this file is not currently supported without changing kustomization files.
 
-### Kubernetes Deployment
 <a name="kubernetes-deployment"></a>
+### Kubernetes Deployment
+
 Deployment manifests are provided in this repository under *deploy/k8s/* along with
 a default BeeGFS Client ConfigMap. The driver is deployed using `kubectl apply
 -k` (kustomize). For more detailed information on how the manifests are organized or 
@@ -113,8 +115,9 @@ Next Steps:
 * For a comprehensive introduction see the [BeeGFS CSI Driver Usage](usage.md)
  documentation.
 
-### Air-Gapped Kubernetes Deployment
 <a name="air-gapped-kubernetes-deployment"></a>
+### Air-Gapped Kubernetes Deployment
+
 This section provides guidance on deploying the BeeGFS CSI driver in
 environments where Kubernetes nodes do not have internet access. 
 
@@ -136,8 +139,9 @@ to them. Adjust the `images[].newTag` fields as necessary to ensure they either
 match images that exist on the Kubernetes nodes or reference your internal 
 registry. Then follow the above commands for Kubernetes deployment.
 
-### Deployment to Kubernetes Clusters With Mixed Nodes
 <a name="mixed-kubernetes-deployment"></a>
+### Deployment to Kubernetes Clusters With Mixed Nodes
+
 In some Kubernetes clusters, not all nodes are capable of running the BeeGFS 
 CSI driver (or it may not be desirable for all nodes to do so). For example:
 * A cluster may be shared by multiple departments and one department may not 
@@ -176,8 +180,8 @@ StatefulSets, Deployments) that depend on BeeGFS MUST be deployed with the same
 nodeAffinity assigned to the driver node service. Provide your users with the 
 labels or nodes they must run their workloads on.
 
-### Deployment to Kubernetes Using the Operator
 <a name="operator-deployment"></a>
+### Deployment to Kubernetes Using the Operator
 
 An [operator](https://operatorframework.io/what/) can be used to deploy the 
 BeeGFS CSI driver to a cluster and manage its configuration/state within that 
@@ -192,8 +196,9 @@ operator to install the driver directly from this repository.
 See the [BeeGFS CSI Driver Operator](../operator/README.md) documentation for 
 details.
 
-## Example Application Deployment
 <a name="example-application-deployment"></a>
+## Example Application Deployment
+
 Verify that a BeeGFS file system is accessible from the Kubernetes nodes.
 Minimally, verify that the BeeGFS servers are up and listening from a
 workstation that can access them:
@@ -349,17 +354,17 @@ Next Steps:
  documentation.
 * For additional examples see *examples/k8s/README.md*. 
 
-## Managing BeeGFS Client Configuration
 <a name="managing-beegfs-client-configuration"></a>
+## Managing BeeGFS Client Configuration
 
 Currently the only tested and supported container orchestrator (CO) for the
 BeeGFS CSI driver is Kubernetes. Notes in the General Configuration section
 below would apply to other COs if supported. For Kubernetes the preferred method
 to apply desired BeeGFS Client configuration is using a Kubernetes ConfigMap and
 Secret, as described in [Kubernetes Configuration](#kubernetes-configuration).
 
-### General Configuration
 <a name="general-configuration"></a>
+### General Configuration
 
 The driver is ready to be used right out of the box, but many environments may
 either require or benefit from additional configuration.
@@ -462,8 +467,9 @@ nodeSpecificConfigs: # OPTIONAL
  fileSystemSpecificConfigs: # as above
 ```
 
-#### ConnAuth Configuration
 <a name="connauth-configuration"></a>
+#### ConnAuth Configuration
+
 For security purposes, the contents of BeeGFS connAuthFiles are stored in a
 separate file. This file is optional, and should only be used if the
 connAuthFile configuration option is used on a file system's other services.
@@ -483,8 +489,9 @@ NOTE: Unlike general configuration, connAuth configuration is only applied at a
 per file system level. There is no default connAuth and the concept of a node 
 specific connAuth doesn't make sense.
 
-### Kubernetes Configuration
 <a name="kubernetes-configuration"></a>
+### Kubernetes Configuration
+
 When deployed into Kubernetes, a single Kubernetes ConfigMap contains the
 configuration for all Kubernetes nodes. When the driver starts up on a node, it
 uses the node's name to filter the global ConfigMap down to a node-specific
@@ -516,16 +523,18 @@ all nodes so that it picks up the latest changes.
 NOTE: To validate the BeeGFS Client configuration file used for a specific PVC, 
 see the [Troubleshooting Guide](troubleshooting.md#k8s-determining-the-beegfs-client-conf-for-a-pvc)
 
-### BeeGFS Client Parameters (beegfsClientConf)
 <a name="beegfs-client-parameters"></a>
+### BeeGFS Client Parameters (beegfsClientConf)
+
 The following beegfs-client.conf parameters appear in the BeeGFS v7.2
 [beegfs-client.conf
 file](https://git.beegfs.io/pub/v7/-/blob/7.2/client_module/build/dist/etc/beegfs-client.conf).
 Other parameters may exist for newer or older BeeGFS versions. The list a
 parameter falls under determines its level of support in the driver.
 
-#### No Effect
 <a name="no-effect"></a>
+#### No Effect
+
 These parameters are specified elsewhere (a Kubernetes StorageClass, etc.) or
 are determined dynamically and have no effect when specified in the
 `beeGFSClientConf` configuration section.
@@ -542,8 +551,9 @@ are determined dynamically and have no effect when specified in the
  nodes.)
 * `connPortShift`
 
-#### Unsupported
 <a name="unsupported"></a>
+#### Unsupported
+
 These parameters are specified elsewhere and may exhibit undocumented behavior
 if specified here.
 
@@ -555,14 +565,16 @@ if specified here.
 * `connTcpOnlyFilterFile` - Overridden by lists in the driver configuration 
  file.
 
-### Tested
 <a name="tested"></a>
+### Tested
+
 These parameters have been tested and verified to have the desired effect.
 
 * `quotaEnabled` - Documented in [Quotas](quotas.md).
 
-#### Untested
 <a name="untested"></a>
+#### Untested
+
 These parameters SHOULD result in the desired effect but have not been tested.
 
 * `connHelperdPortTCP`
@@ -596,11 +608,11 @@ These parameters SHOULD result in the desired effect but have not been tested.
 * `tuneUseGlobalFileLocks`
 * `sysACLsEnabled`
 
-## Notes for Kubernetes Administrators
 <a name="kubernetes-administrator-notes"></a>
+## Notes for Kubernetes Administrators
 
-### Security and Networking Considerations
 <a name="security-considerations"></a>
+### Security and Networking Considerations
 
 **The driver must be allowed to mount and unmount file systems.**
 
@@ -634,8 +646,9 @@ Inbound UDP traffic from nodes serving up BeeGFS file systems to arbitrary
 ports on all BeeGFS clients must be allowed. Each volume requires its own port
 and it is not currently possible to configure an allowed port range.
 
-## Removing the Driver from Kubernetes
 <a name="removing-the-driver-from-kubernetes"></a>
+## Removing the Driver from Kubernetes
+
 If you're experiencing any issues, find functionality lacking, or our
 documentation is unclear, we'd appreciate if you let us know:
 https://github.com/NetApp/beegfs-csi-driver/issues.