diff --git a/.github/workflows/test-site-to-site.yml b/.github/workflows/test-site-to-site.yml index 31f082bf..dc0336f3 100644 --- a/.github/workflows/test-site-to-site.yml +++ b/.github/workflows/test-site-to-site.yml @@ -87,12 +87,19 @@ jobs: - name: Confirm certificate rotation run: | OLDCERTSTART=$(kubectl -n alpha exec nifi-0 -c server -- curl -kv https://localhost:8443 2>&1 | /bin/grep start) - echo Certificate "$OLDCERTSTART" + echo Old Certificate "$OLDCERTSTART" cmctl -n alpha renew nifi-0 + # Remove below block of code when issue NIFI-11536 is fixed + # i.e. in version 1.23 https://issues.apache.org/jira/browse/NIFI-11536 + ######################################################################### + kubectl -n alpha rollout restart statefulset/nifi + kubectl -n alpha rollout status --watch statefulset/nifi --timeout=5m + sleep 5m + ######################################################################### for n in [ 0 1 2 3 4 5 6 7 8 9 ] do NEWCERTSTART=$(kubectl -n alpha exec nifi-0 -c server -- curl -kv https://localhost:8443 2>&1 | /bin/grep start) - echo Certificate "$NEWCERTSTART" + echo New Certificate "$NEWCERTSTART" if [ \""$OLDCERTSTART"\" != \""$NEWCERTSTART"\" ] then exit 0 diff --git a/Chart.yaml b/Chart.yaml index 22cf8abc..ffea66f0 100644 --- a/Chart.yaml +++ b/Chart.yaml @@ -1,8 +1,8 @@ --- apiVersion: v2 name: nifi -version: 1.1.4 -appVersion: 1.16.3 +version: 1.1.5 +appVersion: 1.20.0 description: Apache NiFi is a software project from the Apache Software Foundation designed to automate the flow of data between software systems. keywords: - nifi diff --git a/README.md b/README.md index d5c51035..62c2dc0a 100644 --- a/README.md +++ b/README.md @@ -35,8 +35,9 @@ The following items can be set via `--set` flag during installation or configure #### Configure how to persist data -- **Disable**: The data does not survive the termination of a pod. -- **Persistent Volume Claim(default)**: A default `StorageClass` is needed in the Kubernetes cluster to dynamically provision the volumes. Specify another StorageClass in the `storageClass` or set `existingClaim` if you have already existing persistent volumes to use. +- **Disable(default)**: The data does not survive the termination of a pod. +- **Persistent Volume Claim**: Enable persistence so that data survives termination of the pod. There is the choice of using one large persistent volume (using subPath) or seven separate persistent volumes for config, data, logs, repos, etc. + A default `StorageClass` is needed in the Kubernetes cluster to dynamically provision the volumes. Specify another StorageClass in the `persistence.storageClass` setting. #### Configure authentication @@ -95,7 +96,7 @@ The following table lists the configurable parameters of the nifi chart and the | `replicaCount` | Number of nifi nodes | `1` | | **Image** | | `image.repository` | nifi Image name | `apache/nifi` | -| `image.tag` | nifi Image tag | `1.16.3` | +| `image.tag` | nifi Image tag | `1.20.0` | | `image.pullPolicy` | nifi Image pull policy | `IfNotPresent` | | `image.pullSecret` | nifi Image pull secret | `nil` | | **SecurityContext** | @@ -155,6 +156,7 @@ The following table lists the configurable parameters of the nifi chart and the | `auth.oidc.clientId` | oidc clientId | `nil` | | `auth.oidc.clientSecret` | oidc clientSecret | `nil` | | `auth.oidc.claimIdentifyingUser` | oidc claimIdentifyingUser | `email` | +| `auth.oidc.preferredJwsAlgorithm` | The preferred algorithm for validating identity tokens. If this value is blank, it will default to RS256 which is required to be supported by the OpenID Connect Provider according to the specification. If this value is HS256, HS384, or HS512, NiFi will attempt to validate HMAC protected tokens using the specified client secret. If this value is none, NiFi will attempt to validate unsecured/plain tokens. | `nil` | | `auth.oidc.admin` | Default OIDC admin identity | `nifi@example.com` | | Note that OIDC authentication to a multi-NiFi-node cluster requires Ingress sticky sessions | See [background](https://community.cloudera.com/t5/Support-Questions/OIDC-With-Azure-AD/m-p/232324#M194163) | Also [how](https://kubernetes.github.io/ingress-nginx/examples/affinity/cookie/) | | **postStart** | @@ -184,6 +186,9 @@ The following table lists the configurable parameters of the nifi chart and the | `persistence.enabled` | Use persistent volume to store data | `false` | | `persistence.storageClass` | Storage class name of PVCs (use the default type if unset) | `nil` | | `persistence.accessMode` | ReadWriteOnce or ReadOnly | `[ReadWriteOnce]` | +| `persistence.subPath.enabled` | Use only one persistent volume with subPath instead of seven separate persistent volumes | `false` | +| `persistence.subPath.name` | Name of the one persistent volume claim when using subPath | `data` | +| `persistence.subPath.size` | Size of the one persistent volume claim when using subPath | `36Gi` | | `persistence.configStorage.size` | Size of persistent volume claim | `100Mi` | | `persistence.authconfStorage.size` | Size of persistent volume claim | `100Mi` | | `persistence.dataStorage.size` | Size of persistent volume claim | `1Gi` | @@ -191,7 +196,6 @@ The following table lists the configurable parameters of the nifi chart and the | `persistence.contentRepoStorage.size` | Size of persistent volume claim | `10Gi` | | `persistence.provenanceRepoStorage.size` | Size of persistent volume claim | `10Gi` | | `persistence.logStorage.size` | Size of persistent volume claim | `5Gi` | -| `persistence.existingClaim` | Use an existing PVC to persist data | `nil` | | **jvmMemory** | | `jvmMemory` | bootstrap jvm size | `2g` | | **SideCar** | @@ -219,6 +223,8 @@ The following table lists the configurable parameters of the nifi chart and the | **env** | | `env` | Additional environment variables for the nifi-container (see [spec](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/#envvar-v1-core) for details) | `[]` | | `envFrom` | Additional environment variables for the nifi-container from config-maps or secrets (see [spec](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/#envfromsource-v1-core) for details) | `[]` | +| **extraOptions** | +| `extraOptions` | Additional bootstrap.conf properties (see [properties](https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#bootstrap_properties) for details) | `[]` | | **extraContainers** | | `extraContainers` | Additional container-specifications that should run within the pod (see [spec](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/#container-v1-core) for details) | `[]` | | **extraLabels** | diff --git a/configs/bootstrap.conf b/configs/bootstrap.conf index e6a2444d..a9183e67 100644 --- a/configs/bootstrap.conf +++ b/configs/bootstrap.conf @@ -81,3 +81,8 @@ notification.max.attempts=5 #nifi.dead.notification.services=email-notification +# Extra bootstrap options +{{- range .Values.extraOptions }} +{{ .name }}={{ .value }} +{{- end }} + diff --git a/configs/nifi.properties b/configs/nifi.properties index d4cfd899..d98a5c16 100644 --- a/configs/nifi.properties +++ b/configs/nifi.properties @@ -197,7 +197,7 @@ nifi.security.user.oidc.connect.timeout=5 secs nifi.security.user.oidc.read.timeout=5 secs nifi.security.user.oidc.client.id={{.Values.auth.oidc.clientId}} nifi.security.user.oidc.client.secret={{.Values.auth.oidc.clientSecret}} -nifi.security.user.oidc.preferred.jwsalgorithm= +nifi.security.user.oidc.preferred.jwsalgorithm={{.Values.auth.oidc.preferredJwsAlgorithm}} nifi.security.user.oidc.claim.identifying.user={{.Values.auth.oidc.claimIdentifyingUser}} nifi.security.user.oidc.additional.scopes={{.Values.auth.oidc.additionalScopes}} {{end}} @@ -238,6 +238,7 @@ nifi.cluster.flow.election.max.wait.time=1 mins nifi.cluster.flow.election.max.candidates= # zookeeper properties, used for cluster management # +nifi.zookeeper.client.ensembleTracker={{.Values.properties.zkClientEnsembleTraker}} nifi.zookeeper.connect.string= nifi.zookeeper.connect.timeout=3 secs nifi.zookeeper.session.timeout=3 secs diff --git a/doc/FAQ.md b/doc/FAQ.md index d3c54fd0..f36a05e2 100644 --- a/doc/FAQ.md +++ b/doc/FAQ.md @@ -1,13 +1,17 @@ FAQ - Frequently Asked Questions -====== +================================ Readyness probe fails ---------------- +--------------------- -When encountering errors like `Readiness probe failed: Node not found with CONNECTED state` or `Multi-Attach error for volume "pvc-xxxxxx-xxx-xxx-xxxx-xxxxxxxxx" Volume is already exclusively attached to one node and can't be attached to another`, it means Kubernetes can't provide the pod access to the persistent data it wants. +When encountering errors like `Readiness probe failed: Node not found with CONNECTED state` or `Multi-Attach error for volume "pvc-xxxxxx-xxx-xxx-xxxx-xxxxxxxxx" Volume is already exclusively attached to one node and can't be attached to another`, it means Kubernetes can't provide the pod access to the persistent data it wants. -When this happens, reach out to your Kubernetes cluster administrators to find and fix the problem manually. +When this happens, reach out to your Kubernetes cluster administrators to find and fix the problem manually. For more background, see https://blog.mayadata.io/recover-from-volume-multi-attach-error-in-on-prem-kubernetes-clusters -(see https://github.com/cetic/helm-nifi/issues/47#issuecomment-1122702262) \ No newline at end of file +(see https://github.com/cetic/helm-nifi/issues/47#issuecomment-1122702262) + +## Session Afffinity + +As mentioned in the official NIFI document regarding [session affinity](https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#session_affinity), it's required to implement this feature for your ingress. Please refer to the ingress controller your are using for how to implement it. One example for GKE is with [issue #271](https://github.com/cetic/helm-nifi/issues/271). If NIFI cluster has more than one node, the session affinity has to be there due to the stateful implementation of each node. diff --git a/doc/INSTALLATION.md b/doc/INSTALLATION.md index 821ba2d4..2b03284e 100644 --- a/doc/INSTALLATION.md +++ b/doc/INSTALLATION.md @@ -16,7 +16,7 @@ helm dep up ``` 2. **Set a sensitiveKey** -In 1.16.3 version, Nifi needs a sensitiveKey to encrypt sensitive information. This key can be setted in the `values.yaml` file: +In 1.20.0 version, Nifi needs a sensitiveKey to encrypt sensitive information. This key can be setted in the `values.yaml` file: ```` properties: diff --git a/templates/statefulset.yaml b/templates/statefulset.yaml index 7a732024..f0db1cd1 100644 --- a/templates/statefulset.yaml +++ b/templates/statefulset.yaml @@ -191,6 +191,7 @@ spec: fi {{ end }} + prop_replace nifi.ui.banner.text $(hostname -s) prop_replace nifi.remote.input.host ${FQDN} prop_replace nifi.cluster.node.address ${FQDN} prop_replace nifi.zookeeper.connect.string ${NIFI_ZOOKEEPER_CONNECT_STRING} @@ -453,20 +454,55 @@ spec: tcpSocket: port: {{ .Values.properties.httpsPort }} volumeMounts: - - name: "logs" - mountPath: /opt/nifi/nifi-current/logs - - name: "data" - mountPath: /opt/nifi/data - - name: "auth-conf" - mountPath: /opt/nifi/nifi-current/auth-conf/ - - name: "config-data" - mountPath: /opt/nifi/nifi-current/config-data - - name: "flowfile-repository" - mountPath: /opt/nifi/flowfile_repository - - name: "content-repository" - mountPath: /opt/nifi/content_repository - - name: "provenance-repository" - mountPath: /opt/nifi/provenance_repository + - mountPath: /opt/nifi/nifi-current/logs + {{- if and .Values.persistence.enabled .Values.persistence.subPath.enabled }} + name: {{ .Values.persistence.subPath.name }} + subPath: logs + {{- else }} + name: "logs" + {{- end }} + - mountPath: /opt/nifi/data + {{- if and .Values.persistence.enabled .Values.persistence.subPath.enabled }} + name: {{ .Values.persistence.subPath.name }} + subPath: data + {{- else }} + name: "data" + {{- end }} + - mountPath: /opt/nifi/nifi-current/auth-conf/ + {{- if and .Values.persistence.enabled .Values.persistence.subPath.enabled }} + name: {{ .Values.persistence.subPath.name }} + subPath: auth-conf + {{- else }} + name: "auth-conf" + {{- end }} + - mountPath: /opt/nifi/nifi-current/config-data + {{- if and .Values.persistence.enabled .Values.persistence.subPath.enabled }} + name: {{ .Values.persistence.subPath.name }} + subPath: config-data + {{- else }} + name: "config-data" + {{- end }} + - mountPath: /opt/nifi/flowfile_repository + {{- if and .Values.persistence.enabled .Values.persistence.subPath.enabled }} + name: {{ .Values.persistence.subPath.name }} + subPath: flowfile-repository + {{- else }} + name: "flowfile-repository" + {{- end }} + - mountPath: /opt/nifi/content_repository + {{- if and .Values.persistence.enabled .Values.persistence.subPath.enabled }} + name: {{ .Values.persistence.subPath.name }} + subPath: content-repository + {{- else }} + name: "content-repository" + {{- end }} + - mountPath: /opt/nifi/provenance_repository + {{- if and .Values.persistence.enabled .Values.persistence.subPath.enabled }} + name: {{ .Values.persistence.subPath.name }} + subPath: provenance-repository + {{- else }} + name: "provenance-repository" + {{- end }} - name: "bootstrap-conf" mountPath: /opt/nifi/nifi-current/conf/bootstrap.conf subPath: "bootstrap.conf" @@ -541,8 +577,13 @@ spec: resources: {{ toYaml .Values.logresources | indent 10 }} volumeMounts: - - name: logs - mountPath: /var/log + - mountPath: /var/log + {{- if and .Values.persistence.enabled .Values.persistence.subPath.enabled }} + name: {{ .Values.persistence.subPath.name }} + subPath: logs + {{- else }} + name: "logs" + {{- end }} - name: bootstrap-log imagePullPolicy: {{ .Values.sidecar.imagePullPolicy | default "Always" | quote }} image: "{{ .Values.sidecar.image }}:{{ .Values.sidecar.tag }}" @@ -553,8 +594,13 @@ spec: resources: {{ toYaml .Values.logresources | indent 10 }} volumeMounts: - - name: logs - mountPath: /var/log + - mountPath: /var/log + {{- if and .Values.persistence.enabled .Values.persistence.subPath.enabled }} + name: {{ .Values.persistence.subPath.name }} + subPath: logs + {{- else }} + name: "logs" + {{- end }} - name: user-log imagePullPolicy: {{ .Values.sidecar.imagePullPolicy | default "Always" | quote }} image: "{{ .Values.sidecar.image }}:{{ .Values.sidecar.tag }}" @@ -565,8 +611,13 @@ spec: resources: {{ toYaml .Values.logresources | indent 10 }} volumeMounts: - - name: logs - mountPath: /var/log + - mountPath: /var/log + {{- if and .Values.persistence.enabled .Values.persistence.subPath.enabled }} + name: {{ .Values.persistence.subPath.name }} + subPath: logs + {{- else }} + name: "logs" + {{- end }} {{- if .Values.certManager.enabled }} - name: cert-manager imagePullPolicy: {{ .Values.image.pullPolicy | quote }} @@ -786,7 +837,21 @@ spec: {{- if .Values.extraVolumes }} {{ toYaml .Values.extraVolumes | indent 6 }} {{- end }} -{{- if .Values.persistence.enabled }} +{{- if and .Values.persistence.enabled .Values.persistence.subPath.enabled }} + volumeClaimTemplates: + - metadata: + name: {{ .Values.persistence.subPath.name }} + spec: + accessModes: + {{- range .Values.persistence.accessModes }} + - {{ . | quote }} + {{- end }} + storageClassName: {{ .Values.persistence.storageClass | quote }} + resources: + requests: + storage: {{ .Values.persistence.subPath.size }} +{{- end }} +{{- if and .Values.persistence.enabled (not .Values.persistence.subPath.enabled) }} volumeClaimTemplates: - metadata: name: logs diff --git a/tests/05-install-cert-manager.bash b/tests/05-install-cert-manager.bash index d30add6f..d8e38d80 100755 --- a/tests/05-install-cert-manager.bash +++ b/tests/05-install-cert-manager.bash @@ -11,7 +11,7 @@ ARCH=$(go env GOARCH) /bin/rm -rf /tmp/cmctl-install mkdir -p /tmp/cmctl-install -curl -L -o /tmp/cmctl-install/cmctl.tar.gz https://github.com/jetstack/cert-manager/releases/latest/download/cmctl-$OS-$ARCH.tar.gz +curl -L -o /tmp/cmctl-install/cmctl.tar.gz https://github.com/cert-manager/cert-manager/releases/latest/download/cmctl-$OS-$ARCH.tar.gz (cd /tmp/cmctl-install ; tar xvzf cmctl.tar.gz ; sudo mv cmctl /usr/local/bin) cmctl experimental install diff --git a/tests/06-alpha.flow.xml b/tests/06-alpha.flow.xml index f8b781f8..2f8c677b 100644 --- a/tests/06-alpha.flow.xml +++ b/tests/06-alpha.flow.xml @@ -24,7 +24,7 @@ org.apache.nifi nifi-standard-nar - 1.16.3 + 1.20.0 1 1 sec @@ -117,7 +117,7 @@ org.apache.nifi nifi-ssl-context-service-nar - 1.16.3 + 1.20.0 true @@ -162,7 +162,7 @@ org.apache.nifi nifi-site-to-site-reporting-nar - 1.16.3 + 1.20.0 5 sec RUNNING diff --git a/tests/06-bravo.flow.xml b/tests/06-bravo.flow.xml index 5e5a3ec3..832aa2d6 100644 --- a/tests/06-bravo.flow.xml +++ b/tests/06-bravo.flow.xml @@ -24,7 +24,7 @@ org.apache.nifi nifi-update-attribute-nar - 1.15.2 + 1.20.0 1 0 sec @@ -62,7 +62,7 @@ org.apache.nifi nifi-update-attribute-nar - 1.15.2 + 1.20.0 1 0 sec @@ -159,7 +159,7 @@ org.apache.nifi nifi-prometheus-nar - 1.15.2 + 1.20.0 5 sec RUNNING diff --git a/values.yaml b/values.yaml index 158bdfed..fed0e673 100644 --- a/values.yaml +++ b/values.yaml @@ -7,7 +7,7 @@ replicaCount: 1 ## image: repository: apache/nifi - tag: "1.16.3" + tag: "1.20.0" pullPolicy: "IfNotPresent" ## Optionally specify an imagePullSecret. @@ -94,6 +94,7 @@ properties: httpsPort: 8443 webProxyHost: # : (If Nifi service is NodePort or LoadBalancer) clusterPort: 6007 + zkClientEnsembleTraker: false # https://issues.apache.org/jira/browse/NIFI-10481 provenanceStorage: "8 GB" provenanceMaxStorageTime: "10 days" siteToSite: @@ -149,6 +150,7 @@ auth: clientSecret: # claimIdentifyingUser: email admin: nifi@example.com + preferredJwsAlgorithm: ## Request additional scopes, for example profile additionalScopes: @@ -253,7 +255,16 @@ persistence: # The default storage class is used if this variable is not set. accessModes: [ReadWriteOnce] - ## Storage Capacities for persistent volumes + + ## Use subPath and have 1 persistent volume instead of 7 volumes - use when your k8s nodes have limited volume slots, to limit waste of space, + ## or your available volume sizes are quite large + # The one disk will have a directory folder for each volumeMount, but this is hidden. Run 'mount' to view each mount. + subPath: + enabled: false + name: data + size: 30Gi + + ## Storage Capacities for persistent volumes (these are ignored if using one volume with subPath) configStorage: size: 100Mi authconfStorage: @@ -329,6 +340,9 @@ env: [] ## Extra environment variables from secrets and config maps envFrom: [] +## Extra options to add to the bootstrap.conf file +extraOptions: [] + # envFrom: # - configMapRef: # name: config-name