Skip to content
This repository has been archived by the owner on Jan 19, 2024. It is now read-only.

Documentation and deployment scenario for cluster internal ssl #99

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@

apiVersion: v1
kind: ConfigMap
metadata:
name: cluster-internal-ssl-config
data:
NEO4J_unsupported_dbms_ssl_system_ignore__dot__files: "true"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does k8s do something funky with the volume mappings? Or why is this needed? I don't see any dotfiles in what you're mounting

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, yes it does, K8s creates the directories for the configmaps/secrets in its own locations and symlinks to them. It stores metadata in dot files which gets included.

The main reason for the wizardry is that they can update the symlink for a whole directory atomically at runtime if the ConfigMap or Secret is changed - obviously that's not relevant to Neo4j but there's no way to turn it off in K8s.


# Cluster ssl policy
NEO4J_dbms_ssl_policy_cluster_enabled: "true"
NEO4J_dbms_ssl_policy_cluster_client__auth: "REQUIRE"
NEO4J_dbms_ssl_policy_cluster_verify__hostname: "true"

# Fabric ssl policy
NEO4J_dbms_ssl_policy_fabric_enabled: "true"
NEO4J_dbms_ssl_policy_fabric_client__auth: "REQUIRE"
NEO4J_dbms_ssl_policy_fabric_verify__hostname: "true"

# Backup ssl policy
NEO4J_dbms_ssl_policy_backup_enabled: "true"
NEO4J_dbms_ssl_policy_backup_client__auth: "REQUIRE"
NEO4J_dbms_ssl_policy_backup_verify__hostname: "false"
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@

apiVersion: v1
kind: Secret
metadata:
name: cluster-internal-ssl-cert-and-key
type: Opaque
stringData:
private.key: |-
-----BEGIN RSA PRIVATE KEY-----
MIIEpAIBAAKCAQEAmE2+K+TO4WwFxeVdc72VOBTLHDt/anuHdgtgZpiXqhsvHlgj
uiWfA7dYcQh4aRI8w0+aup9GfSsPfnpYWuXriVslRcV9NmpuUt+fHYyA4gZT6Unu
dXZnr5W2knE2nAFer9u0X7AtVrwtEn6tdhjgfg4mKWbZyL9MKNMxxBuhU3GaetBH
/YlrbovVEAUZededl7+/ePs8mHSOA/VnAXAoGqQ8kiQtQ8Z+3wwyZs2Gb6VFMcuf
htMTxZJUFePa/YEaoSD5pUJJKVX45oAIkIGHtHoocwDV3ont/YsZB6Tj4G3N8bRu
RD4bpg0By5rvurMPwowUt9lkHh/6PbZkyG6KIwIDAQABAoIBAQCHyskD2b2avvVm
vFnWF/IzTlbJlULFbd4ZIYuR7ftLb3FTXMJ99Y0RgycXoLW6+Me0XAVY3ym57+qg
mfStFtIqZVmWG77IBZzXxwnXDq7a10l5drFliWxo4NMnPkmyToZdxUXNCwdhjeWh
19BQu11tBrB/uXPzyJveym5Uq03rVr6IqWCgetWHgvjC/M0y5avWGSjP2qzAc+Dk
xZjQqWAlUIvTtp5egTXtk5SkhxTdgh64az2gkJqwHBQBBE3dyro0aCUGTD2RS9df
6LeWhQmr0UoaCCl837lzh7FcOk/KhEvlRHwWgu1x6brxOZFdykt/Bt2r3IlLHvBB
R51JcHgBAoGBAMZat/dMm1L+lyUNe+mSTnR9nY95lbTSuMJrfGspNNhtO1zVKMQV
ZFGOtXOn2B+1f8NJcsOl0b7SbjQymJxGBjqC4on+31pgqwBqPr27D5IqYmDrF9Dd
5sj7syoypGHx6JkHQWE00EdoXrKedPcfUJbx035YAAqoE5AtwMle3FZxAoGBAMSQ
60Vuov5ZsoNmrrBlfYedDkjF5obP9xfBPdTFt0/ktuOBP1PHVybwrsd5BNoW2zgw
FZud7bmbstPGBsN5Z7QZm2pmPZ+VUIFoEPYodOcGeEd7NmSYSk1MtClG7l6UtMaJ
7AkEN7IU7BR0sv+Dtdk66PhKSLaon6Q+QpTje3vTAoGBALvEbvfcfgC/3qaFsDI4
fKpLq1aBW1V0UNBC3eG3fT4PkS1c351XPsLx3BUi4zWJI+vi4JASrY39N7OT3eG5
a/YBpp/JNPgiIF5hNQl4RdIw6zYh9kaTeP/zPPSKQhAx5uTN+HcjfrLKOzLNS54P
98McIwAsH8X2u6Y1mZVGhkARAoGAMYAtP3b1JQiBpAWfyFxGmHg8uKbdvuVwXFMV
txdzanM2e2R5BigVEoFaAnG/fwxyeFvjlSTYUP2csygTW/ae3wPz13+X1TBM7cm/
O75Eckl20Ml+kSaoz36ZgCuUq8zXGYhyIHMnc3lBWoVo7l/E08e6E4zhct5UFZB4
Q/ZlinECgYBaCBhUCMSmoqnJ8s3EpHCsYZUjFhEyQmNraG5kFVTcANDg/NNkdqQp
HMHMBsvFWlgiEAcngM8LBC8yUtX1a/FL8YbmxZ9bqVlln0E7Z/+OawcR4LdyhMuh
WL+fj/Kny16wyIWck+QR7cpXtXS8vFP+jI2mcKxOoB+hTVJexbWhlg==
-----END RSA PRIVATE KEY-----
public.crt: |-
-----BEGIN CERTIFICATE-----
MIIEEzCCAvugAwIBAgIJAMxkASGTREE2MA0GCSqGSIb3DQEBCwUAMHsxCzAJBgNV
BAYTAkdCMQswCQYDVQQIDAJHQjEPMA0GA1UEBwwGTG9uZG9uMSEwHwYDVQQKDBhN
eSBDZXJ0aWZpY2F0ZSBBdXRob3JpdHkxEDAOBgNVBAsMB0RpZ2l0YWwxGTAXBgNV
BAMMEG15X2NhX19yb290X2NlcnQwHhcNMjAwODIwMDkyNTA2WhcNMjUwODE5MDky
NTA2WjCBsTEiMCAGA1UEAwwZZGVmYXVsdC5zdmMuY2x1c3Rlci5sb2NhbDEuMCwG
CSqGSIb3DQEJARYfYWRtaW5AZGVmYXVsdC5zdmMuY2x1c3Rlci5sb2NhbDEYMBYG
A1UECgwPRXhhbXBsZSBDb21wYW55MRUwEwYDVQQLDAxFeGFtcGxlIFVuaXQxDTAL
BgNVBAcMBENpdHkxDjAMBgNVBAgMBVN0YXRlMQswCQYDVQQGEwJVUzCCASIwDQYJ
KoZIhvcNAQEBBQADggEPADCCAQoCggEBAJhNvivkzuFsBcXlXXO9lTgUyxw7f2p7
h3YLYGaYl6obLx5YI7olnwO3WHEIeGkSPMNPmrqfRn0rD356WFrl64lbJUXFfTZq
blLfnx2MgOIGU+lJ7nV2Z6+VtpJxNpwBXq/btF+wLVa8LRJ+rXYY4H4OJilm2ci/
TCjTMcQboVNxmnrQR/2Ja26L1RAFGXnXnZe/v3j7PJh0jgP1ZwFwKBqkPJIkLUPG
ft8MMmbNhm+lRTHLn4bTE8WSVBXj2v2BGqEg+aVCSSlV+OaACJCBh7R6KHMA1d6J
7f2LGQek4+BtzfG0bkQ+G6YNAcua77qzD8KMFLfZZB4f+j22ZMhuiiMCAwEAAaNj
MGEwHwYDVR0jBBgwFoAU/6oW29HV8UxJMX7HfipHrhbfDHIwCQYDVR0TBAIwADAL
BgNVHQ8EBAMCBPAwJgYDVR0RBB8wHYIbKi5kZWZhdWx0LnN2Yy5jbHVzdGVyLmxv
Y2FsMA0GCSqGSIb3DQEBCwUAA4IBAQAardQ4u2ezwxWeUHrK7hDI7kjWVB8vvaqC
W3Gi9scdMzZExnMk/adk07aYjF6jso8m7+MH0MFIO7d2q4r1gPIOR9ToEkcwg/VO
8qMCtMm7TBpM7uyR9GoirZ4QPsvh3f2qDd0BH/i3/aHJFguo4L3SOHsiNB23GsQ/
Rqe5DqDbCr3osHoT8E4cDXUxdQO0rbAMsr79ME7oaJBFh0+reH1UI8LK7FWqm5pi
atbTCMXzH650Zc4yNh+m0/lHmii8kKZXuNWZ0su1xA6jfVRViqB2yvLvB78NRUKH
iq/a6qatJorvd6akaxAMupp18BUtLeyshEXkv2EoN3MFgBfl/jif
-----END CERTIFICATE-----

---

apiVersion: v1
kind: ConfigMap
metadata:
name: cluster-internal-ssl-trusted-certs
data:
#Add trusted public certificates as new keys to this configmap
root.crt: |-
-----BEGIN CERTIFICATE-----
MIIDyTCCArGgAwIBAgIJANRHW99Q6S6EMA0GCSqGSIb3DQEBCwUAMHsxCzAJBgNV
BAYTAkdCMQswCQYDVQQIDAJHQjEPMA0GA1UEBwwGTG9uZG9uMSEwHwYDVQQKDBhN
eSBDZXJ0aWZpY2F0ZSBBdXRob3JpdHkxEDAOBgNVBAsMB0RpZ2l0YWwxGTAXBgNV
BAMMEG15X2NhX19yb290X2NlcnQwHhcNMjAwODIwMDkyNTA1WhcNMzAwODE4MDky
NTA1WjB7MQswCQYDVQQGEwJHQjELMAkGA1UECAwCR0IxDzANBgNVBAcMBkxvbmRv
bjEhMB8GA1UECgwYTXkgQ2VydGlmaWNhdGUgQXV0aG9yaXR5MRAwDgYDVQQLDAdE
aWdpdGFsMRkwFwYDVQQDDBBteV9jYV9fcm9vdF9jZXJ0MIIBIjANBgkqhkiG9w0B
AQEFAAOCAQ8AMIIBCgKCAQEA3hcl3lVYOoAb6B0ji0DRRfUD2bq/kJveT6V96B1p
ItAeikaVDDrYm4w8mcg/rb/3ny5Sr5p71lBbQAygXDvOwQLon+BQuEfMPmUmmWPC
IVQIQ3wHHmF2vs+oBGamDASlX8dLrm1BCPuiL4XtPOejomHdVucbGhtSUe3APRyz
AGTjj/HiysEHWiTn5PnCSRduYjof9lraosolsW4NrnYiX2f5miC6DREqsnHgUivA
Q/Q3q26fPXGxLIanIU6P1wDlGrm0C//FCNSLlYNlDRzDsW5NHClT1xI047edBsMF
MwGM12cscIqupujcWwdIvApzjeAF7MpRfyQtBIyFR4ebewIDAQABo1AwTjAdBgNV
HQ4EFgQU/6oW29HV8UxJMX7HfipHrhbfDHIwHwYDVR0jBBgwFoAU/6oW29HV8UxJ
MX7HfipHrhbfDHIwDAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAQEAPGfZ
5/a4Y9/j4TMAqO7VRLNpt8YhIful54cI75srpBdG+AG/XD2uhDT/X5cJo/0c1ssZ
JXzg4Ht5GTrs+LmxgksnLRelF2dd1yPS2aJWLenIrorFkMkf+p4440pXzWf/qMqO
j9dpQsa588mw7+bFrujys/6OTu8ocSiO+EXq9H+D6HAtQMJ0SJpE0/RnAYJ3sxPu
7Eb/AUdzjeIKlZBxsTsrtHEZAhgrJPSxhyYm7ZGyWaurhegiHpFSCrvbIK2IRiaH
YfqOKWO4QP/JSQkldSs1PXNRtSqQ10k+XY+A5Tzy5yTzuCodtVmMLEovull5hg7h
nGwJ+IlGC+Q0z94LUw==
-----END CERTIFICATE-----

---

apiVersion: v1
kind: ConfigMap
metadata:
name: cluster-internal-ssl-revoked-certs
data:
# Add revoked public certificates as new keys to this configmap e.g. `revoked1.crt: |- ...`
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
imageTag: "4.0.8-enterprise"

core:
configMap: "cluster-internal-ssl-config"
additionalVolumes:
- name: cluster-internal-cert-and-key
secret:
secretName: cluster-internal-ssl-cert-and-key
- name: cluster-internal-revoked-certs
configMap:
name: cluster-internal-ssl-revoked-certs
- name: cluster-internal-trusted-certs
configMap:
name: cluster-internal-ssl-trusted-certs
additionalVolumeMounts:
- name: cluster-internal-cert-and-key
mountPath: /ssl/cluster
readOnly: true
- name: cluster-internal-revoked-certs
mountPath: /ssl/cluster/revoked
readOnly: true
- name: cluster-internal-trusted-certs
mountPath: /ssl/cluster/trusted
readOnly: true
- name: cluster-internal-cert-and-key
mountPath: /ssl/fabric
readOnly: true
- name: cluster-internal-revoked-certs
mountPath: /ssl/fabric/revoked
readOnly: true
- name: cluster-internal-trusted-certs
mountPath: /ssl/fabric/trusted
readOnly: true
- name: cluster-internal-cert-and-key
mountPath: /ssl/backup
readOnly: true
- name: cluster-internal-revoked-certs
mountPath: /ssl/backup/revoked
readOnly: true
- name: cluster-internal-trusted-certs
mountPath: /ssl/backup/trusted
readOnly: true

readReplica:
configMap: "cluster-internal-ssl-config"
additionalVolumes:
- name: cluster-internal-cert-and-key
secret:
secretName: cluster-internal-ssl-cert-and-key
- name: cluster-internal-revoked-certs
configMap:
name: cluster-internal-ssl-revoked-certs
- name: cluster-internal-trusted-certs
secret:
secretName: cluster-internal-ssl-trusted-certs
additionalVolumeMounts:
- name: cluster-internal-cert-and-key
mountPath: /ssl/cluster
readOnly: true
- name: cluster-internal-revoked-certs
mountPath: /ssl/cluster/revoked
readOnly: true
- name: cluster-internal-trusted-certs
mountPath: /ssl/cluster/trusted
readOnly: true
- name: cluster-internal-cert-and-key
mountPath: /ssl/fabric
readOnly: true
- name: cluster-internal-revoked-certs
mountPath: /ssl/fabric/revoked
readOnly: true
- name: cluster-internal-trusted-certs
mountPath: /ssl/fabric/trusted
readOnly: true
- name: cluster-internal-cert-and-key
mountPath: /ssl/backup
readOnly: true
- name: cluster-internal-revoked-certs
mountPath: /ssl/backup/revoked
readOnly: true
- name: cluster-internal-trusted-certs
mountPath: /ssl/backup/trusted
readOnly: true

acceptLicenseAgreement: "yes"
neo4jPassword: mySecretPassword
17 changes: 17 additions & 0 deletions deployment-scenarios/cluster-internal-ssl/instructions.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
This directory contains quick instructions and an example of how to set up cluster internal ssl
with Neo4j + Helm.

Step 1: Create custom ConfigMap.

kubectl apply -f deployment-scenarios/cluster-internal-ssl/cluster-internal-ssl-configmap.yaml

Step 2: Create ssl certificate ConfigMaps and key Secret.

This directory is populated with some example keys and certificates. You must use your own securely generated keys and certificates.

However if you want to use the example credentials FOR TESTING PURPOSES ONLY then you can install them using you can install using:
kubectl apply -f deployment-scenarios/cluster-internal-ssl/cluster-internal-ssl-files.yaml

Step 3: Install a Neo4j cluster using the provided parameters.

helm install cluster-internal-ssl-example -f deployment-scenarios/cluster-internal-ssl/cluster-internal-ssl.yaml .
12 changes: 12 additions & 0 deletions doc/docs/modules/ROOT/pages/operations.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,18 @@ The helm chart supports values for `additionalVolumes` and `additionalVolumeMoun

Use of additional volumes and mounts is not supported though, and in order to use this feature you must be very comfortable with filesystem basics in Kubernetes and Neo4j directory configuration.

## Transport Layer Security (TLS/SSL)

You can store public certificates in ConfigMaps and private keys in Kubernetes Secrets and use the helm chart's `additonalVolumes` and `additionalVolumeMounts` values to mount them into the Neo4j container.

The following neo4j config setting is required to support directly mounting certificates or keys from Kubernetes:

* `unsupported.dbms.ssl.system.ignore_dot_files=true`

Full details of SSL configuration can be found in the Neo4j operations manual. See the section "SSL Framework".

For an example of using certificates and keys stored in Kubernetes to secure internal traffic in a Neo4j Causal cluster please see the https://github.com/neo4j-contrib/neo4j-helm/blob/master/deployment-scenarios/cluster-internal-ssl/[cluster-internal-ssl deployment scenario].

## Fabric

In Neo4j 4.0+, https://neo4j.com/docs/operations-manual/current/fabric/introduction/[fabric] is a feature that can be enabled with regular configuration in `neo4j.conf`. All of the fabric configuration that is referenced in the manual can be done via custom ConfigMaps described in this documentation.
Expand Down