Skip to content

Commit

Permalink
Backport : [PMK-6474] : Restrict TLS cipher suites for kube-rbac-prox…
Browse files Browse the repository at this point in the history
…y container (#213)
  • Loading branch information
cruizen committed Sep 25, 2024
1 parent b86b7f5 commit 201d4ac
Show file tree
Hide file tree
Showing 24 changed files with 244 additions and 167 deletions.
96 changes: 61 additions & 35 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,37 +1,61 @@
# Luigi

Luigi is a Kubernetes Operator to deploy, manage, and upgrade advanced networking plugins. The default Kubernetes networking model with one CNI and cluster-wide network can be too restrictive for many advanced networking use cases like NFV or virtualization

There are many discrete plugins and solutions, but knowing which ones to use, deploying and managing them can be tedious. Secondary CNIs? Multus? SRIOV? Device plugins? OVS? Which IPAM? What's the current Linux networking state of my nodes? How do I configure my nodes in order to support all of these?

# How to deploy
This will require an already working K8s cluster with DNS and a primary CNI up and running.
## How to deploy

This will require an already working K8s cluster with DNS and a primary CNI up and running.
Deploy the manifest found in samples in this repo:
```

```shell
kubectl apply -f https://raw.githubusercontent.com/platform9/luigi/master/samples/luigi-plugins-operator.yaml
```

A deployment of 1 replica will be created in the luigi-system namespace.

Or, To get started sign up for Platform9 Managed Kubernetes(PMK) for free at platform9.com/signup, see more about our Telco 5G offerings at platform9.com/solutions/telco-5g or contact us at platform9.com/contact. With PMK, Luigi will already be deployed and managed itself

# Plugins supported
## How to build

Platform9 publishes the images to Docker Hub under platform9 organisation.

- Luigi image is published as
`platform9/luigi-plugins:<version tag>`
- Hostplumber image is published as
`platform9/hostplumber:<version tag>`

The version tag is set to the git tag in git repository.
In the absence of a tag, it is set to [git branch]-pmk-[git revision id]

Override the version tag using env variable

- For Luigi:
`OVERRIDE_LUIGI_VERSION`
- For Hostplumber:
`OVERRIDE_HOSTPLUMBER_VERSION`

## Plugins supported

The scope of each plugin is beyond this documentation. But if you know you need it, luigi will deploy the following:

- HostPlumber: A subset of Luigi, an operator to configure/prep networking on the node and retrieve node details
- See: https://github.com/platform9/luigi/blob/master/hostplumber/README.md
- Use to create SRIOV VFs, configure OVS, create VLAN interfaces, etc...
- Recommended unless you have your own tooling to configure nodes
- Multus
- Almost always required - the only way K8S can support multiple CNIs and networks
- SRIOV CNI
- SRIOV Device Plugin
- OpenVSwitch daemon & CLI tools
- OVS CNI plugin
- Macvlan, IPvlan
- Whereabouts IPAM driver
- Required for dynamic IP assignment without an external DHCP service.
- Node Feature Discovery

# Configuration:
- HostPlumber: A subset of Luigi, an operator to configure/prep networking on the node and retrieve node details
- See: [README.md](https://github.com/platform9/luigi/blob/master/hostplumber/README.md)
- Use to create SRIOV VFs, configure OVS, create VLAN interfaces, etc...
- Recommended unless you have your own tooling to configure nodes
- Multus
- Almost always required - the only way K8S can support multiple CNIs and networks
- SRIOV CNI
- SRIOV Device Plugin
- OpenVSwitch daemon & CLI tools
- OVS CNI plugin
- Macvlan, IPvlan
- Whereabouts IPAM driver
- Required for dynamic IP assignment without an external DHCP service.
- Node Feature Discovery

## Configuration

**namespace**: Each plugin will take in a namespace override to deploy, default namespace otherwise

Expand All @@ -42,21 +66,23 @@ The scope of each plugin is beyond this documentation. But if you know you need
**privateRegistryBase**: Some airgapped env's may have a custom container registry. If this is specified, it will replace the public container registry URL (docker.io, gcr.io, quay, etc..) with this path

Each plugin may or may not have some further specific configuration. Here are the current options as of release v0.3:
- HostPlumber - none
- Multus - none
- SRIOV - none
- Node-feature-discovery - none
- OVS - none
- Whereabouts
- ipReconcilerSchedule - specify the CronJob schedule of the whereabouts IP cleanup Job
- ipReconcilerNodeSelector - specify the nodeSelector Labels on which to schedule the ip-reconciler

# NetworkPlugins CRD:

- HostPlumber - none
- Multus - none
- SRIOV - none
- Node-feature-discovery - none
- OVS - none
- Whereabouts
- ipReconcilerSchedule - specify the CronJob schedule of the whereabouts IP cleanup Job
- ipReconcilerNodeSelector - specify the nodeSelector Labels on which to schedule the ip-reconciler

## NetworkPlugins CRD

In it's current phase, only one instance of the CRD is supported. It will reflect the final, desired state of all plugins to be deployed.

If it is present, Luigi will ensure that the plugin is deployed and upgraded. If missing and re-applied, Luigi will remove the plugin if it was previously managing it.

```
```YAML
apiVersion: plumber.k8s.pf9.io/v1
kind: NetworkPlugins
metadata:
Expand All @@ -78,7 +104,7 @@ spec:
The above will deploy all the plugins specified in the default namespace. To override the namespace, and deploy in kube-system:
```
```YAML
apiVersion: plumber.k8s.pf9.io/v1
kind: NetworkPlugins
metadata:
Expand All @@ -98,9 +124,9 @@ spec:
namespace: "kube-system"
```
That is it! Now that you have the secondary CNIs and other related plugins deployed, you may need to prep the nodes before you can actually create Multus Networks and assign them to Pods. In order to do so, use Luigi's own HostPlumber plugin: https://github.com/platform9/luigi/blob/master/hostplumber/README.md
That is it! Now that you have the secondary CNIs and other related plugins deployed, you may need to prep the nodes before you can actually create Multus Networks and assign them to Pods. In order to do so, use Luigi's own HostPlumber plugin. See [README for HostPlumber](https://github.com/platform9/luigi/blob/master/hostplumber/README.md)
## Dev note
##### Dev note
This project needs to migrate to Kubebuilder/v4.
webhooks where added manually `make generate && make manifestes` will not add required feild for webhook in crds and luigi deployment. refer `samples/luigi-plugins-operator-v2.yaml`
webhooks where added manually `make generate && make manifestes` will not add required field for webhook in crds and luigi deployment. refer `samples/luigi-plugins-operator-v2.yaml`
4 changes: 3 additions & 1 deletion config/default/manager_auth_proxy_patch.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,14 @@ spec:
spec:
containers:
- name: kube-rbac-proxy
image: gcr.io/kubebuilder/kube-rbac-proxy:v0.11.0
image: quay.io/brancz/kube-rbac-proxy:v0.18.1
args:
- "--secure-listen-address=0.0.0.0:8443"
- "--upstream=http://127.0.0.1:8080/"
- "--logtostderr=true"
- "--v=0"
- --tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
- --tls-min-version=VersionTLS12
ports:
- containerPort: 8443
protocol: TCP
Expand Down
4 changes: 2 additions & 2 deletions controllers/networkplugins_controller.go
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ const (
KubemacpoolNamespace = "dhcp-controller-system"
DefaultMetricsPort = "8080"
MultusImage = "docker.io/platform9/multus:v3.7.2-pmk-2644970"
WhereaboutsImage = "docker.io/platform9/whereabouts:v0.6.3"
WhereaboutsImage = "docker.io/platform9/whereabouts:v0.6.3-pmk-3299438"
SriovCniImage = "docker.io/platform9/sriov-cni:v2.6.2-pmk-2877848"
SriovDpImage = "docker.io/platform9/sriov-network-device-plugin:v3.3.2-pmk-2877839"
OvsImage = "quay.io/platform9/openvswitch:v2.17.5-3"
Expand All @@ -67,7 +67,7 @@ const (
KubemacpoolImage = "quay.io/kubevirt/kubemacpool:v0.41.0"
KubemacpoolRangeStart = "02:55:43:00:00:00"
KubemacpoolRangeEnd = "02:55:43:FF:FF:FF"
KubeRbacProxyImage = "docker.io/brancz/kube-rbac-proxy:v0.18.0"
KubeRbacProxyImage = "quay.io/brancz/kube-rbac-proxy:v0.18.1"
NfdImage = "docker.io/platform9/node-feature-discovery:v0.11.3-pmk-2877967"
TemplateDir = "/etc/plugin_templates/"
CreateDir = TemplateDir + "create/"
Expand Down
4 changes: 3 additions & 1 deletion dhcp-controller/config/default/manager_auth_proxy_patch.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,14 @@ spec:
capabilities:
drop:
- "ALL"
image: gcr.io/kubebuilder/kube-rbac-proxy:v0.12.0
image: quay.io/brancz/kube-rbac-proxy:v0.18.1
args:
- "--secure-listen-address=0.0.0.0:8443"
- "--upstream=http://127.0.0.1:8080/"
- "--logtostderr=true"
- "--v=0"
- --tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
- --tls-min-version=VersionTLS12
ports:
- containerPort: 8443
protocol: TCP
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,14 @@ spec:
capabilities:
drop:
- "ALL"
image: gcr.io/kubebuilder/kube-rbac-proxy:v0.12.0
image: quay.io/brancz/kube-rbac-proxy:v0.18.1
args:
- "--secure-listen-address=0.0.0.0:8443"
- "--upstream=http://127.0.0.1:8080/"
- "--logtostderr=true"
- "--v=0"
- --tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
- --tls-min-version=VersionTLS12
ports:
- containerPort: 8443
protocol: TCP
Expand Down
14 changes: 7 additions & 7 deletions dhcp-controller/dhcpserver/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -7,16 +7,16 @@ require (
github.com/gocarina/gocsv v0.0.0-20220927221512-ad3251f9fa25
github.com/onsi/ginkgo/v2 v2.15.0
github.com/onsi/gomega v1.31.1
k8s.io/api v0.26.10
k8s.io/apimachinery v0.26.10
k8s.io/client-go v0.26.10
k8s.io/api v0.26.15
k8s.io/apimachinery v0.26.15
k8s.io/client-go v0.26.15
kubevirt.io/api v0.58.0
sigs.k8s.io/controller-runtime v0.14.7
)

require (
github.com/beorn7/perks v1.0.1 // indirect
github.com/cespare/xxhash/v2 v2.1.2 // indirect
github.com/cespare/xxhash/v2 v2.2.0 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/emicklei/go-restful/v3 v3.9.0 // indirect
github.com/evanphx/json-patch/v5 v5.6.0 // indirect
Expand All @@ -28,7 +28,7 @@ require (
github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/protobuf v1.5.3 // indirect
github.com/golang/protobuf v1.5.4 // indirect
github.com/google/gnostic v0.5.7-v3refs // indirect
github.com/google/go-cmp v0.6.0 // indirect
github.com/google/gofuzz v1.1.0 // indirect
Expand Down Expand Up @@ -67,8 +67,8 @@ require (
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
k8s.io/apiextensions-apiserver v0.26.10 // indirect
k8s.io/component-base v0.26.10 // indirect
k8s.io/apiextensions-apiserver v0.26.15 // indirect
k8s.io/component-base v0.26.15 // indirect
k8s.io/klog/v2 v2.80.1 // indirect
k8s.io/kube-openapi v0.0.0-20221012153701-172d655c2280 // indirect
k8s.io/utils v0.0.0-20221128185143-99ec85e7a448 // indirect
Expand Down
27 changes: 14 additions & 13 deletions dhcp-controller/dhcpserver/go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -50,8 +50,9 @@ github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
github.com/cespare/xxhash/v2 v2.1.1/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/cespare/xxhash/v2 v2.1.2 h1:YRXhKfTDauu4ajMg1TPgFO5jnlC2HCbmLXMcTG5cbYE=
github.com/cespare/xxhash/v2 v2.1.2/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/cespare/xxhash/v2 v2.2.0 h1:DC2CZ1Ep5Y4k3ZQ899DldepgrayRUGE6BBZ/cd9Cj44=
github.com/cespare/xxhash/v2 v2.2.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI=
github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI=
github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU=
Expand Down Expand Up @@ -150,8 +151,8 @@ github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw
github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
github.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=
github.com/golang/protobuf v1.5.3 h1:KhyjKVUg7Usr/dYsdSqoFveMYd5ko72D+zANwlG1mmg=
github.com/golang/protobuf v1.5.3/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=
github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
github.com/google/gnostic v0.5.7-v3refs h1:FhTMOKj2VhjpouxvWJAV1TL304uMlb9zcDqkl6cEI54=
Expand Down Expand Up @@ -689,18 +690,18 @@ honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt
honnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=
honnef.co/go/tools v0.0.1-2020.1.4/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=
k8s.io/api v0.23.3/go.mod h1:w258XdGyvCmnBj/vGzQMj6kzdufJZVUwEM1U2fRJwSQ=
k8s.io/api v0.26.10 h1:skTnrDR0r8dg4MMLf6YZIzugxNM0BjFsWKPkNc5kOvk=
k8s.io/api v0.26.10/go.mod h1:ou/H3yviqrHtP/DSPVTfsc7qNfmU06OhajytJfYXkXw=
k8s.io/apiextensions-apiserver v0.26.10 h1:wAriTUc6l7gUqJKOxhmXnYo/VNJzk4oh4QLCUR4Uq+k=
k8s.io/apiextensions-apiserver v0.26.10/go.mod h1:N2qhlxkhJLSoC4f0M1/1lNG627b45SYqnOPEVFoQXw4=
k8s.io/api v0.26.15 h1:tjMERUjIwkq+2UtPZL5ZbSsLkpxUv4gXWZfV5lQl+Og=
k8s.io/api v0.26.15/go.mod h1:CtWOrFl8VLCTLolRlhbBxo4fy83tjCLEtYa5pMubIe0=
k8s.io/apiextensions-apiserver v0.26.15 h1:QePn6+5mihx8sXQLaOXzvF4XPv2RGGj8Pv+O4P75GPU=
k8s.io/apiextensions-apiserver v0.26.15/go.mod h1:PbhgN0XidyF+9vCTUmNgVFK0MMEYqlHLZ4AJeBfiNMo=
k8s.io/apimachinery v0.23.3/go.mod h1:BEuFMMBaIbcOqVIJqNZJXGFTP4W6AycEpb5+m/97hrM=
k8s.io/apimachinery v0.26.10 h1:aE+J2KIbjctFqPp3Y0q4Wh2PD+l1p2g3Zp4UYjSvtGU=
k8s.io/apimachinery v0.26.10/go.mod h1:iT1ZP4JBP34wwM+ZQ8ByPEQ81u043iqAcsJYftX9amM=
k8s.io/client-go v0.26.10 h1:4mDzl+1IrfRxh4Ro0s65JRGJp14w77gSMUTjACYWVRo=
k8s.io/client-go v0.26.10/go.mod h1:sh74ig838gCckU4ElYclWb24lTesPdEDPnlyg5vcbkA=
k8s.io/apimachinery v0.26.15 h1:GPxeERYBSqSZlj3xIkX4L6mBjzZ9q8JPnJ+Vj15qe+g=
k8s.io/apimachinery v0.26.15/go.mod h1:O/uIhIOWuy6ndHqQ6qbkjD7OgeMhVtlk8+Z66ZcmJQc=
k8s.io/client-go v0.26.15 h1:A2Yav2v+VZQfpEsf5ESFp2Lqq5XACKBDrwkG+jEtOg0=
k8s.io/client-go v0.26.15/go.mod h1:KJs7snLEyKPlypqTQG/ngcaqE6h3/6qTvVHDViRL+iI=
k8s.io/code-generator v0.23.3/go.mod h1:S0Q1JVA+kSzTI1oUvbKAxZY/DYbA/ZUb4Uknog12ETk=
k8s.io/component-base v0.26.10 h1:vl3Gfe5aC09mNxfnQtTng7u3rnBVrShOK3MAkqEleb0=
k8s.io/component-base v0.26.10/go.mod h1:/IDdENUHG5uGxqcofZajovYXE9KSPzJ4yQbkYQt7oN0=
k8s.io/component-base v0.26.15 h1:32XJyv5fo/lbDZhYU1HyISXTgdSUkbW5cO4DhfR6Y/8=
k8s.io/component-base v0.26.15/go.mod h1:9V+nBzUtTNtRuYfYmQQEhuKrjhL80i2l6F2H2qUsHAI=
k8s.io/gengo v0.0.0-20210813121822-485abfe95c7c/go.mod h1:FiNAH4ZV3gBg2Kwh89tzAEV2be7d5xI0vBa/VySYy3E=
k8s.io/gengo v0.0.0-20211129171323-c02415ce4185/go.mod h1:FiNAH4ZV3gBg2Kwh89tzAEV2be7d5xI0vBa/VySYy3E=
k8s.io/klog/v2 v2.0.0/go.mod h1:PBfzABfn139FHAV07az/IF9Wp1bkk3vpT2XSJ76fSDE=
Expand Down
14 changes: 7 additions & 7 deletions dhcp-controller/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -10,16 +10,16 @@ require (
github.com/onsi/ginkgo/v2 v2.15.0
github.com/onsi/gomega v1.31.1
github.com/sirupsen/logrus v1.9.0
k8s.io/api v0.26.10
k8s.io/apimachinery v0.26.10
k8s.io/client-go v0.26.10
k8s.io/api v0.26.15
k8s.io/apimachinery v0.26.15
k8s.io/client-go v0.26.15
kubevirt.io/api v0.58.0
sigs.k8s.io/controller-runtime v0.14.7
)

require (
github.com/beorn7/perks v1.0.1 // indirect
github.com/cespare/xxhash/v2 v2.1.2 // indirect
github.com/cespare/xxhash/v2 v2.2.0 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/emicklei/go-restful/v3 v3.9.0 // indirect
github.com/evanphx/json-patch/v5 v5.6.0 // indirect
Expand All @@ -31,7 +31,7 @@ require (
github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/protobuf v1.5.3 // indirect
github.com/golang/protobuf v1.5.4 // indirect
github.com/google/gnostic v0.5.7-v3refs // indirect
github.com/google/go-cmp v0.6.0 // indirect
github.com/google/gofuzz v1.1.0 // indirect
Expand Down Expand Up @@ -70,8 +70,8 @@ require (
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
k8s.io/apiextensions-apiserver v0.26.10 // indirect
k8s.io/component-base v0.26.10 // indirect
k8s.io/apiextensions-apiserver v0.26.15 // indirect
k8s.io/component-base v0.26.15 // indirect
k8s.io/klog/v2 v2.80.1 // indirect
k8s.io/kube-openapi v0.0.0-20221012153701-172d655c2280 // indirect
k8s.io/utils v0.0.0-20221128185143-99ec85e7a448 // indirect
Expand Down
Loading

0 comments on commit 201d4ac

Please sign in to comment.