Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Hacking on ironic component #10

Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
83 changes: 83 additions & 0 deletions components/13-ironic/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
# OpenStack Ironic

So unfortunately OpenStack Helm doesn't publish helm charts that can be consumed like
regular helm charts. You must instead clone two of their git repos side by side and
build the dependencies manually. They additionally don't split out secrets but instead
template them into giant config files or even executable scripts that then get stored
as secrets, a clear violation of <https://12factor.net>. As a result we cannot store
a declarative config of Keystone and allow users to supply their own secrets.
Comment on lines +3 to +8
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it just because they don't publish them in a repo? If so, could we just host it ourselves internally?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's certainly an option. I'm happy to get some ideas around it.


Due to the above issues, for now we'll skip the ArgoCD ability for this deployment.

## Get OpenStack Helm Ready

You may have done this for another OpenStack component and can share the same
git clones. This assumes you're doing this from the top level of this repo.

```bash
# clone the two repos because they reference the infra one as a relative path
# so you can't use real helm commands
git clone https://github.com/openstack/openstack-helm
git clone https://github.com/openstack/openstack-helm-infra
# update the dependencies cause we can't use real helm references
./scripts/openstack-helm-depend-sync.sh ironic
```

## Deploy Ironic

Since we cannot refer to the secrets by name, we must look them up live from the cluster
so that we can injected them into the templated configs. Upstream should really allow
secrets to be passed by reference. As a result of this we cannot use GitOps to generate
these charts and have them applied to the cluster.

Secrets Reference:

- openstack-default-user is created by the messaging-topology-operator which is
executed by the rabbitmq-queues component. The name stems from the RabbitMQ
cluster from the rabbitmq-cluster component. `${CLUSTER_NAME}-default-user`

### Create the rabbitmq and mariadb instances

```bash
kubectl -n openstack apply -k components/13-ironic
```

### Apply the ironic helm template with our custom aio-values.yaml

```bash
helm --namespace openstack template \
ironic \
./openstack-helm/ironic/ \
-f components/13-ironic/aio-values.yaml \
--set endpoints.identity.auth.admin.password="$(kubectl --namespace openstack get secret keystone-admin -o jsonpath='{.data.password}' | base64 -d)" \
--set endpoints.oslo_db.auth.admin.password="$(kubectl --namespace openstack get secret mariadb -o jsonpath='{.data.root-password}' | base64 -d)" \
--set endpoints.oslo_db.auth.keystone.password="$(kubectl --namespace openstack get secret keystone-db-password -o jsonpath='{.data.password}' | base64 -d)" \
--set endpoints.oslo_messaging.auth.admin.password="$(kubectl --namespace openstack get secret openstack-default-user -o jsonpath='{.data.password}' | base64 -d)" \
--set endpoints.oslo_messaging.hosts.default="openstack" \
--set endpoints.oslo_messaging.auth.keystone.password="$(kubectl --namespace openstack get secret keystone-rabbitmq-password -o jsonpath='{.data.password}' | base64 -d)" \
--post-renderer $(git rev-parse --show-toplevel)/scripts/openstack-helm-sealed-secrets.sh \
| kubectl -n openstack apply -f -
```

At this point Ironic will go through some initialization and start up.

## Validating Keystone

You can run an OpenStack client in the cluster to validate it is running correctly.

```bash
# start up a pod with the client
kubectl -n openstack apply -f https://raw.githubusercontent.com/rackerlabs/genestack/main/manifests/utils/utils-openstack-client-admin.yaml
```

Show the catalog list

```bash
kubectl exec -it openstack-admin-client -n openstack -- openstack catalog list
```

Show the service list

```bash
kubectl exec -it openstack-admin-client -n openstack -- openstack service list
```
107 changes: 107 additions & 0 deletions components/13-ironic/aio-values.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
---

images:
tags:
ironic_manage_cleaning_network: "docker.io/openstackhelm/heat:2023.1-ubuntu_jammy"
ironic_retrive_cleaning_network: "docker.io/openstackhelm/heat:2023.1-ubuntu_jammy"
ironic_retrive_swift_config: "docker.io/openstackhelm/heat:2023.1-ubuntu_jammy"
bootstrap: "docker.io/openstackhelm/heat:2023.1-ubuntu_jammy"
db_init: "docker.io/openstackhelm/heat:2023.1-ubuntu_jammy"
db_drop: "docker.io/openstackhelm/heat:2023.1-ubuntu_jammy"
ironic_db_sync: "docker.io/openstackhelm/ironic:2023.1-ubuntu_jammy"
ironic_api: "docker.io/openstackhelm/ironic:2023.1-ubuntu_jammy"
ironic_conductor: "docker.io/openstackhelm/ironic:2023.1-ubuntu_jammy"
ironic_pxe: "docker.io/openstackhelm/ironic:2023.1-ubuntu_jammy"
ironic_pxe_init: "docker.io/openstackhelm/ironic:2023.1-ubuntu_jammy"
ironic_pxe_http: docker.io/nginx:1.13.3
ks_user: "docker.io/openstackhelm/heat:2023.1-ubuntu_jammy"
ks_service: "docker.io/openstackhelm/heat:2023.1-ubuntu_jammy"
ks_endpoints: "docker.io/openstackhelm/heat:2023.1-ubuntu_jammy"
rabbit_init: docker.io/rabbitmq:3.7-management
dep_check: quay.io/airshipit/kubernetes-entrypoint:v1.0.0
image_repo_sync: docker.io/docker:17.07.0
pull_policy: "IfNotPresent"
local_registry:
active: false
exclude:
- dep_check
- image_repo_sync

bootstrap:
image:
enabled: false
openstack:
enabled: false
network:
enabled: false
openstack:
enabled: false
object_store:
enabled: false
openstack:
enabled: false

conf:
ironic:
ironic_conductor:
automated_clean: false
conductor:
automated_clean: false
dhcp:
dhcp_provider: none
logging:
logger_root:
level: DEBUG
logger_ironic:
level: DEBUG

endpoints:
identity:
name: keystone

network:
api:
ingress:
public: true
classes:
namespace: "nginx"
cluster: "nginx-openstack"
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
external_policy_local: false
node_port:
enabled: false
pxe:
device: ens1f0

dependencies:
dynamic:
common:
local_image_registry:
jobs: null
static:
api:
jobs:
- ironic-db-sync
- ironic-ks-user
- ironic-ks-endpoints
services:
- endpoint: internal
service: oslo_db
- endpoint: internal
service: oslo_messaging
conductor:
jobs:
- ironic-db-sync
- ironic-ks-user
- ironic-ks-endpoints
services:
- endpoint: internal
service: oslo_db
- endpoint: internal
service: oslo_messaging

manifests:
job_manage_cleaning_network: false
job_rabbit_init: false
secret_registry: false
52 changes: 52 additions & 0 deletions components/13-ironic/ironic-mariadb-db.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
---
apiVersion: mariadb.mmontes.io/v1alpha1
kind: Database
metadata:
name: ironic
namespace: openstack
spec:
# If you want the database to be created with a different name than the resource name
# name: data-custom
mariaDbRef:
name: mariadb # name of the MariaDB kind
waitForIt: true
characterSet: utf8
collate: utf8_general_ci
retryInterval: 5s
---
apiVersion: mariadb.mmontes.io/v1alpha1
kind: User
metadata:
name: ironic
namespace: openstack
spec:
# If you want the user to be created with a different name than the resource name
# name: user-custom
mariaDbRef:
name: mariadb # name of the MariaDB kind
waitForIt: true
passwordSecretKeyRef:
name: ironic-db-password
key: password
# This field is immutable and defaults to 10, 0 means unlimited.
maxUserConnections: 0
host: "%"
retryInterval: 5s
---
apiVersion: mariadb.mmontes.io/v1alpha1
kind: Grant
metadata:
name: ironic-grant
namespace: openstack
spec:
mariaDbRef:
name: mariadb # name of the MariaDB kind
waitForIt: true
privileges:
- "ALL"
database: "ironic"
table: "*"
username: ironic
grantOption: true
host: "%"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we restrict this to pod networking subnet?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that would make sense. No idea how to look that up.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kubectl get nodes -o jsonpath='{.items[*].spec.podCIDR}'

would be one way, but it's also subject to change so maybe just 10.x.x.x would make sense

retryInterval: 5s
59 changes: 59 additions & 0 deletions components/13-ironic/ironic-rabbitmq-queue.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
---
apiVersion: rabbitmq.com/v1beta1
kind: User
metadata:
name: ironic
namespace: openstack
spec:
tags:
- management # available tags are 'management', 'policymaker', 'monitoring' and 'administrator'
- policymaker
rabbitmqClusterReference:
name: rabbitmq # rabbitmqCluster must exist in the same namespace as this resource
namespace: openstack
importCredentialsSecret:
name: ironic-rabbitmq-password
---
apiVersion: rabbitmq.com/v1beta1
kind: Vhost
metadata:
name: ironic-vhost
namespace: openstack
spec:
name: "ironic" # vhost name; required and cannot be updated
defaultQueueType: quorum # default queue type for this vhost; require RabbitMQ version 3.11.12 or above
rabbitmqClusterReference:
name: rabbitmq # rabbitmqCluster must exist in the same namespace as this resource
namespace: openstack
---
apiVersion: rabbitmq.com/v1beta1
kind: Queue
metadata:
name: ironic-queue
namespace: openstack
spec:
name: ironic-qq # name of the queue
vhost: "ironic" # default to '/' if not provided
type: quorum # without providing a queue type, rabbitmq creates a classic queue
autoDelete: false
durable: true # seting 'durable' to false means this queue won't survive a server restart
rabbitmqClusterReference:
name: rabbitmq # rabbitmqCluster must exist in the same namespace as this resource
namespace: openstack
---
apiVersion: rabbitmq.com/v1beta1
kind: Permission
metadata:
name: ironic-permission
namespace: openstack
spec:
vhost: "ironic" # name of a vhost
userReference:
name: "ironic" # name of a user.rabbitmq.com in the same namespace; must specify either spec.userReference or spec.user
permissions:
write: ".*"
configure: ".*"
read: ".*"
rabbitmqClusterReference:
name: rabbitmq # rabbitmqCluster must exist in the same namespace as this resource
namespace: openstack
7 changes: 7 additions & 0 deletions components/13-ironic/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- ironic-mariadb-db.yaml
- ironic-rabbitmq-queue.yaml
Loading