-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: Hacking on ironic component #10
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,83 @@ | ||
# OpenStack Ironic | ||
|
||
So unfortunately OpenStack Helm doesn't publish helm charts that can be consumed like | ||
regular helm charts. You must instead clone two of their git repos side by side and | ||
build the dependencies manually. They additionally don't split out secrets but instead | ||
template them into giant config files or even executable scripts that then get stored | ||
as secrets, a clear violation of <https://12factor.net>. As a result we cannot store | ||
a declarative config of Keystone and allow users to supply their own secrets. | ||
|
||
Due to the above issues, for now we'll skip the ArgoCD ability for this deployment. | ||
|
||
## Get OpenStack Helm Ready | ||
|
||
You may have done this for another OpenStack component and can share the same | ||
git clones. This assumes you're doing this from the top level of this repo. | ||
|
||
```bash | ||
# clone the two repos because they reference the infra one as a relative path | ||
# so you can't use real helm commands | ||
git clone https://github.com/openstack/openstack-helm | ||
git clone https://github.com/openstack/openstack-helm-infra | ||
# update the dependencies cause we can't use real helm references | ||
./scripts/openstack-helm-depend-sync.sh ironic | ||
``` | ||
|
||
## Deploy Ironic | ||
|
||
Since we cannot refer to the secrets by name, we must look them up live from the cluster | ||
so that we can injected them into the templated configs. Upstream should really allow | ||
secrets to be passed by reference. As a result of this we cannot use GitOps to generate | ||
these charts and have them applied to the cluster. | ||
|
||
Secrets Reference: | ||
|
||
- openstack-default-user is created by the messaging-topology-operator which is | ||
executed by the rabbitmq-queues component. The name stems from the RabbitMQ | ||
cluster from the rabbitmq-cluster component. `${CLUSTER_NAME}-default-user` | ||
|
||
### Create the rabbitmq and mariadb instances | ||
|
||
```bash | ||
kubectl -n openstack apply -k components/13-ironic | ||
``` | ||
|
||
### Apply the ironic helm template with our custom aio-values.yaml | ||
|
||
```bash | ||
helm --namespace openstack template \ | ||
ironic \ | ||
./openstack-helm/ironic/ \ | ||
-f components/13-ironic/aio-values.yaml \ | ||
--set endpoints.identity.auth.admin.password="$(kubectl --namespace openstack get secret keystone-admin -o jsonpath='{.data.password}' | base64 -d)" \ | ||
--set endpoints.oslo_db.auth.admin.password="$(kubectl --namespace openstack get secret mariadb -o jsonpath='{.data.root-password}' | base64 -d)" \ | ||
--set endpoints.oslo_db.auth.keystone.password="$(kubectl --namespace openstack get secret keystone-db-password -o jsonpath='{.data.password}' | base64 -d)" \ | ||
--set endpoints.oslo_messaging.auth.admin.password="$(kubectl --namespace openstack get secret openstack-default-user -o jsonpath='{.data.password}' | base64 -d)" \ | ||
--set endpoints.oslo_messaging.hosts.default="openstack" \ | ||
--set endpoints.oslo_messaging.auth.keystone.password="$(kubectl --namespace openstack get secret keystone-rabbitmq-password -o jsonpath='{.data.password}' | base64 -d)" \ | ||
--post-renderer $(git rev-parse --show-toplevel)/scripts/openstack-helm-sealed-secrets.sh \ | ||
| kubectl -n openstack apply -f - | ||
``` | ||
|
||
At this point Ironic will go through some initialization and start up. | ||
|
||
## Validating Keystone | ||
|
||
You can run an OpenStack client in the cluster to validate it is running correctly. | ||
|
||
```bash | ||
# start up a pod with the client | ||
kubectl -n openstack apply -f https://raw.githubusercontent.com/rackerlabs/genestack/main/manifests/utils/utils-openstack-client-admin.yaml | ||
``` | ||
|
||
Show the catalog list | ||
|
||
```bash | ||
kubectl exec -it openstack-admin-client -n openstack -- openstack catalog list | ||
``` | ||
|
||
Show the service list | ||
|
||
```bash | ||
kubectl exec -it openstack-admin-client -n openstack -- openstack service list | ||
``` |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,107 @@ | ||
--- | ||
|
||
images: | ||
tags: | ||
ironic_manage_cleaning_network: "docker.io/openstackhelm/heat:2023.1-ubuntu_jammy" | ||
ironic_retrive_cleaning_network: "docker.io/openstackhelm/heat:2023.1-ubuntu_jammy" | ||
ironic_retrive_swift_config: "docker.io/openstackhelm/heat:2023.1-ubuntu_jammy" | ||
bootstrap: "docker.io/openstackhelm/heat:2023.1-ubuntu_jammy" | ||
db_init: "docker.io/openstackhelm/heat:2023.1-ubuntu_jammy" | ||
db_drop: "docker.io/openstackhelm/heat:2023.1-ubuntu_jammy" | ||
ironic_db_sync: "docker.io/openstackhelm/ironic:2023.1-ubuntu_jammy" | ||
ironic_api: "docker.io/openstackhelm/ironic:2023.1-ubuntu_jammy" | ||
ironic_conductor: "docker.io/openstackhelm/ironic:2023.1-ubuntu_jammy" | ||
ironic_pxe: "docker.io/openstackhelm/ironic:2023.1-ubuntu_jammy" | ||
ironic_pxe_init: "docker.io/openstackhelm/ironic:2023.1-ubuntu_jammy" | ||
ironic_pxe_http: docker.io/nginx:1.13.3 | ||
ks_user: "docker.io/openstackhelm/heat:2023.1-ubuntu_jammy" | ||
ks_service: "docker.io/openstackhelm/heat:2023.1-ubuntu_jammy" | ||
ks_endpoints: "docker.io/openstackhelm/heat:2023.1-ubuntu_jammy" | ||
rabbit_init: docker.io/rabbitmq:3.7-management | ||
dep_check: quay.io/airshipit/kubernetes-entrypoint:v1.0.0 | ||
image_repo_sync: docker.io/docker:17.07.0 | ||
pull_policy: "IfNotPresent" | ||
local_registry: | ||
active: false | ||
exclude: | ||
- dep_check | ||
- image_repo_sync | ||
|
||
bootstrap: | ||
image: | ||
enabled: false | ||
openstack: | ||
enabled: false | ||
network: | ||
enabled: false | ||
openstack: | ||
enabled: false | ||
object_store: | ||
enabled: false | ||
openstack: | ||
enabled: false | ||
|
||
conf: | ||
ironic: | ||
ironic_conductor: | ||
automated_clean: false | ||
conductor: | ||
automated_clean: false | ||
dhcp: | ||
dhcp_provider: none | ||
logging: | ||
logger_root: | ||
level: DEBUG | ||
logger_ironic: | ||
level: DEBUG | ||
|
||
endpoints: | ||
identity: | ||
name: keystone | ||
|
||
network: | ||
api: | ||
ingress: | ||
public: true | ||
classes: | ||
namespace: "nginx" | ||
cluster: "nginx-openstack" | ||
annotations: | ||
nginx.ingress.kubernetes.io/rewrite-target: / | ||
external_policy_local: false | ||
node_port: | ||
enabled: false | ||
pxe: | ||
device: ens1f0 | ||
|
||
dependencies: | ||
dynamic: | ||
common: | ||
local_image_registry: | ||
jobs: null | ||
static: | ||
api: | ||
jobs: | ||
- ironic-db-sync | ||
- ironic-ks-user | ||
- ironic-ks-endpoints | ||
services: | ||
- endpoint: internal | ||
service: oslo_db | ||
- endpoint: internal | ||
service: oslo_messaging | ||
conductor: | ||
jobs: | ||
- ironic-db-sync | ||
- ironic-ks-user | ||
- ironic-ks-endpoints | ||
services: | ||
- endpoint: internal | ||
service: oslo_db | ||
- endpoint: internal | ||
service: oslo_messaging | ||
|
||
manifests: | ||
job_manage_cleaning_network: false | ||
job_rabbit_init: false | ||
secret_registry: false |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,52 @@ | ||
--- | ||
apiVersion: mariadb.mmontes.io/v1alpha1 | ||
kind: Database | ||
metadata: | ||
name: ironic | ||
namespace: openstack | ||
spec: | ||
# If you want the database to be created with a different name than the resource name | ||
# name: data-custom | ||
mariaDbRef: | ||
name: mariadb # name of the MariaDB kind | ||
waitForIt: true | ||
characterSet: utf8 | ||
collate: utf8_general_ci | ||
retryInterval: 5s | ||
--- | ||
apiVersion: mariadb.mmontes.io/v1alpha1 | ||
kind: User | ||
metadata: | ||
name: ironic | ||
namespace: openstack | ||
spec: | ||
# If you want the user to be created with a different name than the resource name | ||
# name: user-custom | ||
mariaDbRef: | ||
name: mariadb # name of the MariaDB kind | ||
waitForIt: true | ||
passwordSecretKeyRef: | ||
name: ironic-db-password | ||
key: password | ||
# This field is immutable and defaults to 10, 0 means unlimited. | ||
maxUserConnections: 0 | ||
host: "%" | ||
retryInterval: 5s | ||
--- | ||
apiVersion: mariadb.mmontes.io/v1alpha1 | ||
kind: Grant | ||
metadata: | ||
name: ironic-grant | ||
namespace: openstack | ||
spec: | ||
mariaDbRef: | ||
name: mariadb # name of the MariaDB kind | ||
waitForIt: true | ||
privileges: | ||
- "ALL" | ||
database: "ironic" | ||
table: "*" | ||
username: ironic | ||
grantOption: true | ||
host: "%" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. should we restrict this to pod networking subnet? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yeah that would make sense. No idea how to look that up. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. kubectl get nodes -o jsonpath='{.items[*].spec.podCIDR}' would be one way, but it's also subject to change so maybe just 10.x.x.x would make sense |
||
retryInterval: 5s |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,59 @@ | ||
--- | ||
apiVersion: rabbitmq.com/v1beta1 | ||
kind: User | ||
metadata: | ||
name: ironic | ||
namespace: openstack | ||
spec: | ||
tags: | ||
- management # available tags are 'management', 'policymaker', 'monitoring' and 'administrator' | ||
- policymaker | ||
rabbitmqClusterReference: | ||
name: rabbitmq # rabbitmqCluster must exist in the same namespace as this resource | ||
namespace: openstack | ||
importCredentialsSecret: | ||
name: ironic-rabbitmq-password | ||
--- | ||
apiVersion: rabbitmq.com/v1beta1 | ||
kind: Vhost | ||
metadata: | ||
name: ironic-vhost | ||
namespace: openstack | ||
spec: | ||
name: "ironic" # vhost name; required and cannot be updated | ||
defaultQueueType: quorum # default queue type for this vhost; require RabbitMQ version 3.11.12 or above | ||
rabbitmqClusterReference: | ||
name: rabbitmq # rabbitmqCluster must exist in the same namespace as this resource | ||
namespace: openstack | ||
--- | ||
apiVersion: rabbitmq.com/v1beta1 | ||
kind: Queue | ||
metadata: | ||
name: ironic-queue | ||
namespace: openstack | ||
spec: | ||
name: ironic-qq # name of the queue | ||
vhost: "ironic" # default to '/' if not provided | ||
type: quorum # without providing a queue type, rabbitmq creates a classic queue | ||
autoDelete: false | ||
durable: true # seting 'durable' to false means this queue won't survive a server restart | ||
rabbitmqClusterReference: | ||
name: rabbitmq # rabbitmqCluster must exist in the same namespace as this resource | ||
namespace: openstack | ||
--- | ||
apiVersion: rabbitmq.com/v1beta1 | ||
kind: Permission | ||
metadata: | ||
name: ironic-permission | ||
namespace: openstack | ||
spec: | ||
vhost: "ironic" # name of a vhost | ||
userReference: | ||
name: "ironic" # name of a user.rabbitmq.com in the same namespace; must specify either spec.userReference or spec.user | ||
permissions: | ||
write: ".*" | ||
configure: ".*" | ||
read: ".*" | ||
rabbitmqClusterReference: | ||
name: rabbitmq # rabbitmqCluster must exist in the same namespace as this resource | ||
namespace: openstack |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
--- | ||
apiVersion: kustomize.config.k8s.io/v1beta1 | ||
kind: Kustomization | ||
|
||
resources: | ||
- ironic-mariadb-db.yaml | ||
- ironic-rabbitmq-queue.yaml |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it just because they don't publish them in a repo? If so, could we just host it ourselves internally?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's certainly an option. I'm happy to get some ideas around it.