Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vintage-migration-changes #354

Merged
merged 36 commits into from
Oct 3, 2023
Merged
Show file tree
Hide file tree
Changes from 25 commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
b2d246b
vintage-migration-changes
calvix Aug 22, 2023
5ed7fc7
vintage-migration-changes
calvix Aug 22, 2023
6673f62
fix
calvix Aug 22, 2023
7cb571b
vintage
calvix Aug 24, 2023
6a1b9c9
vintage-migration-changes
calvix Aug 24, 2023
1192c25
Merge branch 'master' into vintage-migration-changes
calvix Aug 24, 2023
e44e7f2
vintage-migration-changes
calvix Aug 25, 2023
0c80114
Merge branch 'vintage-migration-changes' of github.com:giantswarm/clu…
calvix Aug 25, 2023
09eb343
Merge branch 'master' into vintage-migration-changes
calvix Aug 29, 2023
2b98865
Merge branch 'vintage-migration-changes' of github.com:giantswarm/clu…
calvix Aug 29, 2023
e7cee29
test-passwd
calvix Aug 29, 2023
fdb217d
vintage-migration-changes
calvix Aug 29, 2023
ef54c8d
fix
calvix Aug 29, 2023
05e695e
Merge branch 'master' into vintage-migration-changes
calvix Aug 29, 2023
9fdaf77
fix-cp-sg
calvix Aug 29, 2023
0b62afa
Merge branch 'vintage-migration-changes' of github.com:giantswarm/clu…
calvix Aug 29, 2023
fa4211f
vintage-migration-changes
calvix Aug 29, 2023
df16bb0
Merge branch 'master' into vintage-migration-changes
calvix Aug 29, 2023
c095c7a
Merge branch 'master' into vintage-migration-changes
calvix Aug 30, 2023
fd5663f
add-api-port
calvix Aug 30, 2023
5d8e86f
fix-scheme
calvix Aug 30, 2023
d52bbbc
add-post-commands
calvix Aug 31, 2023
9e1ce03
Merge branch 'master' into vintage-migration-changes
calvix Sep 12, 2023
9c34334
retry
calvix Sep 12, 2023
d1556b5
clean
calvix Sep 14, 2023
cd87402
Merge branch 'master' into vintage-migration-changes
calvix Sep 26, 2023
422c720
Merge branch 'master' into vintage-migration-changes
calvix Sep 27, 2023
955911d
Update helm/cluster-aws/values.schema.json
calvix Oct 2, 2023
f1b2b30
security-groups
calvix Oct 2, 2023
c062d14
Merge branch 'vintage-migration-changes' of github.com:giantswarm/clu…
calvix Oct 2, 2023
da86fa3
files
calvix Oct 2, 2023
f2a64ef
Merge branch 'master' into vintage-migration-changes
calvix Oct 2, 2023
98ab11b
fix
calvix Oct 2, 2023
f4696c2
vintage-migration-changes
calvix Oct 2, 2023
ff35a60
fix-files
calvix Oct 2, 2023
c9c23e8
remove-user
calvix Oct 3, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions helm/cluster-aws/templates/_aws_cluster.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -46,9 +46,24 @@ spec:
toPort: -1
vpc:
availabilityZoneUsageLimit: {{ .Values.connectivity.availabilityZoneUsageLimit }}
{{- if .Values.connectivity.network.vpcId }}
id: {{ .Values.connectivity.network.vpcId }}
{{- else }}
cidrBlock: {{ .Values.connectivity.network.vpcCidr }}
{{- end }}
{{- if .Values.connectivity.network.internetGatewayId }}
internetGatewayId: {{ .Values.connectivity.network.internetGatewayId }}
{{- end }}
subnets:
{{- range $j, $subnet := .Values.connectivity.subnets }}
{{- if $subnet.id }}
- id: {{ $subnet.id }}
isPublic: {{ $subnet.isPublic }}
routeTableId: {{ $subnet.routeTableId }}
{{- if $subnet.natGatewayId }}
natGatewayId: {{ $subnet.natGatewayId }}
{{- end }}
{{- else }}
{{- range $i, $cidr := $subnet.cidrBlocks }}
- cidrBlock: "{{ $cidr.cidr }}"
{{- if eq (len $cidr.availabilityZone) 1 }}
Expand All @@ -64,6 +79,7 @@ spec:
{{- end }}
{{- end }}
{{- end }}
{{- end }}
sshKeyName: ssh-key
s3Bucket:
controlPlaneIAMInstanceProfile: control-plane-{{ include "resource.default.name" $ }}
Expand Down
36 changes: 34 additions & 2 deletions helm/cluster-aws/templates/_control_plane.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,10 @@ template:
size: {{ .Values.controlPlane.rootVolumeSizeGB }}
type: gp3
iamInstanceProfile: control-plane-{{ include "resource.default.name" $ }}
{{- if .Values.controlPlane.additionalSecurityGroupID }}
calvix marked this conversation as resolved.
Show resolved Hide resolved
additionalSecurityGroups:
- id: {{ .Values.controlPlane.additionalSecurityGroupID }}
{{- end }}
sshKeyName: ""
subnet:
filters:
Expand Down Expand Up @@ -93,6 +97,9 @@ spec:
certSANs:
- "api.{{ include "resource.default.name" $ }}.{{ required "The baseDomain value is required" .Values.baseDomain }}"
- 127.0.0.1
{{- if .Values.controlPlane.apiExtraCertSANs -}}
{{- toYaml .Values.controlPlane.apiExtraCertSANs | nindent 10 }}
{{- end }}
extraArgs:
cloud-provider: external
service-account-issuer: "https://irsa.{{ include "resource.default.name" $ }}.{{ required "The baseDomain value is required" .Values.baseDomain }}"
Expand Down Expand Up @@ -122,6 +129,9 @@ spec:
service-account-lookup: "true"
tls-cipher-suites: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
service-cluster-ip-range: {{ .Values.connectivity.network.services.cidrBlocks | first }}
{{- if .Values.controlPlane.apiExtraArgs -}}
{{- toYaml .Values.controlPlane.apiExtraArgs | nindent 10 }}
{{- end }}
extraVolumes:
- name: auditlog
hostPath: /var/log/apiserver
Expand Down Expand Up @@ -157,6 +167,9 @@ spec:
extraArgs:
listen-metrics-urls: "http://0.0.0.0:2381"
quota-backend-bytes: "8589934592"
{{- if .Values.internal.migration.etcdExtraArgs -}}
{{- toYaml .Values.internal.migration.etcdExtraArgs | nindent 12 }}
{{- end }}
networking:
serviceSubnet: {{ join "," .Values.connectivity.network.services.cidrBlocks }}
files:
Expand All @@ -167,13 +180,23 @@ spec:
{{- if .Values.connectivity.proxy.enabled }}{{- include "proxyFiles" . | nindent 4 }}{{- end }}
{{- include "kubernetesFiles" . | nindent 4 }}
{{- include "registryFiles" . | nindent 4 }}
{{- if .Values.internal.migration.controlPlaneExtraFiles }}
{{- range $file := .Values.internal.migration.controlPlaneExtraFiles }}
calvix marked this conversation as resolved.
Show resolved Hide resolved
- path: {{ $file.path }}
permissions: "0644"
contentFrom:
secret:
name: {{ $file.secretName }}
key: {{ $file.secretKey }}
{{- end -}}
{{- end }}
initConfiguration:
skipPhases:
- addon/kube-proxy
- addon/coredns
localAPIEndpoint:
advertiseAddress: ""
bindPort: 0
bindPort: {{ .Values.internal.migration.apiBindPort }}
nodeRegistration:
kubeletExtraArgs:
cloud-provider: external
Expand All @@ -194,6 +217,9 @@ spec:
{{- end }}
joinConfiguration:
discovery: {}
controlPlane:
localAPIEndpoint:
bindPort: {{ .Values.internal.migration.apiBindPort }}
nodeRegistration:
kubeletExtraArgs:
cloud-provider: external
Expand All @@ -210,11 +236,17 @@ spec:
{{- end }}
{{- end }}
preKubeadmCommands:
{{- include "flatcarKubeadmPreCommands" . | nindent 4 }}
{{- include "sshPreKubeadmCommands" . | nindent 4 }}
{{- if .Values.internal.migration.controlPlanePreKubeadmCommands -}}
{{- toYaml .Values.internal.migration.controlPlanePreKubeadmCommands | nindent 4 }}
{{- end }}
{{- include "flatcarKubeadmPreCommands" . | nindent 4 }}
{{- if .Values.connectivity.proxy.enabled }}{{- include "proxyCommand" $ | nindent 4 }}{{- end }}
postKubeadmCommands:
{{- include "kubeletConfigPostKubeadmCommands" . | nindent 4 }}
{{- if .Values.internal.migration.controlPlanePostKubeadmCommands -}}
{{- toYaml .Values.internal.migration.controlPlanePostKubeadmCommands | nindent 4 }}
{{- end }}
users:
{{- include "sshUsers" . | nindent 4 }}
replicas: 3
Expand Down
4 changes: 4 additions & 0 deletions helm/cluster-aws/templates/_helpers.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -213,6 +213,10 @@ room for such suffix.
- name: giantswarm
groups: sudo
sudo: ALL=(ALL) NOPASSWD:ALL
- name: calvix
calvix marked this conversation as resolved.
Show resolved Hide resolved
groups: sudo
sudo: ALL=(ALL) NOPASSWD:ALL
passwd: $6$xyz$wveLFJmfL7cE7akXUh0JDlYC4E6XcnrYy9fgmOWWml2Qy4jvFwnzV54sByW8nV4TbdnNCTMJWovPu9GG1ir3j0
{{- end -}}

{{- define "ami" }}
Expand Down
4 changes: 4 additions & 0 deletions helm/cluster-aws/templates/_machine_pools.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,10 @@ spec:
size: {{ $value.rootVolumeSizeGB | default 300 }}
type: gp3
sshKeyName: ""
{{- if $value.additionalSecurityGroupID }}
calvix marked this conversation as resolved.
Show resolved Hide resolved
additionalSecurityGroups:
- id: {{ $value.additionalSecurityGroupID }}
{{- end }}
minSize: {{ $value.minSize | default 1 }}
maxSize: {{ $value.maxSize | default 3 }}
mixedInstancesPolicy:
Expand Down
125 changes: 125 additions & 0 deletions helm/cluster-aws/values.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,11 @@
"type": "object",
"title": "Node pool",
"properties": {
"additionalSecurityGroupID": {
"type": "string",
"title": "machine pool additional security group id",
"description": "Additional security groups id that will be added to the machine pool nodes."
},
"availabilityZones": {
"type": "array",
"title": "Availability zones",
Expand Down Expand Up @@ -251,6 +256,11 @@
"type": "object",
"title": "Network",
"properties": {
"internetGatewayId": {
"type": "string",
"title": "Internet Gateway ID",
"description": "ID of the Internet gateway for the VPC."
},
"pods": {
"type": "object",
"title": "Pods",
Expand Down Expand Up @@ -302,6 +312,11 @@
"title": "VPC subnet",
"description": "IPv4 address range to assign to this cluster's VPC, in CIDR notation.",
"default": "10.0.0.0/16"
},
"vpcId": {
"type": "string",
"title": "VPC id",
"description": "ID of the VPC, where the cluster will be deployed."
calvix marked this conversation as resolved.
Show resolved Hide resolved
}
}
},
Expand Down Expand Up @@ -374,10 +389,25 @@
}
}
},
"id": {
"type": "string",
"title": "ID of the subnet",
"description": "ID of the subnets, used inc ase we want to reuse already existing subnet."
calvix marked this conversation as resolved.
Show resolved Hide resolved
},
"isPublic": {
"type": "boolean",
"title": "Public"
},
"natGatewayId": {
"type": "string",
"title": "ID of the NAT Gateway",
"description": "ID of the NAT Gateway used for this existing subnet."
},
"routeTableId": {
"type": "string",
"title": "ID of route table",
"description": "ID of the route table, assigned to the existing subnet. Must be provided when defining subnet via ID."
},
"tags": {
"type": "object",
"title": "Tags",
Expand Down Expand Up @@ -479,6 +509,37 @@
"type": "object",
"title": "Control plane",
"properties": {
"additionalSecurityGroupID": {
"type": "string",
"title": "Control Plane additional security group id",
"description": "Additional security groups id that will be added to the control plane nodes."
},
"apiExtraArgs": {
"type": "object",
"title": "API extra arguments",
"description": "Extra arguments passed to the kubernetes API server.",
"patternProperties": {
"^.+:.+$": {
"type": "string",
"title": "argument"
}
},
"example": [
"audit-log-maxsize: 100"
]
},
"apiExtraCertSANs": {
"type": "array",
"title": "API extra cert SANs",
"description": "Extra certs SANs passed to the kubeadmcontrolplane CR.",
"items": {
"type": "string",
"title": "cert SAN",
"example": [
"test.mydomain.com"
]
}
},
"apiMode": {
"type": "string",
"title": "API mode",
Expand Down Expand Up @@ -621,6 +682,70 @@
],
"default": "1.24.14"
},
"migration": {
"type": "object",
"title": "Migration values",
"description": "Section used for migration of cluster from vintage to CAPI",
"properties": {
"apiBindPort": {
"type": "integer",
"title": "Kubernetes API bind port",
"description": "Kubernetes API bind port used for kube api pod",
"default": 6443
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

vintage listen on 443 and in the migration step we need to also set this to port 443, could be removed later

},
"controlPlaneExtraFiles": {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mostly for adding migration script for etcd

"type": "array",
"title": "Control Plane extra files",
"description": "Additional fiels that will be provisioned to control-plane nodes, reference is from secret in the same namespace.",
"items": {
"type": "object",
"title": "file",
"properties": {
"path": {
"type": "string",
"title": "file path"
},
"secretKey": {
"type": "string",
"title": "secret key for file content"
},
"secretName": {
"type": "string",
"title": "secret name for file content"
}
}
}
},
"controlPlanePostKubeadmCommands": {
"type": "array",
"title": "Control Plane Post Kubeadm Commands",
"description": "Additional Post-Kubeadm Commands executed on the control plane node.",
"items": {
"type": "string",
"title": "command"
}
},
"controlPlanePreKubeadmCommands": {
"type": "array",
"title": "Control Plane Pre Kubeadm Commands",
"description": "Additional Pre-Kubeadm Commands executed on the control plane node.",
"items": {
"type": "string",
"title": "command"
}
},
"etcdExtraArgs": {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to support different etcd configurations? I was thinking if we need this value at all. Couldn't we just use whatever flags we need in clusterConfiguration.etcd.local.extraArgs and avoid making it configurable?

Copy link
Contributor Author

@calvix calvix Oct 2, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, because this extra arguments are only needed for the migration, so it cannot be set globally for all clusters, I need an option to turn specify them on demand fro the migration

anything under .internal should not be used by customers and anything under migration should be possible to remove after migration

"type": "object",
"title": "Etcd extra arguments",
"patternProperties": {
"^.+:.+$": {
"type": "string",
"title": "argument"
}
}
}
}
},
"nodePools": {
"type": "object",
"title": "Default node pool",
Expand Down
2 changes: 2 additions & 0 deletions helm/cluster-aws/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,8 @@ controlPlane:
rootVolumeSizeGB: 120
internal:
kubernetesVersion: 1.24.14
migration:
apiBindPort: 6443
nodePools:
def00:
customNodeLabels:
Expand Down