Images download incorrect when the cpu architecture of the k8s and kubespary are different

[leeonfu]

What happened?

When installing an arm64 k8s cluster remotely on an amd64 host, the download_localhost mode incorrectly downloads the Docker images.

What did you expect to happen?

It is essential to download the images for the correct architecture.

How can we reproduce it (as minimally and precisely as possible)?

Run an amd64 local server and it will be used to install k8s cluster on arm64 server by remoting.

OS

Local:
PRETTY_NAME="Ubuntu 22.04.5 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.5 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy
Linux WPF3PXEG6 5.15.153.1-microsoft-standard-WSL2 #1 SMP Fri Mar 29 23:14:13 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

===============================
Remote, actually is k8s host.
NAME="openEuler"
VERSION="22.03 LTS"
ID="openEuler"
VERSION_ID="22.03"
PRETTY_NAME="openEuler 22.03 LTS"
ANSI_COLOR="0;31"
Linux node1 5.10.0-60.18.0.50.oe2203.aarch64 #1 SMP Wed Mar 30 02:43:08 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux

Version of Ansible

ansible [core 2.16.10]
config file = /home/leeon/kubespray/ansible.cfg
configured module search path = ['/home/leeon/kubespray/library']
ansible python module location = /home/leeon/kubespray/venv/lib/python3.10/site-packages/ansible
ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections
executable location = /home/leeon/kubespray/venv/bin/ansible
python version = 3.10.12 (main, Sep 11 2024, 15:47:36) [GCC 11.4.0] (/home/leeon/kubespray/venv/bin/python)
jinja version = 3.1.4
libyaml = True

Version of Python

3.10.12

Version of Kubespray (commit)

3874de0

Network plugin used

calico

Full inventory with variables

N/A

Command used to invoke ansible

ansible-playbook -i inventory/mycluster/inventory.ini cluster.yml -e "{ 'download_localhost': true, 'download_run_once': true }" -vvvv

Output of ansible run

TASK [download : Download_container | Download image if required] ************************************************************************************************************************************
task path: /home/fuuuulia/kubespray/roles/download/tasks/download_container.yml:57
changed: [node1 -> localhost] => {
"attempts": 1,
"changed": true,
"cmd": [
"/usr/local/bin/ctr",
"-n",
"k8s.io",
"images",
"pull",
"--hosts-dir",
"/etc/containerd/certs.d",
"quay.io/calico/node:v3.28.1"
],
"delta": "0:00:01.919447",
"end": "2024-10-24 16:10:37.332828",
"invocation": {
"module_args": {
"_raw_params": "/usr/local/bin/ctr -n k8s.io images pull --hosts-dir /etc/containerd/certs.d quay.io/calico/node:v3.28.1",

---

  "quay.io/calico/node:v3.28.1: resolving      |\u001b[32m\u001b[0m--------------------------------------| ",
    "elapsed: 1.3 s               total:   0.0 B (0.0 B/s)                                         ",
    "quay.io/calico/node:v3.28.1: resolving      |\u001b[32m\u001b[0m--------------------------------------| ",
    "elapsed: 1.4 s               total:   0.0 B (0.0 B/s)                                         ",
    "quay.io/calico/node:v3.28.1:                                                      resolved       |\u001b[32m++++++++++++++++++++++++++++++++++++++\u001b[0m| ",
    "index-sha256:d8c644a8a3eee06d88825b9a9fec6e7cd3b7c276d7f90afa8685a79fb300e7e3:    exists         |\u001b[32m++++++++++++++++++++++++++++++++++++++\u001b[0m| ",
    "manifest-sha256:f72bd42a299e280eed13231cc499b2d9d228ca2f51f6fd599d2f4176049d7880: exists         |\u001b[32m++++++++++++++++++++++++++++++++++++++\u001b[0m| ",
    "elapsed: 1.6 s                                                                    total:   0.0 B (0.0 B/s)                                         ",
    "quay.io/calico/node:v3.28.1:                                                      resolved       |\u001b[32m++++++++++++++++++++++++++++++++++++++\u001b[0m| ",
    "index-sha256:d8c644a8a3eee06d88825b9a9fec6e7cd3b7c276d7f90afa8685a79fb300e7e3:    exists         |\u001b[32m++++++++++++++++++++++++++++++++++++++\u001b[0m| ",
    "manifest-sha256:f72bd42a299e280eed13231cc499b2d9d228ca2f51f6fd599d2f4176049d7880: exists         |\u001b[32m++++++++++++++++++++++++++++++++++++++\u001b[0m| ",
    "layer-sha256:e8e9886e0a234d203edd09db20d140fc6df1e01569c49057108fb37ac57a676a:    exists         |\u001b[32m++++++++++++++++++++++++++++++++++++++\u001b[0m| ",
    "config-sha256:8bbeb9e1ee3287b8f750c10383f53fa1ec6f942aaea2a900f666d5e4e63cf4cc:   exists         |\u001b[32m++++++++++++++++++++++++++++++++++++++\u001b[0m| ",
    "elapsed: 1.6 s                                                                    total:   0.0 B (0.0 B/s)                                         ",
    "quay.io/calico/node:v3.28.1:                                                      resolved       |\u001b[32m++++++++++++++++++++++++++++++++++++++\u001b[0m| ",
    "index-sha256:d8c644a8a3eee06d88825b9a9fec6e7cd3b7c276d7f90afa8685a79fb300e7e3:    exists         |\u001b[32m++++++++++++++++++++++++++++++++++++++\u001b[0m| ",
    "manifest-sha256:f72bd42a299e280eed13231cc499b2d9d228ca2f51f6fd599d2f4176049d7880: exists         |\u001b[32m++++++++++++++++++++++++++++++++++++++\u001b[0m| ",
    "layer-sha256:e8e9886e0a234d203edd09db20d140fc6df1e01569c49057108fb37ac57a676a:    exists         |\u001b[32m++++++++++++++++++++++++++++++++++++++\u001b[0m| ",
    "config-sha256:8bbeb9e1ee3287b8f750c10383f53fa1ec6f942aaea2a900f666d5e4e63cf4cc:   exists         |\u001b[32m++++++++++++++++++++++++++++++++++++++\u001b[0m| ",
    "elapsed: 1.7 s                                                                    total:   0.0 B (0.0 B/s)                                         ",
    "unpacking linux/**amd64** sha256:d8c644a8a3eee06d88825b9a9fec6e7cd3b7c276d7f90afa8685a79fb300e7e3...",
    "done: 18.823917ms\t"

Anything else we need to know

No response

[thehedhly]

Hi @leeonfu , one quick workaround is to override the default calico vars specifically the Calico node Image Tag defined under https://github.com/kubernetes-sigs/kubespray/blob/master/roles/kubespray-defaults/defaults/main/download.yml#L238.

[leeonfu]

Hi @thehedhly ,
Definitely, you are right!
Actually, it requires a maintainer to fix it thoroughly.

[leeonfu]

Hi @VannTen ,
I think you are very interested in this issue.
Take it?