Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add s390x for ubuntu and centos artifacts #203

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Add s390x for ubuntu and centos artifacts #203

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c25d68a
Adding s390x arch for ubuntu and centos artifacts
aditi-sharma-1 Oct 11, 2024
eaa5499
Added GOARCH variable to Makefile for dynamic architecture detection …
aditi-sharma-1 Oct 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions Makefile
View file
Open in desktop
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you put these changes into a separate commit please?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @0xFelix, Thanks for the review. Sure, will make these changes in a separate commit.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Thank you.

Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,15 @@ $(LOCALBIN):
## Tool Binaries
GOFUMPT ?= $(LOCALBIN)/gofumpt

GOARCH ?= amd64

all: test medius

clean:
rm -rf bin

medius:
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o bin/medius kubevirt.io/containerdisks/cmd/medius
CGO_ENABLED=0 GOOS=linux GOARCH=$(GOARCH) go build -o bin/medius kubevirt.io/containerdisks/cmd/medius

fmt: gofumpt
go mod tidy -compat=1.22
Expand All @@ -38,7 +40,7 @@ getginkgo:
go get github.com/onsi/ginkgo/v2@$(GINKGO_VERSION)

test: lint
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go run github.com/onsi/ginkgo/v2/ginkgo@$(GINKGO_VERSION) -v -timeout $(GINKGO_TIMEOUT) ./...
CGO_ENABLED=0 GOOS=linux GOARCH=$(GOARCH) go run github.com/onsi/ginkgo/v2/ginkgo@$(GINKGO_VERSION) -v -timeout $(GINKGO_TIMEOUT) ./...

.PHONY: gofumpt
gofumpt: $(GOFUMPT) ## Download gofumpt locally if necessary.
Expand Down
27 changes: 27 additions & 0 deletions artifacts/centosstream/centos-stream_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,33 @@ var _ = Describe("CentosStream", func() {
},
},
),
Entry("centos-stream:9 s390x", "9", "s390x", "testdata/centos-stream9-s390x.checksum",
&api.ArtifactDetails{
SHA256Sum: "17322e2562832b57bb2554a5b7056fba6d06db662728c487496d83845d7f016c",
DownloadURL: "https://cloud.centos.org/centos/9-stream/s390x/images/CentOS-Stream-GenericCloud-9-latest.s390x.qcow2",
AdditionalUniqueTags: []string{"9-latest"},
ImageArchitecture: "s390x",
},
&docs.UserData{
Username: "cloud-user",
},
map[string]string{
common.DefaultInstancetypeEnv: "u1.medium",
common.DefaultPreferenceEnv: "centos.stream9",
},
&api.Metadata{
Name: "centos-stream",
Version: "9",
Description: description,
ExampleUserData: docs.UserData{
Username: "cloud-user",
},
EnvVariables: map[string]string{
common.DefaultInstancetypeEnv: "u1.medium",
common.DefaultPreferenceEnv: "centos.stream9",
},
},
),
)
})

Expand Down
72 changes: 72 additions & 0 deletions artifacts/centosstream/testdata/centos-stream9-s390x.checksum
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
# CentOS-Stream-Container-Base-9-20231002.0.s390x.tar.xz: 34815024 bytes
SHA256 (CentOS-Stream-Container-Base-9-20231002.0.s390x.tar.xz) = 0a3d52ff27a202bd3af081573451fb48842fc4a671f3a53978c8530621a26a2e
# CentOS-Stream-Container-Base-9-20240101.0.s390x.tar.xz: 35435032 bytes
SHA256 (CentOS-Stream-Container-Base-9-20240101.0.s390x.tar.xz) = 258322e0fb4ef1057e20df2ef604921e28894888abd5845c56b3760932485278
# CentOS-Stream-Container-Base-9-20240325.0.s390x.tar.xz: 34282952 bytes
SHA256 (CentOS-Stream-Container-Base-9-20240325.0.s390x.tar.xz) = eed0500097fbc8df52116ffb9b86c61e124e2af76e9fb91dbf55178649d87988
# CentOS-Stream-Container-Base-9-20240527.0.s390x.tar.xz: 34835540 bytes
SHA256 (CentOS-Stream-Container-Base-9-20240527.0.s390x.tar.xz) = 3a301d3cf4bdf578e0fa038bb28efd72ee991f46fe36725a2e2f936d197fd6b7
# CentOS-Stream-Container-Base-9-20240703.1.s390x.tar.xz: 35407456 bytes
SHA256 (CentOS-Stream-Container-Base-9-20240703.1.s390x.tar.xz) = 453243ae060548fcf5b7ae6511a5bebb01dadae77a1fb4e142f216ff3ada826d
# CentOS-Stream-Container-Base-9-20240828.0.s390x.tar.xz: 35128812 bytes
SHA256 (CentOS-Stream-Container-Base-9-20240828.0.s390x.tar.xz) = e0a336c6128d55a948d037703b5b1ded5b80b81875e8ecada6ec9088238b68f7
# CentOS-Stream-Container-Base-9-20240909.0.s390x.tar.xz: 34846072 bytes
SHA256 (CentOS-Stream-Container-Base-9-20240909.0.s390x.tar.xz) = 9d19c7690ecbc85c4f18151c7f2bbe04982b46b25d25f92a3467fdd74e9f4d30
# CentOS-Stream-Container-Base-9-20240916.0.s390x.tar.xz: 36690200 bytes
SHA256 (CentOS-Stream-Container-Base-9-20240916.0.s390x.tar.xz) = 16da6e91bb76c0570cc2f87bacd1f9f85f041685f7996a079fa29a232f66b9a0
# CentOS-Stream-Container-Base-9-20240923.0.s390x.tar.xz: 36855220 bytes
SHA256 (CentOS-Stream-Container-Base-9-20240923.0.s390x.tar.xz) = 7f4a0140d85a24de39627bb8f7e4c3bd79453c3b286b9485b9beb77ba95f13d3
# CentOS-Stream-Container-Base-9-20240930.0.s390x.tar.xz: 36811344 bytes
SHA256 (CentOS-Stream-Container-Base-9-20240930.0.s390x.tar.xz) = 4aaf0fcc8d75e07b0fcc4e64aac5f6323c0bc92103dc1310a7289bbefc9b3c16
# CentOS-Stream-Container-Base-9-20241007.0.s390x.tar.xz: 37067652 bytes
SHA256 (CentOS-Stream-Container-Base-9-20241007.0.s390x.tar.xz) = 2c0a5e21ac43ed9ee9c1745eeb88e84fffe879d103ef8a60e9ab70d078aaab14
# CentOS-Stream-Container-Base-9-latest.s390x.tar.xz: 37067652 bytes
SHA256 (CentOS-Stream-Container-Base-9-latest.s390x.tar.xz) = 2c0a5e21ac43ed9ee9c1745eeb88e84fffe879d103ef8a60e9ab70d078aaab14
# CentOS-Stream-Container-Minimal-9-20231002.0.s390x.tar.xz: 22794792 bytes
SHA256 (CentOS-Stream-Container-Minimal-9-20231002.0.s390x.tar.xz) = d5778725dfa76bb5ef1022ffe7d148658b90609e2484560100a19133b322186a
# CentOS-Stream-Container-Minimal-9-20240101.0.s390x.tar.xz: 22799504 bytes
SHA256 (CentOS-Stream-Container-Minimal-9-20240101.0.s390x.tar.xz) = 56d30a08f1b7336aa66418db7fb9df6ec0cdcf2fb89a91399f0aaac815349e7b
# CentOS-Stream-Container-Minimal-9-20240325.0.s390x.tar.xz: 23310596 bytes
SHA256 (CentOS-Stream-Container-Minimal-9-20240325.0.s390x.tar.xz) = cc43fea1f0a0eee0ae0b6b882fc140541b222fcc4ee3070ca2372e376affff45
# CentOS-Stream-Container-Minimal-9-20240527.0.s390x.tar.xz: 23707568 bytes
SHA256 (CentOS-Stream-Container-Minimal-9-20240527.0.s390x.tar.xz) = 0b3654b30be3b7ed4f0e2011c37a5f64203066c10edf71768525d3bcdaa56797
# CentOS-Stream-Container-Minimal-9-20240703.1.s390x.tar.xz: 23586912 bytes
SHA256 (CentOS-Stream-Container-Minimal-9-20240703.1.s390x.tar.xz) = f2718bb3171d14f1858aaed2c870827dd1a06c94147886a381a7b628c4b070f6
# CentOS-Stream-Container-Minimal-9-20240828.0.s390x.tar.xz: 23449328 bytes
SHA256 (CentOS-Stream-Container-Minimal-9-20240828.0.s390x.tar.xz) = b9f54e946511b5d5997a1111a68c2fde30fa059963f5f6e042e477d99d4c9dc1
# CentOS-Stream-Container-Minimal-9-20240909.0.s390x.tar.xz: 23906544 bytes
SHA256 (CentOS-Stream-Container-Minimal-9-20240909.0.s390x.tar.xz) = 670c5701f74d400ca49b6d80fa6539f9802b7b642c801387f3e06b65b6cfc1d2
# CentOS-Stream-Container-Minimal-9-20240916.0.s390x.tar.xz: 35632424 bytes
SHA256 (CentOS-Stream-Container-Minimal-9-20240916.0.s390x.tar.xz) = baac66acb2ab302622f20a0cdff2d9c491983eccb19927b10b1e782a401c2974
# CentOS-Stream-Container-Minimal-9-20240923.0.s390x.tar.xz: 35216100 bytes
SHA256 (CentOS-Stream-Container-Minimal-9-20240923.0.s390x.tar.xz) = 365c03224460af6000cf9585ceeb4d2ffaf53599d782a9fc21c1637d19e9b307
# CentOS-Stream-Container-Minimal-9-20240930.0.s390x.tar.xz: 35947188 bytes
SHA256 (CentOS-Stream-Container-Minimal-9-20240930.0.s390x.tar.xz) = 9d6009a0b24870f0f12eb291ac4af51c52a8cf9e36a1b269aa56d67bb3ff54df
# CentOS-Stream-Container-Minimal-9-20241007.0.s390x.tar.xz: 35391744 bytes
SHA256 (CentOS-Stream-Container-Minimal-9-20241007.0.s390x.tar.xz) = f307dfc13fd65d9f0fd2a36fdd120b303c1b6d68fe2d034aaf8f24592c61d075
# CentOS-Stream-Container-Minimal-9-latest.s390x.tar.xz: 35391744 bytes
SHA256 (CentOS-Stream-Container-Minimal-9-latest.s390x.tar.xz) = f307dfc13fd65d9f0fd2a36fdd120b303c1b6d68fe2d034aaf8f24592c61d075
# CentOS-Stream-GenericCloud-9-20231002.0.s390x.qcow2: 849577472 bytes
SHA256 (CentOS-Stream-GenericCloud-9-20231002.0.s390x.qcow2) = 4f58b07f1e41f17177402335d5951f7645ac94a26e1afcfc58f69be31be2825c
# CentOS-Stream-GenericCloud-9-20240101.0.s390x.qcow2: 886444032 bytes
SHA256 (CentOS-Stream-GenericCloud-9-20240101.0.s390x.qcow2) = 1d02316aa5eee741082ac3aaf87324399723744e577bf795f8fba00f29368aa6
# CentOS-Stream-GenericCloud-9-20240325.0.s390x.qcow2: 933608960 bytes
SHA256 (CentOS-Stream-GenericCloud-9-20240325.0.s390x.qcow2) = 0cacba5d1e5148cb970901cf292c6aa05551f91539536b1d982589314971d317
# CentOS-Stream-GenericCloud-9-20240527.0.s390x.qcow2: 942020608 bytes
SHA256 (CentOS-Stream-GenericCloud-9-20240527.0.s390x.qcow2) = 5b02b1a4bbab1cc2caba14ce59d83fdd145e83e2ac2aac2147699e4ec6e6bdbb
# CentOS-Stream-GenericCloud-9-20240703.1.s390x.qcow2: 924992000 bytes
SHA256 (CentOS-Stream-GenericCloud-9-20240703.1.s390x.qcow2) = 96d713d108a025116215806ca11f26b7449fe0b35827cf26e8278a0886655996
# CentOS-Stream-GenericCloud-9-20240828.0.s390x.qcow2: 972301824 bytes
SHA256 (CentOS-Stream-GenericCloud-9-20240828.0.s390x.qcow2) = 802653119649af31c2f625d0cb0fe9a6b9980b1e51d52556e2bba7e11e61e7d6
# CentOS-Stream-GenericCloud-9-20240909.0.s390x.qcow2: 965258752 bytes
SHA256 (CentOS-Stream-GenericCloud-9-20240909.0.s390x.qcow2) = c6c487e84df3e8bdd01c187e1a6ac6c72ecee25528aaad516f2b0ec10ca5934a
# CentOS-Stream-GenericCloud-9-20240916.0.s390x.qcow2: 959359488 bytes
SHA256 (CentOS-Stream-GenericCloud-9-20240916.0.s390x.qcow2) = 81d486a833ae87698280836eb1e05e1b536012a4f78eecd14db1b76f42ca06df
# CentOS-Stream-GenericCloud-9-20240923.0.s390x.qcow2: 955930624 bytes
SHA256 (CentOS-Stream-GenericCloud-9-20240923.0.s390x.qcow2) = 8d446522df5da261109178200fa4a4fa5b49a3461cf52dfd76a639674012d8c9
# CentOS-Stream-GenericCloud-9-20240930.0.s390x.qcow2: 948032000 bytes
SHA256 (CentOS-Stream-GenericCloud-9-20240930.0.s390x.qcow2) = 07c87e2ac3528ea9c785632642d321fd9bc7eda272b7faba589ba1acf53acf99
# CentOS-Stream-GenericCloud-9-20241007.0.s390x.qcow2: 946031104 bytes
SHA256 (CentOS-Stream-GenericCloud-9-20241007.0.s390x.qcow2) = 17322e2562832b57bb2554a5b7056fba6d06db662728c487496d83845d7f016c
# CentOS-Stream-GenericCloud-9-latest.s390x.qcow2: 946031104 bytes
SHA256 (CentOS-Stream-GenericCloud-9-latest.s390x.qcow2) = 17322e2562832b57bb2554a5b7056fba6d06db662728c487496d83845d7f016c
23 changes: 23 additions & 0 deletions artifacts/ubuntu/ubuntu_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,29 @@ var _ = Describe("Ubuntu", func() {
},
},
),
Entry("ubuntu:22.04 s390x", "22.04", "s390x", "testdata/SHA256SUM",
&api.ArtifactDetails{
SHA256Sum: "192c18a58917622e12a3bb6aaf246fcc6a76d9562eb9f49d34df81fbc59610af",
DownloadURL: "https://cloud-images.ubuntu.com/releases/22.04/release/ubuntu-22.04-server-cloudimg-s390x.img",
ImageArchitecture: "s390x",
},
map[string]string{
common.DefaultInstancetypeEnv: "u1.medium",
common.DefaultPreferenceEnv: "ubuntu",
},
&api.Metadata{
Name: "ubuntu",
Version: "22.04",
Description: description,
ExampleUserData: docs.UserData{
Username: "ubuntu",
},
EnvVariables: map[string]string{
common.DefaultInstancetypeEnv: "u1.medium",
common.DefaultPreferenceEnv: "ubuntu",
},
},
),
)
})

Expand Down
4 changes: 4 additions & 0 deletions cmd/medius/common/registry.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,27 +28,31 @@ var staticRegistry = []Entry{
Artifacts: []api.Artifact{
centosstream.New("9", "x86_64", &docs.UserData{Username: "cloud-user"}, defaultEnvVariables("u1.medium", "centos.stream9")),
centosstream.New("9", "aarch64", &docs.UserData{Username: "cloud-user"}, defaultEnvVariables("u1.medium", "centos.stream9")),
centosstream.New("9", "s390x", &docs.UserData{Username: "cloud-user"}, defaultEnvVariables("u1.medium", "centos.stream9")),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do instancetypes in their current form apply to the s390x architecture? If not they should be dropped here and below. Same for the preferences.

Did you verify the created images work on s390x? (medius images publish + medius images verify)

Copy link
Author

@aditi-sharma-1 aditi-sharma-1 Oct 14, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @0xFelix ,
Sure, looking into it. Thank you.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you check if VMs created with these instancetypes / preferences work on s390x?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @0xFelix ,
Yes, I have checked, and it's working. I am attaching the logs below for your reference. Thank you.
s390x-build.txt

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I first executed bin/medius images push --target-registry=localhost:5000 --dry-run=false --insecure-skip-tls --force --focus=ubuntu:22.04, I encountered the following error:

error introspecting image "quay.io/containerdisks/ubuntu:22.04": Error parsing manifest for image: choosing image instance: no image found in manifest list for architecture "s390x", variant "", OS "linux"

Since I was trying to build it locally on s390x, I commented out the section of code in cmd/medius/images/push.go (from line 291 to line 304) that checks the quay.io manifest and modified the function to return true. I have included the git diff output in the logs for your reference. After making this change, the execution was successful. I have also done a docker pull to verify the images (ubuntu:22.04 and centos-stream:9) created for the s390x architecture.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please let me know if you have any suggestions or specific requirements for this implementation. I appreciate your guidance! Thank you.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I commented out the section of code [...] that checks the quay.io manifest

There is a --source-registry flag to change the source registry. That might have helped, or for next time.

Did you run medius images verify to verify the images? I'm interested to see if the built images were able to boot successfully.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @0xFelix ,
I am trying to execute medius images verify but facing some errors. I'm currently investigating the issue. I will keep you updated about the progress.

Thank you for your understanding!

},
UseForDocs: true,
},
{
Artifacts: []api.Artifact{
ubuntu.New("24.04", "x86_64", defaultEnvVariables("u1.medium", "ubuntu")),
ubuntu.New("24.04", "aarch64", defaultEnvVariables("u1.medium", "ubuntu")),
ubuntu.New("24.04", "s390x", defaultEnvVariables("u1.medium", "ubuntu")),
},
UseForDocs: true,
},
{
Artifacts: []api.Artifact{
ubuntu.New("22.04", "x86_64", defaultEnvVariables("u1.medium", "ubuntu")),
ubuntu.New("22.04", "aarch64", defaultEnvVariables("u1.medium", "ubuntu")),
ubuntu.New("22.04", "s390x", defaultEnvVariables("u1.medium", "ubuntu")),
},
UseForDocs: false,
},
{
Artifacts: []api.Artifact{
ubuntu.New("20.04", "x86_64", defaultEnvVariables("u1.medium", "ubuntu")),
ubuntu.New("20.04", "aarch64", defaultEnvVariables("u1.medium", "ubuntu")),
ubuntu.New("20.04", "s390x", defaultEnvVariables("u1.medium", "ubuntu")),
},
UseForDocs: false,
},
Expand Down
2 changes: 2 additions & 0 deletions pkg/architecture/arch.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ func GetImageArchitecture(arch string) string {
return "amd64"
case "aarch64":
return "arm64"
case "s390x":
return "s390x"
default:
panic(fmt.Sprintf("can't map unknown architecture %s to image architecture", arch))
}
Expand Down