Synchronise yoga with upstream #76
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
The job definition does not exist anymore and zuul config reports error because of this, so let's get rid of this from the check queue (gate queue does not even have it). Change-Id: I0b211b7c8fcfebeaf9f1bc9bbee0c6527faa53d9
py39 jobs (on ubuntu-focal) started to fail due to recent virtualenv release (20.26.4 that bundles setuptools 74.1.2) on Yoga, because we have 'packaging==21.3' in this branch that is not compatible with the new setuptools [1]. setuptools is bundled in virtualenv, so it has to be capped via the virtualenv package. tox also needed to be capped (<4) as gate uses tox 3.28.0, but with capping virtualenv we pull in latest tox as well, which would cause other errors. [1] pypa/setuptools#4483 Change-Id: I668557933c51123ce84275a0c718b6e79b3df174
This adds a safety check and detection mechanism to the tools/test_format_inspector.py utility for verifying those features outside of glance. Change-Id: I447e7e51315472f8fa6013d4c4852f54c1e0c43d (cherry picked from commit b27040b87e43ff4acfb6870ef0d13d54e5ca5caa) (cherry picked from commit b394ef00c3426771092d099cf1d96077bfa4b919) (cherry picked from commit aa12d39b453068d9ee8367591eb60d4a15cddece) (cherry picked from commit a1c94b4) (cherry picked from commit a5ec69b)
We have to address two issues simultaneously: 1. handle multiple cirros images The change[1] configures multiple images for the devstack-tempest job and all the jobs deriving from it. This causes problem with the current post-check-metadata-injection.yaml playbook since we only expect one cirros image and perform steps accordingly. This change modifies the playbook to handle multiple cirros images. Failure in the glance-multistore-cinder-import job can be seen here[2]. [1] https://review.opendev.org/c/openstack/tempest/+/831018 [2] https://zuul.opendev.org/t/openstack/build/68a4d3ec6ce04c87b21d73a333f5b5cd/log/job-output.txt#23161 This was change I70b4a970d7e1. 2. change the the job config to address resource constraints * partial backport of I073216d1bbdd to change swap size (we don't include the change from that patch that was needed to address changes in Bobcat-era devstack) * backporting Ieb96eb6ceb6f to change tempest concurrency Remove openstack-tox-functional-py36-fips as there is no job definition for it. Change-Id: I6cd5b8bbf39f0cfae466f0800faea5a74e960e69 (cherry picked from commit 065e2d7) (cherry picked from commit 28b05a7)
VMDK: When parsing a VMDK file to calculate its size, the format_inspector determines the location of the Descriptor section by reading two uint64 from the headers of the file and uses them to create the descriptor CaptureRegion. It would be possible to craft a VMDK file that commands the format_inspector to create a very big CaptureRegion, thus exhausting resources on the glance-api process. This patch binds the beginning of the descriptor to 0x200 and limits the size of the CaptureRegion to 1MB, similar to how the VMDK descriptor is parsed by qemu. VHDX: It is a bit more involved, but similar: when looking for the VIRTUAL_DISK_SIZE metadata, the format_inspector was creating an unbounded CaptureRegion. In the same way as it seems to be done in Qemu, we now limit the upper bound of this CaptureRegion. Closes-Bug: #2006490 Change-Id: I3ec5a33df20e1cfb6673f4ff1c7c91aacd065532 (cherry picked from commit d4d33ee) (cherry picked from commit 06a1820)
Stream optimized VMDKs are also monolithic disks images, and start with the same sparse extend header as normal monolithic sparse files, so we can parse the virtual disk size in the same manner. See "VMware Virtual Disks Virtual Disk Format 1.1" p. 17. > Header and Footer > The header and the footer are both described by the same SparseExtentHeader > structure shown in Hosted Sparse Extent Header on page 8. Closes-Bug: #2052291 Change-Id: I7d63951ff080dc699b8d11babc0a5998d90621e4 Co-Authored-By: Rajiv Mucheli <[email protected]> (cherry picked from commit 5e7e6bf) (cherry picked from commit 9d45e8d4b87e992be23974a831811bff563ce721) (cherry picked from commit ef22a74) (cherry picked from commit bcfc1e4)
Change-Id: I6326a3e85c1ba4cb1da944a4323769f2399ed2c1 Closes-Bug: #2059809 (cherry picked from commit 2ca29af4433e9fa99a0a48e230d8d25d6eaa4a87) (cherry picked from commit c3586f3a122f6cb0663217b12b52203e74e2e4fa) (cherry picked from commit a92c438fb5ba55440b38cae7c8b4361b58daa9dd) (cherry picked from commit dba3bdb) (cherry picked from commit 6a38aef)
This adds two properties to the QcowInspector that makes it able to indicate whether the file specifies a backing_file or data_file in the header. Both conditions are considered unsafe for our usage. To ease checking of this condition, a classmethod is added that takes a local filename and digests just enough of the file to assert that both conditions are false. Note for yoga backport:With older qemu installed one of the qemu-img create commands fails, let's skip it from unmaintained/yoga and below that. Change-Id: Iaf86b525397d41bd116999cabe0954a0a7efac65 Related-Bug: #2059809 (cherry picked from commit ae536bb394793c9a7a219cb498e03d5c81dbbbb7) (cherry picked from commit 2eba54e0821106097dfeceb424e53943fd090483) (cherry picked from commit 89dbbc838d606f461087e1494d19ddbcf9db0a38) (cherry picked from commit 4860024) (cherry picked from commit f32d5b8)
This makes us check the extent filenames to make sure they don't have any banned characters in them (i.e. slashes). It also makes us reject VMDK files with a footer. Since we process these files as a stream, we can't honor a footer that directs us to find the descriptor block in a location we've already processed. Thus, if a file indicates it has a footer, consider it a policy exception and unsupported. Change-Id: I4a1c6dff7854c49940a0ac7988722aa6befc04fa (cherry picked from commit c1bf35dffb7f4c3090b1f04cf0e27cb431330c3e) (cherry picked from commit d3f1d6159c0218ac01e8d881e2ec4da71fc952ee) (cherry picked from commit 2dd4d138d4b8e1d9ca69fc0dda3711553a65d912) (cherry picked from commit 812e56d) (cherry picked from commit 05650e1)
Recent changes does not allow image conversion to convert an image having AMI disk format for image. Passing list of disk-formats to each import job which will now use qcow2 as default disk-format to create the image. Conflicts: .zuul.yaml Change-Id: I712b304ebe168705ec2793626f5d64135ad6d458 (cherry picked from commit 244ccb6) (cherry picked from commit f98e9ae) (cherry picked from commit 02732c2)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR contains a snapshot of yoga from upstream unmaintained/yoga.