Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bootc status failed with error Reading deployment metadata: Missing base image ref ostree/container/blob/e05a2 #800

Closed
henrywang opened this issue Sep 22, 2024 · 9 comments

Comments

@henrywang
Copy link
Contributor

henrywang commented Sep 22, 2024

This issue is for anaconda installation only (bib anaconda-iso included), bootc install does not have this issue. Test result: https://gitlab.com/redhat/rhel/bifrost/tests/bootc-image/-/issues/219

Deploy bootc image with anaconda kickstart and boot the system. System can be booted without error. Check ostree status with rpm-ostree successful. But check with bootc status failed.

$ sudo bootc status
ERROR: Status: Computing status: Booted deployment: Reading deployment metadata: Missing base image ref ostree/container/blob/e05a222db71963792f1e74c3ca70a0009ae55e8794de45e8f8d7c9cea416936e"
$ rpm-ostree status
Deployments:
    ● ostree-unverified-registry:192.168.100.1:5000/hidden:22t5
                       Digest: sha256:d1aa7635c6fa50c70e8cfd110e229d1a2329a48609616e1bc585b9c8ed8bf3f4
                      Version: 9.20240920.0 (2024-09-20T21:59:00Z)
  1. This issue has been found since bootc-202409192026.g85b2ca5256-1.el9.x86_64.
  2. The bootc-202409191923.geea3996f8f-1.el9.x86_64 does not have this issue.
  3. Check bootc commit from eea3996f8f to 85b2ca5256. The a8737d9 might be the root case.
@cgwalters
Copy link
Collaborator

Yes I saw this on one machine too, it's really concerning.

We may have broken the semantics for how we write refs as part of the upgrade in ostreedev/ostree-rs-ext#663

@cgwalters
Copy link
Collaborator

PR in ostreedev/ostree-rs-ext#666

cgwalters added a commit to cgwalters/oci-spec-rs that referenced this issue Sep 23, 2024
Conceptually `Digest()` is just a parsed string wrapper. Many
cases want to get access to the full value without allocating,
and the introduction of the type was a regression from that point of
view in 0.7.

Luckily, we can change our internals in such a way that it's
safe to add the impl. Do that by holding onto the full value,
with a duplicate small boxed str for the algorithm only in
the degenerate case that it's an unknown type.

If we had this it would have made containers/bootc#800
less likely.

Signed-off-by: Colin Walters <[email protected]>
@cgwalters
Copy link
Collaborator

And youki-dev/oci-spec-rs#229

@cgwalters
Copy link
Collaborator

Should be fixed by #804 - thanks for the report.

@henrywang
Copy link
Contributor Author

Verified on bootc-202409242020.g34e104d217-1.el9.x86_64.

@cgwalters
Copy link
Collaborator

This issue is for anaconda installation only (bib anaconda-iso included),

To explain the reason for this (it's important) - it's because the way Anaconda works today we rely on the version of the ostree-container code which is embedded in the ISO (ref rhinstaller/anaconda#5197 ). So we have version skew.

But we really need to test the version-skew case in general, i.e. we want an "upgrade from stable" test that needs to gate PRs on this repo by default probably.

@henrywang
Copy link
Contributor Author

But we really need to test the version-skew case in general, i.e. we want an "upgrade from stable" test that needs to gate PRs on this repo by default probably.

Ack. It's in my list. The test flow would be start vm from fedora/cs-bootc image, and upgrade bootc with dnf upgrade (after bootc usr-overlay), then try bootc status. Adding this test into integration test is more reasonable.

utam0k pushed a commit to youki-dev/oci-spec-rs that referenced this issue Oct 8, 2024
Conceptually `Digest()` is just a parsed string wrapper. Many
cases want to get access to the full value without allocating,
and the introduction of the type was a regression from that point of
view in 0.7.

Luckily, we can change our internals in such a way that it's
safe to add the impl. Do that by holding onto the full value,
with a duplicate small boxed str for the algorithm only in
the degenerate case that it's an unknown type.

If we had this it would have made containers/bootc#800
less likely.

Signed-off-by: Colin Walters <[email protected]>
@henrywang
Copy link
Contributor Author

To explain the reason for this (it's important) - it's because the way Anaconda works today we rely on the version of the ostree-container code which is embedded in the ISO (ref rhinstaller/anaconda#5197 ). So we have version skew.

Hi @cgwalters, do you mean the ostree-rs-ext version bumped by rpm-ostree (which is used by anadonda) is different from ostree-rs-ext version bumped by bootc in system. That caused this issue. Do I understand correct? Thanks.

@cgwalters
Copy link
Collaborator

Hi @cgwalters, do you mean the ostree-rs-ext version bumped by rpm-ostree (which is used by anadonda) is different from ostree-rs-ext version bumped by bootc in system. That caused this issue. Do I understand correct? Thanks.

Yep!

henrywang added a commit to henrywang/bootc that referenced this issue Oct 11, 2024
Test comes from issue containers#800
Both rpm-ostree and bootc bump ostree-rs-ext. This test is to avoid
version-skew issue

Signed-off-by: Xiaofeng Wang <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants