Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

main: Also automatically remount rw /sysroot for ostree pull etc. #2027

Merged

Conversation

cgwalters
Copy link
Member

See coreos/fedora-coreos-tracker#343
When we added the read-only sysroot support it broke using "raw"
ostree pull and ostree refs --create and all of the core repo
CLIs that just operate on a repo and not a sysroot.

Fixing this is a bit ugly as it "layer crosses" things even more.
Extract a helper function that works in both cases.

@cgwalters cgwalters force-pushed the pull-auto-mnt-namespace branch from 229ddad to 2749887 Compare March 13, 2020 00:52
@cgwalters cgwalters changed the title WIP: main: Also automatically remount rw /sysroot for ostree pull etc. main: Also automatically remount rw /sysroot for ostree pull etc. Mar 13, 2020
@cgwalters
Copy link
Member Author

Lifted WIP, this is tested and working.

@jlebon
Copy link
Member

jlebon commented Mar 13, 2020

Can we do #2029 first? That way we can be sure that the same kola tests that previously failed now pass.

@jlebon
Copy link
Member

jlebon commented Mar 13, 2020

Patch looks reasonable to me otherwise!
/approve

See coreos/fedora-coreos-tracker#343
When we added the read-only sysroot support it broke using "raw"
`ostree pull` and `ostree refs --create` and all of the core repo
CLIs that just operate on a repo and not a sysroot.

Fixing this is a bit ugly as it "layer crosses" things even more.
Extract a helper function that works in both cases.
@cgwalters cgwalters force-pushed the pull-auto-mnt-namespace branch from 2749887 to 6a27dff Compare March 13, 2020 20:53
@jlebon
Copy link
Member

jlebon commented Mar 13, 2020

Nice, ostree failures are now passing! Though hmm, looks like rpm-ostree rollback and rpm-ostree install in some cases are breaking on EROFS. Will look into this! If it's only rpm-ostree that needs fixes, we'll have to force merge this one too to unblock us.

@cgwalters
Copy link
Member Author

The problem is now interlock with coreos/rpm-ostree#2015

@jlebon
Copy link
Member

jlebon commented Mar 14, 2020

Right, it would need the new rpm-ostree to pass.

Anyway, I've confirmed locally that this patch + git master rpm-ostree is green:

kola -b fcos -p qemu-unpriv --cosa-build builds/31.20200313.dev.1/x86_64/meta.json --output-dir tmp/kola run
=== RUN   coreos.tls.fetch-urls
=== RUN   rhcos.selinux.boolean.persist
=== RUN   fcos.internet
=== RUN   coreos.selinux.boolean
=== RUN   ostree.unlock
=== RUN   coreos.ignition.groups
=== RUN   podman.base
=== RUN   rpmostree.status
=== RUN   podman.network-single
=== RUN   fcos.users.shells
=== RUN   podman.workflow
=== RUN   coreos.ignition.resource.remote
=== RUN   coreos.auth.verify
=== RUN   ostree.hotfix
=== RUN   rpmostree.upgrade-rollback
=== RUN   coreos.ignition.once
=== RUN   fcos.filesystem
=== RUN   coreos.ignition.sethostname
=== RUN   coreos.ignition.v2.users
=== RUN   basic
=== RUN   ostree.remote
=== RUN   rootfs.uuid
=== RUN   coreos.ignition.mount.disks
=== RUN   rpmostree.install-uninstall
=== RUN   coreos.ignition.mount.partitions
=== RUN   coreos.selinux.enforce
=== RUN   systemd.sysusers.gshadow
--- PASS: coreos.tls.fetch-urls (27.41s)
=== RUN   rpmostree.upgrade-rollback/upgrade
=== RUN   rpmostree.upgrade-rollback/rollback
--- PASS: rpmostree.upgrade-rollback (74.70s)
    --- PASS: rpmostree.upgrade-rollback/upgrade (24.23s)
    --- PASS: rpmostree.upgrade-rollback/rollback (24.18s)
--- PASS: systemd.sysusers.gshadow (26.54s)
--- PASS: coreos.selinux.enforce (45.67s)
--- PASS: coreos.ignition.mount.partitions (44.84s)
=== RUN   rpmostree.install-uninstall/install
=== RUN   rpmostree.install-uninstall/uninstall
--- PASS: rpmostree.install-uninstall (160.08s)
    --- PASS: rpmostree.install-uninstall/install (108.35s)
    --- PASS: rpmostree.install-uninstall/uninstall (25.21s)
--- PASS: coreos.ignition.mount.disks (55.07s)
=== RUN   rootfs.uuid/RandomUUID
--- PASS: rootfs.uuid (26.63s)
    --- PASS: rootfs.uuid/RandomUUID (0.17s)
=== RUN   ostree.remote/add
=== RUN   ostree.remote/list
=== RUN   ostree.remote/show-url
=== RUN   ostree.remote/refs
=== RUN   ostree.remote/summary
=== RUN   ostree.remote/delete
--- PASS: ostree.remote (32.27s)
    --- PASS: ostree.remote/add (0.37s)
    --- PASS: ostree.remote/list (0.14s)
    --- PASS: ostree.remote/show-url (0.14s)
    --- PASS: ostree.remote/refs (1.93s)
    --- PASS: ostree.remote/summary (2.85s)
    --- PASS: ostree.remote/delete (0.50s)
=== RUN   basic/DbusPerms
=== RUN   basic/NetworkScripts
=== RUN   basic/ServicesActive
=== RUN   basic/ReadOnly
=== RUN   basic/Useradd
=== RUN   basic/MachineID
=== RUN   basic/PortSSH
--- PASS: basic (28.13s)
    --- PASS: basic/DbusPerms (0.24s)
    --- PASS: basic/NetworkScripts (0.14s)
    --- PASS: basic/ServicesActive (0.13s)
    --- PASS: basic/ReadOnly (0.12s)
    --- PASS: basic/Useradd (0.95s)
    --- PASS: basic/MachineID (0.14s)
    --- PASS: basic/PortSSH (0.13s)
--- PASS: coreos.ignition.v2.users (26.88s)
--- PASS: coreos.ignition.sethostname (25.87s)
=== RUN   fcos.filesystem/suid
=== RUN   fcos.filesystem/sgid
=== RUN   fcos.filesystem/writablefiles
=== RUN   fcos.filesystem/writabledirs
=== RUN   fcos.filesystem/stickydirs
=== RUN   fcos.filesystem/blacklist
--- PASS: fcos.filesystem (27.43s)
    --- PASS: fcos.filesystem/suid (0.48s)
    --- PASS: fcos.filesystem/sgid (0.26s)
    --- PASS: fcos.filesystem/writablefiles (0.26s)
    --- PASS: fcos.filesystem/writabledirs (0.22s)
    --- PASS: fcos.filesystem/stickydirs (0.24s)
    --- PASS: fcos.filesystem/blacklist (0.27s)
--- PASS: coreos.ignition.once (45.61s)
--- PASS: rpmostree.status (27.09s)
=== RUN   ostree.hotfix/unlock
=== RUN   ostree.hotfix/install
=== RUN   ostree.hotfix/uninstall
=== RUN   ostree.hotfix/persist
=== RUN   ostree.hotfix/rollback
--- PASS: ostree.hotfix (73.43s)
    --- PASS: ostree.hotfix/unlock (5.58s)
    --- PASS: ostree.hotfix/install (1.05s)
    --- PASS: ostree.hotfix/uninstall (0.59s)
    --- PASS: ostree.hotfix/persist (19.65s)
    --- PASS: ostree.hotfix/rollback (20.34s)
--- PASS: coreos.auth.verify (25.72s)
--- PASS: coreos.ignition.resource.remote (26.17s)
=== RUN   podman.workflow/run
=== RUN   podman.workflow/exec
=== RUN   podman.workflow/stop
=== RUN   podman.workflow/remove
=== RUN   podman.workflow/delete
--- PASS: podman.workflow (46.89s)
    --- PASS: podman.workflow/run (18.79s)
            cluster.go:122: Trying to pull docker.io/library/nginx...
            cluster.go:122: Getting image source signatures
            cluster.go:122: Copying blob sha256:a616aa3b0bf26c4eaf2609f5647e27480f834b8a84456f9aaff8cc03eba0cbbf
            cluster.go:122: Copying blob sha256:68ced04f60ab5c7a5f1d0b0b4e7572c5a4c8cce44866513d30d9df1a15277d6b
            cluster.go:122: Copying blob sha256:28252775b2955bcd1a204d2cb81e5b3696533ecf85bd50cd2edb78780d4da686
            cluster.go:122: Copying config sha256:6678c7c2e56c970388f8d5a398aa30f2ab60e85f20165e101053c3d3a11e6663
            cluster.go:122: Writing manifest to image destination
            cluster.go:122: Storing signatures
    --- PASS: podman.workflow/exec (0.34s)
    --- PASS: podman.workflow/stop (1.26s)
    --- PASS: podman.workflow/remove (0.53s)
    --- PASS: podman.workflow/delete (0.82s)
            cluster.go:122: grep: docker.io/library/nginx:latest: No such file or directory
            cluster.go:122: bash: line 1: Deleted:: command not found
--- PASS: fcos.users.shells (25.77s)
--- PASS: podman.network-single (220.48s)
        cluster.go:122: Getting image source signatures
        cluster.go:122: Copying blob sha256:d1088dfb956ce3f097c6c4e5e947e09eeca8bcde127a9a16174e8b5272f9fd80
        cluster.go:122: Copying config sha256:e0c1f89a381cc42257620c5ef199bdf94e112a65c4ee88fba234bae532db72a1
        cluster.go:122: Writing manifest to image destination
        cluster.go:122: Storing signatures
=== RUN   ostree.unlock/unlock
=== RUN   ostree.unlock/install
=== RUN   ostree.unlock/uninstall
=== RUN   ostree.unlock/discard
--- PASS: ostree.unlock (48.92s)
    --- PASS: ostree.unlock/unlock (0.71s)
    --- PASS: ostree.unlock/install (0.98s)
    --- PASS: ostree.unlock/uninstall (0.60s)
    --- PASS: ostree.unlock/discard (20.45s)
=== RUN   podman.base/info
=== RUN   podman.base/resources
--- PASS: podman.base (39.23s)
    --- PASS: podman.base/info (0.85s)
    --- PASS: podman.base/resources (12.67s)
            cluster.go:122: Getting image source signatures
            cluster.go:122: Copying blob sha256:5b44a56794056ee1922f9772d16c3414651fd5506b126625c269c3ca0b09c8ac
            cluster.go:122: Copying config sha256:4b2256c96f27b3bb83f61bbd795fe0ec85bbef6ee498a4a11e443a3d3f251b97
            cluster.go:122: Writing manifest to image destination
            cluster.go:122: Storing signatures
            cluster.go:122: Your kernel does not support Block I/O weight or the cgroup is not mounted. Weight di.
--- PASS: coreos.ignition.groups (26.90s)
=== RUN   fcos.internet/PodmanEcho
=== RUN   fcos.internet/PodmanWgetHead
--- PASS: fcos.internet (30.99s)
    --- PASS: fcos.internet/PodmanEcho (3.73s)
    --- PASS: fcos.internet/PodmanWgetHead (0.95s)
--- PASS: coreos.selinux.boolean (44.80s)
--- PASS: rhcos.selinux.boolean.persist (48.95s)
PASS, output in tmp/kola
qemu path: builds/31.20200313.dev.1/x86_64/fedora-coreos-31.20200313.dev.1-qemu.x86_64.qcow2
kola -b fcos -p qemu-unpriv --cosa-build builds/31.20200313.dev.1/x86_64/meta.json --output-dir tmp/kola-upgrade e
2020-03-14T00:24:02Z cli: Started logging at level INFO
fedora-coreos-31.20200310.20.0-qemu.x86_64.qcow2: [============] 476 MB/476 MB
=== RUN   fcos.upgrade.basic
=== RUN   fcos.upgrade.basic/setup
=== RUN   fcos.upgrade.basic/upgrade-from-previous
=== RUN   fcos.upgrade.basic/upgrade-from-current
--- PASS: fcos.upgrade.basic (110.37s)
    --- PASS: fcos.upgrade.basic/setup (29.24s)
    --- PASS: fcos.upgrade.basic/upgrade-from-previous (30.82s)
    --- PASS: fcos.upgrade.basic/upgrade-from-current (23.59s)
PASS, output in tmp/kola-upgrade

/override continuous-integration/jenkins/pr-merge
/lgtm

@openshift-ci-robot
Copy link
Collaborator

@jlebon: Overrode contexts on behalf of jlebon: continuous-integration/jenkins/pr-merge

In response to this:

Right, it would need the new rpm-ostree to pass.

Anyway, I've confirmed locally that this patch + git master rpm-ostree is green:

kola -b fcos -p qemu-unpriv --cosa-build builds/31.20200313.dev.1/x86_64/meta.json --output-dir tmp/kola run
=== RUN   coreos.tls.fetch-urls
=== RUN   rhcos.selinux.boolean.persist
=== RUN   fcos.internet
=== RUN   coreos.selinux.boolean
=== RUN   ostree.unlock
=== RUN   coreos.ignition.groups
=== RUN   podman.base
=== RUN   rpmostree.status
=== RUN   podman.network-single
=== RUN   fcos.users.shells
=== RUN   podman.workflow
=== RUN   coreos.ignition.resource.remote
=== RUN   coreos.auth.verify
=== RUN   ostree.hotfix
=== RUN   rpmostree.upgrade-rollback
=== RUN   coreos.ignition.once
=== RUN   fcos.filesystem
=== RUN   coreos.ignition.sethostname
=== RUN   coreos.ignition.v2.users
=== RUN   basic
=== RUN   ostree.remote
=== RUN   rootfs.uuid
=== RUN   coreos.ignition.mount.disks
=== RUN   rpmostree.install-uninstall
=== RUN   coreos.ignition.mount.partitions
=== RUN   coreos.selinux.enforce
=== RUN   systemd.sysusers.gshadow
--- PASS: coreos.tls.fetch-urls (27.41s)
=== RUN   rpmostree.upgrade-rollback/upgrade
=== RUN   rpmostree.upgrade-rollback/rollback
--- PASS: rpmostree.upgrade-rollback (74.70s)
   --- PASS: rpmostree.upgrade-rollback/upgrade (24.23s)
   --- PASS: rpmostree.upgrade-rollback/rollback (24.18s)
--- PASS: systemd.sysusers.gshadow (26.54s)
--- PASS: coreos.selinux.enforce (45.67s)
--- PASS: coreos.ignition.mount.partitions (44.84s)
=== RUN   rpmostree.install-uninstall/install
=== RUN   rpmostree.install-uninstall/uninstall
--- PASS: rpmostree.install-uninstall (160.08s)
   --- PASS: rpmostree.install-uninstall/install (108.35s)
   --- PASS: rpmostree.install-uninstall/uninstall (25.21s)
--- PASS: coreos.ignition.mount.disks (55.07s)
=== RUN   rootfs.uuid/RandomUUID
--- PASS: rootfs.uuid (26.63s)
   --- PASS: rootfs.uuid/RandomUUID (0.17s)
=== RUN   ostree.remote/add
=== RUN   ostree.remote/list
=== RUN   ostree.remote/show-url
=== RUN   ostree.remote/refs
=== RUN   ostree.remote/summary
=== RUN   ostree.remote/delete
--- PASS: ostree.remote (32.27s)
   --- PASS: ostree.remote/add (0.37s)
   --- PASS: ostree.remote/list (0.14s)
   --- PASS: ostree.remote/show-url (0.14s)
   --- PASS: ostree.remote/refs (1.93s)
   --- PASS: ostree.remote/summary (2.85s)
   --- PASS: ostree.remote/delete (0.50s)
=== RUN   basic/DbusPerms
=== RUN   basic/NetworkScripts
=== RUN   basic/ServicesActive
=== RUN   basic/ReadOnly
=== RUN   basic/Useradd
=== RUN   basic/MachineID
=== RUN   basic/PortSSH
--- PASS: basic (28.13s)
   --- PASS: basic/DbusPerms (0.24s)
   --- PASS: basic/NetworkScripts (0.14s)
   --- PASS: basic/ServicesActive (0.13s)
   --- PASS: basic/ReadOnly (0.12s)
   --- PASS: basic/Useradd (0.95s)
   --- PASS: basic/MachineID (0.14s)
   --- PASS: basic/PortSSH (0.13s)
--- PASS: coreos.ignition.v2.users (26.88s)
--- PASS: coreos.ignition.sethostname (25.87s)
=== RUN   fcos.filesystem/suid
=== RUN   fcos.filesystem/sgid
=== RUN   fcos.filesystem/writablefiles
=== RUN   fcos.filesystem/writabledirs
=== RUN   fcos.filesystem/stickydirs
=== RUN   fcos.filesystem/blacklist
--- PASS: fcos.filesystem (27.43s)
   --- PASS: fcos.filesystem/suid (0.48s)
   --- PASS: fcos.filesystem/sgid (0.26s)
   --- PASS: fcos.filesystem/writablefiles (0.26s)
   --- PASS: fcos.filesystem/writabledirs (0.22s)
   --- PASS: fcos.filesystem/stickydirs (0.24s)
   --- PASS: fcos.filesystem/blacklist (0.27s)
--- PASS: coreos.ignition.once (45.61s)
--- PASS: rpmostree.status (27.09s)
=== RUN   ostree.hotfix/unlock
=== RUN   ostree.hotfix/install
=== RUN   ostree.hotfix/uninstall
=== RUN   ostree.hotfix/persist
=== RUN   ostree.hotfix/rollback
--- PASS: ostree.hotfix (73.43s)
   --- PASS: ostree.hotfix/unlock (5.58s)
   --- PASS: ostree.hotfix/install (1.05s)
   --- PASS: ostree.hotfix/uninstall (0.59s)
   --- PASS: ostree.hotfix/persist (19.65s)
   --- PASS: ostree.hotfix/rollback (20.34s)
--- PASS: coreos.auth.verify (25.72s)
--- PASS: coreos.ignition.resource.remote (26.17s)
=== RUN   podman.workflow/run
=== RUN   podman.workflow/exec
=== RUN   podman.workflow/stop
=== RUN   podman.workflow/remove
=== RUN   podman.workflow/delete
--- PASS: podman.workflow (46.89s)
   --- PASS: podman.workflow/run (18.79s)
           cluster.go:122: Trying to pull docker.io/library/nginx...
           cluster.go:122: Getting image source signatures
           cluster.go:122: Copying blob sha256:a616aa3b0bf26c4eaf2609f5647e27480f834b8a84456f9aaff8cc03eba0cbbf
           cluster.go:122: Copying blob sha256:68ced04f60ab5c7a5f1d0b0b4e7572c5a4c8cce44866513d30d9df1a15277d6b
           cluster.go:122: Copying blob sha256:28252775b2955bcd1a204d2cb81e5b3696533ecf85bd50cd2edb78780d4da686
           cluster.go:122: Copying config sha256:6678c7c2e56c970388f8d5a398aa30f2ab60e85f20165e101053c3d3a11e6663
           cluster.go:122: Writing manifest to image destination
           cluster.go:122: Storing signatures
   --- PASS: podman.workflow/exec (0.34s)
   --- PASS: podman.workflow/stop (1.26s)
   --- PASS: podman.workflow/remove (0.53s)
   --- PASS: podman.workflow/delete (0.82s)
           cluster.go:122: grep: docker.io/library/nginx:latest: No such file or directory
           cluster.go:122: bash: line 1: Deleted:: command not found
--- PASS: fcos.users.shells (25.77s)
--- PASS: podman.network-single (220.48s)
       cluster.go:122: Getting image source signatures
       cluster.go:122: Copying blob sha256:d1088dfb956ce3f097c6c4e5e947e09eeca8bcde127a9a16174e8b5272f9fd80
       cluster.go:122: Copying config sha256:e0c1f89a381cc42257620c5ef199bdf94e112a65c4ee88fba234bae532db72a1
       cluster.go:122: Writing manifest to image destination
       cluster.go:122: Storing signatures
=== RUN   ostree.unlock/unlock
=== RUN   ostree.unlock/install
=== RUN   ostree.unlock/uninstall
=== RUN   ostree.unlock/discard
--- PASS: ostree.unlock (48.92s)
   --- PASS: ostree.unlock/unlock (0.71s)
   --- PASS: ostree.unlock/install (0.98s)
   --- PASS: ostree.unlock/uninstall (0.60s)
   --- PASS: ostree.unlock/discard (20.45s)
=== RUN   podman.base/info
=== RUN   podman.base/resources
--- PASS: podman.base (39.23s)
   --- PASS: podman.base/info (0.85s)
   --- PASS: podman.base/resources (12.67s)
           cluster.go:122: Getting image source signatures
           cluster.go:122: Copying blob sha256:5b44a56794056ee1922f9772d16c3414651fd5506b126625c269c3ca0b09c8ac
           cluster.go:122: Copying config sha256:4b2256c96f27b3bb83f61bbd795fe0ec85bbef6ee498a4a11e443a3d3f251b97
           cluster.go:122: Writing manifest to image destination
           cluster.go:122: Storing signatures
           cluster.go:122: Your kernel does not support Block I/O weight or the cgroup is not mounted. Weight di.
--- PASS: coreos.ignition.groups (26.90s)
=== RUN   fcos.internet/PodmanEcho
=== RUN   fcos.internet/PodmanWgetHead
--- PASS: fcos.internet (30.99s)
   --- PASS: fcos.internet/PodmanEcho (3.73s)
   --- PASS: fcos.internet/PodmanWgetHead (0.95s)
--- PASS: coreos.selinux.boolean (44.80s)
--- PASS: rhcos.selinux.boolean.persist (48.95s)
PASS, output in tmp/kola
qemu path: builds/31.20200313.dev.1/x86_64/fedora-coreos-31.20200313.dev.1-qemu.x86_64.qcow2
kola -b fcos -p qemu-unpriv --cosa-build builds/31.20200313.dev.1/x86_64/meta.json --output-dir tmp/kola-upgrade e
2020-03-14T00:24:02Z cli: Started logging at level INFO
fedora-coreos-31.20200310.20.0-qemu.x86_64.qcow2: [============] 476 MB/476 MB
=== RUN   fcos.upgrade.basic
=== RUN   fcos.upgrade.basic/setup
=== RUN   fcos.upgrade.basic/upgrade-from-previous
=== RUN   fcos.upgrade.basic/upgrade-from-current
--- PASS: fcos.upgrade.basic (110.37s)
   --- PASS: fcos.upgrade.basic/setup (29.24s)
   --- PASS: fcos.upgrade.basic/upgrade-from-previous (30.82s)
   --- PASS: fcos.upgrade.basic/upgrade-from-current (23.59s)
PASS, output in tmp/kola-upgrade

/override continuous-integration/jenkins/pr-merge
/lgtm

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cgwalters, jlebon

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-merge-robot openshift-merge-robot merged commit 5164dcb into ostreedev:master Mar 14, 2020
@jlebon
Copy link
Member

jlebon commented Mar 14, 2020

#2030

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants