Add the ability to define packages to be layered.

[jmarrero]

As part of the layering initiative we would like to be able to declare packages as part of the schema. The scope of this issue is the build time. For that purpose butane will generate an tree file in the ignition config that can be consumed with coreos/rpm-ostree#3340

[lucab]

This is coming from coreos/fedora-coreos-tracker#1054.

#81 has some prior discussion and examples on it.

[bgilbert]

If we want to eventually support this in both the fcos and openshift specs, note that we'll need to get openshift/machine-config-operator#2675 (or something like it) unstuck. openshift specs derive their Ignition spec version from the fcos version they inherit from, and openshift stable specs are currently pinned to fcos 1.3.0 because the MCO currently maxes out at Ignition spec 3.2.0.

[jmarrero]

@jlebon during your demo today your example was dropping a ltrace.yaml file. I assume we don't want a file per package right? @cgwalters I think suggested a SHA of the file. We could also use a UUID.yaml as filename. If I just output to /etc/rpm-ostree/origin.d/SHAorUUID.yaml that is what would be the "server side" expected file to output on #304 ?

packages:
   - strace
   - usbguard

Also, unless I misunderstood you suggested that we did not need the systemd service with the treefile approach on the client side. But even if we are adding a treefile using ignition --live-apply we still need to call rpm-ostree after boot to process the treefile so we still need a service no?

[jlebon]

@jlebon during your demo today your example was dropping a ltrace.yaml file. I assume we don't want a file per package right? @cgwalters I think suggested a SHA of the file. We could also use a UUID.yaml as filename.

No strong opinion on filename. Indeed we should just combine them in a single file instead of separate ones per package. I'd go for something like extensions.yaml to keep it simple and have some traceability. Might also be worth including a YAML comment at the top of the file to say it was generated by Butane.

If I just output to /etc/rpm-ostree/origin.d/SHAorUUID.yaml that is what would be the "server side" expected file to output on #304 ?

packages:
   - strace
   - usbguard

LGTM

Also, unless I misunderstood you suggested that we did not need the systemd service with the treefile approach on the client side. But even if we are adding a treefile using ignition --live-apply we still need to call rpm-ostree after boot to process the treefile so we still need a service no?

I'd keep the firstboot case scoped out of this for now. I'd like to investigate using coreos/rpm-ostree#3006 for that, though even if we do go with a systemd service approach, I think it'd make more sense to have that service live in f-c-c (or possibly even rpm-ostree - see coreos/rpm-ostree#2326 (comment)).

The idea here with the client treefiles is that it's a new rpm-ostree thing. Butane just happens to be the first one to use it, but there will be other use cases, and not all may have Butane sugar for it.

[jmarrero]

@bgilbert you got a preference on what filename butane will spit out? I got an example with SHA here: fd9e08a and a commit with UUID which I like the filename more but adds a dep. : fbc041f

[bgilbert]

The UUID one generates non-reproducible Ignition configs, which we haven't done so far, so I think we should avoid it. The hash one is okay, but: is it important that we add uniqueness to the filename?

[jmarrero]

The idea is that we could have multiple files added by different layers. For example person1 created the first layered image which will have that file in place. Second person creates a new image using the first one as base and also layers packages. So we would have two files in the dropins directory of the second image.

[bgilbert]

Ah, right. SHA seems like a good choice then.

[cgwalters]

If we want to eventually support this in both the fcos and openshift specs, note that we'll need to get openshift/machine-config-operator#2675 (or something like it) unstuck.

This is a very interesting topic. Ideally, we avoid generating MachineConfig from this. I'd say a big goal of this effort is to get out of having the MCO reimplement a subset of Ignition. The expected flow is written down here https://github.com/mkenigs/coreos-derive/tree/main/prototype
(There's some diagrams that need publishing)

But basically we run ignition-liveapply as part of a container build.

Now...probably where this should be going is that we have a CRD that's just a wrapper for butane. And the MCO instead vendors butane (which actually it already does) but the role of that is simply to store it in etcd and pass it into a build.

Definitely needs some thought and care though.

[bgilbert]

Ideally, we avoid generating MachineConfig from this. I'd say a big goal of this effort is to get out of having the MCO reimplement a subset of Ignition.

Right. The MCO version dependency is a problem only during the transition period, and only if we want the same openshift stable spec to be able to generate MachineConfigs and to use the extension sugar. If we don't need that, it's a non-issue.

[jmarrero]

#304 & #316 solve this issue.