Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

alpha: add rpms to sdk #85

Merged
merged 1 commit into from
Oct 30, 2023
Merged

alpha: add rpms to sdk #85

merged 1 commit into from
Oct 30, 2023

Conversation

webern
Copy link
Contributor

@webern webern commented Sep 28, 2023

Issue #, if available:

Closes #78

Description of changes:

To create an early-access version of twoliter build variant, we need an SDK that contains the Bottlerocket rpms. This script and Dockerfile creates it.

Testing

Ran these on both x86_64 and aarch64 hosts.

docker run --rm -it xyz.dkr.ecr.us-west-2.amazonaws.com/twoliter-sdk-aarch64:v1.16.0 bash
docker run --rm -it xyz.dkr.ecr.us-west-2.amazonaws.com/twoliter-sdk-x86_64:v1.16.0 bash

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@webern webern requested review from ecpullen and bcressey September 28, 2023 00:58
@webern webern marked this pull request as draft October 6, 2023 23:07
@webern
Copy link
Contributor Author

webern commented Oct 6, 2023

https://github.com/bottlerocket-os/twoliter/compare/639741fa4d0c501eca59693fc52994944ed4838c..2a54bb38cfe68fb8d359a36884e77fa726cb6aa7

Working on targeting both arches and running on both arches (multiarch-manifest)

@webern
Copy link
Contributor Author

webern commented Oct 6, 2023

local/alpha-sdk.dockerfile Outdated Show resolved Hide resolved
@webern webern force-pushed the alpha-sdk branch 3 times, most recently from a23afc7 to 7774045 Compare October 10, 2023 22:43
@webern webern marked this pull request as ready for review October 11, 2023 17:46
local/README.md Outdated Show resolved Hide resolved
local/alpha-sdk.dockerfile Show resolved Hide resolved

COPY build/rpms/ /twoliter/alpha/build/rpms/
COPY sbkeys/generate-local-sbkeys /twoliter/alpha/sbkeys/generate-local-sbkeys
COPY sbkeys/generate-aws-sbkeys /twoliter/alpha/sbkeys/generate-aws-sbkeys
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: can omit generate-aws-sbkeys because it's not invoked by any make tasks

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was required for twoliter build variant because the current build system assumes those scripts exist in tree.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should just need generate-local-sbkeys.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm also confused by this. Don't we, and won't a Twoliter Alpha customer, need the generate-aws-sbkeys script?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't we, and won't a Twoliter Alpha customer, need the generate-aws-sbkeys script?

It'll have to be invoked by hand for now since the inputs are quite complicated. Eventually I expect we'll want commands like twoliter generate repository-keys and twoliter generate secure-boot-keys that take a structured input file rather than CLI args.

Comment on lines 60 to 64
--sdk-registry The namespace or Docker registry where the SDK is found. For
example, in the following string 'public.ecr.aws' is the
registry:
--alpha-name The name of the Twoliter alpha SDK container.
public.ecr.aws/bottlerocket/bottlerocket-sdk-x86_64:v0.50.0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
--sdk-registry The namespace or Docker registry where the SDK is found. For
example, in the following string 'public.ecr.aws' is the
registry:
--alpha-name The name of the Twoliter alpha SDK container.
public.ecr.aws/bottlerocket/bottlerocket-sdk-x86_64:v0.50.0
--sdk-registry The namespace or Docker registry where the SDK is found. For
example, in the following string 'public.ecr.aws' is the
registry:
public.ecr.aws/bottlerocket/bottlerocket-sdk-x86_64:v0.50.0
--alpha-name The name of the Twoliter alpha SDK container.

[[ -n ${alpha_registry} ]] || usage_error 'required: --alpha-registry'
[[ -n ${alpha_version} ]] || usage_error 'required: --alpha-version'

variant="${variant:=aws-k8s-1.27}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's default to one with all the new feature flags enabled:

Suggested change
variant="${variant:=aws-k8s-1.27}"
variant="${variant:=aws-dev}"

local/alpha-sdk.sh Show resolved Hide resolved
docker build \
--tag "${tag}" \
--build-arg "SDK=${sdk}" \
--build-arg "HOST_GOARCH=${host_arch}" \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
--build-arg "HOST_GOARCH=${host_arch}" \
--build-arg "HOST_GOARCH=${host_arch}" \
--build-arg "TARGET_ARCH=${target_arch}" \

local/alpha-sdk.dockerfile Show resolved Hide resolved
Comment on lines +127 to +158
cargo make \
-e "BUILDSYS_VARIANT=${variant}" \
-e "BUILDSYS_ARCH=${target_arch}" \
build-variant
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we do a preemptive cargo make clean before the loop so that the build directory is in a known-empty state? Otherwise we'll end up copying artifacts in a way that won't necessarily be reproducible.

@webern
Copy link
Contributor Author

webern commented Oct 12, 2023

https://github.com/bottlerocket-os/twoliter/compare/5215482625db906536efbee42564ae831728d549..cd6fea8eaa24d204b19ac68dfc8720856bb5ffe5

  • copy only target arch rpms
  • cargo make clean (by default, but can skip)
  • default to getting the SDK version from Bottlerocket's Twoliter.toml

@webern
Copy link
Contributor Author

webern commented Oct 13, 2023

To create an early-access version of twoliter build variant, we need
an SDK that contains the Bottlerocket rpms. This script and Dockerfile
creates it.
@webern
Copy link
Contributor Author

webern commented Oct 13, 2023

@bcressey I think this has everything you requested. I had trouble globbing, but if you don't pass --skip-clean then cargo make clean runs between target arches achieving the same thing.

@ecpullen looks like there is an issue with the cargo deny install, can you check it out and maybe pin the cargo deny install to a version that isn't broken?

@webern webern merged commit 54e6a0d into bottlerocket-os:develop Oct 30, 2023
1 check passed
@webern webern deleted the alpha-sdk branch October 30, 2023 18:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

alpha: create rpm-enhanced sdk
3 participants