Skip to content

Commit

Permalink
Merge branch 'main' into patch-1
Browse files Browse the repository at this point in the history
  • Loading branch information
@AidanDelaney
AidanDelaney authored Nov 30, 2023
2 parents f35db36 + 7c5f256 commit 071a5b9
Show file tree
Hide file tree
Showing 15 changed files with 381 additions and 621 deletions.
161 changes: 56 additions & 105 deletions content/docs/buildpack-author-guide/create-buildpack/adding-bill-of-materials.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,43 +15,39 @@ One of the benefits of buildpacks is they can also populate the app image with m

You can find some of this information using `pack` via its `inspect-image` command. The bill-of-materials information will be available using `pack sbom download`.

<!-- test:exec -->
```bash
pack inspect-image test-ruby-app
pack inspect-image test-node-js-app
```
<!--+- "{{execute}}"+-->
You should see the following:

<!-- test:assert=contains;ignore-lines=... -->
```text
Run Images:
cnbs/sample-base-run:jammy
...
Buildpacks:
ID VERSION HOMEPAGE
examples/ruby 0.0.1 -
examples/node-js 0.0.1 -
Processes:
TYPE SHELL COMMAND ARGS WORK DIR
web (default) bash bundle exec ruby app.rb /workspace
worker bash bundle exec ruby worker.rb /workspace
web (default) bash node-js app.js /workspace
```

Apart from the above standard metadata, buildpacks can also populate information about the dependencies they have provided in form of a `Bill-of-Materials`. Let's see how we can use this to populate information about the version of `ruby` that was installed in the output app image.
Apart from the above standard metadata, buildpacks can also populate information about the dependencies they have provided in form of a `Bill-of-Materials`. Let's see how we can use this to populate information about the version of `node-js` that was installed in the output app image.

To add the `ruby` version to the output of `pack download sbom`, we will have to provide a [Software `Bill-of-Materials`](https://en.wikipedia.org/wiki/Software_bill_of_materials) (`SBOM`) containing this information. There are three "standard" ways to report SBOM data. You'll need to choose to use one of [CycloneDX](https://cyclonedx.org/), [SPDX](https://spdx.dev/) or [Syft](https://github.com/anchore/syft) update the `ruby.sbom.<ext>` (where `<ext>` is the extension appropriate for your SBOM standard, one of `cdx.json`, `spdx.json` or `syft.json`) at the end of your `build` script. Discussion of which SBOM format to choose is outside the scope of this tutorial, but we will note that the SBOM format you choose to use is likely to be the output format of any SBOM scanner (eg: [`syft cli`](https://github.com/anchore/syft)) you might choose to use. In this example we will use the CycloneDX json format.
To add the `node-js` version to the output of `pack download sbom`, we will have to provide a [Software `Bill-of-Materials`](https://en.wikipedia.org/wiki/Software_bill_of_materials) (`SBOM`) containing this information. There are three "standard" ways to report SBOM data. You'll need to choose to use one of [CycloneDX](https://cyclonedx.org/), [SPDX](https://spdx.dev/) or [Syft](https://github.com/anchore/syft) update the `node-js.sbom.<ext>` (where `<ext>` is the extension appropriate for your SBOM standard, one of `cdx.json`, `spdx.json` or `syft.json`) at the end of your `build` script. Discussion of which SBOM format to choose is outside the scope of this tutorial, but we will note that the SBOM format you choose to use is likely to be the output format of any SBOM scanner (eg: [`syft cli`](https://github.com/anchore/syft)) you might choose to use. In this example we will use the CycloneDX json format.

First, annotate the `buildpack.toml` to specify that it emits CycloneDX:

<!-- test:file=ruby-buildpack/buildpack.toml -->
<!-- test:file=node-js-buildpack/buildpack.toml -->
```toml
# Buildpack API version
api = "0.8"

# Buildpack ID and metadata
[buildpack]
id = "examples/ruby"
id = "examples/node-js"
version = "0.0.1"
sbom-formats = [ "application/vnd.cyclonedx+json" ]

Expand All @@ -69,179 +65,134 @@ Then, in our buildpack implementation we will generate the necessary SBOM metada
```bash
# ...

# Append a Bill-of-Materials containing metadata about the provided ruby version
cat >> "$layersdir/ruby.sbom.cdx.json" << EOL
# Append a Bill-of-Materials containing metadata about the provided node-js version
cat >> "${layersdir}/node-js.sbom.cdx.json" << EOL
{
"bomFormat": "CycloneDX",
"specVersion": "1.4",
"version": 1,
"components": [
{
"type": "library",
"name": "ruby",
"version": "$ruby_version"
"name": "node-js",
"version": "${node_js_version}"
}
]
}
EOL
```

We can also add an SBOM entry for each dependency listed in `Gemfile.lock`. Here we use `jq` to add a new record to the `components` array in `bundler.sbom.cdx.json`:
We can also add an SBOM entry for each dependency listed in `package.json`. Here we use `jq` to add a new record to the `components` array in `bundler.sbom.cdx.json`:

```bash
crubybom="${layersdir}/ruby.sbom.cdx.json"
cat >> ${rubybom} << EOL
node-jsbom="${layersdir}/node-js.sbom.cdx.json"
cat >> ${node-jsbom} << EOL
{
"bomFormat": "CycloneDX",
"specVersion": "1.4",
"version": 1,
"components": [
{
"type": "library",
"name": "ruby",
"version": "$ruby_version"
"name": "node-js",
"version": "${node_js_version}"
}
]
}
EOL
if [[ -f Gemfile.lock ]] ; then
for gem in $(gem dep -q | grep ^Gem | sed 's/^Gem //')
do
version=${gem##*-}
name=${gem%-${version}}
DEP=$(jq --arg name "${name}" --arg version "${version}" \
'.components[.components| length] |= . + {"type": "library", "name": $name, "version": $version}' \
"${rubybom}")
echo ${DEP} > "${rubybom}"
done
fi
```

Your `ruby-buildpack/bin/build`<!--+"{{open}}"+--> script should look like the following:
Your `node-js-buildpack/bin/build`<!--+"{{open}}"+--> script should look like the following:

<!-- test:file=ruby-buildpack/bin/build -->
<!-- test:file=node-js-buildpack/bin/build -->
```bash
#!/usr/bin/env bash
set -eo pipefail

echo "---> Ruby Buildpack"
echo "---> NodeJS Buildpack"

# ======= MODIFIED =======
# 1. GET ARGS
layersdir=$1
plan=$3

# 2. CREATE THE LAYER DIRECTORY
rubylayer="$layersdir"/ruby
mkdir -p "$rubylayer"

# 3. DOWNLOAD RUBY
ruby_version=$(cat "$plan" | yj -t | jq -r '.entries[] | select(.name == "ruby") | .metadata.version')
echo "---> Downloading and extracting Ruby $ruby_version"
ruby_url=https://s3-external-1.amazonaws.com/heroku-buildpack-ruby/heroku-22/ruby-$ruby_version.tgz
wget -q -O - "$ruby_url" | tar -xzf - -C "$rubylayer"

# 4. MAKE RUBY AVAILABLE DURING LAUNCH
echo -e '[types]\nlaunch = true' > "$layersdir/ruby.toml"

# 5. MAKE RUBY AVAILABLE TO THIS SCRIPT
export PATH="$rubylayer"/bin:$PATH
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH:+${LD_LIBRARY_PATH}:}"$rubylayer/lib"

# 6. INSTALL GEMS
# Compares previous Gemfile.lock checksum to the current Gemfile.lock
bundlerlayer="$layersdir/bundler"
local_bundler_checksum=$((sha256sum Gemfile.lock || echo 'DOES_NOT_EXIST') | cut -d ' ' -f 1)
remote_bundler_checksum=$(cat "$layersdir/bundler.toml" | yj -t | jq -r .metadata.checksum 2>/dev/null || echo 'DOES_NOT_EXIST')
# Always set the types table so that we re-use the appropriate layers
echo -e '[types]\ncache = true\nlaunch = true' >> "$layersdir/bundler.toml"
if [[ -f Gemfile.lock && $local_bundler_checksum == $remote_bundler_checksum ]] ; then
# Determine if no gem dependencies have changed, so it can reuse existing gems without running bundle install
echo "---> Reusing gems"
bundle config --local path "$bundlerlayer" >/dev/null
bundle config --local bin "$bundlerlayer/bin" >/dev/null
node_js_layer="${layersdir}"/node-js
mkdir -p "${node_js_layer}"

# 3. DOWNLOAD node-js
default_node_js_version="18.18.1"
node_js_version=$(cat "$plan" | yj -t | jq -r '.entries[] | select(.name == "node-js") | .metadata.version' || echo ${default_node_js_version})
node_js_url=https://nodejs.org/dist/v${node_js_version}/node-v${node_js_version}-linux-x64.tar.xz
remote_nodejs_version=$(cat "${layersdir}/node-js.toml" 2> /dev/null | yj -t | jq -r .metadata.nodejs_version 2>/dev/null || echo 'NOT FOUND')
if [[ "${node_js_url}" != *"${remote_nodejs_version}"* ]] ; then
echo "-----> Downloading and extracting NodeJS"
wget -q -O - "${node_js_url}" | tar -xJf - --strip-components 1 -C "${node_js_layer}"
else
# Determine if there has been a gem dependency change and install new gems to the bundler layer; re-using existing and un-changed gems
echo "---> Installing gems"
mkdir -p "$bundlerlayer"
cat >> "$layersdir/bundler.toml" << EOL
echo "-----> Reusing NodeJS"
fi

# 4. MAKE node-js AVAILABLE DURING LAUNCH and CACHE the LAYER
cat > "${layersdir}/node-js.toml" << EOL
[types]
cache = true
launch = true
[metadata]
checksum = "$local_bundler_checksum"
nodejs_version = "${node_js_version}"
EOL
bundle config set --local path "$bundlerlayer" && bundle install && bundle binstubs --all --path "$bundlerlayer/bin"

fi
# 7. SET DEFAULT START COMMAND
cat > "$layersdir/launch.toml" << EOL
# our web process
# 5. SET DEFAULT START COMMAND
cat >> "${layersdir}/launch.toml" << EOL
[[processes]]
type = "web"
command = "bundle exec ruby app.rb"
command = "node app.js"
default = true
# our worker process
[[processes]]
type = "worker"
command = "bundle exec ruby worker.rb"
EOL

# ========== ADDED ===========
# 8. ADD A SBOM
rubybom="${layersdir}/ruby.sbom.cdx.json"
cat >> ${rubybom} << EOL
# 6. ADD A SBOM
node_jsbom="${layersdir}/node-js.sbom.cdx.json"
cat >> ${node_jsbom} << EOL
{
"bomFormat": "CycloneDX",
"specVersion": "1.4",
"version": 1,
"components": [
{
"type": "library",
"name": "ruby",
"version": "$ruby_version"
"name": "node-js",
"version": "${node_js_version}"
}
]
}
EOL
if [[ -f Gemfile.lock ]] ; then
for gem in $(gem dep -q | grep ^Gem | sed 's/^Gem //')
do
version=${gem##*-}
name=${gem%-${version}}
DEP=$(jq --arg name "${name}" --arg version "${version}" \
'.components[.components| length] |= . + {"type": "library", "name": $name, "version": $version}' \
"${rubybom}")
echo ${DEP} > "${rubybom}"
done
fi
```

Then rebuild your app using the updated buildpack:

<!-- test:exec -->
```bash
pack build test-ruby-app --path ./ruby-sample-app --buildpack ./ruby-buildpack
pack build test-node-js-app --path ./node-js-sample-app --buildpack ./node-js-buildpack
```
<!--+- "{{execute}}"+-->

Viewing your bill-of-materials requires extracting (or `download`ing) the bill-of-materials from your local image. This command can take some time to return.

<!-- test:exec -->
```bash
pack sbom download test-ruby-app
pack sbom download test-node-js-app
```
<!--+- "{{execute}}"+-->

The SBOM information is now downloaded to the local file system:

<!-- test:exec -->
```bash
cat layers/sbom/launch/examples_ruby/ruby/sbom.cdx.json | jq -M
cat layers/sbom/launch/examples_node-js/node-js/sbom.cdx.json | jq -M
```

You should find that the included `ruby` version is `3.1.0` as expected.
You should find that the included `node-js` version is `18.18.1` as expected.

<!-- test:assert=contains;ignore-lines=... -->
```text
{
"bomFormat": "CycloneDX",
Expand All @@ -250,9 +201,9 @@ You should find that the included `ruby` version is `3.1.0` as expected.
"components": [
{
"type": "library",
"name": "ruby",
"version": "3.1.0"
},
"name": "node-js",
"version": "18.18.1"
}
...
]
}
Expand All @@ -264,7 +215,7 @@ Congratulations! You’ve created your first configurable Cloud Native Buildpack

Now that you've finished your buildpack, how about extending it? Try:

- Caching the downloaded Ruby version
- Caching the downloaded NodeJS version
- [Packaging your buildpack for distribution][package-a-buildpack]

[package-a-buildpack]: /docs/buildpack-author-guide/package-a-buildpack/
Loading

0 comments on commit 071a5b9

Please sign in to comment.