Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor update-template-ubuntu.sh and add update-template-debian.sh #2731

Draft
wants to merge 24 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
20a5ce1
cache-common-inc.sh: Prevent `download_to_cache` from rechecking URLs…
norio-nomura Oct 13, 2024
34373ce
update-template-ubuntu.sh: add `ubuntu_` prefix to some symbols
norio-nomura Oct 13, 2024
e45e287
update-template-ubuntu.sh: change some parameters order in some function
norio-nomura Oct 13, 2024
6aaa9b9
update-template-ubuntu.sh: avoid reusing parameters for flags
norio-nomura Oct 13, 2024
4b142f1
update-template-ubuntu.sh: add `ubuntu_{flavor,version}_from_location`
norio-nomura Oct 14, 2024
1d9fda8
update-template-ubuntu.sh: add `ubuntu_location_url_spec`
norio-nomura Oct 14, 2024
63b9453
update-template-ubuntu.sh: temporarily remove URL caching for refacto…
norio-nomura Oct 14, 2024
7db10a1
update-template-ubuntu.sh: add `ubuntu_{arch,path_suffix}_from_location`
norio-nomura Oct 14, 2024
b57a428
update-template-ubuntu.sh: change some functions to return json
norio-nomura Oct 14, 2024
452b604
update-template-ubuntu.sh: refactor checking `url_spec` condition
norio-nomura Oct 14, 2024
d0a1d46
hack/cache-common-inc.sh: change `download_to_cache` to use JSON inst…
norio-nomura Oct 14, 2024
1c24332
update-template-ubuntu.sh: change downloading `unpacked/SHA256SUMS` t…
norio-nomura Oct 14, 2024
e0fda02
update-template-ubuntu.sh: add `ubuntu_image_entry_for_image_kernel_f…
norio-nomura Oct 14, 2024
15c30d9
update-template-ubuntu.sh: change to `ubuntu_image_entry_with_kernel_…
norio-nomura Oct 15, 2024
6773d17
update-template-ubuntu.sh: change to use `json_vars` for creating JSON
norio-nomura Oct 17, 2024
3aeb1a2
update-template-ubuntu.sh: support version aliases: latest, lts
norio-nomura Oct 17, 2024
92f0196
update-template-ubuntu.sh: change to fully utilize `set -e`
norio-nomura Oct 17, 2024
d8c47ee
update-template-ubuntu.sh: simplify script for `limactl edit --set`
norio-nomura Oct 17, 2024
302555f
update-template-ubuntu.sh: add `image_entry` caching
norio-nomura Oct 17, 2024
8c7512e
update-template-ubuntu.sh: check for `.kernel` in `image_entry` befor…
norio-nomura Oct 17, 2024
6fb8dcb
update-template-ubuntu.sh: add error message for when `com.ubuntu.clo…
norio-nomura Oct 17, 2024
6708113
update-template-ubuntu.sh: include `_with_kernel` to cache key if ker…
norio-nomura Oct 18, 2024
19791f9
cache-common-inc.sh: move `error_exit` from `update-template-ubuntu.sh`
norio-nomura Oct 18, 2024
2af6bcd
update-template.sh: add `update-template.sh` script to update all dis…
norio-nomura Oct 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 40 additions & 14 deletions hack/cache-common-inc.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
#!/usr/bin/env bash

# print the error message and exit with status 1
function error_exit() {
echo "Error: $*" >&2
exit 1
}

# e.g.
# ```console
# $ download_template_if_needed templates/default.yaml
Expand Down Expand Up @@ -209,8 +215,7 @@ function location_to_sha256() {
elif command -v shasum >/dev/null; then
sha256="$(echo -n "${location}" | shasum -a 256 | cut -d' ' -f1)"
else
echo "sha256sum or shasum not found" >&2
exit 1
error_exit "sha256sum or shasum not found"
fi
echo "${sha256}"
)
Expand Down Expand Up @@ -351,16 +356,32 @@ function hash_file() {
# /Users/user/Library/Caches/lima/download/by-url-sha256/346ee1ff9e381b78ba08e2a29445960b5cd31c51f896fc346b82e26e345a5b9a/data # on macOS
# /home/user/.cache/lima/download/by-url-sha256/346ee1ff9e381b78ba08e2a29445960b5cd31c51f896fc346b82e26e345a5b9a/data # on others
function download_to_cache() {
local code_time_type_url
code_time_type_url=$(
curl -sSLI -w "%{http_code}\t%header{Last-Modified}\t%header{Content-Type}\t%{url_effective}" "$1" -o /dev/null
)
local cache_path
cache_path=$(location_to_cache_path "$1")
# before checking remote location, check if the data file is already downloaded and the time file is updated within 10 minutes
if [[ -f ${cache_path}/data && -n "$(find "${cache_path}/time" -mmin -10 || true)" ]]; then
echo "${cache_path}/data"
return
fi

# check the remote location
local curl_info_json write_out
write_out='{
"http_code":%{http_code},
"last_modified":"%header{Last-Modified}",
"content_type":"%{content_type}",
"url":"%{url_effective}",
"filename":"%{filename_effective}"
}'
curl_info_json=$(curl -sSLI -w "${write_out}" "$1" -o /dev/null)

local code time type url
IFS=$'\t' read -r code time type url filename <<<"${code_time_type_url}"
[[ ${code} == 200 ]] || exit 1
code=$(jq -r '.http_code' <<<"${curl_info_json}")
time=$(jq -r '.last_modified' <<<"${curl_info_json}")
type=$(jq -r '.content_type' <<<"${curl_info_json}")
url=$(jq -r '.url' <<<"${curl_info_json}")
[[ ${code} == 200 ]] || error_exit "Failed to download $1"

local cache_path
cache_path=$(location_to_cache_path "${url}")
[[ -d ${cache_path} ]] || mkdir -p "${cache_path}"

Expand All @@ -369,18 +390,23 @@ function download_to_cache() {
[[ -f ${cache_path}/time && "$(<"${cache_path}/time")" == "${time}" ]] || needs_download=1
[[ -f ${cache_path}/type && "$(<"${cache_path}/type")" == "${type}" ]] || needs_download=1
if [[ ${needs_download} -eq 1 ]]; then
local code_time_type_url_filename
code_time_type_url_filename=$(
curl_info_json=$(
echo "downloading ${url}" >&2
curl -SL -w "%{http_code}\t%header{Last-Modified}\t%header{Content-Type}\t%{url_effective}\t%{filename_effective}" --no-clobber -o "${cache_path}/data" "${url}"
curl -SL -w "${write_out}" --no-clobber -o "${cache_path}/data" "${url}"
)
local filename
IFS=$'\t' read -r code time type url filename <<<"${code_time_type_url_filename}"
[[ ${code} == 200 ]] || exit 1
code=$(jq -r '.http_code' <<<"${curl_info_json}")
time=$(jq -r '.last_modified' <<<"${curl_info_json}")
type=$(jq -r '.content_type' <<<"${curl_info_json}")
url=$(jq -r '.url' <<<"${curl_info_json}")
filename=$(jq -r '.filename' <<<"${curl_info_json}")
[[ ${code} == 200 ]] || error_exit "Failed to download ${url}"
[[ "${cache_path}/data" == "${filename}" ]] || mv "${filename}" "${cache_path}/data"
# sha256.digest seems existing if expected digest is available. so, not creating it here.
# sha256sum "${cache_path}/data" | awk '{print "sha256:"$1}' >"${cache_path}/sha256.digest"
echo -n "${time}" >"${cache_path}/time"
else
touch "${cache_path}/time"
fi
[[ -f ${cache_path}/type ]] || echo -n "${type}" >"${cache_path}/type"
[[ -f ${cache_path}/url ]] || echo -n "${url}" >"${cache_path}/url"
Expand Down
Loading
Loading