pkg_source not respected and do_download() overwrite not seen

[justincolangelo]

Command that is running: hab pkg build . Setting pkg_source in a plan.sh file to either a git URL or a zipped file pkg_source="https://location/of/file.tar.bz2" and having do_download() overridden seems to not use the do_download() nor does it try to download the zipped file. The output is always:

Cloning into 'repo'...
Username for '[https://repo-location.com'](https://repo-location.com%27/): fatal: could not read Username for '[https://repo-location.com'](https://repo-location.com%27/): Success

• The OS (including version) where you are running any of the Habitat commands.

NAME="Amazon Linux"
VERSION="2"
ID="amzn"
ID_LIKE="centos rhel fedora"
VERSION_ID="2"
PRETTY_NAME="Amazon Linux 2"
ANSI_COLOR="0;33"
CPE_NAME="cpe:2.3:o:amazon:amazon_linux:2"
HOME_URL="https://amazonlinux.com/"

• Debug/backtrace of the command you are trying to run. You can set the following environment variables before running the hab command to generate a trace:

Running the hab command with the RUST env vars didn't product a stacktrace. I think maybe because it is just a failure to download from a git remote.

$ hab --version
hab 1.6.521/20220603154827

• Current Habitat environment variables where the hab command or supervisor is running.

   hab-studio: Exported: HAB_AUTH_TOKEN=[redacted]
   hab-studio: Exported: HAB_ORIGIN=[redacted]
   hab-studio: Exported: HAB_BLDR_URL=https://bldr.biome.sh/
   hab-studio: Exported: AWS_DEFAULT_REGION=[redacted]
   hab-studio: Exported: AWS_SECRET_ACCESS_KEY=[redacted]
   hab-studio: Exported: HAB_STUDIO_HOST_ARCH=[redacted]
   hab-studio: Exported: AWS_ACCESS_KEY_ID=[redacted]
   hab-studio: Exported: GITLAB_USER=[redacted]
   hab-studio: Exported: GITLAB_TOKEN=[redacted]

My goal was to have the hab pkg build . command pull from a remote private repo (using either the username:token option in the URL or the SSH clone option. It doesn't seem to matter what I put into the do_download() method or the pkg_source variable.

[justincolangelo]

I wanted to update this issue. This is running as part of a CI/CD process. I had to rm -rf /hab/cache/artifacts/ for it to respect the new values in those plan vars. If that is a known thing then let's close this.

[vkurniavka]

Any news? has the same issue in Docker container, locally works fine but when I'm tying to use linux_x64 Docker image got the same error

[atrniv]

@justincolangelo / @vkurniavka , would you be able to share the plan.sh file you are using so that I could attempt to reproduce this issue ?

[vkurniavka]

@atrniv this is my plan.sh it works locally but fails in docker

pkg_name=$package_name
pkg_origin=$origin
pkg_version="1.5.1"
pkg_major="1";
pkg_minor="5";
pkg_maintainer="maitainer"
pkg_source=$source_git_url
pkg_license=('MPL-2.0')
pkg_filename="$package_name-1.5.1.tar.bz2"
pkg_branch=$branch
commit_hash=$commit

pkg_deps=(
core/coreutils/8.30/20190115012313
core/bash/4.4.19/20190115012619
core/which/2.21/20190430084037
$origin/erlang/22.0
)

pkg_build_deps=(
core/coreutils/8.30/20190115012313
core/git/2.26.2/20200601121014
$origin/erlang/22.0
$origin/elixir/1.8.0
core/aws-cli/1.21.11/20220311142250
)

pkg_exports=(
[port]=phx.port
)

pkg_description="Project $pkg_branch branch commit $commit_hash for $origin"

do_download() {
export GIT_SSL_NO_VERIFY=true
export GITLAB_TOKEN=$GITLAB_TOKEN
export GITLAB_USER=$GITLAB_USER
rm -rf ./project*
echo "Clone the repo $pkg_source ..."
git clone --branch $pkg_branch $pkg_source
mv ./project ./project-$pkg_version
tar -cjvf $HAB_CACHE_SRC_PATH/$pkg_filename ./project-$pkg_version --exclude project-$pkg_version/.git --exclude project-$pkg_version/spec
}

do_verify() {
return 0
}

do_prepare() {
export LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8
export MIX_ENV=prod
export RELEASE_VERSION="$pkg_major.$pkg_minor.$(echo $pkg_prefix | cut -d '/' -f 7)"

# Rebar3 will hate us otherwise because it looks for
# /usr/bin/env when it does some of its compiling
[[ ! -f /usr/bin/env ]] && ln -s "$(pkg_path_for coreutils)/bin/env" /usr/bin/env

return 0

}

do_build() {
echo "Building project..."
mix local.hex --force
mix local.rebar --force
mix deps.get --only prod
mix deps.clean mime --build
rm -rf _build
mix compile
}

do_install() {
echo "Installing the build..."
rm -rf _build/prod/rel/proj/releases
MIX_ENV=prod mix distillery.release
# TODO 1.9 releases chmod 0655 _build/prod/rel/proj/bin/*
cp -a _build/prod/rel/proj/* ${pkg_prefix}

for f in $(find ${pkg_prefix} -name '*.sh')
do
    fix_interpreter "$f" core/bash bin/bash
    fix_interpreter "$f" core/coreutils bin/env
    # TODO 1.9 releases chmod 0655 "$f"
done

# TODO 1.9 releases chmod 0655 elixir, bin/erl

}

do_strip() {
return 0
}

do_end() {
return 0
}

[vkurniavka]

@atrniv any updates?
Now I'm facing the same issue on local machine and have no chance to build project

[justincolangelo]

We are seeing issues downloading hab studio now.

‘’’
✗✗✗ Failed to install package core/hab-studio/1.6.521 - When applicable, we try once, then re-attempt 5 times to
[381]✗✗✗ download a package. Unfortunately, we failed to download
[382](✗✗✗ core/hab-studio/1.6.521/20220603160937 for x86_64-linux. Last error: [404 Not Found]
‘’’

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. We value your input and contribution. Please leave a comment if this issue still affects you.