Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(ci): add direnv virtual env integration #13380

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions build/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -275,6 +275,16 @@ kong_template_file(
template = "//build:templates/venv.fish",
)

kong_template_file(
name = "direnv",
output = "%s-venv.envrc" % KONG_VAR["BUILD_NAME"],
substitutions = {
"{{build_name}}": KONG_VAR["BUILD_NAME"],
"{{workspace_path}}": KONG_VAR["WORKSPACE_PATH"],
},
template = "//build:templates/direnv",
)

kong_template_file(
name = "venv-commons",
is_executable = True,
Expand All @@ -291,6 +301,7 @@ kong_rules_group(
":kong",
":venv.sh",
":venv.fish",
":direnv",
":venv-commons",
flrgh marked this conversation as resolved.
Show resolved Hide resolved
],
visibility = ["//visibility:public"],
Expand Down
69 changes: 66 additions & 3 deletions build/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,11 +50,24 @@ Once the build is complete, you will see four `bazel-*` folders in the current d

### Development environment

To use the build as a virtual development environment, run:
To use the build as a virtual development environment, build the `//build:venv`
target:

```bash
bazel build //build:venv --verbose_failures
. ./bazel-bin/build/kong-dev-venv.sh
```

This generates files in `./bazel-bin/build` that can be used to initialize your
development environment.

#### bash/fish shell

```sh
# bash
source ./bazel-bin/build/kong-dev-venv.sh

# fish
source ./bazel-bin/build/kong-dev-venv.fish
```

This operation primarily accomplishes the following:
Expand All @@ -63,6 +76,56 @@ This operation primarily accomplishes the following:
2. Set and specify the runtime path for Kong.
3. Provide Bash functions to start and stop the database and other third-party dependency services required for Kong development environment using Docker, read more: [Start Kong](../DEVELOPER#start-kong).

#### direnv

The `//build:venv` target also creates a file at `bazel-bin/build/{{build_name}}-venv-envrc`
which is suitable to include in your [direnv](https://direnv.net) workflow.

Example `.envrc` file:

```sh
BUILD_NAME=kong-dev
source_env_if_exists "bazel-bin/build/${BUILD_NAME}-venv.envrc"
```

Example with a kong checkout at `~/git/kong/kong` and an `.envrc` file:

```
$ direnv reload
direnv: loading ~/git/kong/kong/bazel-bin/build/kong-dev-venv.envrc
LuaRocks: /home/michaelm/git/kong/kong/bazel-bin/build/kong-dev
OpenResty: /home/michaelm/git/kong/kong/bazel-bin/build/kong-dev/openresty
OpenSSL: /home/michaelm/git/kong/kong/bazel-bin/external/openssl/openssl
direnv: export +CRYPTO_DIR +KONG_OPENRESTY_PATH +KONG_TEST_OPENRESTY_PATH +LIBRARY_PREFIX +OPENSSL_DIR +OPENSSL_INCDIR +_KONG_REPO ~LUAROCKS_CONFIG ~LUA_CPATH ~LUA_PATH ~MANPATH ~PATH

$ which kong resty nginx openssl lua luajit luarocks busted luacheck
~/git/kong/kong/bin/kong
~/git/kong/kong/bazel-bin/build/kong-dev/openresty/bin/resty
~/git/kong/kong/bazel-bin/build/kong-dev/openresty/nginx/sbin/nginx
~/git/kong/kong/bazel-bin/external/openssl/openssl/bin/openssl
~/git/kong/kong/bazel-bin/build/kong-dev/openresty/luajit/bin/lua
~/git/kong/kong/bazel-bin/build/kong-dev/openresty/luajit/bin/luajit
~/git/kong/kong/bazel-bin/build/kong-dev/bin/luarocks
~/git/kong/kong/bin/busted
~/git/kong/kong/bazel-bin/build/kong-dev/bin/luacheck
```

The following environment variables will be managed by direnv:
* `CRYPTO_DIR`
* `KONG_OPENRESTY_PATH`
* `KONG_TEST_OPENRESTY_PATH`
* `LIBRARY_PREFIX`
* `LUAROCKS_CONFIG`
* `LUA_CPATH`
* `LUA_PATH`
* `MANPATH`
* `OPENSSL_DIR`
* `OPENSSL_INCDIR`
* `PATH`

**NOTE:** direnv does not support exported shell functions, so `start_services`
and `stop_services` functions are not provided.

### C module and Nginx development

When we are developing part of the Kong installation, especially some Nginx C modules, installing stuffs like luarocks is not necessary. We can use the following steps:
Expand Down Expand Up @@ -98,7 +161,7 @@ Each targets under `//build:install` installs an independent component that
composes the Kong runtime environment. We can query `deps(//build:install, 1)`
recursively to find the target that only build and install specific component.
This would be useful if one is debugging the issue of a specific target without
the need to build whole Kong runtime environment.
the need to build whole Kong runtime environment.

We can use the target labels to build the dependency directly, for example:

Expand Down
168 changes: 168 additions & 0 deletions build/templates/direnv
Original file line number Diff line number Diff line change
@@ -0,0 +1,168 @@
build_name="{{build_name}}"
workspace_path="{{workspace_path}}"
build_root=$workspace_path/bazel-bin/build
build_ext=$workspace_path/bazel-bin/external
kong_venv=$build_root/$build_name

if [[ ! -d $kong_venv ]]; then
echo "Kong install at $kong_venv is in an unknown state, exiting"
return
fi

# it's like direnv's path_add function, but it supports custom separators
_add_path() {
local -r var=$1
local -r path=$2
local -r sep=${3:-;}

if [[ -z "${!var:-}" ]]; then
export "$var=$path"
return
fi

local -a old
IFS="${sep}" read -ra old <<<"${!var-}"

local -a new=()

for p in "${old[@]}"; do
if [[ "$p" == "$path" ]]; then
continue
fi
new+=("$p")
done

printf -v "$var" '%s' "$path" "${new[@]/#/$sep}"

# required for direnv to register the exported var
export "$var=${!var}"
}

_add_path_if_exists() {
local var=$1
local path=$2 # trying to keep the same signature as _add_path()

path=$(realpath -m "$path")

if [[ ! -d $path ]]; then
echo "$var element $path does not exist"
return 0
fi

_add_path "$@"
}

_add_lua_path() {
local path=$1

path=$(realpath -m "$path")

if [[ ! -d $path ]]; then
echo "LUA_PATH dir $path does not exist"
return 0
fi

_add_path LUA_PATH "$path/?/init.lua" ";"
_add_path LUA_PATH "$path/?.lua" ";"
}

_add_lua_cpath() {
local path=$1

path=$(realpath -m "$path")

if [[ ! -d $path ]]; then
echo "LUA_CPATH dir $path does not exist"
return 0
fi

_add_path LUA_CPATH "$path/?.so" ";"
}

_add_lua_paths() {
local -r path=$1

_add_lua_path "$path"
_add_lua_cpath "$path"
}

# generate luarocks config file and virtual env bindir
common=${kong_venv}-venv/lib/venv-commons
if [[ -x $common ]]; then
watch_file "$common"
"$common" "$kong_venv" /dev/null
fi

unset LUA_PATH LUA_CPATH
export LUA_PATH LUA_CPATH

rocks=$kong_venv
if [[ -d $rocks ]]; then
echo "LuaRocks: $rocks"

if [[ -e $rocks/rocks_config ]]; then
export LUAROCKS_CONFIG=$rocks/rocks_config
fi

_add_path_if_exists PATH "$rocks"/bin ":"
_add_lua_path "$rocks/share/lua/5.1"
_add_lua_cpath "$rocks/lib/lua/5.1"
else
echo "LuaRocks dir ($rocks) not found"
fi

_add_path_if_exists PATH "$kong_venv"/venv/bin ":"

if [[ -d $kong_venv/kong ]]; then
# keep "make dev" happy
export LIBRARY_PREFIX=$kong_venv/kong
fi

# openresty
resty=${kong_venv}/openresty
if [[ -d $resty ]]; then
echo "OpenResty: $resty"

export KONG_OPENRESTY_PATH=$resty
export KONG_TEST_OPENRESTY_PATH=$resty

_add_path_if_exists PATH "$resty"/bin ":"
_add_path_if_exists PATH "$resty"/nginx/sbin ":"
_add_path_if_exists PATH "$resty"/luajit/bin ":"
_add_path_if_exists MANPATH "$resty"/luajit/share/man ":"

for path in "$resty"/luajit/share/luajit-*; do
_add_lua_path "$path"
done

_add_lua_cpath "$resty"/luajit/lib
_add_lua_paths "$resty"/lualib
_add_lua_paths "$resty"/site/lualib
else
echo "OpenResty ($resty) not found"
fi

# openssl
openssl=$build_ext/openssl/openssl
if [[ -d $openssl ]]; then
echo "OpenSSL: $openssl"

_add_path_if_exists PATH "$openssl"/bin ":"

# needed for lua openssl
export OPENSSL_DIR=$openssl
export OPENSSL_INCDIR=$openssl/include

# needed for lua cqueues
export CRYPTO_DIR=$openssl
else
echo "openssl ($openssl) not found"
fi

# always prepend ./
_add_lua_path "$workspace_path"
LUA_PATH="${LUA_PATH%%;};"

_add_path_if_exists PATH "$workspace_path"/bin ":"

# vim: ft=sh
Loading