From fe8fb4156310a1a36589d3b611640407a7889f13 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Mon, 15 Jan 2024 07:35:36 +0000 Subject: [PATCH 01/23] modified: docs/reference/cos.md --- docs/reference/cos.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/reference/cos.md b/docs/reference/cos.md index 7a2a14224..690f85436 100644 --- a/docs/reference/cos.md +++ b/docs/reference/cos.md @@ -10,7 +10,7 @@ The dashboard presents the following rows: - General: Displays general metrics about the charm and runners, such as: - Lifecycle counters: Tracks the frequency of Runner initialisation, start, stop, and crash events. - Idle runners after reconciliation: Reflects the count of Runners marked as idle during the last reconciliation event. Note: This data updates post-reconciliation events and isn't real-time. - - Duration observations: Each data point aggregates the last hour and shows the 50th, 90th, 95th percentile and maximum durations for: + - Duration observations: Each data point aggregates the last hour, showcasing minimum, maximum, and average durations for: - Runner installation - Runner idle duration - Charm reconciliation duration From adc5b280d53f9d6949b360286546fcf5eb7f15e2 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Fri, 19 Jan 2024 09:49:00 +0000 Subject: [PATCH 02/23] modified: docs/reference/cos.md --- docs/reference/cos.md | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/docs/reference/cos.md b/docs/reference/cos.md index 690f85436..e854ad1e8 100644 --- a/docs/reference/cos.md +++ b/docs/reference/cos.md @@ -5,7 +5,10 @@ ### Runner and Charm Insights Upon [COS](https://charmhub.io/topics/canonical-observability-stack) integration, this charm initiates the transmission of various metrics—refer to the relevant [specification](https://discourse.charmhub.io/t/specification-isd075-github-runner-cos-integration/12084) for comprehensive details—regarding the runner instances and the charm itself. -The dashboard presents the following rows: +There are two dashboards. One for fine-granular metrics, called "GitHub Self-Hosted Runner Metrics", and one for long-term metrics, +called "GitHub Self-Hosted Runner Metrics (Long-Term)". + +The "GitHub Self-Hosted Runner Metrics" metrics dashboard presents the following rows: - General: Displays general metrics about the charm and runners, such as: - Lifecycle counters: Tracks the frequency of Runner initialisation, start, stop, and crash events. @@ -21,6 +24,10 @@ The dashboard presents the following rows: - Job duration observation - Number of jobs per repository +The "GitHub Self-Hosted Runner Metrics (Long-Term)" metrics dashboard displays the following rows: + +- General: Displays a panel showing the number of jobs per day. + While the dashboard visualises a subset of potential metrics, these metrics are logged in a file named `/var/log/github-runner-metrics.log`. Use following Loki query to retrieve lines from this file: From 5ea4ff6bc02b599f9074daf894f591bd22299954 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Mon, 22 Jan 2024 06:28:51 +0000 Subject: [PATCH 03/23] modified: docs/reference/cos.md --- docs/reference/cos.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/docs/reference/cos.md b/docs/reference/cos.md index e854ad1e8..833e7aa67 100644 --- a/docs/reference/cos.md +++ b/docs/reference/cos.md @@ -13,7 +13,7 @@ The "GitHub Self-Hosted Runner Metrics" metrics dashboard presents the following - General: Displays general metrics about the charm and runners, such as: - Lifecycle counters: Tracks the frequency of Runner initialisation, start, stop, and crash events. - Idle runners after reconciliation: Reflects the count of Runners marked as idle during the last reconciliation event. Note: This data updates post-reconciliation events and isn't real-time. - - Duration observations: Each data point aggregates the last hour, showcasing minimum, maximum, and average durations for: + - Duration observations: Each data point aggregates the last hour and shows the 50th, 90th, 95th percentile and maximum durations for: - Runner installation - Runner idle duration - Charm reconciliation duration @@ -26,7 +26,11 @@ The "GitHub Self-Hosted Runner Metrics" metrics dashboard presents the following The "GitHub Self-Hosted Runner Metrics (Long-Term)" metrics dashboard displays the following rows: -- General: Displays a panel showing the number of jobs per day. +- General: Contains the following panels: + - Total Jobs + - Total unique repositories + - Timeseries chart displaying the number of jobs per day + - Percentage of jobs with low queue time (less than 60 seconds) From cbf4a0388b915de12cc456dec3be3e0599ea99bf Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Mon, 22 Jan 2024 07:52:35 +0000 Subject: [PATCH 04/23] modified: docs/index.md // new: docs/how-to/configure-runner-storage.md --- docs/how-to/configure-runner-storage.md | 33 +++++++++++++++++++++++++ docs/index.md | 1 + 2 files changed, 34 insertions(+) create mode 100644 docs/how-to/configure-runner-storage.md diff --git a/docs/how-to/configure-runner-storage.md b/docs/how-to/configure-runner-storage.md new file mode 100644 index 000000000..c732247a6 --- /dev/null +++ b/docs/how-to/configure-runner-storage.md @@ -0,0 +1,33 @@ +# How to configure runner storage + +To prevent GitHub Action job from exhausting the disk IO of the juju machine hosting the charm, the charm provides two storage options to be configured as the LXD instance root disk: + +- Random access memory as disk +- Storage provided by juju + +This is configured with the [`runner-storage`](https://charmhub.io/github-runner/configure#runner-storage) option. The configuration should be set during deployment and cannot be changed. + +## Random access memory as disk + +The random access memory of the juju machine is configured as LXD storage and used as the root disk for the LXD instances. + +The `runner-storage` configuration needs to be set to `memory` during deployment, and the juju machine constraints should have enough memory for the virtual machine memory and disk. See [Managing resource usage](https://charmhub.io/github-runner/docs/managing-resource-usage). + +An example deployment: + +```shell +juju deploy github-runner --constraints="cores=4 mem=16G root-disk=20G virt-type=virtual-machine" --config token= --config path= --config runner-storage=memory --config vm-memory=2GiB --config vm-disk=10GiB +``` + +## Storage provided by juju + +The juju storage needs to be mounted during deployment, and the `runner-storage` configuration should be set to `juju-storage` during deployment. + +An example deployment: + +```shell +juju deploy github-runner --constraints="cores=4 mem=6G root-disk=30G virt-type=virtual-machine" --config token= --config path= --config runner-storage=juju-storage --config vm-memory=2GiB --config vm-memory=10GiB --storage runner=rootfs +``` + +The above example uses `rootfs`, which is using the root disk of the juju machine. Hence the root-disk size was increase to 30G. +In production environment, a separated storage managed by juju should be used. See [how to manage juju storage](https://juju.is/docs/juju/manage-storage). \ No newline at end of file diff --git a/docs/index.md b/docs/index.md index 3d98d7bc7..00c6239ac 100644 --- a/docs/index.md +++ b/docs/index.md @@ -52,6 +52,7 @@ Thinking about using the GitHub runner charm for your next project? [Get in touc 1. [How to change GitHub personal access token](how-to/change-token.md) 1. [How to comply with security requirements](how-to/comply-security.md) 1. [How to restrict self-hosted runner network access](how-to/configure-denylist.md) + 1. [How to configure runner storage](how-to/configure-runner-storage.md) 1. [How to contribute](how-to/contribute.md) 1. [How to deploy on ARM64](how-to/deploy-on-arm64.md) 1. [How to integrate with COS](how-to/integrate-with-cos.md) From b080a1f75195c05236ae91ba85ba58c1866110ef Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Mon, 29 Jan 2024 09:00:21 +0000 Subject: [PATCH 05/23] modified: docs/how-to/configure-runner-storage.md --- docs/how-to/configure-runner-storage.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/how-to/configure-runner-storage.md b/docs/how-to/configure-runner-storage.md index c732247a6..4466bd61e 100644 --- a/docs/how-to/configure-runner-storage.md +++ b/docs/how-to/configure-runner-storage.md @@ -29,5 +29,4 @@ An example deployment: juju deploy github-runner --constraints="cores=4 mem=6G root-disk=30G virt-type=virtual-machine" --config token= --config path= --config runner-storage=juju-storage --config vm-memory=2GiB --config vm-memory=10GiB --storage runner=rootfs ``` -The above example uses `rootfs`, which is using the root disk of the juju machine. Hence the root-disk size was increase to 30G. -In production environment, a separated storage managed by juju should be used. See [how to manage juju storage](https://juju.is/docs/juju/manage-storage). \ No newline at end of file +The above example uses `rootfs`, which is using the root disk of the juju machine. Hence the root-disk size was increase to 30G. \ No newline at end of file From 9caaddb83504c1d874d87342030d9c2c2938df9d Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Mon, 29 Jan 2024 10:42:58 +0000 Subject: [PATCH 06/23] modified: docs/index.md // new: docs/reference/integrations.md,docs/explanation/ssh-debug.md,docs/how-to/debug-with-ssh.md --- docs/explanation/ssh-debug.md | 18 ++++++++++++ docs/how-to/debug-with-ssh.md | 53 ++++++++++++++++++++++++++++++++++ docs/index.md | 3 ++ docs/reference/integrations.md | 11 +++++++ 4 files changed, 85 insertions(+) create mode 100644 docs/explanation/ssh-debug.md create mode 100644 docs/how-to/debug-with-ssh.md create mode 100644 docs/reference/integrations.md diff --git a/docs/explanation/ssh-debug.md b/docs/explanation/ssh-debug.md new file mode 100644 index 000000000..be2956099 --- /dev/null +++ b/docs/explanation/ssh-debug.md @@ -0,0 +1,18 @@ +# SSH Debug + +SSH debugging allows a user to identify and resolve issues or errors that occur through the secure +shell (SSH) connection between a client and a server. + +To enhance the security of the runner and the infrastructure behind the runner, only user ssh-keys +registered on [Authorized Keys](https://github.com/tmate-io/tmate-ssh-server/pull/93) are allowed +by default on [tmate-ssh-server charm](https://charmhub.io/tmate-ssh-server/). + +Authorized keys are registered via [action-tmate](https://github.com/canonical/action-tmate/)'s +`limit-access-to-actor` feature. This feature uses GitHub users's SSH key to launch an instance +of tmate session with `-a` option, which adds the user's SSH key to `~/.ssh/authorized_keys`. + +### Firewall rules + +By default, if there are any overlapping IPs within the `denylist` config option with the IP +assigned to `tmate-ssh-server`, an exception to that IP will be made so that the `debug-ssh` +relation can be setup correctly. \ No newline at end of file diff --git a/docs/how-to/debug-with-ssh.md b/docs/how-to/debug-with-ssh.md new file mode 100644 index 000000000..2b891c601 --- /dev/null +++ b/docs/how-to/debug-with-ssh.md @@ -0,0 +1,53 @@ +# How to debug with ssh + +The charm exposes an integration `debug-ssh` interface which can be used with +[tmate-ssh-server charm](https://charmhub.io/tmate-ssh-server/) to pre-configure runners with +environment variables to be picked up by [action-tmate](https://github.com/canonical/action-tmate/) +for automatic configuration. + +## Prerequisites + +To enhance the security of self-hosted runners and its infrastracture, only authorized connections +can be established. Hence, action-tmate users must have +[ssh-key registered](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/adding-a-new-ssh-key-to-your-github-account) +on the GitHub account. + +## Deploying + +Use the following command to deploy and integrate github-runner with tmate-ssh-server. + +```shell +juju deploy tmate-ssh-server +juju integrate tmate-ssh-server github-runner +``` + +Idle runners will be flushed and restarted. Busy runners will be configured automatically on next +spawn. + +## Using the action + +Create a workflow that looks like the following within your workflow to enable action-tmate. + +```yaml +name: SSH Debug workflow example + +on: [pull_request] + +jobs: + build: + runs-on: [self-hosted] + steps: + - uses: actions/checkout@v3 + - name: Setup tmate session + uses: canonical/action-tmate@main +``` + +The output of the action looks like the following. + +``` + +SSH: ssh -p 10022 @ +or: ssh -i -p10022 @ +``` + +Read more about [action-tmate's usage here](https://github.com/canonical/action-tmate). \ No newline at end of file diff --git a/docs/index.md b/docs/index.md index 00c6239ac..d14b93ae5 100644 --- a/docs/index.md +++ b/docs/index.md @@ -47,6 +47,7 @@ Thinking about using the GitHub runner charm for your next project? [Get in touc 1. [Explanation](explanation) 1. [ARM64](explanation/arm64.md) 1. [Charm architecture](explanation/charm-architecture.md) + 1. [SSH Debug](explanation/ssh-debug.md) 1. [How To](how-to) 1. [How to change repository or organization](how-to/change-path.md) 1. [How to change GitHub personal access token](how-to/change-token.md) @@ -54,6 +55,7 @@ Thinking about using the GitHub runner charm for your next project? [Get in touc 1. [How to restrict self-hosted runner network access](how-to/configure-denylist.md) 1. [How to configure runner storage](how-to/configure-runner-storage.md) 1. [How to contribute](how-to/contribute.md) + 1. [How to debug with ssh](how-to/debug-with-ssh.md) 1. [How to deploy on ARM64](how-to/deploy-on-arm64.md) 1. [How to integrate with COS](how-to/integrate-with-cos.md) 1. [How to comply with repository policies](how-to/repo-policy.md) @@ -63,6 +65,7 @@ Thinking about using the GitHub runner charm for your next project? [Get in touc 1. [ARM64](reference/arm64.md) 1. [Configurations](reference/configurations.md) 1. [COS Integration](reference/cos.md) + 1. [Integrations](reference/integrations.md) 1. [Tutorial](tutorial) 1. [Managing resource usage](tutorial/managing-resource-usage.md) 1. [Quick start](tutorial/quick-start.md) \ No newline at end of file diff --git a/docs/reference/integrations.md b/docs/reference/integrations.md new file mode 100644 index 000000000..6988e03bf --- /dev/null +++ b/docs/reference/integrations.md @@ -0,0 +1,11 @@ +# Integrations + +### debug-ssh + +_Interface_: debug-ssh +_Supported charms_: [tmate-ssh-server](https://charmhub.io/tmate-ssh-server) + +Debug-ssh integration provides necessary information for runners to provide ssh reverse-proxy +applications to setup inside the runner. + +Example debug-ssh integrate command: `juju integrate github-runner tmate-ssh-server` \ No newline at end of file From 5a74183ec3852e24c5de90bd81e12e09dbc752dd Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Tue, 6 Feb 2024 15:42:53 +0000 Subject: [PATCH 07/23] modified: docs/tutorial/quick-start.md --- docs/tutorial/quick-start.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tutorial/quick-start.md b/docs/tutorial/quick-start.md index 50e83fd27..78faa1134 100644 --- a/docs/tutorial/quick-start.md +++ b/docs/tutorial/quick-start.md @@ -12,7 +12,7 @@ - GitHub Account. - Juju 3 installed. -- Juju controller on OpenStack or LXD (See [How to run on LXD cloud](https://charmhub.io/github-runner/docs/run-on-lxd)), and a juju model. +- Juju controller on OpenStack or LXD (see [How to run on LXD cloud](https://charmhub.io/github-runner/docs/how-to-run-on-lxd)) and a juju model. For more information about how to install and use Juju, see [Get started with Juju](https://juju.is/docs/olm/get-started-with-juju). From 5322897152e4543830cf049f52e1d474610eadb1 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Wed, 7 Feb 2024 14:34:23 +0000 Subject: [PATCH 08/23] modified: docs/explanation/charm-architecture.md --- docs/explanation/charm-architecture.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/explanation/charm-architecture.md b/docs/explanation/charm-architecture.md index 67d042bfe..ccdaf6f58 100644 --- a/docs/explanation/charm-architecture.md +++ b/docs/explanation/charm-architecture.md @@ -66,6 +66,9 @@ The charm requires a GitHub personal access token for the [`token` configuration The token is also passed to [repo-policy-compliance](https://github.com/canonical/repo-policy-compliance) to access GitHub API for the service. +Note that the GitHub API uses a [rate-limiting mechanism](https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api?apiVersion=2022-11-28). When this is reached, the charm may not be able to perform the necessary operations and may go into +BlockedStatus. The charm will automatically recover from this state once the rate limit is reset, but using a different token with a higher rate limit may be a better solution depending on your deployment requirements. + ## GitHub repository setting check The [repo-policy-compliance](https://github.com/canonical/repo-policy-compliance) is a [Flask application](https://flask.palletsprojects.com/) hosted on [Gunicorn](https://gunicorn.org/) that provides a RESTful HTTP API to check the settings of GitHub repositories. This ensures the GitHub repository settings do not allow the execution of code not reviewed by maintainers on the self-hosted runners. From c38ea58cdef9e196c11bf61ccb9becdb243312f4 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Wed, 14 Feb 2024 06:05:49 +0000 Subject: [PATCH 09/23] modified: docs/reference/cos.md --- docs/reference/cos.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/reference/cos.md b/docs/reference/cos.md index 833e7aa67..7be49240f 100644 --- a/docs/reference/cos.md +++ b/docs/reference/cos.md @@ -32,6 +32,7 @@ The "GitHub Self-Hosted Runner Metrics (Long-Term)" metrics dashboard displays t - Timeseries chart displaying the number of jobs per day - Percentage of jobs with low queue time (less than 60 seconds) +Both dashboards allow for filtering by runner flavor by specifying a regular expression on the `Flavor` variable. While the dashboard visualises a subset of potential metrics, these metrics are logged in a file named `/var/log/github-runner-metrics.log`. Use following Loki query to retrieve lines from this file: From adc63591befc9129d397b61767b29f0966b7f800 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Tue, 20 Feb 2024 11:09:26 +0000 Subject: [PATCH 10/23] modified: docs/explanation/charm-architecture.md --- docs/explanation/charm-architecture.md | 25 ++++++++++++++++++++++--- 1 file changed, 22 insertions(+), 3 deletions(-) diff --git a/docs/explanation/charm-architecture.md b/docs/explanation/charm-architecture.md index ccdaf6f58..2c238f707 100644 --- a/docs/explanation/charm-architecture.md +++ b/docs/explanation/charm-architecture.md @@ -48,9 +48,28 @@ The configurations applied in the image include: ## Network configuration -The charm respects the HTTP(S) proxy configuration of the model configuration of Juju. The configuration can be set with [`juju model-config`](https://juju.is/docs/juju/juju-model-config) using the following keys: `juju-http-proxy`, `juju-https-proxy`, `juju-no-proxy`. The GitHub self-hosted runner applications are configured to use the proxy configuration. - -If an HTTP(S) proxy is used, all HTTP(S) requests in the GitHub workflow will be transparently routed to the proxy with [aproxy](https://github.com/canonical/aproxy). Iptables are set up to route network traffic to the destination on ports 80 and 443 to the aproxy. The aproxy will route received packets to the configured HTTP(S) proxy. The service is installed on each runner virtual machine and configured according to the proxy configuration from the Juju model. +The charm respects the HTTP(S) proxy configuration of the model configuration of Juju. The configuration can be set with [`juju model-config`](https://juju.is/docs/juju/juju-model-config) using the following keys: `juju-http-proxy`, `juju-https-proxy`, `juju-no-proxy`. +The GitHub self-hosted runner applications will be configured to utilise the proxy configuration. +This involves setting environment variables such as `http_proxy`, `https_proxy`, `no_proxy`, `HTTP_PROXY`, `HTTPS_PROXY`, and `NO_PROXY` +in various locations within the runner environment, such as `/etc/environment`. + +However, employing this approach with environment variables has its drawbacks. +Not all applications within a workflow may adhere to these variables as they +[lack standardisation](https://about.gitlab.com/blog/2021/01/27/we-need-to-talk-no-proxy/). +This inconsistency can result in failed workflows, prompting the introduction of aproxy, as detailed in the subsection below. + +### aproxy +If the proxy configuration is utilised and [aproxy](https://github.com/canonical/aproxy) is specified through the charm's configuration option, +all HTTP(S) requests to standard ports (80, 443) within the GitHub workflow will be automatically directed +to the specified HTTP(s) proxy. Network traffic destined for ports 80 and 443 is redirected to aproxy using iptables. +aproxy then forwards received packets to the designated HTTP(S) proxy. +Beyond that, the environment variables (`http_proxy`, `https_proxy`, `no_proxy`, `HTTP_PROXY`, `HTTPS_PROXY`, `NO_PROXY`) +will no longer be defined in the runner environment. +It's worth noting that this setup deviates from the behaviour when not using aproxy, +where these variables are set in the runner environment. In that scenario, traffic to non-standard ports +would also be directed to the HTTP(s) proxy, unlike when using aproxy. + +### denylist The nftables on the Juju machine are configured to deny traffic from the runner virtual machine to IPs on the [`denylist` configuration](https://charmhub.io/github-runner/configure#denylist). The runner will always have access to essential services such as DHCP and DNS, regardless of the denylist configuration. From 911c9a4de710090afa70a5a9f7da395e2988967f Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Wed, 28 Feb 2024 11:06:21 +0000 Subject: [PATCH 11/23] modified: docs/tutorial/quick-start.md --- docs/tutorial/quick-start.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/tutorial/quick-start.md b/docs/tutorial/quick-start.md index 78faa1134..7bb47e4b9 100644 --- a/docs/tutorial/quick-start.md +++ b/docs/tutorial/quick-start.md @@ -35,6 +35,7 @@ The registration token can be requested by calling the [GitHub API](https://docs ### Deploy the GitHub runner charm The charm requires a GitHub personal access token with `repo` access, which can be created following the instructions [here](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens#creating-a-personal-access-token-classic). +A user with `admin` access for the repository/org is required, otherwise, the repo-policy-compliance will fail the job. Once the personal access token is created, the charm can be deployed with: From 5e87d6c029674d4c52281e0d4ff3f6bee40bd715 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Tue, 26 Mar 2024 11:28:11 +0000 Subject: [PATCH 12/23] modified: docs/tutorial/quick-start.md --- docs/tutorial/quick-start.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tutorial/quick-start.md b/docs/tutorial/quick-start.md index 7bb47e4b9..0cbf2f06b 100644 --- a/docs/tutorial/quick-start.md +++ b/docs/tutorial/quick-start.md @@ -80,7 +80,7 @@ If the workflow failed at the `Set up runner` step with the following message: > This job has failed to pass a repository policy compliance check as defined in the https://github.com/canonical/repo-policy-compliance repository. The specific failure is listed below. Please update the settings on this project to fix the relevant policy. -The repository setting does not comply with the best practice enforce by the charm. See [How to comply with repository policies](https://charmhub.io/github-runner/docs/repo-policy). +The repository setting does not comply with the best practice enforce by the charm. See [How to comply with repository policies](https://charmhub.io/github-runner/docs/how-to-repo-policy). #### Removing the charm From 3f8f3edffd264f37232d6d4c8f8da558b0c1ff20 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Thu, 28 Mar 2024 10:19:37 +0000 Subject: [PATCH 13/23] new: docs/how-to/add-custom-labels.md --- docs/how-to/add-custom-labels.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) create mode 100644 docs/how-to/add-custom-labels.md diff --git a/docs/how-to/add-custom-labels.md b/docs/how-to/add-custom-labels.md new file mode 100644 index 000000000..8dd2133c1 --- /dev/null +++ b/docs/how-to/add-custom-labels.md @@ -0,0 +1,15 @@ +# How to add custom labels + +This charm supports adding custom labels to the runners. + +By using [`juju config`](https://juju.is/docs/juju/juju-config) to change the +[charm configuration labels](https://charmhub.io/github-runner/configure#labels), additional +custom labels can be attached to the self-hosted runners. + +```shell +juju config labels= +``` + +An example of a COMMA_SEPARATED_LABELS value would be "large,gpu", "small,arm64". +Accepted values are alphanumeric values with underscores (_), whitespaces before and after the the +word will be automatically trimmed. \ No newline at end of file From 754e22b95ed15563e79ef07a60d1eec102a951cc Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Mon, 29 Apr 2024 02:52:21 +0000 Subject: [PATCH 14/23] modified: docs/index.md // new: docs/how-to/set-base-image.md --- docs/how-to/set-base-image.md | 14 ++++++++++++++ docs/index.md | 9 +++++++-- 2 files changed, 21 insertions(+), 2 deletions(-) create mode 100644 docs/how-to/set-base-image.md diff --git a/docs/how-to/set-base-image.md b/docs/how-to/set-base-image.md new file mode 100644 index 000000000..cb21cab30 --- /dev/null +++ b/docs/how-to/set-base-image.md @@ -0,0 +1,14 @@ +# How to set base image + +This charm supports deploying the runners on different base images. + +By using [`juju config`](https://juju.is/docs/juju/juju-config) to change the +[charm configuration labels](https://charmhub.io/github-runner/configure#base-image), the runner +can be deployed with a different Ubuntu base image. Latest two LTS images "jammy" and "noble" are +supported. The default base image is "jammy". + +```shell +juju config base-image= +``` + +An example of a BASE_IMAGE_TAG_OR_NAME value would be "jammy", "22.04", "noble", "24.04". \ No newline at end of file diff --git a/docs/index.md b/docs/index.md index d14b93ae5..61f7ac159 100644 --- a/docs/index.md +++ b/docs/index.md @@ -7,10 +7,12 @@ Just like GitHub's, the self-hosted runners managed by the charm are isolated in Some of the charm dependencies upgrades on a schedule to migrate security risks. The landscape-client charm can be deployed with this charm to ensure other dependencies are up to date. -The charm maintains a set of ephemeral self-hosted runners, each isolated in a single-use virtual machine instance. To prevent disk IO exhaustion, random access memory is used as disk for the virtual machine instances. In addition, resource limits for the self-hosted runners can be configured. +The charm maintains a set of ephemeral self-hosted runners, each isolated in a single-use virtual machine instance. In addition, resource limits for the self-hosted runners can be configured. See [charm architecture](https://charmhub.io/github-runner/docs/charm-architecture) for more information. +The charm operates in a stateless manner. It can be redeployed without losing any data and there is no need to backup the charm's state. + The charm also supports observability through the optional `cos-agent` integration. Metrics and logs about the runners and the charm itself are collected and sent to the [Canonical Observability Stack](https://charmhub.io/topics/canonical-observability-stack) for analysis and visualisation. @@ -37,7 +39,7 @@ The GitHub runner charm is a member of the Ubuntu family. It's an open-source pr - [Code of conduct](https://ubuntu.com/community/code-of-conduct) - [Get support](https://discourse.charmhub.io/) -- [Join our online chat](https://chat.charmhub.io/charmhub/channels/charm-dev) +- [Join our online chat](https://matrix.to/#/#charmhub-charmdev:ubuntu.com) - [Contribute](Contribute) Thinking about using the GitHub runner charm for your next project? [Get in touch](https://chat.charmhub.io/charmhub/channels/charm-dev)! @@ -49,6 +51,7 @@ Thinking about using the GitHub runner charm for your next project? [Get in touc 1. [Charm architecture](explanation/charm-architecture.md) 1. [SSH Debug](explanation/ssh-debug.md) 1. [How To](how-to) + 1. [How to add custom labels](how-to/add-custom-labels.md) 1. [How to change repository or organization](how-to/change-path.md) 1. [How to change GitHub personal access token](how-to/change-token.md) 1. [How to comply with security requirements](how-to/comply-security.md) @@ -60,11 +63,13 @@ Thinking about using the GitHub runner charm for your next project? [Get in touc 1. [How to integrate with COS](how-to/integrate-with-cos.md) 1. [How to comply with repository policies](how-to/repo-policy.md) 1. [How to run on LXD cloud](how-to/run-on-lxd.md) + 1. [How to set base image](how-to/set-base-image.md) 1. [Reference](reference) 1. [Actions](reference/actions.md) 1. [ARM64](reference/arm64.md) 1. [Configurations](reference/configurations.md) 1. [COS Integration](reference/cos.md) + 1. [External Access](reference/external-access.md) 1. [Integrations](reference/integrations.md) 1. [Tutorial](tutorial) 1. [Managing resource usage](tutorial/managing-resource-usage.md) From f317591a7999e742b65b5b9638a27595355a0763 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Tue, 28 May 2024 07:47:43 +0000 Subject: [PATCH 15/23] modified: docs/tutorial/quick-start.md --- docs/tutorial/quick-start.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/tutorial/quick-start.md b/docs/tutorial/quick-start.md index 0cbf2f06b..071e285bc 100644 --- a/docs/tutorial/quick-start.md +++ b/docs/tutorial/quick-start.md @@ -36,6 +36,7 @@ The registration token can be requested by calling the [GitHub API](https://docs The charm requires a GitHub personal access token with `repo` access, which can be created following the instructions [here](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens#creating-a-personal-access-token-classic). A user with `admin` access for the repository/org is required, otherwise, the repo-policy-compliance will fail the job. +For information on token scopes, see [How to change GitHub personal access token](how-to/change-token.md). Once the personal access token is created, the charm can be deployed with: From 8cfaf93208dbe376ddbf573b8139bf182d324b73 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Tue, 4 Jun 2024 06:06:15 +0000 Subject: [PATCH 16/23] modified: docs/reference/cos.md // new: docs/reference/token-scopes.md --- docs/reference/cos.md | 23 +++++++++++++++---- docs/reference/token-scopes.md | 42 ++++++++++++++++++++++++++++++++++ 2 files changed, 60 insertions(+), 5 deletions(-) create mode 100644 docs/reference/token-scopes.md diff --git a/docs/reference/cos.md b/docs/reference/cos.md index 7be49240f..61002d780 100644 --- a/docs/reference/cos.md +++ b/docs/reference/cos.md @@ -12,7 +12,8 @@ The "GitHub Self-Hosted Runner Metrics" metrics dashboard presents the following - General: Displays general metrics about the charm and runners, such as: - Lifecycle counters: Tracks the frequency of Runner initialisation, start, stop, and crash events. - - Idle runners after reconciliation: Reflects the count of Runners marked as idle during the last reconciliation event. Note: This data updates post-reconciliation events and isn't real-time. + - Available runners: A horizontal bar graph showing the number of runners available during the last reconciliation event. Note: This data is updated after each reconciliation event and is not real-time. + - Idle runners after reconciliation: A time series graph showing the number of runners marked as idle during the last reconciliation event over time. Note: This data is updated after each reconciliation event and is not real-time. - Duration observations: Each data point aggregates the last hour and shows the 50th, 90th, 95th percentile and maximum durations for: - Runner installation - Runner idle duration @@ -20,7 +21,7 @@ The "GitHub Self-Hosted Runner Metrics" metrics dashboard presents the following - Job queue duration - how long a job waits in the queue before a runner picks it up - Jobs: Displays certain metrics about the jobs executed by the runners. These metrics can be displayed per repository by specifying a regular expression on the `Repository` variable. The following metrics are displayed: - - Pie charts: Share of jobs by completion status, job conclusion, flavor, repo policy check failure http codes and github events. + - Proportion charts: Share of jobs by completion status, job conclusion, application, repo policy check failure http codes and github events over time. - Job duration observation - Number of jobs per repository @@ -28,11 +29,12 @@ The "GitHub Self-Hosted Runner Metrics (Long-Term)" metrics dashboard displays t - General: Contains the following panels: - Total Jobs + - Runners created per application: Shows the number of runners created per charm application. - Total unique repositories - Timeseries chart displaying the number of jobs per day - Percentage of jobs with low queue time (less than 60 seconds) -Both dashboards allow for filtering by runner flavor by specifying a regular expression on the `Flavor` variable. +Both dashboards allow for filtering by charm application by specifying a regular expression on the `Application` variable. While the dashboard visualises a subset of potential metrics, these metrics are logged in a file named `/var/log/github-runner-metrics.log`. Use following Loki query to retrieve lines from this file: @@ -41,11 +43,22 @@ While the dashboard visualises a subset of potential metrics, these metrics are {filename="/var/log/github-runner-metrics.log"} ``` -These log events contain valuable details such as flavor (pertinent for multiple runner applications), GitHub events triggering workflows along with their respective repositories, and more. Customising metric visualisation is possible to suit specific needs. +These log events contain valuable details such as charm application, GitHub events triggering workflows along with their respective repositories, and more. Customising metric visualisation is possible to suit specific needs. ### Machine Host Metrics The `grafana-agent` autonomously transmits machine host metrics, which are visualised in the `System Resources` dashboard. ## Logs -The `grafana-agent` effectively transmits all logs located at `/var/log/**/*log`, from the charm unit to Loki. Additionally, it collects logs concerning crashed runners with accessible but unshut LXD virtual machines. \ No newline at end of file +The `grafana-agent` effectively transmits all logs located at `/var/log/**/*log`, from the charm unit to Loki. Additionally, it collects logs concerning crashed runners with accessible but unshut LXD virtual machines. + + +## Alerts + +The charm contains a number of alerts that are sent to COS using the `grafana-agent`. +Please refer to the COS documentation for more information on how to set up alerts. + +Alerts are divided into two categories: + +- Capacity Alerts: Alerts you when there is a shortage of a particular type of runner. +- Failure Alerts: Notification of runner crashes or repo policy related failures. \ No newline at end of file diff --git a/docs/reference/token-scopes.md b/docs/reference/token-scopes.md new file mode 100644 index 000000000..029ce186b --- /dev/null +++ b/docs/reference/token-scopes.md @@ -0,0 +1,42 @@ +# Token scopes + +## Fine grained access token scopes + +### Organizational Runners + +The following are the permissions scopes required for the GitHub runners when registering as an +organisational runner. + +Organisation: + +- Self-hosted runners: read & write + +Repository: + +- Administration: read +- Contents: read +- Pull requests: read + +### Repository Runners + +The following are the permissions scopes required for the GitHub runners when registering as an +repository runner. + +- Contents: read +- Metadata: read +- Pull requests: read + +## Personal access token scopes + +### Organizational Runners + +To use this charm for GitHub organisations, the following scopes should be selected: + +- `repo` +- `admin:org` + +### Repository Runners + +To use this charm for GitHub repositories, the following scopes should be selected: + +- `repo` \ No newline at end of file From 05dc8461f892365078562463387f04dc870ccad9 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Fri, 12 Jul 2024 14:03:35 +0000 Subject: [PATCH 17/23] modified: docs/reference/token-scopes.md --- docs/reference/token-scopes.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/reference/token-scopes.md b/docs/reference/token-scopes.md index 029ce186b..a5465ce34 100644 --- a/docs/reference/token-scopes.md +++ b/docs/reference/token-scopes.md @@ -13,6 +13,7 @@ Organisation: Repository: +- Actions: read (required if COS integration is enabled and private repositories exist) - Administration: read - Contents: read - Pull requests: read @@ -22,6 +23,8 @@ Repository: The following are the permissions scopes required for the GitHub runners when registering as an repository runner. +- Actions: read (required if COS integration is enabled and the repository is private) +- Administration: read & write - Contents: read - Metadata: read - Pull requests: read From 4ca8264a5eee074cbdafac7366bb10899948d2b2 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Mon, 22 Jul 2024 06:22:56 +0000 Subject: [PATCH 18/23] modified: docs/index.md,docs/how-to/run-on-lxd.md // new: docs/how-to/openstack-runner.md --- docs/how-to/openstack-runner.md | 59 +++++++++++++++++++++++++++++++++ docs/how-to/run-on-lxd.md | 12 +++++-- docs/index.md | 1 + 3 files changed, 70 insertions(+), 2 deletions(-) create mode 100644 docs/how-to/openstack-runner.md diff --git a/docs/how-to/openstack-runner.md b/docs/how-to/openstack-runner.md new file mode 100644 index 000000000..53dd770a2 --- /dev/null +++ b/docs/how-to/openstack-runner.md @@ -0,0 +1,59 @@ +# How to spawn OpenStack runner + +The charm can be configured to use OpenStack cloud for creating runners. +The charm must be deployed with the correct configuration and once the OpenStack integration is +enabled the charm cannot be changed to use other virtualization methods. + +## Configuration + +There are three configuration that the charm needs to be deployed with to enable OpenStack integration: `openstack-clouds-yaml`, `openstack-flavor`, and `openstack-network`. + +## Integration + +The image will take about 10-15 minutes to build and be fully integrated. Deploy the +`github-runner-image-builder` charm and wait for the image to be successfully provided via the +relation data. + +```bash +juju deploy github-runner-image-builder +juju integrate github-runner-image-builder github-runner +juju status github-runner +``` + +The image will take about 10-15 minutes to build and be ready via the relation. + +### OpenStack clouds.yaml + +The `openstack-clouds-yaml` configuration contains the authorization information needed for the charm to log in to the openstack cloud. +The first cloud in the `clouds.yaml` is used by the charm. + +Here is a sample of the `clouds.yaml`: + +```yaml +clouds: + cloud: + auth: + auth_url: https://keystone.cloud.com:5000/v3 + project_name: github-runner + username: github-runner + password: PASSWORD + user_domain_name: Default + project_domain_name: Default + region_name: cloud +``` + +The `clouds.yaml` documentation is [here](https://docs.openstack.org/python-openstackclient/pike/configuration/index.html#clouds-yaml). + +### OpenStack Flavor + +The `openstack-flavor` configuration sets the flavor used to create the OpenStack virtual machine when spawning new runners. +The flavor is tied with the vCPU, memory, and storage. +The flavors documentation is [here](https://docs.openstack.org/nova/rocky/user/flavors.html). + +### OpenStack Network + +The `openstack-network` configuration sets the network used to create the OpenStack virtual machine when spawning new runners. + +Note that the network should be configured to allow traffic from the charm deployment (juju machine) to the OpenStack virtual machine, and traffic from the OpenStack virtual machine to GitHub. + +The network documentation is [here](https://docs.openstack.org/neutron/latest/admin/intro-os-networking.html). \ No newline at end of file diff --git a/docs/how-to/run-on-lxd.md b/docs/how-to/run-on-lxd.md index 35aa2937e..1351e944d 100644 --- a/docs/how-to/run-on-lxd.md +++ b/docs/how-to/run-on-lxd.md @@ -7,7 +7,15 @@ By default, juju machines on LXD are containers. To run this charm on LXD, add `virt-type=virtual-machine` to the constraints during deployment: ```shell -juju deploy github-runner --constraints="cores=2 mem=16G virt-type=virtual-machine" --config token= --config path= +juju deploy github-runner --constraints="cores=2 mem=16G virt-type=virtual-machine" \ +--config token= --config path= ``` -This constraint ensures the juju machine hosting the charm is a LXD virtual machine. See [Managing resource usage](https://charmhub.io/github-runner/docs/managing-resource-usage) for recommendation on `cores` and `mem` constraint. \ No newline at end of file +This constraint ensures the juju machine hosting the charm is a LXD virtual machine. See +[Managing resource usage](https://charmhub.io/github-runner/docs/managing-resource-usage) for +recommendation on `cores` and `mem` constraint. + +### Notes + +The name of the application must not be longer than 29 characters. This is due to the nature of LXD +pathing that must not exceed 108 bytes. 79 characters are reserved for path naming convention. \ No newline at end of file diff --git a/docs/index.md b/docs/index.md index 61f7ac159..a1d8be988 100644 --- a/docs/index.md +++ b/docs/index.md @@ -61,6 +61,7 @@ Thinking about using the GitHub runner charm for your next project? [Get in touc 1. [How to debug with ssh](how-to/debug-with-ssh.md) 1. [How to deploy on ARM64](how-to/deploy-on-arm64.md) 1. [How to integrate with COS](how-to/integrate-with-cos.md) + 1. [How to spawn OpenStack runner](how-to/openstack-runner.md) 1. [How to comply with repository policies](how-to/repo-policy.md) 1. [How to run on LXD cloud](how-to/run-on-lxd.md) 1. [How to set base image](how-to/set-base-image.md) From 9bdf20dbef3219747830f6cae30e94f4f5e6c135 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Wed, 25 Sep 2024 06:51:26 +0000 Subject: [PATCH 19/23] modified: docs/how-to/deploy-on-arm64.md,docs/how-to/add-custom-labels.md,docs/tutorial/managing-resource-usage.md,docs/how-to/integrate-with-cos.md,docs/how-to/configure-runner-storage.md,docs/reference/arm64.md,docs/explanation/ssh-debug.md,docs/how-to/run-on-lxd.md --- docs/explanation/ssh-debug.md | 2 +- docs/how-to/add-custom-labels.md | 2 +- docs/how-to/configure-runner-storage.md | 14 +++++++------- docs/how-to/deploy-on-arm64.md | 6 +++--- docs/how-to/integrate-with-cos.md | 10 +++++----- docs/how-to/run-on-lxd.md | 8 +++----- docs/reference/arm64.md | 2 +- docs/tutorial/managing-resource-usage.md | 12 ++++++++---- 8 files changed, 29 insertions(+), 27 deletions(-) diff --git a/docs/explanation/ssh-debug.md b/docs/explanation/ssh-debug.md index be2956099..261be32bc 100644 --- a/docs/explanation/ssh-debug.md +++ b/docs/explanation/ssh-debug.md @@ -15,4 +15,4 @@ of tmate session with `-a` option, which adds the user's SSH key to `~/.ssh/auth By default, if there are any overlapping IPs within the `denylist` config option with the IP assigned to `tmate-ssh-server`, an exception to that IP will be made so that the `debug-ssh` -relation can be setup correctly. \ No newline at end of file +relation can be set up correctly. \ No newline at end of file diff --git a/docs/how-to/add-custom-labels.md b/docs/how-to/add-custom-labels.md index 8dd2133c1..46e243f49 100644 --- a/docs/how-to/add-custom-labels.md +++ b/docs/how-to/add-custom-labels.md @@ -11,5 +11,5 @@ juju config labels= ``` An example of a COMMA_SEPARATED_LABELS value would be "large,gpu", "small,arm64". -Accepted values are alphanumeric values with underscores (_), whitespaces before and after the the +Accepted values are alphanumeric values with underscores (_); whitespaces before and after the the word will be automatically trimmed. \ No newline at end of file diff --git a/docs/how-to/configure-runner-storage.md b/docs/how-to/configure-runner-storage.md index 4466bd61e..bd7309b9b 100644 --- a/docs/how-to/configure-runner-storage.md +++ b/docs/how-to/configure-runner-storage.md @@ -1,17 +1,17 @@ # How to configure runner storage -To prevent GitHub Action job from exhausting the disk IO of the juju machine hosting the charm, the charm provides two storage options to be configured as the LXD instance root disk: +To prevent the GitHub Action job from exhausting the disk IO of the Juju machine hosting the charm, the charm provides two storage options to be configured as the LXD instance root disk: - Random access memory as disk -- Storage provided by juju +- Storage provided by Juju This is configured with the [`runner-storage`](https://charmhub.io/github-runner/configure#runner-storage) option. The configuration should be set during deployment and cannot be changed. ## Random access memory as disk -The random access memory of the juju machine is configured as LXD storage and used as the root disk for the LXD instances. +The random access memory of the Juju machine is configured as LXD storage and used as the root disk for the LXD instances. -The `runner-storage` configuration needs to be set to `memory` during deployment, and the juju machine constraints should have enough memory for the virtual machine memory and disk. See [Managing resource usage](https://charmhub.io/github-runner/docs/managing-resource-usage). +The `runner-storage` configuration needs to be set to `memory` during deployment, and the Juju machine constraints should have enough memory for the virtual machine memory and disk. See [Managing resource usage](https://charmhub.io/github-runner/docs/managing-resource-usage). An example deployment: @@ -19,9 +19,9 @@ An example deployment: juju deploy github-runner --constraints="cores=4 mem=16G root-disk=20G virt-type=virtual-machine" --config token= --config path= --config runner-storage=memory --config vm-memory=2GiB --config vm-disk=10GiB ``` -## Storage provided by juju +## Storage provided by Juju -The juju storage needs to be mounted during deployment, and the `runner-storage` configuration should be set to `juju-storage` during deployment. +The Juju storage needs to be mounted during deployment, and the `runner-storage` configuration should be set to `juju-storage` during deployment. An example deployment: @@ -29,4 +29,4 @@ An example deployment: juju deploy github-runner --constraints="cores=4 mem=6G root-disk=30G virt-type=virtual-machine" --config token= --config path= --config runner-storage=juju-storage --config vm-memory=2GiB --config vm-memory=10GiB --storage runner=rootfs ``` -The above example uses `rootfs`, which is using the root disk of the juju machine. Hence the root-disk size was increase to 30G. \ No newline at end of file +The above example uses `rootfs`, which is using the root disk of the Juju machine. Hence the root-disk size was increase to 30G. \ No newline at end of file diff --git a/docs/how-to/deploy-on-arm64.md b/docs/how-to/deploy-on-arm64.md index 9d8fbd7cb..e862f1749 100644 --- a/docs/how-to/deploy-on-arm64.md +++ b/docs/how-to/deploy-on-arm64.md @@ -7,13 +7,13 @@ deployment currently only supports ARM64 bare-metal machines due to the limitati The following uses AWS's [m7g.metal](https://aws.amazon.com/blogs/aws/now-available-bare-metal-arm-based-ec2-instances/) instance to deploy the GitHub Runner on ARM64 architecture. -### Prerequisites +## Requirements 1. Juju with ARM64 bare metal instance availability. - On AWS: `juju bootstrap aws ` 2. GitHub [Personal Access Token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens) 3. Repository to register the GitHub runners. -### Deployment steps +## Deployment steps Run the following command: ```shell @@ -26,5 +26,5 @@ The units may take several minutes to settle. Furthermore, due to charm restart the Unit may become lost for a few minutes. This is an expected behavior and the unit should automatically re-register itself onto the Juju controller after a successful reboot. -Goto the repository > Settings (tab) > Actions (left menu dropdown) > Runners and verify that the +Go to the repository > Settings (tab) > Actions (left menu dropdown) > Runners and verify that the runner has successfully registered and is online. \ No newline at end of file diff --git a/docs/how-to/integrate-with-cos.md b/docs/how-to/integrate-with-cos.md index fd685b5fb..dd02f83b4 100644 --- a/docs/how-to/integrate-with-cos.md +++ b/docs/how-to/integrate-with-cos.md @@ -4,17 +4,17 @@ This guide demonstrates the process of integrating with the [Canonical Observabi The `cos-agent` integration can be consumed by the [grafana-agent](https://charmhub.io/grafana-agent) charm, which is responsible for transmitting logs, Prometheus metrics, and Grafana dashboards to the COS stack. -**Note:** The Github Runner charm and grafana-agent charm function as machine charms, while the COS stack comprises Kubernetes charms. Therefore, establishing [cross-model integrations](https://juju.is/docs/juju/manage-cross-model-integrations) is necessary, along with potential firewall rule configurations to allow inter-model traffic. +> NOTE: The Github Runner charm and `grafana-agent` charm function as machine charms, while the COS stack comprises Kubernetes charms. Therefore, establishing [cross-model integrations](https://juju.is/docs/juju/manage-cross-model-integrations) is necessary, along with potential firewall rule configurations to allow inter-model traffic. -### Prerequisites +## Requirements 1. Deploy the Github Runner Charm with the application name `github-runner` in the `machine-model`. 2. Deploy the COS stack on a Kubernetes cloud (refer to [this tutorial](https://charmhub.io/topics/canonical-observability-stack/tutorials/install-microk8s)). - Ensure `loki`, `prometheus`, `grafana`, and `traefik` charms are deployed within a model named `k8s-model`. - - Integration between `loki` and `traefik` is required to enable `grafana-agent` to transmit logs by setting a public IP for the loki service accessible from the machine cloud. + - Integration between `loki` and `traefik` is required to enable `grafana-agent` to transmit logs by setting a public IP for the Loki service accessible from the machine cloud. - Confirm that both models exist in the same Juju controller. If not, adjust the model names by appending the respective controller name (followed by ":") in the subsequent steps. Ensure you have the necessary [permissions](https://juju.is/docs/juju/manage-cross-model-integrations#heading--control-access-to-an-offer) to consume the offers. -### Steps +## Steps 1. Deploy the `grafana-agent` charm in the machine model. ```shell @@ -47,4 +47,4 @@ The `cos-agent` integration can be consumed by the [grafana-agent](https://charm ``` You should now be able to access a Grafana Dashboard named `GitHub Self-Hosted Runner Metrics`, displaying metrics, and another named `System Resources` exhibiting host resources in Grafana. -Additionally, you can explore Loki logs using Grafana's Explore function. For detailed information about the specific metrics in the `GitHub Self-Hosted Runner Metrics` dashboard, refer to [Metrics](https://charmhub.io/github-runner/docs/cos). \ No newline at end of file +Additionally, you can explore Loki logs using Grafana's Explore function. For detailed information about the specific metrics in the `GitHub Self-Hosted Runner Metrics` dashboard, refer to [Metrics](https://charmhub.io/github-runner/docs/reference-cos). \ No newline at end of file diff --git a/docs/how-to/run-on-lxd.md b/docs/how-to/run-on-lxd.md index 1351e944d..7b9362e90 100644 --- a/docs/how-to/run-on-lxd.md +++ b/docs/how-to/run-on-lxd.md @@ -2,7 +2,7 @@ This machine charm needs to run on virtual machines with nested virtualization enabled. -By default, juju machines on LXD are containers. +By default, Juju machines on LXD are containers. To run this charm on LXD, add `virt-type=virtual-machine` to the constraints during deployment: @@ -11,11 +11,9 @@ juju deploy github-runner --constraints="cores=2 mem=16G virt-type=virtual-machi --config token= --config path= ``` -This constraint ensures the juju machine hosting the charm is a LXD virtual machine. See +This constraint ensures the Juju machine hosting the charm is a LXD virtual machine. See [Managing resource usage](https://charmhub.io/github-runner/docs/managing-resource-usage) for recommendation on `cores` and `mem` constraint. -### Notes - -The name of the application must not be longer than 29 characters. This is due to the nature of LXD +> NOTE: The name of the application must not be longer than 29 characters. This is due to the nature of LXD pathing that must not exceed 108 bytes. 79 characters are reserved for path naming convention. \ No newline at end of file diff --git a/docs/reference/arm64.md b/docs/reference/arm64.md index cc928f62d..253cff298 100644 --- a/docs/reference/arm64.md +++ b/docs/reference/arm64.md @@ -2,5 +2,5 @@ ### AWS bare metal instances -Use any of the [ARM64 metal instances](https://aws.amazon.com/ec2/instance-types/) to provide juju +Use any of the [ARM64 metal instances](https://aws.amazon.com/ec2/instance-types/) to provide Juju with ARM64 bare metal instances. Some of the examples include: a1.metal, m7g.metal. \ No newline at end of file diff --git a/docs/tutorial/managing-resource-usage.md b/docs/tutorial/managing-resource-usage.md index 134c8c3e1..aa678267f 100644 --- a/docs/tutorial/managing-resource-usage.md +++ b/docs/tutorial/managing-resource-usage.md @@ -8,13 +8,13 @@ The minimum requirements for a single virtual machine are: - 1 [virtual machine vCPU](https://charmhub.io/github-runner/configure#vm-cpu) - 2GiB of [virtual machine memory](https://charmhub.io/github-runner/configure#vm-memory) -- 6GiB of [virtual machine disk](https://charmhub.io/github-runner/configure#vm-disk) +- 10GiB of [virtual machine disk](https://charmhub.io/github-runner/configure#vm-disk) ## Juju machine resource usage -It is recommended the juju machine has a minimum of 4GiB of memory dedicated to itself. Generally, 20GiB of disk is provisioned for the juju machine for the Juju logs. +It is recommended the Juju machine has a minimum of 4GiB of memory dedicated to itself. Generally, 20GiB of disk is provisioned for the Juju machine for the Juju logs. -The juju machine will also need the enough resources to host the virtual machines. The memory of the juju machine is used as the disk for the virtual machines. +The Juju machine will also need the enough resources to host the virtual machines. The recommended combined resource usage is: @@ -22,6 +22,10 @@ The recommended combined resource usage is: - memory: number of virtual machines * (memory per virtual machine + disk per virtual machine) + 4GiB - disk: 20GiB +If memory is used as [runner storage](https://charmhub.io/github-runner/docs/configure-runner-storage): + +- memory: number of virtual machines * (memory per virtual machine + disk per virtual machine) + 4GiB + ## Juju machine constraints -During [deployment of the charm](https://juju.is/docs/juju/juju-deploy), constraints can be used to specify the juju machine resource requirements. For example, `juju deploy github-runner --constraints="cores=4 mem=16G disk=20G"`. \ No newline at end of file +During [deployment of the charm](https://juju.is/docs/juju/juju-deploy), constraints can be used to specify the Juju machine resource requirements. For example, `juju deploy github-runner --constraints="cores=4 mem=16G disk=20G"`. \ No newline at end of file From e7e22b06a0ece16cb392aec1914376b9b0beea80 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Wed, 25 Sep 2024 06:52:29 +0000 Subject: [PATCH 20/23] modified: docs/index.md // new: docs/changelog.md,docs/reference/external-access.md --- docs/changelog.md | 10 ++++++++++ docs/index.md | 10 ++++++---- docs/reference/external-access.md | 16 ++++++++++++++++ 3 files changed, 32 insertions(+), 4 deletions(-) create mode 100644 docs/changelog.md create mode 100644 docs/reference/external-access.md diff --git a/docs/changelog.md b/docs/changelog.md new file mode 100644 index 000000000..66e16d2b6 --- /dev/null +++ b/docs/changelog.md @@ -0,0 +1,10 @@ +# Changelog + +### 2024-09-18 + +- Changed code to be able to spawn a runner in reactive mode. +- Removed reactive mode support for LXD as it is not currently in development. + +## 2024-09-09 + +- Added changelog for tracking user-relevant changes. \ No newline at end of file diff --git a/docs/index.md b/docs/index.md index a1d8be988..dfedff4c5 100644 --- a/docs/index.md +++ b/docs/index.md @@ -9,7 +9,7 @@ Some of the charm dependencies upgrades on a schedule to migrate security risks. The charm maintains a set of ephemeral self-hosted runners, each isolated in a single-use virtual machine instance. In addition, resource limits for the self-hosted runners can be configured. -See [charm architecture](https://charmhub.io/github-runner/docs/charm-architecture) for more information. +See [charm architecture](https://charmhub.io/github-runner/docs/explanation-charm-architecture) for more information. The charm operates in a stateless manner. It can be redeployed without losing any data and there is no need to backup the charm's state. @@ -18,7 +18,7 @@ Metrics and logs about the runners and the charm itself are collected and sent t This charm will make operating GitHub self-hosted runners simple and straightforward for DevOps or SRE teams through Juju's clean interface. -The charm enforces a set of GitHub repository settings as best practice. This is planned to be opt-in in the future. See [How to comply with repository policies](https://charmhub.io/github-runner/docs/repo-policy). +The charm enforces a set of GitHub repository settings as best practice. This is planned to be opt-in in the future. See [How to comply with repository policies](https://charmhub.io/github-runner/docs/how-to-repo-policy). ## In this documentation @@ -42,7 +42,7 @@ The GitHub runner charm is a member of the Ubuntu family. It's an open-source pr - [Join our online chat](https://matrix.to/#/#charmhub-charmdev:ubuntu.com) - [Contribute](Contribute) -Thinking about using the GitHub runner charm for your next project? [Get in touch](https://chat.charmhub.io/charmhub/channels/charm-dev)! +Thinking about using the GitHub runner charm for your next project? [Get in touch](https://matrix.to/#/#charmhub-charmdev:ubuntu.com)! # Contents @@ -72,6 +72,8 @@ Thinking about using the GitHub runner charm for your next project? [Get in touc 1. [COS Integration](reference/cos.md) 1. [External Access](reference/external-access.md) 1. [Integrations](reference/integrations.md) + 1. [Token scopes](reference/token-scopes.md) 1. [Tutorial](tutorial) 1. [Managing resource usage](tutorial/managing-resource-usage.md) - 1. [Quick start](tutorial/quick-start.md) \ No newline at end of file + 1. [Quick start](tutorial/quick-start.md) +1. [Changelog](changelog.md) \ No newline at end of file diff --git a/docs/reference/external-access.md b/docs/reference/external-access.md new file mode 100644 index 000000000..938c48854 --- /dev/null +++ b/docs/reference/external-access.md @@ -0,0 +1,16 @@ +# External Access + +The GitHub Runner Charm itself requires access to the + +- GitHub API (e.g. to register and remove runners). +- GitHub website (e.g. to download the runner binary or other applications like yq) +- Ubuntu package repositories (e.g. to install packages) +- Snap store (e.g. to install LXD or aproxy) +- [Ubuntu Cloud Images](https://cloud-images.ubuntu.com/) (for the image used by a runner) +- npm registry (e.g. to download and install specific packages) + +In addition, access is required depending on the requirements of the workloads that the runners +will be running (as they will be running on the same machine as the charm). + +More details on network configuration can be found in the +[charm architecture documentation](https://charmhub.io/github-runner/docs/charm-architecture). \ No newline at end of file From 68bdf886f1a1226829af1b45f79106465b0fe59c Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Mon, 7 Oct 2024 07:15:19 +0000 Subject: [PATCH 21/23] modified: docs/tutorial/quick-start.md --- docs/tutorial/quick-start.md | 38 ++++++++++++++++++++++++------------ 1 file changed, 25 insertions(+), 13 deletions(-) diff --git a/docs/tutorial/quick-start.md b/docs/tutorial/quick-start.md index 071e285bc..12f734a1c 100644 --- a/docs/tutorial/quick-start.md +++ b/docs/tutorial/quick-start.md @@ -1,4 +1,4 @@ -# Quick start +# Deploy the GitHub runner charm for the first time ## What you'll do @@ -12,7 +12,7 @@ - GitHub Account. - Juju 3 installed. -- Juju controller on OpenStack or LXD (see [How to run on LXD cloud](https://charmhub.io/github-runner/docs/how-to-run-on-lxd)) and a juju model. +- Juju controller on OpenStack or LXD (see [How to run on LXD cloud](https://charmhub.io/github-runner/docs/how-to-run-on-lxd)). For more information about how to install and use Juju, see [Get started with Juju](https://juju.is/docs/olm/get-started-with-juju). @@ -26,12 +26,20 @@ To create a GitHub repository, log in to [GitHub](https://github.com) with your ### Activate GitHub APIs related to self-hosted runner -***This must be done for the GitHub runner charm to function correctly.*** +> ***⚠️ This must be done for the GitHub runner charm to function correctly.*** The GitHub runner charm relies on GitHub APIs for self-hosted runners. Some of the APIs will only be functional after a self-hosted runner registration token is requested for the repository for the first time. The registration token can be requested by calling the [GitHub API](https://docs.github.com/en/rest/actions/self-hosted-runners?apiVersion=2022-11-28#create-a-registration-token-for-a-repository). Alternatively, it can also be requested by visiting the "New self-hosted runner" page for the repository (`https://github.com/{OWNER}/{REPO}/settings/actions/runners/new`). This can be done by following the instruction to the 4th step provided [here](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/adding-self-hosted-runners#adding-a-self-hosted-runner-to-a-repository). +### Set up the tutorial model + +To easily clean up the resources and to separate your workload from the contents of this tutorial, set up a new Juju model with the following command. + +``` +juju add-model github-runner-tutorial +``` + ### Deploy the GitHub runner charm The charm requires a GitHub personal access token with `repo` access, which can be created following the instructions [here](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens#creating-a-personal-access-token-classic). @@ -41,12 +49,18 @@ For information on token scopes, see [How to change GitHub personal access token Once the personal access token is created, the charm can be deployed with: ```shell -juju deploy github-runner --constraints="cores=4 mem=16G root-disk=20G" --config token= --config path= +juju deploy github-runner --constraints="cores=4 mem=16G root-disk=20G virt-type=virtual-machine" --config token= --config path= --config runner-storage=memory --config vm-memory=2GiB --config vm-disk=10GiB ``` Replacing the `` with the personal access token, and `` the GitHub account name and GitHub repository separated with `/`. -The `--constraints` option for the `juju deploy` sets the resource requirements for the juju machine hosting the charm application. This is used to accommodate different sizes of self-hosted runners. For details, refer to [Managing resource usage](https://charmhub.io/github-runner/docs/managing-resource-usage). +The `--constraints` option for the `juju deploy` sets the resource requirements for the Juju machine hosting the charm application. This is used to accommodate different sizes of self-hosted runners. For details, refer to [Managing resource usage](https://charmhub.io/github-runner/docs/managing-resource-usage). + +The `--storage` option mounts a juju storage to be used as the disk for LXD instances hosting the self-hosted runners. Refer [How to configure runner storage](https://charmhub.io/github-runner/docs/configure-runner-storage) for more information. + +The charm performs various installation and configuration on startup. The charm might upgrade the kernel of the Juju machine and reboot the Juju machine. During reboot, the Juju machine will go into the `down` state; this is a part of the normal reboot process and the Juju machine should be restarted after a while. + +Monitor the deployment status using `juju status --watch 5s`. The deployment finishes when the status shows "Active". Once the charm reaches active status, visit the runner page for the GitHub repository (`https://github.com/{OWNER}/{REPO}/settings/actions/runners`) according to the instructions [here](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/using-self-hosted-runners-in-a-workflow#viewing-available-runners-for-a-repository). A single new runner should be available as it is the default number of self-hosted runners created. @@ -56,9 +70,7 @@ The charm will spawn new runners on a schedule. During this time, the charm will Once the self-hosted runner is available on GitHub, it can be used to run GitHub Actions jobs similar to runners provided by GitHub. The only difference being the label specified in the `runs-on` of a job. -The self-hosted runner managed by the charm will have the following labels: `self-hosted`, `linux`, and the application name. - -In the above deployment, the application name was not specified, hence the default value of `github-runner` was used. As such, `github-runner` will be a label for the self-hosted runner managed by the application instance. +In addition to the labels added by the GitHub runner application by default, the charm will include labels from the [`labels` charm configuration](https://charmhub.io/github-runner/configurations#labels). To test out the self-hosted runner, create the following file under the path `.github/workflows/runner_test.yaml` in the repository with the following content: @@ -70,7 +82,7 @@ on: jobs: hello-world-test: - runs-on: [self-hosted, github-runner] + runs-on: [self-hosted] steps: - run: echo "hello world" ``` @@ -81,12 +93,12 @@ If the workflow failed at the `Set up runner` step with the following message: > This job has failed to pass a repository policy compliance check as defined in the https://github.com/canonical/repo-policy-compliance repository. The specific failure is listed below. Please update the settings on this project to fix the relevant policy. -The repository setting does not comply with the best practice enforce by the charm. See [How to comply with repository policies](https://charmhub.io/github-runner/docs/how-to-repo-policy). +The repository setting does not comply with the best practice enforce by the charm. See [How to comply with repository policies](https://charmhub.io/github-runner/docs/repo-policy). -#### Removing the charm +### Clean up the environment -The charm and the self-hosted runners can be removed with the following command: +The Juju model, charm and the self-hosted runners can be removed with the following command: ```shell -juju remove-application github-runner +juju destroy-model --destroy-storage github-runner-tutorial ``` \ No newline at end of file From d11fac371128223288721d307d06d545f6b0e872 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Mon, 21 Oct 2024 12:37:43 +0000 Subject: [PATCH 22/23] modified: docs/reference/configurations.md,docs/how-to/contribute.md,docs/index.md,docs/reference/integrations.md,docs/tutorial/quick-start.md,docs/reference/actions.md --- docs/how-to/contribute.md | 31 +------------- docs/index.md | 69 +++++++++++++++----------------- docs/reference/actions.md | 4 +- docs/reference/configurations.md | 4 +- docs/reference/integrations.md | 7 +++- docs/tutorial/quick-start.md | 44 ++++++++++++++++---- 6 files changed, 82 insertions(+), 77 deletions(-) diff --git a/docs/how-to/contribute.md b/docs/how-to/contribute.md index 6d7c2e0a4..2db63dff2 100644 --- a/docs/how-to/contribute.md +++ b/docs/how-to/contribute.md @@ -1,32 +1,3 @@ # How to contribute -## Overview - -This document explains the processes and practices recommended for contributing enhancements to the GitHub Runner operator. - -* Generally, before developing enhancements to this charm, you should consider [opening an issue](https://github.com/canonical/github-runner-operator/issues) explaining your use case. -* If you would like to chat with us about your use-cases or proposed implementation, you can reach us at [Canonical Mattermost public channel](https://chat.charmhub.io/charmhub/channels/charm-dev) or [Discourse](https://discourse.charmhub.io/). -* Familiarizing yourself with the [Charmed Operator Framework](https://juju.is/docs/sdk) library will help you a lot when working on new features or bug fixes. -* All enhancements require review before being merged. Code review typically examines - * code quality - * test coverage - * user experience for Juju administrators of this charm. -For more details, check our [contributing guide](https://github.com/canonical/is-charms-contributing-guide/blob/main/CONTRIBUTING.md). - -## Developing - -For any problems with this charm, please [report bugs here](https://github.com/canonical/github-runner-operator/issues). - -The code for this charm can be downloaded as follows: - -```shell -git clone https://github.com/canonical/github-runner-operator.git -``` - -Prior to working on the charm ensure juju is connected to an LXD cloud, see the [upstream documentation](https://juju.is/docs/lxd-cloud) for details. - -To test the charm, unit test can be ran with `tox -e unit` and the integration test on juju 3.1 can be ran with `tox -e integration-juju3.1`. - -## Canonical Contributor Agreement - -Canonical welcomes contributions to the GitHub Runner Operator. Please check out our [contributor agreement](https://ubuntu.com/legal/contributors) if you’re interested in contributing to the solution. \ No newline at end of file +See the [contributing guide](https://github.com/canonical/github-runner-operator/blob/main/CONTRIBUTING.md) on GitHub. \ No newline at end of file diff --git a/docs/index.md b/docs/index.md index dfedff4c5..d25ca4fc1 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,23 +1,20 @@ -A [Juju](https://juju.is/) [charm](https://juju.is/docs/olm/charmed-operators) for deploying and managing [GitHub self-hosted runners](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/about-self-hosted-runners) on virtual machines. +# GitHub Runner Operator -This charm simplifies the initial deployment and "day N" operations of GitHub self-hosted runners. The charm makes it easy to manage self-hosted runners with security and hardware resource usage in mind. +A [Juju](https://juju.is/) [charm](https://juju.is/docs/olm/charmed-operators) for deploying and managing [GitHub self-hosted runners](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/about-self-hosted-runners) on virtual machines. The charm maintains a set of ephemeral self-hosted runners, each isolated in a single-use virtual machine instance. -Operating a self-hosted runner comes with [certain security concerns according to GitHub](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/about-self-hosted-runners#self-hosted-runner-security). -Just like GitHub's, the self-hosted runners managed by the charm are isolated in a single-use virtual machine. - -Some of the charm dependencies upgrades on a schedule to migrate security risks. The landscape-client charm can be deployed with this charm to ensure other dependencies are up to date. - -The charm maintains a set of ephemeral self-hosted runners, each isolated in a single-use virtual machine instance. In addition, resource limits for the self-hosted runners can be configured. +Like any Juju charm, this charm supports one-line deployment, configuration, integration, scaling, and more. +For the github-runner-operator charm, this includes: +* Stateless operation. +* Configurable resource limits. +* Ability to redeploy without losing any data (no need to back up). +* Supported observability through the `cos-agent` integration. +* Scheduled dependences upgrades to mitigate security risks. Furthermore, the landscape-client charm can be deployed with this charm to ensure other dependencies are kept up to date. -See [charm architecture](https://charmhub.io/github-runner/docs/explanation-charm-architecture) for more information. - -The charm operates in a stateless manner. It can be redeployed without losing any data and there is no need to backup the charm's state. +Operating a self-hosted runner comes with [certain security concerns according to GitHub](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/about-self-hosted-runners#self-hosted-runner-security). +Just like GitHub's runners, the self-hosted runners managed by the charm are isolated in a single-use virtual machine. -The charm also supports observability through the optional `cos-agent` integration. Metrics and logs about the runners and the charm itself are collected and sent to the [Canonical Observability Stack](https://charmhub.io/topics/canonical-observability-stack) for analysis and visualisation. -This charm will make operating GitHub self-hosted runners simple and straightforward for DevOps or SRE teams through Juju's clean interface. - The charm enforces a set of GitHub repository settings as best practice. This is planned to be opt-in in the future. See [How to comply with repository policies](https://charmhub.io/github-runner/docs/how-to-repo-policy). ## In this documentation @@ -46,25 +43,24 @@ Thinking about using the GitHub runner charm for your next project? [Get in touc # Contents -1. [Explanation](explanation) - 1. [ARM64](explanation/arm64.md) - 1. [Charm architecture](explanation/charm-architecture.md) - 1. [SSH Debug](explanation/ssh-debug.md) -1. [How To](how-to) - 1. [How to add custom labels](how-to/add-custom-labels.md) - 1. [How to change repository or organization](how-to/change-path.md) - 1. [How to change GitHub personal access token](how-to/change-token.md) - 1. [How to comply with security requirements](how-to/comply-security.md) - 1. [How to restrict self-hosted runner network access](how-to/configure-denylist.md) - 1. [How to configure runner storage](how-to/configure-runner-storage.md) - 1. [How to contribute](how-to/contribute.md) - 1. [How to debug with ssh](how-to/debug-with-ssh.md) - 1. [How to deploy on ARM64](how-to/deploy-on-arm64.md) - 1. [How to integrate with COS](how-to/integrate-with-cos.md) - 1. [How to spawn OpenStack runner](how-to/openstack-runner.md) - 1. [How to comply with repository policies](how-to/repo-policy.md) - 1. [How to run on LXD cloud](how-to/run-on-lxd.md) - 1. [How to set base image](how-to/set-base-image.md) +1. [Tutorial](tutorial) + 1. [Managing resource usage](tutorial/managing-resource-usage.md) + 1. [Quick start](tutorial/quick-start.md) +1. [How to](how-to) + 1. [Add custom labels](how-to/add-custom-labels.md) + 1. [Change repository or organization](how-to/change-path.md) + 1. [Change GitHub personal access token](how-to/change-token.md) + 1. [Comply with security requirements](how-to/comply-security.md) + 1. [Restrict self-hosted runner network access](how-to/configure-denylist.md) + 1. [Configure runner storage](how-to/configure-runner-storage.md) + 1. [Contribute](how-to/contribute.md) + 1. [Debug with SSH](how-to/debug-with-ssh.md) + 1. [Deploy on ARM64](how-to/deploy-on-arm64.md) + 1. [Integrate with COS](how-to/integrate-with-cos.md) + 1. [Spawn OpenStack runner](how-to/openstack-runner.md) + 1. [Comply with repository policies](how-to/repo-policy.md) + 1. [Run on LXD cloud](how-to/run-on-lxd.md) + 1. [Set base image](how-to/set-base-image.md) 1. [Reference](reference) 1. [Actions](reference/actions.md) 1. [ARM64](reference/arm64.md) @@ -73,7 +69,8 @@ Thinking about using the GitHub runner charm for your next project? [Get in touc 1. [External Access](reference/external-access.md) 1. [Integrations](reference/integrations.md) 1. [Token scopes](reference/token-scopes.md) -1. [Tutorial](tutorial) - 1. [Managing resource usage](tutorial/managing-resource-usage.md) - 1. [Quick start](tutorial/quick-start.md) +1. [Explanation](explanation) + 1. [ARM64](explanation/arm64.md) + 1. [Charm architecture](explanation/charm-architecture.md) + 1. [SSH Debug](explanation/ssh-debug.md) 1. [Changelog](changelog.md) \ No newline at end of file diff --git a/docs/reference/actions.md b/docs/reference/actions.md index e2e78e54a..a20c82f30 100644 --- a/docs/reference/actions.md +++ b/docs/reference/actions.md @@ -1,3 +1,5 @@ # Actions -See [Actions](https://charmhub.io/github-runner/actions). \ No newline at end of file +See [Actions](https://charmhub.io/github-runner/actions). + +> Read more about actions in the Juju docs: [Action](https://juju.is/docs/juju/action) \ No newline at end of file diff --git a/docs/reference/configurations.md b/docs/reference/configurations.md index 68043cf7c..4e66f9b89 100644 --- a/docs/reference/configurations.md +++ b/docs/reference/configurations.md @@ -1,3 +1,5 @@ # Configurations -See [Configure](https://charmhub.io/github-runner/configure). \ No newline at end of file +See [Configurations](https://charmhub.io/github-runner/configure). + +> Read more about configurations in the Juju docs: [Configuration](https://juju.is/docs/juju/configuration) \ No newline at end of file diff --git a/docs/reference/integrations.md b/docs/reference/integrations.md index 6988e03bf..69b781833 100644 --- a/docs/reference/integrations.md +++ b/docs/reference/integrations.md @@ -2,10 +2,13 @@ ### debug-ssh -_Interface_: debug-ssh +_Interface_: debug-ssh _Supported charms_: [tmate-ssh-server](https://charmhub.io/tmate-ssh-server) Debug-ssh integration provides necessary information for runners to provide ssh reverse-proxy applications to setup inside the runner. -Example debug-ssh integrate command: `juju integrate github-runner tmate-ssh-server` \ No newline at end of file +Example debug-ssh integrate command: +``` +juju integrate github-runner tmate-ssh-server +``` \ No newline at end of file diff --git a/docs/tutorial/quick-start.md b/docs/tutorial/quick-start.md index 12f734a1c..25ff979c5 100644 --- a/docs/tutorial/quick-start.md +++ b/docs/tutorial/quick-start.md @@ -2,7 +2,7 @@ ## What you'll do -- Setup a GitHub repository +- Set up a GitHub repository - Activate the GitHub APIs related to self-hosted runner - Deploy the [GitHub runner charm](https://charmhub.io/github-runner) - Ensure GitHub repository setting are secure @@ -11,13 +11,32 @@ ## Requirements - GitHub Account. -- Juju 3 installed. -- Juju controller on OpenStack or LXD (see [How to run on LXD cloud](https://charmhub.io/github-runner/docs/how-to-run-on-lxd)). - -For more information about how to install and use Juju, see [Get started with Juju](https://juju.is/docs/olm/get-started-with-juju). +- A working station, e.g., a laptop, with amd64 architecture and at least 16 GB of RAM. +- Juju 3 installed and bootstrapped to an LXD controller. You can accomplish this process by following this guide: [Set up / Tear down your test environment](https://juju.is/docs/juju/set-up--tear-down-your-test-environment) +For more information about how to install Juju, see [Get started with Juju](https://juju.is/docs/olm/get-started-with-juju). ## Steps +### Add more RAM to the Multipass VM +> NOTE: If you're working locally, you don't need to do this step. +The blueprint used for deploying Multipass VM is configured with 8 GB of RAM. To add more RAM to the VM, follow these steps: +Stop the VM: +``` +multipass stop my-juju-vm +``` +Set the RAM to 16 GB with the following command: +``` +multipass set local.my-juju-vm.memory=16G +``` + +### Shell into the Multipass VM +> NOTE: If you're working locally, you don't need to do this step. + +To be able to work inside the Multipass VM first you need to log in with the following command: +``` +multipass shell my-juju-vm +``` + ### Create GitHub repository The GitHub self-hosted runner spawned by the charm needs to connect to a GitHub repository or organization. GitHub repositories are used as it is simpler to manage. @@ -34,6 +53,11 @@ The registration token can be requested by calling the [GitHub API](https://docs ### Set up the tutorial model +Switch to the LXD controller(`lxd` is the default LXD controller name if you are using the Multipass VM, if you bootstrapped LXD yourself please use the name set for it): +``` +juju switch lxd +``` + To easily clean up the resources and to separate your workload from the contents of this tutorial, set up a new Juju model with the following command. ``` @@ -48,7 +72,7 @@ For information on token scopes, see [How to change GitHub personal access token Once the personal access token is created, the charm can be deployed with: -```shell +``` juju deploy github-runner --constraints="cores=4 mem=16G root-disk=20G virt-type=virtual-machine" --config token= --config path= --config runner-storage=memory --config vm-memory=2GiB --config vm-disk=10GiB ``` @@ -68,7 +92,7 @@ The charm will spawn new runners on a schedule. During this time, the charm will ### Run a simple workflow on the self-hosted runner -Once the self-hosted runner is available on GitHub, it can be used to run GitHub Actions jobs similar to runners provided by GitHub. The only difference being the label specified in the `runs-on` of a job. +Once the self-hosted runner is available on GitHub, the runner can be used to run GitHub Actions jobs similar to runners provided by GitHub. The only difference being the label specified in the `runs-on` of a job. In addition to the labels added by the GitHub runner application by default, the charm will include labels from the [`labels` charm configuration](https://charmhub.io/github-runner/configurations#labels). @@ -101,4 +125,10 @@ The Juju model, charm and the self-hosted runners can be removed with the follow ```shell juju destroy-model --destroy-storage github-runner-tutorial +``` + +If you used Multipass, to remove the Multipass instance you created for this tutorial, run the following command outside of the VM. + +``` +multipass delete --purge my-juju-vm ``` \ No newline at end of file From 5ffef143bb53194cf9254ddc148f6b1383b148e7 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Mon, 11 Nov 2024 08:21:41 +0000 Subject: [PATCH 23/23] modified: docs/how-to/openstack-runner.md --- docs/how-to/openstack-runner.md | 21 ++++++++++++--------- 1 file changed, 12 insertions(+), 9 deletions(-) diff --git a/docs/how-to/openstack-runner.md b/docs/how-to/openstack-runner.md index 53dd770a2..7201309f8 100644 --- a/docs/how-to/openstack-runner.md +++ b/docs/how-to/openstack-runner.md @@ -1,8 +1,8 @@ # How to spawn OpenStack runner The charm can be configured to use OpenStack cloud for creating runners. -The charm must be deployed with the correct configuration and once the OpenStack integration is -enabled the charm cannot be changed to use other virtualization methods. +The charm must be deployed with the correct configuration, and once the OpenStack integration is +enabled, the charm cannot be changed to use other virtualization methods. ## Configuration @@ -24,7 +24,7 @@ The image will take about 10-15 minutes to build and be ready via the relation. ### OpenStack clouds.yaml -The `openstack-clouds-yaml` configuration contains the authorization information needed for the charm to log in to the openstack cloud. +The `openstack-clouds-yaml` configuration contains the authorization information needed for the charm to log in to the OpenStack cloud. The first cloud in the `clouds.yaml` is used by the charm. Here is a sample of the `clouds.yaml`: @@ -44,16 +44,19 @@ clouds: The `clouds.yaml` documentation is [here](https://docs.openstack.org/python-openstackclient/pike/configuration/index.html#clouds-yaml). -### OpenStack Flavor +### OpenStack Flavour -The `openstack-flavor` configuration sets the flavor used to create the OpenStack virtual machine when spawning new runners. -The flavor is tied with the vCPU, memory, and storage. -The flavors documentation is [here](https://docs.openstack.org/nova/rocky/user/flavors.html). +The `openstack-flavor` configuration sets the flavour used to create the OpenStack virtual machine when spawning new runners. +The flavour is tied with the vCPU, memory, and storage. +The flavours documentation is [here](https://docs.openstack.org/nova/rocky/user/flavors.html). ### OpenStack Network The `openstack-network` configuration sets the network used to create the OpenStack virtual machine when spawning new runners. -Note that the network should be configured to allow traffic from the charm deployment (juju machine) to the OpenStack virtual machine, and traffic from the OpenStack virtual machine to GitHub. +Note that the network should be configured to allow traffic from the charm deployment (Juju machine) to the OpenStack virtual machine, and traffic from the OpenStack virtual machine to GitHub. -The network documentation is [here](https://docs.openstack.org/neutron/latest/admin/intro-os-networking.html). \ No newline at end of file +The network documentation is [here](https://docs.openstack.org/neutron/latest/admin/intro-os-networking.html). + +> NOTE: The name of the application must not be longer than 50 characters. A valid runner name is 64 characters or less in length and does not include '"', '/', ':', +'<', '>', '\', '|', '*' and '?'. 14 characters are reserved for Juju unit number and unique identifier. \ No newline at end of file