Skip to content

Commit

Permalink
Remove the uptest tool and the associated libraries.
Browse files Browse the repository at this point in the history
- Update README.md so that it mentions about the remaining tools
  and the reusable CI workflows in the repository.

Signed-off-by: Alper Rifat Ulucinar <[email protected]>
  • Loading branch information
ulucinar committed Mar 12, 2024
1 parent 60b6090 commit 32a5756
Show file tree
Hide file tree
Showing 19 changed files with 22 additions and 1,746 deletions.
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ GO_REQUIRED_VERSION = 1.21
# Uncomment below if you need to override the version.
# GOLANGCILINT_VERSION ?= 1.54.0

GO_STATIC_PACKAGES = $(GO_PROJECT)/cmd/uptest $(GO_PROJECT)/cmd/crddiff $(GO_PROJECT)/cmd/buildtagger $(GO_PROJECT)/cmd/updoc $(GO_PROJECT)/cmd/ttr $(GO_PROJECT)/cmd/perf $(GO_PROJECT)/cmd/linter/lint-provider-family
GO_STATIC_PACKAGES = $(GO_PROJECT)/cmd/crddiff $(GO_PROJECT)/cmd/buildtagger $(GO_PROJECT)/cmd/updoc $(GO_PROJECT)/cmd/ttr $(GO_PROJECT)/cmd/perf $(GO_PROJECT)/cmd/linter/lint-provider-family
GO_LDFLAGS += -X $(GO_PROJECT)/internal/version.Version=$(VERSION)
GO_SUBDIRS += cmd internal
GO111MODULE = on
Expand Down
152 changes: 21 additions & 131 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,134 +1,24 @@
# UPTEST

The end to end integration testing tool for Crossplane providers and configurations.

Uptest comes as a binary which can be installed from the releases section. It runs end-to-end tests
by applying the provided examples and waiting for the expected conditions. Other than that, it enables templating to
insert dynamic values into the examples and supports running scripts as hooks just before and right after applying
the examples.

## Usage

```shell
$ uptest e2e --help
usage: uptest e2e [<flags>] [<manifest-list>]

Run e2e tests for manifests by applying them to a control plane and waiting until a given condition is met.

Flags:
--help Show context-sensitive help (also try --help-long and --help-man).
--data-source="" File path of data source that will be used for injection some values.
--setup-script="" Script that will be executed before running tests.
--teardown-script="" Script that will be executed after running tests.
--default-timeout=1200 Default timeout in seconds for the test. Timeout could be overridden per resource using
"uptest.upbound.io/timeout" annotation.
--default-conditions="Ready" Comma seperated list of default conditions to wait for a successful test. Conditions could be
overridden per resource using "uptest.upbound.io/conditions" annotation.

Args:
[<manifest-list>] List of manifests. Value of this option will be used to trigger/configure the tests.The possible usage:
'provider-aws/examples/s3/bucket.yaml,provider-gcp/examples/storage/bucket.yaml': The comma separated resources
are used as test inputs. If this option is not set, 'MANIFEST_LIST' env var is used as default.
```
Uptest expects a running control-plane (a.k.a. k8s + crossplane) where required providers are running and/or required
configuration were applied.
Example run:
```shell
uptest e2e examples/user.yaml,examples/bucket.yaml --setup-script="test/hooks/setup.sh"
```
### Injecting Dynamic Values (and Datasource)
Uptest supports injecting dynamic values into the examples by using a data source. The data source is a yaml file
storing key-value pairs. The values can be used in the examples by using the following syntax:
```
${data.key}
```
Example data source file content:
```yaml
aws_account_id: 123456789012
aws_region: us-east-1
```
Example manifest:
```yaml
apiVersion: athena.aws.upbound.io/v1beta1
kind: DataCatalog
metadata:
labels:
testing.upbound.io/example-name: example
name: example
spec:
forProvider:
description: Example Athena data catalog
parameters:
function: arn:aws:lambda:${data.aws_region}:${data.aws_account_id}:function:upbound-example-function
region: us-west-1
tags:
Name: example-athena-data-catalog
type: LAMBDA
```
Uptest also supports generating random strings as follows:
```
${Rand.RFC1123Subdomain}
```
Example Manifest:
```yaml
apiVersion: s3.aws.upbound.io/v1beta1
kind: Bucket
metadata:
name: ${Rand.RFC1123Subdomain}
labels:
testing.upbound.io/example-name: s3
spec:
forProvider:
region: us-west-1
objectLockEnabled: true
tags:
Name: SampleBucket
```
### Hooks
There are 6 types of hooks that can be used to customize the test flow:
1. `setup-script`: This hook will be executed before running the tests case. It is useful to set up the control plane
before running the tests. For example, you can use it to create a provider config and your cloud credentials. This
can be configured via `--setup-script` flag as a relative path to where uptest is executed.
2. `teardown-script`: This hook will be executed after running the tests case. This can be configured via
`--teardown-script` flag as a relative path to where uptest is executed.
3. `pre-assert-hook`: This hook will be executed before running the assertions and after applying a specific manifest.
This can be configured via `uptest.upbound.io/pre-assert-hook` annotation on the manifest as a relative path to the
manifest file.
4. `post-assert-hook`: This hook will be executed after running the assertions. This can be configured via
`uptest.upbound.io/post-assert-hook` annotation on the manifest as a relative path to the manifest file.
5. `pre-delete-hook`: This hook will be executed just before deleting the resource. This can be configured via
`uptest.upbound.io/pre-delete-hook` annotation on the manifest as a relative path to the manifest file.
6. `post-delete-hook`: This hook will be executed right after the resource is deleted. This can be configured via
`uptest.upbound.io/post-delete-hook` annotation on the manifest as a relative path to the manifest file.
> All hooks need to be executables, please make sure to set the executable bit on your scripts, e.g. with `chmod +x`.
### Troubleshooting
Uptest uses [kuttl](https://kuttl.dev/) under the hood and generates a `kuttl` test case based on the provided input.
You can inspect the generated kuttl test case by checking the temporary test directory which is printed in the beginning
of uptest e2e output. For example:
```shell
Running kuttl tests at /var/folders/_5/jc7399qx6cn6t5535npv9z4c0000gn/T/uptest-e2e
```
# official-providers-ci
Repository for the CI tooling of the Upbound official providers repositories:
- `crddiff`: A tool for checking breaking API changes between two CRD OpenAPI v3 schemas. The schemas can come from either two revisions of a CRD, or from the versions declared in a single CRD.
- `buildtagger`: A tool for generating build tags (constraints) for the source modules of the official provider families.
- `lint-provider-family`: A linter for the official provider families. Checks whether all CRDs generated for a provider family are packaged in the corresponding service-scoped provider and checks the provider metadata.
- `perf`: A tool for running performance experiments in the official provider repositories and for collecting & reporting the CPU & Memory utilizations and time to readiness (TTR) for MRs in these experiments.
- `ttr`: A tool that reports the time-to-readiness (TTR) measurements for a subset of the managed resources in a Kubernetes cluster.
- `updoc`: Upbound enhanced document processor.

This repository is also the home of the Upbound official providers reusable workflows:
- `.github/workflows/provider-ci.yml`: A reusable CI workflow for building, linting & validating the official providers.
- `.github/workflows/pr-comment-trigger.yml`: A reusable workflow for triggering `uptest` runs using a specified set of example manifests via pull request comments.
- `.github/workflows/provider-publish-service-artifacts.yml`: A reusable workflow for building the official provider families and pushing their packages to the Upbound registry.
- `.github/workflows/native-provider-bump.yml`: A reusable workflow for bumping the underlying Terraform provider versions of upjet-based official providers.
- `.github/workflows/provider-backport.yml`: A reusable workflow for opening backport PRs in the specified release branches by inspecting the labels on merged PRs.
- `.github/workflows/issue-triage.yml`: A reusable workflow for identifying and labeling issues opened by the community.
- `.github/workflows/pr-triage.yml`: A reusable workflow for identifying and labeling pull requests opened by the community.
- `.github/workflows/provider-tag.yml`: A reusable workflow for tagging commits in the release process.
- `.github/workflows/provider-updoc.yml`: A reusable workflow for running `updoc` and publishing the provider documentation to the [Upbound marketplace](https://marketplace.upbound.io/providers).
- `.github/workflows/scan.yml`: A reusable workflow for running [Trivy](https://trivy.dev) scans in the official provider repositories.
- `.github/workflows/provider-commands.yml`: A reusable workflow for opening backport PRs in the specified release branches via PR comments.

## Report a Bug

Expand Down
108 changes: 0 additions & 108 deletions cmd/uptest/main.go

This file was deleted.

Loading

0 comments on commit 32a5756

Please sign in to comment.