Update test scripts and instructions to use them.

- Updated the `run-e2e` script to allow running tests without recording
  goldens
- Updated `record-gcp` to call `run-e2e` instead of reproducing most of
  its functionality
- Renamed `compare-mock` to `record-mock` to better reflect what it does
- Added `compare-mock` and `compare-gcp` to provide convenient ways of
  running each in non-record mode
- Updated mockgcp docs to reference the scripts instead of using the
  full command line
- Updated direct controller step 1 doc with the new scripts

dev/tasks/run-e2e

@@ -77,9 +77,15 @@ if [[ -z "${TEST_TIMEOUT:-}" ]]; then
 fi
 echo "Using TEST_TIMEOUT=${TEST_TIMEOUT}"
 
+# Check if WRITE_GOLDEN_OUTPUT is unset (does NOT match empty string, so an empty WRITE_GOLDEN_OUTPUT can be used to disable writing output)
+if [[ -z "${WRITE_GOLDEN_OUTPUT+set}" ]]; then
+  WRITE_GOLDEN_OUTPUT=1
+fi
+echo "WRITE_GOLDEN_OUTPUT=${WRITE_GOLDEN_OUTPUT}"
+
 # Run e2e tests against real gcp, capturing output
 GOLDEN_REQUEST_CHECKS=1 \
 GOLDEN_OBJECT_CHECKS=1 \
-WRITE_GOLDEN_OUTPUT=1 \
+WRITE_GOLDEN_OUTPUT=${WRITE_GOLDEN_OUTPUT} \
 RUN_E2E=1 \
  go test ./tests/e2e -timeout ${TEST_TIMEOUT} -v -run $RUN_TESTS

docs/develop-resources/README.md

@@ -6,6 +6,8 @@ In the past few months, we have made a tremendous amount of efforts to make addi
 
 There are definitely more work need to be done, but we'd like to share the steps about adding the direct resource so that it can benefit more users. We will conitnue improving this guide and making the step changes to make it simpler and easier to use. Please stay tuned for the upcoming changes.
 
+Be sure to work through the [Contributing guide](../../CONTRIBUTING.md) before starting.
+
 # Contents
 
 ## Introduction

docs/develop-resources/guides/1-add-mockgcp-tests.md

@@ -40,11 +40,10 @@ hack/record-gcp fixtures/<your_resource_test>
 
 #### Record mock GCP log. 
 
-Note: This will override the output!
-(remove the` WRITE_GOLDEN_OUTPUT=1` in hack/compare-mock if you don’t want to override)
+Note: This will overwrite the golden output! (use `hack/compare-mock` to just compare instead)
 
 ```
-hack/compare-mock fixtures/<your_resource_test>
+hack/record-mock fixtures/<your_resource_test>
 ```
 
 ## Exit Criteria

hack/compare-gcp

@@ -0,0 +1,17 @@
+#!/bin/bash
+
+set -o errexit
+set -o nounset
+set -o pipefail
+
+REPO_ROOT="$(git rev-parse --show-toplevel)"
+cd ${REPO_ROOT}
+
+# Run the tests specified on the command line
+export RUN_TESTS=TestAllInSeries/$1
+
+# Clear existing artifacts
+rm -rf $(pwd)/artifactz/realgcp
+
+# Run e2e tests against real gcp
+WRITE_GOLDEN_OUTPUT="" ARTIFACTS=$(pwd)/artifactz/realgcp E2E_GCP_TARGET=real dev/tasks/run-e2e

hack/compare-mock

@@ -26,5 +26,5 @@ export RUN_TESTS=TestAllInSeries/$1
 # Clear existing artifacts
 rm -rf $(pwd)/artifactz/mock
 
-# Run e2e tests against our mockgcp, capturing output
-E2E_GCP_TARGET=mock dev/tasks/run-e2e
+# Run e2e tests against our mockgcp
+WRITE_GOLDEN_OUTPUT="" E2E_GCP_TARGET=mock dev/tasks/run-e2e

hack/record-gcp

@@ -1,44 +1,17 @@
 #!/bin/bash
 
-set -e
-set -x
+set -o errexit
+set -o nounset
+set -o pipefail
 
-export KUBEBUILDER_ASSETS=$(go run sigs.k8s.io/controller-runtime/tools/setup-envtest@latest use -p path)
+REPO_ROOT="$(git rev-parse --show-toplevel)"
+cd ${REPO_ROOT}
 
-rm -rf $(pwd)/artifactz/realgcp
-
-RUN_TESTS=TestAllInSeries/$1
-
-if [[ -z "${GCP_PROJECT_ID:-}" ]]; then
-  GCP_PROJECT_ID=$(gcloud config get-value project)
-fi
-echo "Using project: ${GCP_PROJECT_ID}"
-
-TEST_FOLDER_ID=$(gcloud projects describe ${GCP_PROJECT_ID} --format='value(parent.id)')
-export TEST_FOLDER_ID
+# Run the tests specified on the command line
+export RUN_TESTS=TestAllInSeries/$1
 
-TEST_ORG_ID=$(gcloud projects get-ancestors ${GCP_PROJECT_ID} | grep organization | cut -f1 -d ' ')
-export TEST_ORG_ID
-
-if [[ -z "${TEST_BILLING_ACCOUNT_ID:-}" ]]; then
-  TEST_BILLING_ACCOUNT_ID=$(gcloud billing projects describe ${GCP_PROJECT_ID} --format='value(billingAccountName)' | cut -f2 -d/)
-fi
-echo "TEST_BILLING_ACCOUNT_ID=${TEST_BILLING_ACCOUNT_ID}"
-export TEST_BILLING_ACCOUNT_ID
-
-if [[ -z "${KCC_USE_DIRECT_RECONCILERS:-}" ]]; then
-  KCC_USE_DIRECT_RECONCILERS="ComputeForwardingRule"
-fi
-export KCC_USE_DIRECT_RECONCILERS
-echo "KCC_USE_DIRECT_RECONCILERS=$KCC_USE_DIRECT_RECONCILERS"
+# Clear existing artifacts
+rm -rf $(pwd)/artifactz/realgcp
 
 # Run e2e tests against real gcp, capturing output
-ARTIFACTS=$(pwd)/artifactz/realgcp \
-E2E_KUBE_TARGET=envtest \
-E2E_GCP_TARGET=real \
-GOLDEN_REQUEST_CHECKS=1 \
-GOLDEN_OBJECT_CHECKS=1 \
-WRITE_GOLDEN_OUTPUT=1 \
-RUN_E2E=1 \
-KCC_USE_DIRECT_RECONCILERS=$KCC_USE_DIRECT_RECONCILERS \
- go test ./tests/e2e -timeout 7200s -v -run $RUN_TESTS
+ARTIFACTS=$(pwd)/artifactz/realgcp E2E_GCP_TARGET=real dev/tasks/run-e2e

hack/record-mock

@@ -0,0 +1,30 @@
+#!/usr/bin/env bash
+# Copyright 2024 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+set -o errexit
+set -o nounset
+set -o pipefail
+
+REPO_ROOT="$(git rev-parse --show-toplevel)"
+cd ${REPO_ROOT}
+
+# Run the tests specified on the command line
+export RUN_TESTS=TestAllInSeries/$1
+
+# Clear existing artifacts
+rm -rf $(pwd)/artifactz/mock
+
+# Run e2e tests against our mockgcp, capturing output
+E2E_GCP_TARGET=mock dev/tasks/run-e2e

mockgcp/README.md

@@ -91,23 +91,26 @@ and it normally logs an error like "foo not set" (in this case, simply add that
 
 1. Capture golden object and HTTP golden logs against real GCP.
 
-   1. Run the following command to generate the golden object (`_generated_object_\[testname\].golden.yaml` file):
-      `E2E_KUBE_TARGET=envtest RUN_E2E=1 E2E_GCP_TARGET=real GOLDEN_OBJECT_CHECKS=1 WRITE_GOLDEN_OUTPUT=1 go test -test.count=1 -timeout 3600s -v ./tests/e2e -run TestAllInSeries/fixtures/[testname]`.
+   1. Run the following command to generate the golden object (`_generated_object_\[testname\].golden.yaml` file) and golden request (`_http.log` file):
+      ```
+      hack/record-gcp fixtures/[testname]
+      ```
 
-   1. Run the following command to generate the golden request (`_http.log` file):
-      `E2E_KUBE_TARGET=envtest RUN_E2E=1 E2E_GCP_TARGET=real GOLDEN_REQUEST_CHECKS=1 WRITE_GOLDEN_OUTPUT=1 go test -test.count=1 -timeout 3600s -v ./tests/e2e -run TestAllInSeries/fixtures/[testname]`.
-
-   1. Ensure both `GOLDEN_OBJECT_CHECKS` and `GOLDEN_REQUEST_CHECKS` pass:
-      `E2E_KUBE_TARGET=envtest RUN_E2E=1 E2E_GCP_TARGET=real GOLDEN_OBJECT_CHECKS=1 GOLDEN_REQUEST_CHECKS=1 go test -test.count=1 -timeout 3600s -v ./tests/e2e -run TestAllInSeries/fixtures/[testname]`.
+   1. Ensure recorded goldens pass:
+      ```
+      hack/compare-gcp fixtures/[testname]
+      ```
 
       1. If the test doesn't pass, check the output log and identify diffs. Normalize the values if needed in
          [tests/e2e/normalize.go](https://github.com/GoogleCloudPlatform/k8s-config-connector/blob/v1.120.1/tests/e2e/normalize.go#L66)
          for GOLDEN_OBJECT_CHECKS and in
          [tests/e2e/unified_test.go](https://github.com/GoogleCloudPlatform/k8s-config-connector/blob/v1.120.1/tests/e2e/unified_test.go#L523)
          for GOLDEN_REQUEST_CHECKS.
 
-1. Ensure the test works when `E2E_GCP_TARGET=mock`:
-   `E2E_KUBE_TARGET=envtest RUN_E2E=1 E2E_GCP_TARGET=mock GOLDEN_OBJECT_CHECKS=1 GOLDEN_REQUEST_CHECKS=1 go test -test.count=1 -timeout 3600s -v ./tests/e2e -run TestAllInSeries/fixtures/[testname]`.
+1. Ensure the test works against mockGCP:
+   ```
+   hack/compare-mock fixtures/[testname]
+   ```
 
    1. If the test doesn't pass, update the CRUD methods in the mock to match the behavior needed by the golden files.