Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

e2e tests maintenance #158

Merged
merged 1 commit into from
Jan 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 6 additions & 12 deletions e2e/Makefile
Original file line number Diff line number Diff line change
@@ -1,22 +1,22 @@
export GO111MODULE=on
IMG ?= linode/linode-cloud-controller-manager:latest
GINKGO_PATH := $(shell go env GOPATH)/bin/ginkgo

REUSED_KUBECONFIG := test/ccm-linode-for-reuse.conf

ifneq ("$(wildcard $(REUSED_KUBECONFIG))","")
CONTROL_PLANE_IP := $(shell grep -E -o "([0-9]{1,3}[\.]){3}[0-9]{1,3}" $(REUSED_KUBECONFIG))
endif

imports: $(GOPATH)/bin/goimports
goimports -w test

.PHONY: test reuse-and-test clean

${GINKGO_PATH}:
go install github.com/onsi/ginkgo/v2/[email protected]

reuse-and-test: SUITE_ARGS='--reuse'

test reuse-and-test: $(GOPATH)/bin/ginkgo check-token
test reuse-and-test: ${GINKGO_PATH} check-token
go list -m; \
ginkgo -r --v --progress --trace --cover $(TEST_ARGS) -- --v=3 --image=${IMG} $(SUITE_ARGS)
ginkgo -r --v --trace --cover $(TEST_ARGS) -- --v=3 --image=${IMG} $(SUITE_ARGS)

clean: check-token
cd test; \
Expand All @@ -30,11 +30,5 @@ check-token:
exit 1; \
fi

install-terraform:
sudo apt-get install wget unzip
wget https://releases.hashicorp.com/terraform/0.11.13/terraform_0.11.13_linux_amd64.zip
unzip terraform_0.11.13_linux_amd64.zip
sudo mv terraform /usr/local/bin/

control-plane-ssh: $(REUSED_KUBECONFIG)
ssh root@$(CONTROL_PLANE_IP)
64 changes: 2 additions & 62 deletions e2e/README.md
Original file line number Diff line number Diff line change
@@ -1,64 +1,4 @@
## How to run these End-to-end (e2e) tests

Install the following packages (macOS examples)

```
brew install terraform
brew install kubectl
brew install hg
brew install golang
```

Add the following environment variables to your shell rc

```
export LINODE_API_TOKEN=<your linode API token>

export GOPATH=$HOME/go
export PATH=$HOME/go/bin:$PATH
export GO111MODULE=on
```

If you need a Linode API token visit this page:
https://cloud.linode.com/profile/tokens

Then, `go get` this repo
`go get github.com/linode/linode-cloud-controller-manager`

That may fail, if it does, navigate to the directory that was created and run `go mod tidy`:

```
cd ~/go/src/github.com/linode/linode-cloud-controller-manager
go mod tidy
```

Then, use the makefile in the directory above this directory to build the CCM (this is to download goimports)

```
cd $GOPATH/src/github.com/linode/linode-cloud-controller-manager
make build
```

By default the tests use $HOME/.ssh/id\_rsa.pub as the public key used to provision the cluster, so it needs to be added to your agent.

```
ssh-add $HOME/.ssh/id_rsa
```

Come back here and run the tests

```
cd e2e
make test
```

To save time on multiple runs by allowing the cluster to remain, use `make reuse-and-test`

### Generating a new server certificate for testing

Some of the tests require a secret containing a TLS certificate, for use in creating or updating a NodeBalancer config using TLS. A CA certificate and server certificate can be found in the `test/certificates` directory. The server certificate, used for TLS NodeBalancer configs, has an expiry of 4 years. You can use the following command to generate a new TLS certificate, using the existing CSR:

```
openssl x509 -req -in test/certificates/server.csr -CA test/certificates/ca.crt -CAkey test/certificates/ca.key -CAcreateserial -out test/certificates/server.crt -days 1440 -sha256 -extfile <(printf "subjectAltName=DNS:linode.test,DNS:www.linode.test")
```
Once a new cert is generated, you will need to replace the existing constants, "serverCert" and "serverKey", in test/framework/secret.go.
TBD: the way we run these e2e tests has recently changed, there is currently
no convenient way to run these with no pre-existing clusters.
3 changes: 0 additions & 3 deletions e2e/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -4,19 +4,16 @@ go 1.17

require (
github.com/appscode/go v0.0.0-20200323182826-54e98e09185a
github.com/codeskyblue/go-sh v0.0.0-20190412065543-76bd3d59ff27
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
github.com/linode/linodego v0.21.1
github.com/onsi/ginkgo/v2 v2.3.1
github.com/onsi/gomega v1.22.0
github.com/pkg/errors v0.9.1
k8s.io/api v0.19.2
k8s.io/apimachinery v0.19.2
k8s.io/client-go v0.19.2
)

require (
github.com/codegangsta/inject v0.0.0-20150114235600-33e0aa1cb7c0 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/go-logr/logr v0.2.0 // indirect
github.com/go-resty/resty/v2 v2.1.1-0.20191201195748-d7b97669fe48 // indirect
Expand Down
3 changes: 0 additions & 3 deletions e2e/go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -36,9 +36,7 @@ github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWR
github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI=
github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU=
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
github.com/codegangsta/inject v0.0.0-20150114235600-33e0aa1cb7c0 h1:sDMmm+q/3+BukdIpxwO365v/Rbspp2Nt5XntgQRXq8Q=
github.com/codegangsta/inject v0.0.0-20150114235600-33e0aa1cb7c0/go.mod h1:4Zcjuz89kmFXt9morQgcfYZAYZ5n8WHjt81YYWIwtTM=
github.com/codeskyblue/go-sh v0.0.0-20190412065543-76bd3d59ff27 h1:HHUr4P/aKh4quafGxDT9LDasjGdlGkzLbfmmrlng3kA=
github.com/codeskyblue/go-sh v0.0.0-20190412065543-76bd3d59ff27/go.mod h1:VQx0hjo2oUeQkQUET7wRwradO6f+fN5jzXgB/zROxxE=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
Expand Down Expand Up @@ -176,7 +174,6 @@ github.com/onsi/gomega v1.22.0 h1:AIg2/OntwkBiCg5Tt1ayyiF1ArFrWFoCSMtMi/wdApk=
github.com/onsi/gomega v1.22.0/go.mod h1:iYAIXgPSaDHak0LCMA+AWBpIKBr8WZicMxnE8luStNc=
github.com/peterbourgon/diskv v2.0.1+incompatible/go.mod h1:uqqh8zWWbv1HBMNONnaR/tNboyR3/BZd58JJSHlUSCU=
github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
Expand Down
6 changes: 3 additions & 3 deletions e2e/test/ccm_e2e_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@ import (
"context"
"e2e_test/test/framework"
"fmt"
"os/exec"
"strconv"

"github.com/codeskyblue/go-sh"
"github.com/linode/linodego"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
Expand Down Expand Up @@ -231,12 +231,12 @@ var _ = Describe("e2e tests", func() {
}

var addNewNode = func() {
_, err := sh.Command("terraform", "apply", "-var", "nodes=3", "-auto-approve").Output()
err := exec.Command("terraform", "apply", "-var", "nodes=3", "-auto-approve").Run()
Expect(err).NotTo(HaveOccurred())
}

var deleteNewNode = func() {
_, err := sh.Command("terraform", "apply", "-var", "nodes=2", "-auto-approve").Output()
err := exec.Command("terraform", "apply", "-var", "nodes=2", "-auto-approve").Run()
Expect(err).NotTo(HaveOccurred())
}

Expand Down
3 changes: 1 addition & 2 deletions e2e/test/framework/service.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@ import (
"context"
"fmt"

"github.com/pkg/errors"
core "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/util/intstr"
Expand Down Expand Up @@ -117,7 +116,7 @@ func (i *lbInvocation) GetLoadBalancerIps() ([]string, error) {
}
}
if serverAddr == nil {
return nil, errors.Errorf("failed to get Status.LoadBalancer.Ingress for service %s/%s", TestServerResourceName, i.Namespace())
return nil, fmt.Errorf("failed to get Status.LoadBalancer.Ingress for service %s/%s", TestServerResourceName, i.Namespace())
}
return serverAddr, nil
}
Expand Down
Loading