From 7d44a4f0111e32b70baa8b59d9bdb77741ebfb7d Mon Sep 17 00:00:00 2001 From: Shankari Date: Sat, 9 Mar 2024 21:27:00 -0800 Subject: [PATCH 01/12] Turn on basic testing MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit I _wanted_ to turn on the automated tests that we have, but it fails because pytest is not installed. @couryrr-afs can you please fix the automated tests so we can include them as well.
Existing automated e2e tests fail ``` $ bash demo-automated-testing.sh [+] Building 0.0s (0/0) docker:desktop-linux [+] Running 2/0 ✔ Container everest-ac-demo-mqtt-server-1 Created 0.0s ✔ Container everest-ac-demo-manager-1 Created 0.0s Attaching to everest-ac-demo-manager-1, everest-ac-demo-mqtt-server-1 everest-ac-demo-mqtt-server-1 | 1710048535: mosquitto version 2.0.10 starting everest-ac-demo-mqtt-server-1 | 1710048535: Config loaded from /mosquitto/config/mosquitto.conf. everest-ac-demo-mqtt-server-1 | 1710048535: Opening ipv4 listen socket on port 1883. everest-ac-demo-mqtt-server-1 | 1710048535: Opening websockets listen socket on port 9001. everest-ac-demo-mqtt-server-1 | 1710048535: mosquitto version 2.0.10 running everest-ac-demo-manager-1 | ./run-test.sh: line 3: pytest: not found everest-ac-demo-manager-1 exited with code 127 ```
Signed-off-by: Shankari --- .github/workflows/e2etest.yaml | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) create mode 100644 .github/workflows/e2etest.yaml diff --git a/.github/workflows/e2etest.yaml b/.github/workflows/e2etest.yaml new file mode 100644 index 00000000..b609408c --- /dev/null +++ b/.github/workflows/e2etest.yaml @@ -0,0 +1,20 @@ +name: e2etest + +on: + schedule: + # * is a special character in YAML so you have to quote this string + - cron: '5 4 * * *' + +jobs: + pull-and-run-tests: + runs-on: ${{ matrix.os }} + strategy: + matrix: + os: [ubuntu-latest, macos-latest] + + steps: + - name: Download and launch + id: curl-download-and-launch + shell: bash + run: | + curl https://raw.githubusercontent.com/everest/everest-demo/main/demo-iso15118-2-ac-plus-ocpp201.sh | bash From 73e5d7120addf2143071035ffbe9b3311051c484 Mon Sep 17 00:00:00 2001 From: Shankari Date: Sat, 9 Mar 2024 21:36:13 -0800 Subject: [PATCH 02/12] Turn it on for pull requests so we can test that it works Signed-off-by: Shankari --- .github/workflows/e2etest.yaml | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/.github/workflows/e2etest.yaml b/.github/workflows/e2etest.yaml index b609408c..0b91ae01 100644 --- a/.github/workflows/e2etest.yaml +++ b/.github/workflows/e2etest.yaml @@ -1,6 +1,10 @@ name: e2etest on: + pull_request: + branches: + - main + schedule: # * is a special character in YAML so you have to quote this string - cron: '5 4 * * *' From 64872a5843a1f4d5c3bcc813a164190780a985cf Mon Sep 17 00:00:00 2001 From: Shankari Date: Sat, 9 Mar 2024 21:40:40 -0800 Subject: [PATCH 03/12] Remove macos since it doesn't have docker preinstalled --- .github/workflows/e2etest.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/e2etest.yaml b/.github/workflows/e2etest.yaml index 0b91ae01..0fad2e3b 100644 --- a/.github/workflows/e2etest.yaml +++ b/.github/workflows/e2etest.yaml @@ -14,7 +14,7 @@ jobs: runs-on: ${{ matrix.os }} strategy: matrix: - os: [ubuntu-latest, macos-latest] + os: [ubuntu-latest] steps: - name: Download and launch From f33a589c2eb0386f1cc5957e8754a4581fdaa256 Mon Sep 17 00:00:00 2001 From: Shankari Date: Sat, 9 Mar 2024 23:24:59 -0800 Subject: [PATCH 04/12] Set a timeout so that the action doesn't run forever When we have true automated tests, we will not need this, but the scripts all currently run without -d and expect UI interaction, so we will timeout. Note that this means that the action will always be canceled, but not much we can do to avoid that now. Signed-off-by: Shankari --- .github/workflows/e2etest.yaml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/.github/workflows/e2etest.yaml b/.github/workflows/e2etest.yaml index 0fad2e3b..9e996375 100644 --- a/.github/workflows/e2etest.yaml +++ b/.github/workflows/e2etest.yaml @@ -20,5 +20,6 @@ jobs: - name: Download and launch id: curl-download-and-launch shell: bash + timeout-minutes: 10 run: | - curl https://raw.githubusercontent.com/everest/everest-demo/main/demo-iso15118-2-ac-plus-ocpp201.sh | bash + timeout curl https://raw.githubusercontent.com/everest/everest-demo/main/demo-iso15118-2-ac-plus-ocpp201.sh | bash From 008e0cb4dd084bc40819d71cb933535656852e04 Mon Sep 17 00:00:00 2001 From: Shankari Date: Sat, 9 Mar 2024 23:24:59 -0800 Subject: [PATCH 05/12] Set a timeout so that the action doesn't run forever When we have true automated tests, we will not need this, but the scripts all currently run without -d and expect UI interaction, so we will timeout. Note that this means that the action will always be canceled, but not much we can do to avoid that now. Signed-off-by: Shankari --- .github/workflows/e2etest.yaml | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/workflows/e2etest.yaml b/.github/workflows/e2etest.yaml index 0fad2e3b..af4fa362 100644 --- a/.github/workflows/e2etest.yaml +++ b/.github/workflows/e2etest.yaml @@ -20,5 +20,6 @@ jobs: - name: Download and launch id: curl-download-and-launch shell: bash + timeout-minutes: 10 run: | curl https://raw.githubusercontent.com/everest/everest-demo/main/demo-iso15118-2-ac-plus-ocpp201.sh | bash From b4fb099c598ea59dcc06464ac231c78dde1c3bc6 Mon Sep 17 00:00:00 2001 From: Shankari Date: Sat, 9 Mar 2024 23:30:28 -0800 Subject: [PATCH 06/12] Remove timeout from command since we have it as an action parameter Signed-off-by: Shankari --- .github/workflows/e2etest.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/e2etest.yaml b/.github/workflows/e2etest.yaml index 9e996375..af4fa362 100644 --- a/.github/workflows/e2etest.yaml +++ b/.github/workflows/e2etest.yaml @@ -22,4 +22,4 @@ jobs: shell: bash timeout-minutes: 10 run: | - timeout curl https://raw.githubusercontent.com/everest/everest-demo/main/demo-iso15118-2-ac-plus-ocpp201.sh | bash + curl https://raw.githubusercontent.com/everest/everest-demo/main/demo-iso15118-2-ac-plus-ocpp201.sh | bash From 5d422599245eec793c78fcbfd0c55f54c85efa67 Mon Sep 17 00:00:00 2001 From: Shankari Date: Sat, 9 Mar 2024 23:43:00 -0800 Subject: [PATCH 07/12] Switch to mac and install docker To be as close to the laptop environment as possible --- .github/workflows/e2etest.yaml | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/.github/workflows/e2etest.yaml b/.github/workflows/e2etest.yaml index af4fa362..7e1eddf1 100644 --- a/.github/workflows/e2etest.yaml +++ b/.github/workflows/e2etest.yaml @@ -14,9 +14,18 @@ jobs: runs-on: ${{ matrix.os }} strategy: matrix: - os: [ubuntu-latest] + os: [macos-latest] steps: + - name: Setup Docker on macOS using Colima, Lima-VM, and Homebrew. + uses: douglascamata/setup-docker-macos-action@v1-alpha # Uses an action in the root directory + id: docker-install + - name: Verify install + id: docker-verify + shell: bash + run: | + docker --version + docker compose --version - name: Download and launch id: curl-download-and-launch shell: bash From 2c9692ebd569e211d1ffb72dcb695e0eaf7626e4 Mon Sep 17 00:00:00 2001 From: Shankari Date: Sat, 9 Mar 2024 23:59:18 -0800 Subject: [PATCH 08/12] Bump up timeout to 20 minutes Since 10 minutes is too short for OSX ``` c9a245e33d92 Extracting [=================> ] 357.1MB/1.048GB c9a245e33d92 Extracting [=================> ] 357.6MB/1.048GB c9a245e33d92 Extracting [=================> ] 358.2MB/1.048GB c9a245e33d92 Extracting [=================> ] 358.7MB/1.048GB c9a245e33d92 Extracting [=================> ] 359.3MB/1.048GB Error: The action 'Download and launch' has timed out after 10 minutes. ``` --- .github/workflows/e2etest.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/e2etest.yaml b/.github/workflows/e2etest.yaml index 7e1eddf1..4cdf82ad 100644 --- a/.github/workflows/e2etest.yaml +++ b/.github/workflows/e2etest.yaml @@ -29,6 +29,6 @@ jobs: - name: Download and launch id: curl-download-and-launch shell: bash - timeout-minutes: 10 + timeout-minutes: 20 run: | curl https://raw.githubusercontent.com/everest/everest-demo/main/demo-iso15118-2-ac-plus-ocpp201.sh | bash From 05528d4e76c1dc69f764bfb83459c88624613d3c Mon Sep 17 00:00:00 2001 From: Shankari Date: Sun, 10 Mar 2024 00:24:03 -0800 Subject: [PATCH 09/12] Bump up the timeout to 30 mins Download takes 20 mins Build should take another 10 mins --- .github/workflows/e2etest.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/e2etest.yaml b/.github/workflows/e2etest.yaml index 4cdf82ad..4e0f8520 100644 --- a/.github/workflows/e2etest.yaml +++ b/.github/workflows/e2etest.yaml @@ -29,6 +29,6 @@ jobs: - name: Download and launch id: curl-download-and-launch shell: bash - timeout-minutes: 20 + timeout-minutes: 30 run: | curl https://raw.githubusercontent.com/everest/everest-demo/main/demo-iso15118-2-ac-plus-ocpp201.sh | bash From b9a4b84273d2b8224e8e32ab8e8b32b611a2463e Mon Sep 17 00:00:00 2001 From: Shankari Date: Sun, 10 Mar 2024 08:48:33 -0700 Subject: [PATCH 10/12] Verify docker compose version to address issue with build mounts Signed-off-by: Shankari --- .github/workflows/e2etest.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/e2etest.yaml b/.github/workflows/e2etest.yaml index 4e0f8520..6aca7536 100644 --- a/.github/workflows/e2etest.yaml +++ b/.github/workflows/e2etest.yaml @@ -25,7 +25,7 @@ jobs: shell: bash run: | docker --version - docker compose --version + docker compose version - name: Download and launch id: curl-download-and-launch shell: bash From 12f393cf500650dc676d95332ea571c144bb84e8 Mon Sep 17 00:00:00 2001 From: Shankari Date: Sun, 10 Mar 2024 09:18:56 -0700 Subject: [PATCH 11/12] Let's check the colima version Signed-off-by: Shankari --- .github/workflows/e2etest.yaml | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/workflows/e2etest.yaml b/.github/workflows/e2etest.yaml index 6aca7536..5afc5489 100644 --- a/.github/workflows/e2etest.yaml +++ b/.github/workflows/e2etest.yaml @@ -26,6 +26,7 @@ jobs: run: | docker --version docker compose version + colima --version - name: Download and launch id: curl-download-and-launch shell: bash From 304e16c27248da9a00c4d0815e863191a7661ce8 Mon Sep 17 00:00:00 2001 From: Shankari Date: Sun, 10 Mar 2024 09:19:37 -0700 Subject: [PATCH 12/12] And create the host folder if we need to Although the error is likely because we are trying to mount a directory outside $HOME and /tmp/colima --- .github/workflows/e2etest.yaml | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/workflows/e2etest.yaml b/.github/workflows/e2etest.yaml index 5afc5489..f30e8bcd 100644 --- a/.github/workflows/e2etest.yaml +++ b/.github/workflows/e2etest.yaml @@ -27,6 +27,7 @@ jobs: docker --version docker compose version colima --version + mkdir -p /var/folders - name: Download and launch id: curl-download-and-launch shell: bash