Dlaliberte playwright 06 22 (#3184)

* Initial install of playwright visual tests * Close to working. * who knows... * after updates * Extend devcontainer docker compose file for playwright (#3192) * Extend devcontainer docker compose file for playwright Create another docker compose file for playwright. That new one extends the existing docker compose Should be able to run: `npm run pwtests` * adjust gh action * adjust gh action * remove volumes * add timeout * fix volumes * Fix docker compose environment * ignore stuff and volume mounting * Attempting to login reliably * Auto cleanups. * Finally starting to work * Thought it was getting better, but maybe not * Seems reliable now * Higher success rate * Add test of entering feature name. Remove new feature page content test. * Add playwright-report dir to Dockerfile. * Comment out mkdir -p /work/playwright-report line. * Update screenshots. * Increasing timeout for entering feature name * try running as user (#3248) * try running as user * again * try * change volume ownership * tidy up * Adjust login process to be more reliable. * Add 5 second delay before first test runs. * Wait longer for first login. * Add auto-logout during login if already logged in * Screenshot after login click * Add testName to screenshots. Upload artifacts. Fix form method="POST". * Add doc to README for Updating the test_html_rendering files. * Maybe increase chance of login. * Add a couple more delays. * Repeat auto-logout during login until no longer logged in, before logging in again. * Add some more delays. * Use loginButton.click(). Add delay before after-click screenshot. * Add console info an errors to login process. * Capture console messages, relay to terminal. * Capture console messages properly. * Add some more console messages during login. * Ignore error on jsonValue. * Reduce most delays. Add more logs. Add top-level playwright call. * More logging to figure out why login fails * More logging to detect if ever logged in. * Add more logging to each stage of testing * Use history.replaceState before changing location. * Use location.reload() after a timeout. * After login, just load home page. * Add playwright-report to artifacts. * Turn off playwright-report during CI * Log response after login * Only log first response after request that should log in. * Look for 'currentuser/permissions' request/response. * Delay after mock_login and before reload of home page * Comment out header and cookie logs. * Act like logged when in DEV_MODE and login attempted. * Fix mock mock login. * Undo fake mock_login. * rearrange docker (#3265) * Updates after changes by James. * Fix previous updates. * Remove pwtests-shutdown from "stop" command. * Remove pwtest from client-src/elements. (moved to packages/playwright/tests) * update paths for run.sh * Add pwtests-shutdown to top-level stop command. * Remove "playwright" from ./wait-for-app.sh * Add logging of user login status. * add playwright_display for debugging * add server debugging * Remove extra "/__screenshots__" in path. * Update the path for test-results artifacts in the playwright.yml * add health checks * fix action * fix run.sh * Add beforeAll method, disabled. * add dsadmin * Add top level pwtests command, not working yet. * cleanup and undo changes * use transactions * always reuse server * temp: limit the workers * Partial cleanup. Reduce timeouts and delays a bit. * More cleanup. Reduce timeout/delay. * Update the correct README file. Update html rendering tests. * Remove unnecessary changes. * set script-shell to fix source not found and call npm run setup (#3292) * set script-shell to fix source not found and call npm run setup There has always been `source: not found` in the previous CI jobs The tests previously worked because npm run deps installs the requirements into the default environment and not the virtual environment. And since we actually did not call `source` correctly, it was okay because the default environment had the packages installed. The problem started to arise because we started to call `source` correctly in some places but not all places. This fixes by setting the shell consistently whenever we call `source` More info: https://morgan.cugerone.com/blog/troubleshooting-source-not-found-when-in-the-context-of-package-json-scripts/ * do each separately * Revert "do each separately" This reverts commit 4fc75f7. * do ci job * consistent to the right datastore mode * verbose * kill the emulator * remove verbose * Reenable Google login * Finish reenabling Google signin button. Add margin for Firefox. * Only show Google signin if playwright is not active * Preserve url on login. Use default num of workers. Retry tests in CI * Some updates after comments * optimize docker changes * Rename account-indicator. Shorten some delays. * revert the workers * Make playwright.yml push and pull_request triggers be like others. * Use getByTestId instead of locator with data-testid. --------- Co-authored-by: James C Scott III <[email protected]> Co-authored-by: James Scott <[email protected]>
GoogleChrome · Sep 6, 2023 · 0baad0b · 0baad0b
1 parent 0ac2427
commit 0baad0b
Show file tree

Hide file tree

Showing 48 changed files with 1,076 additions and 93 deletions.
diff --git a/.devcontainer/Dockerfile b/.devcontainer/Dockerfile
@@ -1,11 +1,14 @@
 # See here for image contents: https://github.com/microsoft/vscode-dev-containers/tree/v0.245.2/containers/python-3/.devcontainer/base.Dockerfile
 
 # [Choice] Python version (use -bullseye variants on local arm64/Apple Silicon): 3, 3.10, 3.9, 3.8, 3.7, 3.6, 3-bullseye, 3.10-bullseye, 3.9-bullseye, 3.8-bullseye, 3.7-bullseye, 3.6-bullseye, 3-buster, 3.10-buster, 3.9-buster, 3.8-buster, 3.7-buster, 3.6-buster
-ARG VARIANT=""
-FROM mcr.microsoft.com/vscode/devcontainers/python:0-${VARIANT}
+# Update 'VARIANT' to pick a version of Python: 3, 3.10, 3.9, 3.8, 3.7, 3.6
+# Append -bullseye or -buster to pin to an OS version.
+# Use -bullseye variants on local arm64/Apple Silicon.
+ARG VARIANT="3.11-bullseye"
+FROM mcr.microsoft.com/vscode/devcontainers/python:0-${VARIANT} as ide
 
 # [Choice] Node.js version: none, lts/*, 16, 14, 12, 10
-ARG NODE_VERSION="none"
+ARG NODE_VERSION="18"
 RUN if [ "${NODE_VERSION}" != "none" ]; then su vscode -c "umask 0002 && . /usr/local/share/nvm/nvm.sh && nvm install ${NODE_VERSION} 2>&1"; fi
 
 RUN apt-get update && export DEBIAN_FRONTEND=noninteractive \ 

diff --git a/.devcontainer/README.md b/.devcontainer/README.md
@@ -29,7 +29,8 @@ You can click on the appropriate badge to get the environment setup:
 | [Visual Studio Code Remote - Containers](https://code.visualstudio.com/docs/remote/create-dev-container) | [![Open in Remote - Containers](https://img.shields.io/static/v1?label=Remote%20-%20Containers&message=Open&color=blue&logo=visualstudiocode)](https://vscode.dev/redirect?url=vscode://ms-vscode-remote.remote-containers/cloneInVolume?url=https://github.com/GoogleChrome/chromium-dashboard) | Locally: Docker and Visual Studio Code | Runs locally. Have more resources | Need to have VS Code and Docker setup |
 | [GitHub Codespaces](https://docs.github.com/en/enterprise-cloud@latest/codespaces) | <ul><li>East US[![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://github.com/codespaces/new?hide_repo_select=true&ref=main&repo=8633551&machine=standardLinux32gb&location=EastUs&devcontainer_path=.devcontainer%2Fdevcontainer.json)</li><li>West US[![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://github.com/codespaces/new?hide_repo_select=true&ref=main&repo=8633551&machine=standardLinux32gb&location=WestUs2&devcontainer_path=.devcontainer%2Fdevcontainer.json)</li></ul> | Access to Codespaces | Nothing to install locally | Limited resources: 4 cores, 8GB RAM, 32GB Storage |
 
-Upon creating the devcontainer by clicking the appropriate badge, `npm run setup` is ran automatically.
+Upon creating the devcontainer by clicking the appropriate badge, `npm run setup`
+is run automatically.
 
 *Note*: Sometimes when you open the terminal, it won't automatically activate
 the python environment. Either 1) Open a new terminal (usually it works the
@@ -61,11 +62,11 @@ There are other ports needed for the IDE itself but mainly, you only need to wat
 ## Upgrading versions of Node and Python
 
 In the docker-compose.yml file:
-- For Python: Change the `VARIANT` argument to the appropirate [variant](https://github.com/microsoft/vscode-dev-containers/tree/main/containers/python-3)
+- For Python: Change the `VARIANT` argument to the appropriate [variant](https://github.com/microsoft/vscode-dev-containers/tree/main/containers/python-3)
 - For Node: Change the `NODE_VERSION` to the appropriate version used by [nvm](https://github.com/nvm-sh/nvm)
 
 ## Troubleshooting
 
-When there are depedency changes and environment changes in requirements.txt or package.json, the Devcontainer needs to be rebuilt:
+When there are dependency changes and environment changes in requirements.txt or package.json, the Devcontainer needs to be rebuilt:
 - `rm -rf node_modules cs-env` in terminal
 - View -> Command Palette -> Rebuild container without cache
diff --git a/.devcontainer/db-docker-compose.yml b/.devcontainer/db-docker-compose.yml
@@ -0,0 +1,17 @@
+version: '3.8'
+
+services:
+  db:
+    image: google/cloud-sdk:402.0.0-emulators
+    restart: unless-stopped
+    command: gcloud beta emulators datastore start --project=cr-status-staging --host-port 0.0.0.0:15606 --no-store-on-disk --use-firestore-in-datastore-mode
+
+  dsadmin:
+    image: ghcr.io/remko/dsadmin:v0.18.1
+    entrypoint: /ko-app/dsadmin -port 8888
+    depends_on:
+      - db
+    network_mode: service:db
+    environment:
+      DATASTORE_PROJECT_ID: cr-status-staging
+      DATASTORE_EMULATOR_HOST: "db:15606"
diff --git a/.devcontainer/devcontainer.json b/.devcontainer/devcontainer.json
@@ -2,8 +2,11 @@
 // https://github.com/microsoft/vscode-dev-containers/tree/v0.245.2/containers/python-3
 {
 	"name": "Chromium Dashboard",
-	"dockerComposeFile": "docker-compose.yml",
-	"service": "app",
+	"dockerComposeFile": [
+		"docker-compose.yml",
+		"db-docker-compose.yml"
+	],
+	"service": "ide",
 	"workspaceFolder": "/workspace",
 
 	// Configure tool-specific properties.
@@ -41,7 +44,8 @@
 			"extensions": [
 				"ms-python.python",
 				"bierner.lit-html",
-				"42Crunch.vscode-openapi"
+				"42Crunch.vscode-openapi",
+				"ms-playwright.playwright"
 			]
 		}
 	},
@@ -64,5 +68,12 @@
 	"postCreateCommand": "npm run clean-setup; . cs-env/bin/activate; npm run setup; python -m pip install pylint==2.13.4",
 
 	// Comment out to connect as root instead. More info: https://aka.ms/vscode-remote/containers/non-root.
-	"remoteUser": "vscode"
+	"remoteUser": "vscode",
+
+	"features": {
+		// https://github.com/devcontainers/features/tree/main/src/docker-in-docker
+		"ghcr.io/devcontainers/features/docker-in-docker:2": {
+			"dockerDashComposeVersion": "v2"
+		}
+	}
 }
diff --git a/.devcontainer/docker-compose.yml b/.devcontainer/docker-compose.yml
@@ -1,18 +1,11 @@
 version: '3.8'
 
 services:
-  app:
+  ide:
     build:
       context: ..
       dockerfile: .devcontainer/Dockerfile
-      args:
-        # Update 'VARIANT' to pick a version of Python: 3, 3.10, 3.9, 3.8, 3.7, 3.6
-        # Append -bullseye or -buster to pin to an OS version.
-        # Use -bullseye variants on local arm64/Apple Silicon.
-        VARIANT: 3.10-bullseye
-        # Optional Node.js version to install
-        NODE_VERSION: "18"
-
+      target: ide
     volumes:
       - ..:/workspace:cached
 
@@ -25,18 +18,3 @@ services:
     environment:
       DATASTORE_PROJECT_ID: cr-status-staging
       DATASTORE_EMULATOR_HOST: localhost:15606
-
-  db:
-    image: google/cloud-sdk:402.0.0-emulators
-    restart: unless-stopped
-    command: gcloud beta emulators datastore start --project=cr-status-staging --host-port 0.0.0.0:15606 --no-store-on-disk --consistency=1.0
-
-  dsadmin:
-    image: ghcr.io/remko/dsadmin:v0.18.1
-    entrypoint: /ko-app/dsadmin -port 8888
-    depends_on:
-      - db
-    network_mode: service:db
-    environment:
-      DATASTORE_PROJECT_ID: cr-status-staging
-      DATASTORE_EMULATOR_HOST: "db:15606"
diff --git a/.dockerignore b/.dockerignore
@@ -0,0 +1,10 @@
+cs-env
+node_modules
+test-results
+playwright-report/
+playwright/.cache/
+
+# Web test runner coverage report
+coverage
+
+__pycache__
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -45,14 +45,14 @@ jobs:
       - name: installation
         run: |
           npm install -g gulp
-          npm ci
-          npm run deps
+          npm config set script-shell bash --global
+          npm run setup
 
       - name: lint
         run: npm run lint
 
       - name: mypy
-        run: mypy --ignore-missing-imports --exclude cs-env/ --exclude gen/ --exclude appengine_config.py .
+        run: npm run mypy
 
       - name: test
         run: npm test

diff --git a/.github/workflows/playwright.yml b/.github/workflows/playwright.yml
@@ -0,0 +1,27 @@
+name: Playwright Tests
+on:
+  push:
+    branches:
+      - main
+  pull_request:
+
+jobs:
+  playwright:
+    name: 'Playwright Tests'
+    runs-on: ubuntu-latest
+    steps:
+        - uses: actions/checkout@v3
+
+        # Steps for Playwright testing with container
+        - uses: actions/setup-node@v3
+          with:
+            node-version: 18
+        - name: Run your tests
+          timeout-minutes: 15
+          run: npm run pwtests-shutdown --workspace=playwright; npm run test --workspace=playwright
+        - uses: actions/upload-artifact@v3
+          if: failure()
+          with:
+            name: test-results
+            path: packages/playwright/test-results/
+            retention-days: 30
diff --git a/.github/workflows/web_tests.yml b/.github/workflows/web_tests.yml
@@ -55,11 +55,9 @@ jobs:
         run: |
           npm install -g gulp
 
-      - run: npm ci
-      - run: npm run deps
+      - run: npm config set script-shell bash --global
+      - run: npm run setup
       - run: npm run build --if-present
 
-      - run: npx playwright install-deps
-
       - name: Run web test runner
         run: npm run webtest
diff --git a/.gitignore b/.gitignore
@@ -2,7 +2,8 @@
 *.pyc
 *.swp
 .sass-cache
-.vscode
+.vscode/*
+!.vscode/launch.json
 *oauth2.data
 .fcm_server_key
 *.iml
@@ -18,7 +19,7 @@ env_vars.yaml
 node_modules
 
 # Generated files
-# static/css/
+static/css/
 static/dist/
 static/js/
 
@@ -35,5 +36,10 @@ coverage
 # venv directory
 cs-env
 
+# playwright
+test-results/
+/playwright-report/
+/playwright/.cache/
+
 # API Key for local development
 ot_api_key.txt
diff --git a/.vscode/launch.json b/.vscode/launch.json
@@ -0,0 +1,24 @@
+{
+    // Use IntelliSense to learn about possible attributes.
+    // Hover to view descriptions of existing attributes.
+    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
+    "version": "0.2.0",
+    "configurations": [
+        {
+            "name": "Python: Remote Attach",
+            "type": "python",
+            "request": "attach",
+            "connect": {
+                "host": "localhost",
+                "port": 5678
+            },
+            "pathMappings": [
+                {
+                    "localRoot": "${workspaceFolder}",
+                    "remoteRoot": "/workspace"
+                }
+            ],
+            "justMyCode": false
+        }
+    ]
+}
diff --git a/Dockerfile b/Dockerfile
@@ -0,0 +1,37 @@
+ARG PYTHON_VARIANT=3.11-bullseye
+FROM python:${PYTHON_VARIANT} as app
+
+RUN groupadd -g 1001 appuser && \
+    useradd appuser -u 1001 -g 1001 -m -d /home/appuser -s /bin/bash
+
+RUN apt-get update && export DEBIAN_FRONTEND=noninteractive \ 
+    # openapi-generator-cli dependencies
+    && apt-get install -y openjdk-17-jre
+
+USER appuser
+
+# Install node
+ENV NVM_DIR=/home/appuser/.nvm \
+    # Need the full major.minor.patch version for NODE_VERISON
+    # https://nodejs.org/en/download/releases
+    NODE_VERSION=18.17.1
+
+RUN mkdir /home/appuser/.nvm && \
+    curl https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.4/install.sh | bash \
+    && . $NVM_DIR/nvm.sh \
+    && nvm install $NODE_VERSION \
+    && nvm alias default $NODE_VERSION \
+    && nvm use default \
+    && npm config set script-shell bash
+
+ENV NODE_PATH=$NVM_DIR/v$NODE_VERSION/lib/node_modules \
+    PATH="${PATH}:$NVM_DIR/versions/node/v$NODE_VERSION/bin"
+
+WORKDIR /workspace
+# Cache the infrequent but time consuming changes early
+COPY --chown=appuser:appuser requirements.txt requirements.dev.txt package.json package-lock.json /workspace/
+COPY --chown=appuser:appuser openapi /workspace/openapi/
+COPY --chown=appuser:appuser gen /workspace/gen/
+RUN npm install -g gulp-cli && npm run setup && cs-env/bin/python -m pip install debugpy==1.6.7.post1
+# Copy the rest
+COPY --chown=appuser:appuser . /workspace
diff --git a/README.md b/README.md
@@ -62,7 +62,31 @@ npm test
 
 This will start a local datastore emulator, run unit tests, and then shut down the emulator.
 
-There are some developing information in developer-documentation.md.
+To update test_html_rendering.html, modify the `test_html_rendering` method in
+the corresponding test file, uncomment the line that looks like:
+
+```python
+ # TESTDATA.make_golden(template_text, 'test_html_rendering.html')
+```
+
+Then run the test again (and maybe one more time), and then you can revert
+your change of the test files.
+
+To run the Playwright visual tests (aka end-to-end tests), the command to use is:
+```bash
+npm run pwtests --packages/playwright
+```
+
+If there are errors, or if you need to update any of the screeenshot images,
+you can update all images for all tests with:
+```bash
+npm run pwtests-update --packages/playwright
+```
+
+Then check the `packages/playwright/test-results` directory for any changes.
+
+
+There is some additional information for developers in developer-documentation.md.
 
 ### Origin Trials
 To test the functionality of this application locally that interacts with data from the Origin Trials API, an API key will need to be acquired. To do this, run the following command:

diff --git a/client-src/elements/chromedash-app.js b/client-src/elements/chromedash-app.js
@@ -100,6 +100,7 @@ class ChromedashApp extends LitElement {
       loading: {type: Boolean},
       appTitle: {type: String},
       googleSignInClientId: {type: String},
+      devMode: {type: String},
       currentPage: {type: String},
       bannerMessage: {type: String},
       bannerTime: {type: Number},
@@ -117,6 +118,7 @@ class ChromedashApp extends LitElement {
     this.loading = true;
     this.appTitle = '';
     this.googleSignInClientId = '',
+    this.devMode = '';
     this.currentPage = '';
     this.bannerMessage = '';
     this.bannerTime = null;
@@ -543,6 +545,7 @@ class ChromedashApp extends LitElement {
               <chromedash-header
                 .user=${this.user}
                 .appTitle=${this.appTitle}
+                .devMode=${this.devMode}
                 .googleSignInClientId=${this.googleSignInClientId}
                 .currentPage=${this.currentPage}
                 @drawer-clicked=${this.handleShowDrawer}>
@@ -557,6 +560,7 @@ class ChromedashApp extends LitElement {
                 .user=${this.user}
                 .currentPage=${this.currentPage}
                 ?defaultOpen=${true}
+                .devMode=${this.devMode}
                 .googleSignInClientId=${this.googleSignInClientId}>
               </chromedash-drawer>
             </div>