From 099bd995851a3aa9688f5e160a744c6924f8ec7a Mon Sep 17 00:00:00 2001 From: Pa1NarK <69745008+pixincreate@users.noreply.github.com> Date: Tue, 7 Jan 2025 12:26:49 +0530 Subject: [PATCH] docs(cypress): update cypress documentation (#6956) Co-authored-by: hyperswitch-bot[bot] <148525504+hyperswitch-bot[bot]@users.noreply.github.com> --- cypress-tests/README.md | 498 +++++++++++++++++++++++++++------------- 1 file changed, 337 insertions(+), 161 deletions(-) diff --git a/cypress-tests/README.md b/cypress-tests/README.md index 0a071aa34a01..87681bb6ea00 100644 --- a/cypress-tests/README.md +++ b/cypress-tests/README.md @@ -1,31 +1,82 @@ -# Cypress Tests +# Hyperswitch Cypress Testing Framework ## Overview -This Tool is a solution designed to automate testing for the [Hyperswitch](https://github.com/juspay/hyperswitch/) using Cypress, an open-source tool capable of conducting API call tests and UI tests. This README provides guidance on installing Cypress and its dependencies. - -## Installation - -### Prerequisites - -Before installing Cypress, ensure that `Node` and `npm` is installed on your machine. To check if it is installed, run the following command: +This is a comprehensive testing framework built with [Cypress](https://cypress.io) to automate testing for [Hyperswitch](https://github.com/juspay/hyperswitch/). The framework supports API testing with features like multiple credential management, configuration management, global state handling, and extensive utility functions. The framework provides extensive support for API testing with advanced features including: + +- [Multiple credential management](#multiple-credential-support) +- [Dynamic configuration management](#dynamic-configuration-management) +- Global state handling +- Extensive utility functions +- Parallel test execution +- Connector-specific implementations + +## Table of Contents + +- [Overview](#overview) +- [Table of Contents](#table-of-contents) +- [Quick Start](#quick-start) +- [Getting Started](#getting-started) + - [Prerequisites](#prerequisites) + - [Installation](#installation) + - [Running Tests](#running-tests) + - [Development Mode (Interactive)](#development-mode-interactive) + - [CI Mode (Headless)](#ci-mode-headless) + - [Execute tests against multiple connectors or in parallel](#execute-tests-against-multiple-connectors-or-in-parallel) +- [Test reports](#test-reports) +- [Folder structure](#folder-structure) +- [Adding tests](#adding-tests) + - [Addition of test for a new connector](#addition-of-test-for-a-new-connector) + - [Developing Core Features or adding new tests](#developing-core-features-or-adding-new-tests) + - [1. Create or update test file](#1-create-or-update-test-file) + - [2. Add New Commands](#2-add-new-commands) + - [Managing global state](#managing-global-state) +- [Debugging](#debugging) + - [1. Interactive Mode](#1-interactive-mode) + - [2. Logging](#2-logging) + - [3. Screenshots](#3-screenshots) + - [4. State Debugging](#4-state-debugging) + - [5. Hooks](#5-hooks) + - [6. Tasks](#6-tasks) +- [Linting](#linting) +- [Best Practices](#best-practices) +- [Additional Resources](#additional-resources) +- [Contributing](#contributing) +- [Appendix](#appendix) + - [Example creds.json](#example-credsjson) + - [Multiple credential support](#multiple-credential-support) + - [Dynamic configuration management](#dynamic-configuration-management) + +## Quick Start + +For experienced users who want to get started quickly: ```shell -node -v -npm -v +git clone https://github.com/juspay/hyperswitch.git +cd hyperswitch/cypress-tests +npm ci +# connector_id must be replaced with the connector name that is being tested (e.g. stripe, paypal, etc.) +CYPRESS_CONNECTOR="connector_id" npm run cypress:ci ``` -If not, download and install `Node` from the official [Node.js website](https://nodejs.org/en/download/package-manager/current). This will also install `npm`. +## Getting Started -### Run Test Cases on your local +## Prerequisites -To run test cases, follow these steps: +- Node.js (18.x or above) +- npm or yarn +- [Hyperswitch development environment](https://github.com/juspay/hyperswitch/blob/main/docs/try_local_system.md) + +> [!NOTE] +> To learn about the hardware requirements and software dependencies for running Cypress, refer to the [official documentation](https://docs.cypress.io/app/get-started/install-cypress). + +## Installation 1. Clone the repository and switch to the project directory: ```shell - git clone https://github.com/juspay/hyperswitch - cd cypress-tests + git clone https://github.com/juspay/hyperswitch.git + cd hyperswitch/cypress-tests ``` 2. Install Cypress and its dependencies to `cypress-tests` directory by running the following command: @@ -34,65 +85,70 @@ To run test cases, follow these steps: npm ci ``` -3. Insert data to `cards_info` table in `hyperswitch_db` + Once installed, verify the installation by running: ```shell - psql --host=localhost --port=5432 --username=db_user --dbname=hyperswitch_db --command "\copy cards_info FROM '.github/data/cards_info.csv' DELIMITER ',' CSV HEADER;" + npx cypress --version ``` -4. Set environment variables for cypress + To learn about the supported commands, execute: ```shell - export CYPRESS_CONNECTOR="connector_id" - export CYPRESS_BASEURL="base_url" - export DEBUG=cypress:cli - export CYPRESS_ADMINAPIKEY="admin_api_key" - export CYPRESS_CONNECTOR_AUTH_FILE_PATH="path/to/creds.json" + npm run ``` -5. Run Cypress test cases - - To run the tests in interactive mode run the following command +3. Set up the cards database: ```shell - npm run cypress + psql --host=localhost --port=5432 --username=db_user --dbname=hyperswitch_db --command "\copy cards_info FROM '.github/data/cards_info.csv' DELIMITER ',' CSV HEADER;" ``` - To run all the tests in headless mode run the following command +4. Set environment variables for cypress ```shell - npm run cypress:ci + export CYPRESS_CONNECTOR="connector_id" + export CYPRESS_BASEURL="base_url" + export DEBUG=cypress:cli + export CYPRESS_ADMINAPIKEY="admin_api_key" + export CYPRESS_CONNECTOR_AUTH_FILE_PATH="path/to/creds.json" ``` - To run payment tests in headless mode run the following command +> [!TIP] +> It is recommended to install [direnv](https://github.com/direnv/direnv) and use a `.envrc` file to store these environment variables with `cypress-tests` directory. This will make it easier to manage environment variables while working with Cypress tests. - ```shell - npm run cypress:payments - ``` +> [!NOTE] +> To learn about how `creds` file should be structured, refer to the [example.creds.json](#example-credsjson) section below. - To run payout tests in headless mode run the following command +## Running Tests - ```shell - npm run cypress:payouts - ``` +Execution of Cypress tests can be done in two modes: Development mode (Interactive) and CI mode (Headless). The tests can be executed against a single connector or multiple connectors in parallel. Time taken to execute the tests will vary based on the number of connectors and the number of tests. For a single connector, the tests will take approximately 07-12 minutes to execute (this also depends on the hardware configurations). - To run routing tests in headless mode run the following command +For Development mode, the tests will run in the Cypress UI where execution of tests can be seen in real-time and provides a larger area for debugging based on the need. In CI mode (Headless), tests run in the terminal without UI interaction and generate reports automatically. - ```shell - npm run cypress:routing - ``` +### Development Mode (Interactive) -In order to run cypress tests against multiple connectors at a time or in parallel: +```shell +npm run cypress +``` -1. Set up `.env` file that exports necessary info: +### CI Mode (Headless) - ```env - export DEBUG=cypress:cli +```shell +# All tests +npm run cypress:ci + +# Specific test suites +npm run cypress:payments # Payment tests +npm run cypress:payment-method-list # Payment method list tests +npm run cypress:payouts # Payout tests +npm run cypress:routing # Routing tests +``` - export CYPRESS_ADMINAPIKEY='admin_api_key' - export CYPRESS_BASEURL='base_url' - export CYPRESS_CONNECTOR_AUTH_FILE_PATH="path/to/creds.json" +### Execute tests against multiple connectors or in parallel +1. Set additional environment variables: + + ```shell export PAYMENTS_CONNECTORS="payment_connector_1 payment_connector_2 payment_connector_3 payment_connector_4" export PAYOUTS_CONNECTORS="payout_connector_1 payout_connector_2 payout_connector_3" export PAYMENT_METHOD_LIST="" @@ -103,72 +159,64 @@ In order to run cypress tests against multiple connectors at a time or in parall ```shell source .env - scripts/execute_cypress.sh + ../scripts/execute_cypress.sh ``` Optionally, `--parallel ` can be passed to run cypress tests in parallel. By default, when `parallel` command is passed, it will be run in batches of `5`. -> [!NOTE] -> To learn about how creds file should be structured, refer to the [example.creds.json](#example-credsjson) section below. - -## Folder Structure +## Test reports -The folder structure of this directory is as follows: +The test reports are generated in the `cypress/reports` directory. The reports are generated in the `mochawesome` format and can be viewed in the browser. +These reports does include: -```text -. # The root directory for the Cypress tests. -├── .gitignore -├── cypress # Contains Cypress-related files and folders. -│ ├── e2e # End-to-end test directory. -│ │ ├── ConnectorTest # Directory for test scenarios related to connectors. -│ │ │ ├── your_testcase1_files_here.cy.js -│ │ │ ├── your_testcase2_files_here.cy.js -│ │ │ └── ... -│ │ └── ConnectorUtils # Directory for utility functions related to connectors. -│ │ ├── connector_detail_files_here.js -│ │ └── utils.js -│ ├── fixtures # Directory for storing test data API request. -│ │ └── your_fixture_files_here.json -│ ├── support # Directory for Cypress support files. -│ │ ├── commands.js # File containing custom Cypress commands and utilities. -│ │ └── e2e.js -│ └── utils -│ └── utility_files_go_here.js -├── cypress.config.js # Cypress configuration file. -├── cypress.env.json # File is used to store environment-specific configuration values,such as base URLs, which can be accessed within your Cypress tests. -├── package.json # Node.js package file. -├── readme.md # This file -└── yarn.lock -``` +- screenshots of the failed tests +- HTML and JSON reports -## Writing Tests +## Folder structure -### Adding Connectors +The folder structure of this directory is as follows: -To add a new connector for testing with Hyperswitch, follow these steps: +```txt +. +├── .prettierrc # prettier configs +├── README.md # this file +├── cypress +│   ├── e2e +│   │   ├── Test # Directory for test scenarios related to connectors. +│   │   │   ├── 00000-test_<0>.cy.js +│   │   │   ├── ... +│   │   │   └── 0000n-test_.cy.js +│   │   └── Utils # Directory for utility functions related to connectors. +│   │   ├── connector_<1>.js +│   │   ├── ... +│   │   └── connector_.js +│   ├── fixtures # Directory for storing test data API request. +│   │   ├── fixture_<1>.json +│   │   ├── ... +│   │   └── fixture_.json +│   ├── support # Directory for Cypress support files. +│   │   ├── commands.js # File containing custom Cypress commands and utilities. +│   │   ├── e2e.js +│   │   └── redirectionHandler.js +│   └── utils +│   ├── RequestBodyUtils.js +│   ├── State.js +│   └── featureFlags.js +├── cypress.config.js # Cypress configuration file. +├── eslint.config.js # linter configuration file. +└── package.json # Node.js package file. +``` -1. Include the connector details in the `creds.json` file: +## Adding tests - example: +### Addition of test for a new connector - ```json - { - "stripe": { - "connector_account_details": { - "auth_type": "HeaderKey", - "api_key": "SK_134" - } - } - } - ``` +1. Include the connector details in the `creds.json` file 2. Add the new connector details to the ConnectorUtils folder (including CardNo and connector-specific information). - Refer to Stripe.js file for guidance: - - ```javascript - /cypress-tests/cypress/e2e/ConnectorUtils/Stripe.js - ``` + To add a new Payment connector, refer to [`Stripe.js`](cypress/e2e/PaymentUtils/Stripe.js) file for reference. + To add a new Payout connector, refer to [`Adyen.js`](cypress-tests/cypress/e2e/PayoutUtils/Adyen.js) file for reference. **File Naming:** Create a new file named .js for your specific connector. @@ -176,107 +224,173 @@ To add a new connector for testing with Hyperswitch, follow these steps: **Handling Unsupported Features:** - - If a connector does not support a specific payment method or feature: - - You can omit the relevant configurations in the .js file. - - The handling of unsupported features will be managed by the commons.js file, which will throw an unsupported or not implemented error as appropriate. + - If a connector does not support a specific payment method or a feature: + - The relevant configurations in the `.js` file can be omitted + - The handling of unsupported or unimplemented features will be managed by the [`Commons.js`](cypress/e2e/PaymentUtils/Commons.js) file, which will throw the appropriate `unsupported` or `not implemented` error + +3. In `Utils.js`, import the new connector details + +4. If the connector has a specific redirection requirement, add relevant redirection logic in `support/redirectionHandler.js` -3. In `Utils.js`, import the new connector details. +### Developing Core Features or adding new tests + +#### 1. Create or update test file + +To add a new test, create a new test file in the `e2e` directory under respective `service`. The test file should follow the naming convention `000-Test.cy.js` and should contain the test cases related to the service. + +```javascript +// cypress/e2e/Test/NewFeature.cy.js +import * as fixtures from "../../fixtures/imports"; +import State from "../../utils/State"; -### Adding Functions +describe("New Feature", () => { + let globalState; -Similarly, add any helper functions or utilities in the `commands.js` in support folder and import them into your tests as needed. + before(() => { + cy.task("getGlobalState").then((state) => { + globalState = new State(state); + }); + }); + + after("flush global state", () => { + cy.task("setGlobalState", globalState.data); + }); + + it("tests new functionality", () => { + // Test implementation + }); +}); +``` -Example: Adding List Mandate function to support `ListMandate` scenario +#### 2. Add New Commands ```javascript -Cypress.Commands.add("listMandateCallTest", (globalState) => { - // declare all the variables and constants - const customerId = globalState.get("customerId"); - // construct the URL for the API call - const url: `${globalState.get("baseUrl")}/customers/${customerId}/mandates` - const api_key = globalState.get("apiKey"); +// cypress/support/commands.js +Cypress.Commands.add("newCommand", (params, globalState) => { + const baseUrl = globalState.get("baseUrl"); + const apiKey = globalState.get("apiKey"); + const url = `${baseUrl}/endpoint`; cy.request({ - method: "GET", + method: "POST", url: url, headers: { - "Content-Type": "application/json", - "api-key": api_key, + "api-key": apiKey, }, - // set failOnStatusCode to false to prevent Cypress from failing the test - failOnStatusCode: false, + body: params, }).then((response) => { - // mandatorliy log the `x-request-id` to the console - logRequestId(response.headers["x-request-id"]); - - expect(response.headers["content-type"]).to.include("application/json"); - - if (response.status === 200) { - // do the necessary validations like below - for (const key in response.body) { - expect(response.body[key]).to.have.property("mandate_id"); - expect(response.body[key]).to.have.property("status"); - } - } else { - // handle the error response - expect(response.status).to.equal(400); - } + // Assertions }); }); ``` -### Adding Scenarios +### Managing global state -To add new test scenarios: +The global state is used to share data between tests. The global state is stored in the `State` class and is accessible across all tests. Can only be accessed in the `before` and `after` hooks. -1. Navigate to the ConnectorTest directory. -2. Create a new test file or modify existing ones to add your scenarios. -3. Write your test scenarios using Cypress commands. +## Debugging -For example, to add a scenario for listing mandates in the `Mandateflows`: +### 1. Interactive Mode + +- Use `npm run cypress` for real-time test execution +- View request/response details in Cypress UI +- Use DevTools for deeper debugging + +### 2. Logging ```javascript -// cypress/ConnectorTest/CreateSingleuseMandate.js -describe("Payment Scenarios", () => { - it("should complete a successful payment", () => { - // Your test logic here - }); -}); +cy.task("cli_log", "Debug message"); +cy.log("Test state:", globalState.data); ``` -In this scenario, you can call functions defined in `command.js`. For instance, to test the `listMandateCallTest` function: +### 3. Screenshots + +- Automatically captured on test failure +- Custom screenshot capture: ```javascript -describe("Payment Scenarios", () => { - it("list-mandate-call-test", () => { - cy.listMandateCallTest(globalState); - }); +cy.screenshot("debug-state"); +``` + +### 4. State Debugging + +- Add state logging in hooks: + +```javascript +beforeEach(() => { + cy.log("Current state:", JSON.stringify(globalState.data)); }); ``` -You can create similar scenarios by calling other functions defined in `commands.js`. These functions interact with utility files like `.js` and include necessary assertions to support various connector scenarios. +### 5. Hooks -### Debugging +- If the `globalState` object does not contain latest data, it must be due to the hooks not being executed in the correct order +- Add `cy.log(globalState)` to the test case to verify the data in the `globalState` object -It is recommended to run `npm run cypress` while developing new test cases to debug and verify as it opens the Cypress UI allowing the developer to run individual tests. This also opens up the possibility to to view the test execution in real-time and debug any issues that may arise by viewing the request and response payloads directly. +> [!NOTE] +> Refer to the Cypress's official documentation for more information on hooks and their execution order [here](https://docs.cypress.io/app/core-concepts/writing-and-organizing-tests#Hooks). + +### 6. Tasks + +- Use `cy.task` to interact with the Node.js environment +- Task can only be used in `support` files and `spec` files. Using them in files outside these directories will result in unexpected behavior or errors like abrupt termination of the test suite + +## Linting -If, for any reason, the `globalState` object does not contain latest data, it must be due to the hooks not being executed in the correct order. In such cases, it is recommended to add `cy.log(globalState)` to the test case to verify the data in the `globalState` object. -Please refer to the Cypress's official documentation for more information on hooks and their execution order [here](https://docs.cypress.io/app/core-concepts/writing-and-organizing-tests#Hooks). +To run the formatting and lint checks, execute the following command: + +```shell +# Format the code +npm run format + +# Check the formatting +npm run format:check + +# Lint the code. This wont fix the logic issues, unused imports or variables +npm run lint -- --fix +``` + +## Best Practices + +1. Use the global state for sharing data between tests +2. Implement proper error handling +3. Use appropriate wait strategies +4. Maintain test independence +5. Follow the existing folder structure +6. Document connector-specific behaviors +7. Use descriptive test and variable names +8. Use custom commands for repetitive tasks +9. Use `cy.log` for debugging and do not use `console.log` ## Additional Resources -For more information on using Cypress and writing effective tests, refer to the official Cypress documentation: [Cypress Documentation](https://docs.cypress.io/) +- [Cypress Documentation](https://docs.cypress.io/) +- [API Testing Best Practices](https://docs.cypress.io/guides/end-to-end-testing/api-testing) +- [Hyperswitch API Documentation](https://hyperswitch.io/docs) + +## Contributing -## Example creds.json +1. Fork the repository +2. Create a feature branch +3. Add tests following the guidelines +4. Submit a pull request + +## Appendix + +### Example creds.json ```json { + // Connector with single credential support and metadata support "adyen": { "connector_account_details": { "auth_type": "SignatureKey", "api_key": "api_key", "key1": "key1", "api_secret": "api_secret" + }, + "metadata": { + "key": "value" } }, "bankofamerica": { @@ -294,12 +408,23 @@ For more information on using Cypress and writing effective tests, refer to the "key1": "key1" } }, + // Connector with multiple credential support "cybersource": { - "connector_account_details": { - "auth_type": "SignatureKey", - "api_key": "api_key", - "key1": "key1", - "api_secret": "api_secret" + "connector_1": { + "connector_account_details": { + "auth_type": "SignatureKey", + "api_key": "api_key", + "key1": "key1", + "api_secret": "api_secret" + } + }, + "connector_2": { + "connector_account_details": { + "auth_type": "SignatureKey", + "api_key": "api_key", + "key1": "key1", + "api_secret": "api_secret" + } } }, "nmi": { @@ -332,3 +457,54 @@ For more information on using Cypress and writing effective tests, refer to the } } ``` + +### Multiple credential support + +- There are some use cases where a connector supports a feature that requires a different set of API keys (example: Network transaction ID for Stripe expects a different API Key to be passed). This forces the need for having multiple credentials that serves different use cases +- This basically means that a connector can have multiple credentials +- At present the maximum number of credentials that can be supported is `2` +- The `creds.json` file should be structured to support multiple credentials for such connectors. The `creds.json` file should be structured as follows: + +```json +{ + "connector_name": { + "connector_1": { + "connector_account_details": { + "auth_type": "SignatureKey", + "api_key": "api_key", + "key1": "key1", + "api_secret": "api_secret" + } + }, + "connector_2": { + "connector_account_details": { + "auth_type": "SignatureKey", + "api_key": "api_key", + "key1": "key1", + "api_secret": "api_secret" + } + } + } +} +``` + +### Dynamic configuration management + +- `Configs` is the new `object` that is introduced to manage the dynamic configurations that are required for the tests +- This is supposed to be passed in an exchange (configuration for a specific can be passed to a test based on the need and this will impact everywhere in the test execution for that connector) +- At present, only 3 configs are supported: + - `DELAY`: This is used to introduce a delay in the test execution. This is useful when a connector requires a delay in order to perform a specific operation + - `CONNECTOR_CREDENTIAL`: This is used to control the connector credentials that are used in the test execution. This is useful only when a connector supports multiple credentials and the test needs to be executed with a specific credential + - `TRIGGER_SKIP`: This is used to skip a test execution (preferably redirection flows). This is useful when a test is does not support a specific redirection flow and needs to be skipped +- Example: In order to refund a payment in Trustpay, a `DELAY` of at least `5` seconds is required. By passing `DELAY` to the `Configs` object for Trustpay, the delay will be applied to all the tests that are executed for Trustpay + +```json +{ + "Configs": { + "DELAY": { + "STATUS": true, + "TIMEOUT": 15000 + } + } +} +```