Skip to content

Commit

Permalink
Add workflow2 integration test for QUAlibrate app
Browse files Browse the repository at this point in the history
  • Loading branch information
@kleinwave
kleinwave committed Dec 15, 2024
1 parent e693ca6 commit 45c13c1
Show file tree
Hide file tree
Showing 7 changed files with 172 additions and 77 deletions.
4 changes: 3 additions & 1 deletion .github/workflows/playwright-tests.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -31,4 +31,6 @@ jobs:
run: npx playwright install --with-deps

- name: Run Playwright Tests
run: npx playwright test
env:
PLAYWRIGHT_GLOBAL_SETUP: './tests/global-setup.ts'
run: npx playwright test
27 changes: 19 additions & 8 deletions frontend/tests/README.md
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
# End to End Workflow Navigation Testing

![Playwright Tests](https://github.com/qua-platform/qualibrate-app/actions/workflows/playwright-tests.yaml/badge.svg)


### Overview
The following tests are designed to validate the functionality and correctness of [user workflows](https://quantum-machines.atlassian.net/wiki/spaces/hlsw/pages/3223912481/QAPP+UX+Design+Brief#User-workflows) in the QUAlibrate application.
This is a front-end testing suite for integration workflows for the QUAlibrate Application. The following tests are designed to validate the functionality and correctness of [user workflows](https://quantum-machines.atlassian.net/wiki/spaces/hlsw/pages/3223912481/QAPP+UX+Design+Brief#User-workflows) section of the online documentation.

---

Expand Down Expand Up @@ -63,24 +66,32 @@ Once the server is running, access the application at http://127.0.0.1:8001/.
- Verify that the graph parameters are displayed.
- You see the calibration nodes populated on the left hands side
- You see the calibration graph populated the right hand side
- Ensure the qubits section is editable to include qubits such as `Q0`, `Q2`, and `Q3`.
- Ensure the qubits section is editable to include qubits such as `q0`, `q2`, and `q3`.
3. **Modify Node-Specific Parameters**:
- Navigate to a specific node in the calibration graph.
- Update a parameter, such as changing the sampling points from `100` to `1000`.
- Navigate to a specific node in the calibration graph and click on the `qubit_spectroscopy` node.
- verify that its parameters are expanded and are now visable
- varify `parameters` qubit parameter has calapsed
- Navigate to the calibration graph and click the middle `Rabi` node
- varify that the `qubit_spectroscopy` parameters have calapsed
- varify that the `Rabi` parameters are visible
- For `Rabi`, update a parameter, such as changing the sampling points from `100` to `1000`.
- Ensure that the updated parameter value is correctly reflected.
- Expand the Ramsey node it too also works correctly
4. **Run the Calibration Graph**:
- Click the "Play" button to start running the graph.
- Verify that the application navigates to the "Graph Status" page.
- Confirm that the graph status shows `Running` and that progress (e.g., "1 out of 3 nodes completed") is displayed.
- Confirm that the graph status shows `running` and that progress (e.g., "1 out of 3 nodes completed") is displayed.
5. **Monitor Graph Execution**:
- Wait for the graph to finish executing.
- Verify that the status updates to `Finished` and displays the total runtime (e.g., `12 seconds`).
- Verify that the status updates to `finished` and displays the total runtime (e.g., `12 seconds`).
6. **View Results**:
- Confirm nodes populated in Execution History are expandable and shows the Status, Parameters, and Outcomes sections.
- Check the results section to ensure that data (e.g., qubit spectroscopy) is displayed.
- Confirm that failed nodes or operations are clearly marked, along with the corresponding parameters.
- ~~Confirm that failed nodes or operations are clearly marked, along with the corresponding parameters.~~
- Confirm the QuAM state window is visible
7. **Inspect Additional Nodes**:
- Navigate through the results of other nodes in the graph (e.g., `Rabi` and `Ramsey`).
- Verify that all available results are displayed, or confirm that no results are present if the node has not generated data.
- ~~Verify that all available results are displayed, or confirm that no results are present if the node has not generated data.~~


## Workflow 3: Viewing Past Data
Expand Down
168 changes: 100 additions & 68 deletions frontend/tests/e2e/workflow2.test.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -27,97 +27,129 @@ test('Workflow2', {
// 2. Select a Calibration Graph:
// Identify and click on a specific calibration graph (e.g., `Single Qubit Tuneup`).
await page.getByText('Single Qubit TuneupParametersQubits:Qubit_spectroscopyRabiRamsey').click();

// Verify that the graph parameters are displayed.
await expect(page.getByRole('textbox', { name: 'qubits' })).toBeVisible();

// Ensure the qubits section is editable to include qubits such as `Q0`, `Q2`, and `Q3`.
// You see the calibration nodes populated on the left hands side
await expect(page.getByText('ParametersQubits:Qubit_spectroscopyRabiRamsey')).toBeVisible();
await expect(page.getByText('ParametersQubits:').first()).toBeVisible();
await expect(page.locator('div').filter({ hasText: /^Qubit_spectroscopy$/ }).first()).toBeVisible();
await expect(page.locator('div').filter({ hasText: /^Rabi$/ }).first()).toBeVisible();
await expect(page.locator('div').filter({ hasText: /^Ramsey$/ }).first()).toBeVisible();
// You see the calibration graph populated the right hand side
await expect(page.locator('canvas').first()).toBeVisible();
// Ensure the qubits section is editable to include qubits such as q0, q2, and q3.
const qubitsInput = page.getByRole('textbox', { name: 'qubits' });
await qubitsInput.click();
await qubitsInput.fill('q0, q2, q3');
await expect(qubitsInput).toHaveValue('q0, q2, q3');

// 3. Modify Node-Specific Parameters:
// Navigate to a specific node in the calibration graph.
const parameterTitle3 = page.locator('div:nth-child(3) > [class^="Parameters-module__parameterTitle__"]');
const arrowIcon3 = parameterTitle3.locator('[class^="Parameters-module__arrowIconWrapper__"]');
const parameterTitle4 = page.locator('div:nth-child(4) > [class^="Parameters-module__parameterTitle__"]');
const arrowIcon4 = parameterTitle4.locator('[class^="Parameters-module__arrowIconWrapper__"]');
await arrowIcon3.first().click();
await arrowIcon4.first().click();

await page.locator('div').filter({ hasText: /^Rabi$/ }).locator('div').click();
// Update a parameter, such as changing the sampling points from `100` to `1000`.
// Navigate to a specific node in the calibration graph and click on the qubit_spectroscopy node.
await page.locator('canvas').first().click({
modifiers: ['ControlOrMeta'],
position: {
x: 365,
y: 63
}
});
// verify that its parameters are expanded and are now visable
await expect(page.locator('div').filter({ hasText: /^Sampling Points:$/ }).first()).toBeVisible();
await expect(page.locator('div').filter({ hasText: /^Noise Factor:$/ }).first()).toBeVisible();
// varify parameters qubit parameter has calapsed
await expect(page.getByRole('textbox', { name: 'qubits' })).toBeHidden();
// Navigate to the calibration graph and click the middle Rabi node
await page.waitForTimeout(1000);
await page.locator('canvas').first().click({
position: {
x: 367,
y: 203
}
});
// varify that the qubit_spectroscopy parameters have calapsed
await expect(page.getByText('Qubit_spectroscopySampling')).toBeHidden();
// varify that the Rabi parameters are visible
await expect(page.getByText('RabiSampling Points:Noise')).toBeVisible();
await expect(page.locator('div').filter({ hasText: /^Sampling Points:$/ }).first()).toBeVisible();
await expect(page.locator('div').filter({ hasText: /^Noise Factor:$/ }).first()).toBeVisible();
await expect(page.locator('div').filter({ hasText: /^Test List:$/ }).first()).toBeVisible();
// For Rabi, update a parameter, such as changing the sampling points from 100 to 1000.
const samplingPointsInput = page.getByPlaceholder('sampling_points');
await samplingPointsInput.click();
await samplingPointsInput.fill('1000');
// Ensure that the updated parameter value is correctly reflected.
await expect(samplingPointsInput).toHaveValue('1000');
// Expand the Ramsey node it too also works correctly
const parameterTitle5 = page.locator('div:nth-child(5) > [class^="Parameters-module__parameterTitle__"]');
const arrowIcon5 = parameterTitle5.locator('[class^="Parameters-module__arrowIconWrapper__"]');
await arrowIcon5.first().click();


// 4. Run the Calibration Graph:
// Click the "Play" button to start running the graph.
await page.locator('.GraphElement-module__iconWrapper__uHkqg > div > svg').first().click();

await page.locator('[class^="GraphElement-module__iconWrapper__"] > div > svg').first().click();
// Verify that the application navigates to the "Graph Status" page.
//~~await expect(page).toHaveURL(/.*graph-status/);~~

await expect(page.getByText('Calibration Graph Progress')).toBeVisible();
// Confirm that the graph status shows `Running`.
await expect(page.getByText('Status: Running')).toBeVisible();

// Confirm progress (e.g., "1 out of 3 nodes completed") is displayed.
await expect(page.getByText(/Graph progress: \d+\/\d+ nodes completed/)).toBeVisible();
await expect(page.getByText('Graph progress: 0/3 node')).toBeVisible();
await expect(page.getByText('No measurements found')).toBeVisible(); // before any node is measured at all
await expect(page.getByText('Graph progress: 1/3 node')).toBeVisible();
await expect(page.locator('[class^="MeasurementElement-module__row__"]')).toBeVisible(); // Qubit_spectroscopy populates in execution history
await expect(page.getByText('Graph progress: 2/3 nodes')).toBeVisible();
await expect(page.locator('[class^="MeasurementElement-module__rowWrapper__"]').first()).toBeVisible(); // Rabi populates in execution history
await expect(page.getByText('Graph progress: 3/3 nodes')).toBeVisible();
await expect(page.locator('[class^="MeasurementElement-module__rowWrapper__"]').first()).toBeVisible(); // Ramsey populates in execution history

// 5. Monitor Graph Execution:
// Wait for the graph to finish executing.
await page.waitForTimeout(5000); // Adjust timeout as per actual runtime.
await expect(page.getByText('Status: Finished')).toBeVisible();

// Verify total runtime is displayed (e.g., `12 seconds`).
await expect(page.getByText('Status: finished')).toBeVisible();
// Verify total runtime is displayed (e.g., Run duration: 12.176s).
await expect(page.getByText(/Run duration: \d+\.\d{1,3}s/)).toBeVisible();

// 6. View Results:
// Check the results section to ensure that data is displayed.
await expect(page.getByRole('textbox', { name: 'results' })).toBeVisible();

// Confirm failed nodes or operations are marked.
await expect(page.getByText('Failed Nodes')).not.toBeVisible(); // Update if necessary.

// 7. Inspect Additional Nodes:
// Navigate through the results of other nodes in the graph.
await page.getByText('Rabi').click();
await expect(page.getByRole('textbox', { name: 'results' })).toBeVisible();

await page.getByText('Ramsey').click();
await expect(page.getByRole('textbox', { name: 'results' })).toBeVisible();




// 2. Select a Calibration Graph:
// Identify and click on a specific calibration graph (e.g., `Single Qubit Tuneup`).
// Verify that the graph parameters are displayed.
// Ensure the qubits section is editable to include qubits such as `Q0`, `Q2`, and `Q3`.


// 3. Modify Node-Specific Parameters:
// Navigate to a specific node in the calibration graph.
// Update a parameter, such as changing the sampling points from `100` to `1000`.
// Ensure that the updated parameter value is correctly reflected.

// 4. Run the Calibration Graph:
// Click the "Play" button to start running the graph.
// Verify that the application navigates to the "Graph Status" page.
// Confirm that the graph status shows `Running` and that progress (e.g., "1 out of 3 nodes completed") is displayed.

// 5. Monitor Graph Execution:
// Wait for the graph to finish executing.
// Verify that the status updates to `Finished` and displays the total runtime (e.g., `12 seconds`).

// 6. View Results:
// Check the results section to ensure that data (e.g., qubit spectroscopy) is displayed.
// Confirm that failed nodes or operations are clearly marked, along with the corresponding parameters.

// 7. Inspect Additional Nodes:
// Confirm nodes populated in Execution History are expandable and shows Status, Parameters, and Outcomes sections.
await page.locator('div:nth-child(3) > [class^="MeasurementElement-module__rowWrapper__"]').click();
// await expect(page.getByText('#408 Qubit_spectroscopyStatus')).toBeVisible();
// await expect(page.getByText(/#\d+ Qubit_spectroscopyStatus/)).toBeVisible();
await expect(page.getByText(/.*Qubit_spectroscopyStatus/)).toBeVisible();
await expect(page.getByText(/Status:Run start: \d{4}-\d{2}-\d{2}/)).toBeVisible();
// Status
await expect(page.getByText('Status:', { exact: true })).toBeVisible();
await expect(page.getByText(/Run start: \d{4}-\d{2}-\d{2} \d{2}:\d{2}:/)).toBeVisible();
// Parameters
await expect(page.getByText('Parameters')).toBeVisible();
await expect(page.getByText('qubits: q0q2q3sampling_points')).toBeVisible();
await expect(page.getByText('qubits: q0q2q3')).toBeVisible();
await expect(page.getByText('sampling_points:')).toBeVisible();
await expect(page.getByText('noise_factor:')).toBeVisible();
// Outcomes
await expect(page.getByText('Outcomes')).toBeVisible();
// Check the results section to ensure that data (e.g., qubit spectroscopy) is displayed.
await expect(page.getByRole('heading', { name: 'Results' })).toBeVisible();
await expect(page.getByPlaceholder('Enter a value').first()).toBeVisible();
await expect(page.getByTestId('data-key-pairfrequency_shift')).toBeVisible();
await expect(page.getByTestId('data-key-pairfrequency_shift')).toContainText(/"frequency_shift":\d+(\.\d+)?/); // Matches the format of any number
await expect(page.getByText('"./results_fig.png"')).toBeVisible();
await expect(page.locator('a')).toBeVisible(); // results figure
await expect(page.getByTestId('data-key-pairarr')).toBeVisible();
// Confirm the QuAM state window is visible
await expect(page.getByRole('heading', { name: 'QuAM' })).toBeVisible();
await expect(page.getByPlaceholder('Enter a value').nth(1)).toBeVisible();
await expect(page.locator('div').filter({ hasText: /^\{\}0 Items$/ }).first()).toBeVisible();

// 7 Inspect Additional Nodes:
// Navigate through the results of other nodes in the graph (e.g., `Rabi` and `Ramsey`).
// Verify that all available results are displayed, or confirm that no results are present if the node has not generated data.

});
await page.locator('canvas').first().click({ // clicking on the Rabi node
position: {
x: 157,
y: 204
}
});
await page.locator('canvas').first().click({ // clicking on the Ramsey node
position: {
x: 158,
y: 348
}
});
});
12 changes: 12 additions & 0 deletions frontend/tests/e2e/workflow3.test.ts
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
import { test, expect } from '@playwright/test';


test('Workflow_3', {
annotation: {
type: 'Third User Workflow',
description: 'Viewing Past Data',
},
}, async ({ page }) => {

// TODO
});
23 changes: 23 additions & 0 deletions frontend/tests/global-setup.ts
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
import { FullConfig } from '@playwright/test';
import { exec } from 'child_process';

async function globalSetup(config: FullConfig) {
console.log('Starting QUAlibrate server...');

// Start the server as a child process
const server = exec('qualibrate start', (error, stdout, stderr) => {
if (error) {
console.error(`Error starting server: ${error.message}`);
return;
}
if (stderr) {
console.error(`Server stderr: ${stderr}`);
}
console.log(`Server stdout: ${stdout}`);
});

// Store server reference for teardown, if needed
(global as any).__SERVER__ = server;
}

export default globalSetup;
10 changes: 10 additions & 0 deletions frontend/tests/global-teardown.ts
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
async function globalTeardown() {
console.log('Stopping QUAlibrate server...');
const server = (global as any).__SERVER__;
if (server) {
server.kill('SIGTERM');
}
}

export default globalTeardown;

5 changes: 5 additions & 0 deletions frontend/tests/playwright.config.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,17 @@ export default defineConfig({
use: {
/* Base URL to use in actions like `await page.goto('/')`. */
// baseURL: 'http://127.0.0.1:3000',
headless: true,
baseURL: 'http://127.0.0.1:8001/', // Update base URL for your app

/* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */
trace: 'on-first-retry',
screenshot: 'only-on-failure', // Take screenshots only on failure
video: 'retain-on-failure', // Record videos only for failed tests
},
globalSetup: './tests/global-setup.ts', // Path to your global setup file
globalTeardown: './tests/global-teardown.ts', // Path to your global teardown file

projects: [
{
name: 'Chromium',
Expand Down

0 comments on commit 45c13c1

Please sign in to comment.