[APM] add cypress-axe to e2e tests to check A11y #126610

MiriamAparicio · 2022-03-01T16:01:06Z

Added cypress-axe to e2e tests to check A11y

For now, we are going to set the parameter skipFailures to true, this will log the errors but won't make the test fails, issues to tackle the failures will be created afterwards

elasticmachine · 2022-03-01T16:01:09Z

Pinging @elastic/kibana-accessibility (Project:Accessibility)

elasticmachine · 2022-03-01T16:01:09Z

Pinging @elastic/apm-ui (Team:apm)

sorenlouv · 2022-03-01T18:01:11Z

x-pack/plugins/apm/ftr_e2e/cypress/integration/read_only_user/dependencies.spec.ts

@@ -93,6 +105,9 @@ describe('Dependencies', () => {
      cy.contains('a', 'postgresql').click({ force: true });

      cy.contains('h1', 'postgresql');
+
+      // set skipFailures to true to not fail the test when there are accessibility failures
+      checkA11y(true);


This should probably be in a separate it

1Copenut

Just one comment about a best practice option for bumping code coverage. Thank you!

1Copenut · 2022-03-01T22:37:08Z

x-pack/plugins/apm/ftr_e2e/cypress/integration/read_only_user/dependencies.spec.ts

+          timeRange
+        )}`
+      );
+      cy.contains('a[role="tab"]', 'Dependencies');


If there is a second tab that can be interacted with, it'd be great to click on that tab and make a second checkA11y call to bump code coverage.

Hi @1Copenut, this is our first approach to the A11y checks in our pages, we are going to add more checks in the future for now we are just checking the first render and not the interactions.

APM Service Overview contains multiple tabs, it's good to know that we can bump the code coverage if we check A11y in all of them.

gbamparop · 2022-03-02T14:27:19Z

x-pack/plugins/apm/ftr_e2e/cypress/support/commands.ts

+
+const axeConfig = {
+  ...AXE_CONFIG,
+  rules: [...AXE_CONFIG.rules],


Isn't this the same with just using AXE_CONFIG?

gbamparop · 2022-03-02T14:30:48Z

...r_e2e/cypress/integration/read_only_user/transactions_overview/transactions_overview.spec.ts

+  it('has no detectable a11y violations on load', () => {
+    cy.visit(serviceTransactionsHref);
+    cy.contains('a[role="tab"]', 'Transactions').should(
+      'have.class',


We might want to avoid depending on eui classes. If we need to check that a tab is selected, we could instead use aria-selected="true" for example.

gbamparop · 2022-03-02T14:33:44Z

x-pack/plugins/apm/ftr_e2e/cypress/support/commands.ts

+  runOnly: [...AXE_OPTIONS.runOnly, 'best-practice'],
+};
+
+export const checkA11y = (skipFailures: boolean) => {


Could we wrap cy.checkA11y in our own command instead of calling the function?

gbamparop · 2022-03-02T14:36:44Z

...s/apm/ftr_e2e/cypress/integration/read_only_user/service_inventory/service_inventory.spec.ts

@@ -53,6 +54,12 @@ describe('When navigating to the service inventory', () => {
    cy.visit(serviceInventoryHref);
  });

+  it('has no detectable a11y violations on load', () => {
+    cy.contains('Services');


Checking some text on the page could be included in another test as it's not specific to accessibility (same with other files)

It shows an error if trying to call checkA11y() directly

You mean without having anything else in the test? What error do you get?

I think cy.contains('Services'); is needed to wait for that element to be visible before running the a11y check. Would be better if this was more obvious from the code

gbamparop

Many thanks for adding this! It's great that we now have automation for accessibility testing and we can start adding more to it! I've added some minor comments on the PR.

sorenlouv · 2022-03-02T15:11:20Z

x-pack/plugins/apm/ftr_e2e/cypress/integration/read_only_user/dependencies.spec.ts

+      );
+      cy.contains('a[role="tab"]', 'Dependencies');
+      // set skipFailures to true to not fail the test when there are accessibility failures
+      checkA11y(true);


Let's make it clear what true means:

Suggested change

checkA11y(true);

checkA11y({ skipFailures: true });

sorenlouv · 2022-03-02T15:14:56Z

x-pack/plugins/apm/ftr_e2e/cypress/support/commands.ts

+  cy.injectAxe();
+  cy.configureAxe(axeConfig);
+  const context = '.kbnAppWrapper'; // Scopes a11y checks to only our app
+  cy.checkA11y(context, axeOptions, undefined, skipFailures);


What's undefined ?

I'd prefer this:

Suggested change

cy.checkA11y(context, axeOptions, undefined, skipFailures);

cy.checkA11y( { context, axeOptions, skipFailures });

But I assume that's out of our control.

I can't change that unfortunately

But we can get rid of the last two params when we don't need to add skipFailures, I added it now until we fix the failures.

@sqren The undefined here is a placeholder for an optional violations callback function that ultimately would be passed to the axe-core lib. The cypress-axe docs do a good job unpacking the arguments if you're curious: https://www.npmjs.com/package/cypress-axe#user-content-parameters-on-cychecka11y-axerun

kibana-ci · 2022-03-02T18:16:12Z

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

💚 Build #27064 succeeded 10d64a1
💚 Build #26860 succeeded 5487206ea72aca1a8047980eb27b96c716f793a8

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

MiriamAparicio added Project:Accessibility Team:APM All issues that need APM UI Team support release_note:skip Skip the PR/issue when compiling release notes backport:skip This commit does not require backporting v8.2.0 labels Mar 1, 2022

MiriamAparicio requested a review from a team as a code owner March 1, 2022 16:01

sorenlouv reviewed Mar 1, 2022

View reviewed changes

1Copenut reviewed Mar 1, 2022

View reviewed changes

MiriamAparicio added 2 commits March 2, 2022 08:53

[APM] add cypress-axe to e2e tests to check A11y

7e282a5

check A11y in dependecy overview

10d64a1

MiriamAparicio force-pushed the add-a11y-test-to-apm branch from 5487206 to 10d64a1 Compare March 2, 2022 10:43

gbamparop reviewed Mar 2, 2022

View reviewed changes

gbamparop approved these changes Mar 2, 2022

View reviewed changes

sorenlouv reviewed Mar 2, 2022

View reviewed changes

PR comments

2255f92

MiriamAparicio merged commit 2880a75 into elastic:main Mar 3, 2022

MiriamAparicio deleted the add-a11y-test-to-apm branch March 3, 2022 10:48

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[APM] add cypress-axe to e2e tests to check A11y #126610

[APM] add cypress-axe to e2e tests to check A11y #126610

MiriamAparicio commented Mar 1, 2022

elasticmachine commented Mar 1, 2022

elasticmachine commented Mar 1, 2022

sorenlouv Mar 1, 2022

1Copenut left a comment

1Copenut Mar 1, 2022

MiriamAparicio Mar 2, 2022

gbamparop Mar 2, 2022

gbamparop Mar 2, 2022 •

edited

Loading

gbamparop Mar 2, 2022

gbamparop Mar 2, 2022

MiriamAparicio Mar 2, 2022

gbamparop Mar 2, 2022

sorenlouv Mar 2, 2022

gbamparop left a comment

sorenlouv Mar 2, 2022

sorenlouv Mar 2, 2022

MiriamAparicio Mar 2, 2022

MiriamAparicio Mar 2, 2022

1Copenut Mar 2, 2022

kibana-ci commented Mar 2, 2022

	cy.checkA11y(context, axeOptions, undefined, skipFailures);
	cy.checkA11y( { context, axeOptions, skipFailures });

[APM] add cypress-axe to e2e tests to check A11y #126610

[APM] add cypress-axe to e2e tests to check A11y #126610

Conversation

MiriamAparicio commented Mar 1, 2022

elasticmachine commented Mar 1, 2022

elasticmachine commented Mar 1, 2022

Choose a reason for hiding this comment

1Copenut left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gbamparop Mar 2, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gbamparop left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kibana-ci commented Mar 2, 2022

💚 Build Succeeded

Metrics [docs]

History

gbamparop Mar 2, 2022 •

edited

Loading