Skip to content

Commit

Permalink
Skip some serverless tests after roles and spaces changes in QA (elas…
Browse files Browse the repository at this point in the history
…tic#190599)

## Summary

This PR skips some serverless tests after changes on roles and spaces
have been applied to the QA environment.

### Details

After role management and multiple spaces have been enabled in the QA
environment, a number of tests started to fail when running against MKI.
Since we run the same set of tests against QA and Staging, we can't just
fix the tests to work with the changes in QA because then they would
fail when running against Staging.
After carefully checking the assertions in the tests (mainly checking
that certain things are NOT available, while we gradually make them
available again env by env), we decided in a small group discussion it's
the best way forward to temporarily skip these tests and revisit them
once the change has been applied to all environments. Some of the tests
might be deleted then, others might be adjusted to the new behavior.

To be clear: this is not ideal and this outcome is a trade-off for this
specific situation and should not be used as future guidance to handle
this kind of problem. We have some ideas of how to deal with changes
that are applied gradually to the environments, but it takes more time
to consolidate these. In the interest of moving forward quickly, the
test skip was considered the best solution for now.
  • Loading branch information
pheyos authored Aug 15, 2024
1 parent ae6c1f6 commit 1aeecaf
Show file tree
Hide file tree
Showing 5 changed files with 21 additions and 7 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,9 @@ export default function ({ getService }: FtrProviderContext) {
svlCommonApi.assertResponseStatusCode(400, status, body);
});

it('#create', async () => {
// Skipped due to change in QA environment for role management and spaces
// TODO: revisit once the change is rolled out to all environments
it.skip('#create', async () => {
const { body, status } = await supertestWithoutAuth
.post('/api/spaces/space')
.set(internalRequestHeader)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -42,15 +42,19 @@ export default function ({ getService }: FtrProviderContext) {
describe('route access', () => {
describe('internal', () => {
describe('disabled', () => {
it('get all privileges', async () => {
// Skipped due to change in QA environment for role management and spaces
// TODO: revisit once the change is rolled out to all environments
it.skip('get all privileges', async () => {
const { body, status } = await supertestWithoutAuth
.get('/api/security/privileges')
.set(svlCommonApi.getInternalRequestHeader())
.set(roleAuthc.apiKeyHeader);
svlCommonApi.assertApiNotFound(body, status);
});

it('get built-in elasticsearch privileges', async () => {
// Skipped due to change in QA environment for role management and spaces
// TODO: revisit once the change is rolled out to all environments
it.skip('get built-in elasticsearch privileges', async () => {
const { body, status } = await supertestWithoutAuth
.get('/internal/security/esPrivileges/builtin')
.set(svlCommonApi.getInternalRequestHeader())
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,9 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) {
});
});

describe('when in single space mode', function () {
// Skipped due to change in QA environment for role management and spaces
// TODO: revisit once the change is rolled out to all environments
describe.skip('when in single space mode', function () {
let dataViewId = '';
before(async () => {
await esArchiver.load(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,9 @@ export default ({ getPageObjects, getService }: FtrProviderContext) => {
}).not.to.throwError();
});

describe('Roles management card', () => {
// Skipped due to change in QA environment for role management and spaces
// TODO: revisit once the change is rolled out to all environments
describe.skip('Roles management card', () => {
it('should not be displayed by default', async () => {
await retry.waitFor('page to be visible', async () => {
return await testSubjects.exists('cards-navigation-page');
Expand All @@ -56,7 +58,9 @@ export default ({ getPageObjects, getService }: FtrProviderContext) => {
});
});

describe('Organization members management card', () => {
// Skipped due to change in QA environment for role management and spaces
// TODO: revisit once the change is rolled out to all environments
describe.skip('Organization members management card', () => {
it('should not be displayed by default', async () => {
await retry.waitFor('page to be visible', async () => {
return await testSubjects.exists('cards-navigation-page');
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,9 @@ export default function ({ getPageObject, getService }: FtrProviderContext) {
const svlCommonNavigation = getService('svlCommonNavigation');
const testSubjects = getService('testSubjects');

describe('space selection', function () {
// Skipped due to change in QA environment for role management and spaces
// TODO: revisit once the change is rolled out to all environments
describe.skip('space selection', function () {
before(async () => {
await svlCommonPage.loginAsViewer();
});
Expand Down

0 comments on commit 1aeecaf

Please sign in to comment.