Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve test run time #92

Open
thalurur opened this issue Jun 30, 2021 · 4 comments
Open

Improve test run time #92

thalurur opened this issue Jun 30, 2021 · 4 comments
Labels
CI/CD maintenance improves code quality, but not the product

Comments

@thalurur
Copy link
Contributor

Is your feature request related to a problem? Please describe.
At the moment running the tests of IndexManagement takes a long time. Need to bring this time down to make the code development process easier

Describe the solution you'd like
Make the different test suites run as different github actions so PRs don't need to wait around 20-30 min to see if the tests succeed. We can parallelize the actions at the current highest feature level.

  • ISM
  • Rollups
  • Transforms

Test cleanup - any test that can be an unit test should be one, go through existing tests and move any integration tests that can be just unit tests

@thalurur thalurur changed the title Imporve test run time Improve test run time Jun 30, 2021
thalurur pushed a commit to thalurur/open-index-management that referenced this issue Oct 22, 2021
* Transform cypress tests (opensearch-project#21)

* Added Cypress tests for Transforms UI

* License update

* Unit tests initial commit

* Added jest tests for Transforms, details, edit and create

* Sample transform fix

* Fixed cypress test transform and sample data load order

Signed-off-by: Eric Lobdell <[email protected]>

* Test fix debug

* Simplified unit test

* Updated transforms cypress test comments and success criteria

Signed-off-by: Eric Lobdell <[email protected]>

* Fixed quotation mark issue in test

Signed-off-by: Eric Lobdell <[email protected]>

* Removed debug mapping and index objects from unit test

Signed-off-by: Eric Lobdell <[email protected]>

* Cypress test improvements and completed edit transform unit test

Signed-off-by: Eric Lobdell <[email protected]>

* Integ test delete fix and EditTransforms jest test fix

Signed-off-by: Eric Lobdell <[email protected]>

* Restoring enable disable page refresh delay to fix failing integ test

Signed-off-by: Eric Lobdell <[email protected]>

* Resolve test constants conflict

Signed-off-by: Eric Lobdell <[email protected]>
@dbbaughe
Copy link
Contributor

So the majority of the long test times comes from the fact that under the hood we are using the testing framework from core that uses the randomization runner which means we can't parallelize our tests (even though the majority if not all could be parallelized).

So as Ravi pointed out we should look into splitting the tests by feature first and then from there we could further split each feature into a subset of tests (such as different REST APIs and other things in ISM).

@dbbaughe
Copy link
Contributor

Current unit tests:
Screen Shot 2022-03-16 at 10 30 15 AM

There are some unit tests in the Steps part that are taking 3-7 seconds each test that needs to be checked (although not significant in terms of the 20-30+ minute total time).

@dbbaughe
Copy link
Contributor

Current integ tests:

Screen Shot 2022-03-16 at 10 35 14 AM

@dbbaughe
Copy link
Contributor

Screen Shot 2022-03-16 at 10 37 00 AM
Screen Shot 2022-03-16 at 10 37 14 AM
Screen Shot 2022-03-16 at 10 37 25 AM

@bowenlan-amzn bowenlan-amzn added maintenance improves code quality, but not the product and removed enhancement New request Severity-Minor Priority-Immediate labels Nov 5, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI/CD maintenance improves code quality, but not the product
Projects
None yet
Development

No branches or pull requests

3 participants