Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Refactor ai-api E2E tests #187

Merged
merged 27 commits into from
Oct 18, 2024
Merged

chore: Refactor ai-api E2E tests #187

merged 27 commits into from
Oct 18, 2024

Conversation

shibeshduw
Copy link
Contributor

@shibeshduw shibeshduw commented Oct 1, 2024

Context

AI/gen-ai-hub-sdk-js-backlog#142.

  • Added sample code for DeploymentApi and ScenarioApi endpoints, some of which are now used in the e2e-tests.
  • These tests can now be run individually for better local testing or sequentially as a whole.
  • Implemented a cleanup feature so that unused deployments are deleted if tests fail or are interrupted.

Definition of Done

  • Code is tested (Unit, E2E)
  • Error handling created / updated & covered by the tests above
  • Documentation updated
  • [ ] (Optional) Aligned changes with the Java SDK
  • (Optional) Release notes updated

.changeset/sixty-pandas-bathe.md Outdated Show resolved Hide resolved
packages/ai-api/README.md Outdated Show resolved Hide resolved
tests/e2e-tests/src/deployment-api.test.ts Outdated Show resolved Hide resolved
tests/e2e-tests/package.json Outdated Show resolved Hide resolved
tests/e2e-tests/src/deployment-api.test.ts Show resolved Hide resolved
tests/e2e-tests/src/deployment-api.test.ts Show resolved Hide resolved
@shibeshduw shibeshduw marked this pull request as draft October 4, 2024 08:35
@shibeshduw shibeshduw marked this pull request as ready for review October 4, 2024 13:55
Copy link
Contributor

@ZhongpinWang ZhongpinWang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will continue reviewing the PR once I have more time. Left some comments.

sample-code/src/ai-api/deployment-api.ts Outdated Show resolved Hide resolved
sample-code/src/ai-api/deployment-api.ts Outdated Show resolved Hide resolved
sample-code/src/index.ts Outdated Show resolved Hide resolved
tests/e2e-tests/package.json Outdated Show resolved Hide resolved
tests/e2e-tests/src/deployment-api.test.ts Outdated Show resolved Hide resolved
tests/e2e-tests/src/deployment-api.test.ts Outdated Show resolved Hide resolved
tests/e2e-tests/src/deployment-api.test.ts Outdated Show resolved Hide resolved
tests/e2e-tests/src/deployment-api.test.ts Outdated Show resolved Hide resolved
tests/e2e-tests/src/deployment-api.test.ts Outdated Show resolved Hide resolved
tests/e2e-tests/src/utils/ai-api-utils.ts Outdated Show resolved Hide resolved
tests/e2e-tests/src/utils/cleanup-deployments.ts Outdated Show resolved Hide resolved
Copy link
Contributor

@marikaner marikaner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cleanup looks good to me, I noticed that currently the samples are not really samples and I think this will not be very helpful to users. I believe it is ok to use the samples for the E2E tests, but the tests should not guide how we structure our samples, because the purpose is a different one: Samples are for users, tests are for automatic quality insurance.

sample-code/src/ai-api/deployment-api.ts Outdated Show resolved Hide resolved
sample-code/src/ai-api/deployment-api.ts Outdated Show resolved Hide resolved
tests/e2e-tests/src/utils/ai-api-utils.ts Outdated Show resolved Hide resolved
@shibeshduw
Copy link
Contributor Author

Was finally able to get to the changes for this. I've changed the sample code to hopefully have something useful now. The new retry values and timeouts seem to work well from my testing.

Copy link
Contributor

@ZhongpinWang ZhongpinWang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@shibeshduw shibeshduw merged commit 65c50fa into main Oct 18, 2024
11 checks passed
@shibeshduw shibeshduw deleted the refactor-ai-api-e2e-tests branch October 18, 2024 08:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants