-
Notifications
You must be signed in to change notification settings - Fork 8.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
FTR for playground #186246
FTR for playground #186246
Conversation
await pageObjects.searchPlayground.PlaygroundStartChatPage.expectSelectIndex(indexName); | ||
it('creates a connector successfully', async () => { | ||
await pageObjects.searchPlayground.PlaygroundStartChatPage.expectOpenConnectorPagePlayground(); | ||
await pageObjects.searchPlayground.PlaygroundStartChatPage.createOpenAIConnector(proxy); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Need to create an api function for creating connector as we don't need to test third-party library. Because if they changed name of field it will broke our test.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Although a different team maintains it, the main reason to add this step in the test is to verify the flyout opens and the user can create a Gen AI connector and use it immediately.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah I agree that we need to test how flyout is opening, because this is related to our code, but testing the flyout itself seems like an antipattern “Third-Party Code Testing.” Which can lead to:
- Unnecessary Maintenance: tests might fail if the third-party library updates or changes, leading to maintenance work unrelated to your actual code.
- Redundancy: Most third-party libraries come with their own tests. Testing them again in integration suite duplicates effort without additional value.
- False Confidence: Passing tests might give a false sense of security about the library’s behavior under all conditions, which isn’t practical to ensure comprehensively.
088617d
to
e389932
Compare
Flaky Test Runner Stats🎉 All tests passed! - kibana-flaky-test-suite-runner#6365[✅] x-pack/test/functional/apps/search_playground/config.ts: 25/25 tests passed. |
const [conversationSimulator] = await Promise.all([ | ||
conversationInterceptor.waitForIntercept(), | ||
]); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we don't need here promise.all. guess just forgot to remove it after reducing titleInterceptor
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Really great job with making llmProxy works 😎
@elasticmachine merge upstream |
@elasticmachine merge upstream |
Flaky Test Runner Stats🎉 All tests passed! - kibana-flaky-test-suite-runner#6393[✅] x-pack/test/functional/apps/search_playground/config.ts: 25/25 tests passed. |
Flaky Test Runner Stats🎉 All tests passed! - kibana-flaky-test-suite-runner#6394[✅] x-pack/test_serverless/functional/test_suites/search/config.ts: 25/25 tests passed. |
155b6d0
to
c49db73
Compare
💛 Build succeeded, but was flaky
Failed CI StepsTest FailuresMetrics [docs]Async chunks
History
To update your PR or re-run it, just comment with: |
Summary
This PR contains high-priority FTR tests for the playground in ESS.
Checklist
Delete any items that are not applicable to this PR.
Risk Matrix
Delete this section if it is not applicable to this PR.
Before closing this PR, invite QA, stakeholders, and other developers to identify risks that should be tested prior to the change/feature release.
When forming the risk matrix, consider some of the following examples and how they may potentially impact the change:
For maintainers