[FEATURE] Implement API Consistency Tests for Remaining Workflow Steps #908
Labels
enhancement
New feature or request
good first issue
Good for newcomers
help wanted
Extra attention is needed
Is your feature request related to a problem?
Currently, only the agent step API consistency test has been completed. There are additional workflow steps that require similar consistency checks to ensure that the API specifications match the required enum parameters. Without these tests, we cannot guarantee that future changes won't break the consistency across other workflow steps.
What solution would you like?
Follow #900 to extend the API consistency test implementation for other workflow steps.
Steps:
Use ApiSpecFetcher to fetch the OpenAPI specification:
Leverage the
ApiSpecFetcher
utility introduced in Add ApiSpecFetcher for Fetching and Comparing API Specifications #900 to programmatically fetch the OpenAPI specification for each workflow step.Utilize the compareRequiredFields method:
Use the
compareRequiredFields
method from ApiSpecFetcher to compare the required input fields from the API specification with the predefined enum values for each workflow step.Integrate ApiConsistencyTest into each step's unit tests:
For each workflow step (e.g., registerAgent, registerModel), ensure an API consistency check is added in the respective unit tests to validate the correctness of required API parameters.
Add a GitHub Action for daily workflow test execution:
Once all steps are completed, set up a GitHub Action to trigger a daily run of the workflow tests to ensure API consistency over time.
Do you have any additional context?
The agent step API consistency test has already been implemented. The same approach can be applied to the remaining steps to ensure all workflow steps are covered. This issue is important to maintain consistency in our API as it evolves.
CheckList
ML-Commons Related
CreateConnectorStep
DeleteAgentStep
(No request body)DeleteModelStep
(No request body)DeployModelStep
(No request body)RegisterAgentStep
RegisterLocalCustomModelStep
RegisterLocalPretrainedModelStep
RegisterLocalSparseEncodingModelStep
RegisterModelGroupStep
RegisterRemoteModelStep
UndeployModelStep
(No request body)The text was updated successfully, but these errors were encountered: