-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Allow running subsets of test cases #18
Conversation
Allow running only a subset of the provided tests using pytest's filtering feature with the '-k' command-line option, which facilitates (de)selecting the test cases to run filtering on their name. Signed-off-by: Phoevos Kalemkeris <[email protected]>
Add a comment in the top-level tox.ini with instructions on how to use the newly added filtering functionality of the test suite. Signed-off-by: Phoevos Kalemkeris <[email protected]>
Introduce a tox testenv for running all tests apart from the ones that assume MLFlow integration. This relies on the newly added filtering functionality of the test suite. Signed-off-by: Phoevos Kalemkeris <[email protected]>
Update READMEs with instructions for filtering test cases using pytest directly or tox. Signed-off-by: Phoevos Kalemkeris <[email protected]>
246c5a2
to
c284717
Compare
Add a `filter` option to pytest to use over the builtin `override_ini`, greatly simplifying invocation. Signed-off-by: Phoevos Kalemkeris <[email protected]>
177ec43
to
b3476d7
Compare
Checked out branch and tried to run some tests according to instructions. Tests failed:
|
The functionality that this PR introduces works correctly, i.e. proper tests are selected when |
@i-chvets, I have seen this error before and it's due to running the test inside a virtual environment. That won't work out of the box, you'd have to set up some Jupyter/ipython related stuff, hence it's not in the instructions. This is supposed to be executed from inside a notebook where you'd usually just pip install the requirements user-wide (That's what the driver automation does ATM anyway). |
Understood. Approving. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Approved. ACK tests should not be executed in virtual env, but in the notebook.
-k
command-line option, which facilitates (de)selecting the test cases to run filtering on their nameREADME
with instructions on using the newly added functionalityCloses #6