From c264093c7f3efefb2d42e2103612fef1c4ac6947 Mon Sep 17 00:00:00 2001 From: Radovan Bast Date: Wed, 27 Sep 2023 21:10:18 +0200 Subject: [PATCH] mv end-to-end test to test design; closes #209 --- content/pytest.md | 85 +----------------------------------------- content/test-design.md | 63 +++++++++++++++++++++++++++++++ 2 files changed, 64 insertions(+), 84 deletions(-) diff --git a/content/pytest.md b/content/pytest.md index a00fae4..9272744 100644 --- a/content/pytest.md +++ b/content/pytest.md @@ -137,90 +137,7 @@ def test_add(): This is OK but the `1.0e-7` can be a bit arbitrary. ```` - -## Exercise end-to-end test (advanced) - -If you have spare time, you can work on this. It isn't required for -anything else. It counts as a bit advanced, because you are writing -one program to run another program and check it's output. This isn't -necessarily hard, but it is different from what most people do! - -````{exercise} Local-3: Create an end-to-end test (advanced, optional) - -Often, you can include tests that run your whole workflow or program. -For example, you might include sample data and check the output -against what you expect. (including sample data is a great idea -anyway, so this helps a lot!) - -We'll use the word-count example repository -, as used in [the -Documentation -lesson](https://coderefinery.github.io/reproducible-research/workflow-management/). - -As a reminder, you can run the script like this to get some output, -which prints to standard output (the terminal): - -```console -$ python3 statistics/count.py data/abyss.txt -``` - -Your goal is to make a test that can run this and let you know if it's -successful or not. You could use Python, or you could use shell -scripting. You can test if these two lines are in the output: `the -4044` and `and 2807`. - -Python hint: -[subprocess.check_output](https://docs.python.org/3/library/subprocess.html#subprocess.check_output) -will run a command and return its output as a string. - -Bash hint: `COMMAND | grep "PATTERN"` ("pipe to grep") will be true if -the pattern is in the command. - -```` - -````{solution} Solution: Local-3 -There are two solutions in the repository already, in the `tests/` -dierectory , one in Python -and one in bash shell. Neither of these are a very advanced or -perfect solution, and you could integrate them with pytest or whatever -other test framework you use. - -The shell one works with shell and prints a bit more output: - -```console -$ sh tests/end-to-end-shell.sh -the 4044 -Success: 'the' found correct number of times -and 2807 -Success: 'and' found correct number of times -Success -``` - -The Python one: -```console -$ python3 tests/end-to-end-python.py -Success -``` - -```` - - -## Spare time? - -We give a lot of time here, since it can be tricky for some people. -If you have more time, try looking at {doc}`test-design` - there are -many more tests to start writing there, and you now know enough to do -that. - -We'll discuss these in more detail later. - - -## Summary - -* Local testing can be very easy. -* Even simple tests can provide checks for obviously breaking things. - - +--- ```{keypoints} - pytest collects and runs all test functions starting with diff --git a/content/test-design.md b/content/test-design.md index 38a8675..efa91ef 100644 --- a/content/test-design.md +++ b/content/test-design.md @@ -707,6 +707,69 @@ more repetition - How would you test a code end-to-end which produces images? ```` +If you have spare time, you can work on this. It isn't required for +anything else. It counts as a bit advanced, because you are writing +one program to run another program and check it's output. This isn't +necessarily hard, but it is different from what most people do! + +````{exercise} Design-9: Create an end-to-end test (advanced, optional) + +Often, you can include tests that run your whole workflow or program. +For example, you might include sample data and check the output +against what you expect. (including sample data is a great idea +anyway, so this helps a lot!) + +We'll use the word-count example repository +, as used in [the +Documentation +lesson](https://coderefinery.github.io/reproducible-research/workflow-management/). + +As a reminder, you can run the script like this to get some output, +which prints to standard output (the terminal): + +```console +$ python3 statistics/count.py data/abyss.txt +``` + +Your goal is to make a test that can run this and let you know if it's +successful or not. You could use Python, or you could use shell +scripting. You can test if these two lines are in the output: `the +4044` and `and 2807`. + +Python hint: +[subprocess.check_output](https://docs.python.org/3/library/subprocess.html#subprocess.check_output) +will run a command and return its output as a string. + +Bash hint: `COMMAND | grep "PATTERN"` ("pipe to grep") will be true if +the pattern is in the command. + +```` + +````{solution} Solution: Design-9 +There are two solutions in the repository already, in the `tests/` +dierectory , one in Python +and one in bash shell. Neither of these are a very advanced or +perfect solution, and you could integrate them with pytest or whatever +other test framework you use. + +The shell one works with shell and prints a bit more output: + +```console +$ sh tests/end-to-end-shell.sh +the 4044 +Success: 'the' found correct number of times +and 2807 +Success: 'and' found correct number of times +Success +``` + +The Python one: +```console +$ python3 tests/end-to-end-python.py +Success +``` +```` + ```{keypoints} - Pure functions (these are functions without side-effects) are easiest to test. - Classes can be tested but it's somewhat more elaborate.