forked from swcarpentry/matlab-novice-inflammation
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy path06-defensive.html
235 lines (221 loc) · 24.3 KB
/
06-defensive.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<meta name="generator" content="pandoc">
<title>Software Carpentry: Programming with MATLAB</title>
<link rel="shortcut icon" type="image/x-icon" href="/favicon.ico" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<link rel="stylesheet" type="text/css" href="css/bootstrap/bootstrap.css" />
<link rel="stylesheet" type="text/css" href="css/bootstrap/bootstrap-theme.css" />
<link rel="stylesheet" type="text/css" href="css/swc.css" />
<link rel="alternate" type="application/rss+xml" title="Software Carpentry Blog" href="http://software-carpentry.org/feed.xml"/>
<meta charset="UTF-8" />
<!-- HTML5 shim, for IE6-8 support of HTML5 elements -->
<!--[if lt IE 9]>
<script src="http://html5shim.googlecode.com/svn/trunk/html5.js"></script>
<![endif]-->
</head>
<body class="lesson">
<div class="container card">
<div class="banner">
<a href="http://software-carpentry.org" title="Software Carpentry">
<img alt="Software Carpentry banner" src="img/software-carpentry-banner.png" />
</a>
</div>
<article>
<div class="row">
<div class="col-md-10 col-md-offset-1">
<h1 class="title">Programming with MATLAB</h1>
<h2 class="subtitle">Defensive Programming</h2>
<div id="learning-objectives" class="objectives panel panel-warning">
<div class="panel-heading">
<h2><span class="glyphicon glyphicon-certificate"></span>Learning Objectives</h2>
</div>
<div class="panel-body">
<ul>
<li>Explain what an assertion is.</li>
<li>Add assertions to programs that correctly check the program’s state.</li>
<li>Correctly add precondition and postcondition assertions to functions.</li>
<li>Explain what test-driven development is, and use it when creating new functions.</li>
<li>Explain why variables should be initialized using actual data values rather than arbitrary constants.</li>
<li>Debug code containing an error systematically.</li>
</ul>
</div>
</div>
<p>Our previous lessons have introduced the basic tools of programming: variables and lists, file I/O, loops, conditionals, and functions. What they <em>haven’t</em> done is show us how to tell whether a program is getting the right answer, and how to tell if it’s <em>still</em> getting the right answer as we make changes to it.</p>
<p>To achieve that, we need to:</p>
<ul>
<li>write programs that check their own operation,</li>
<li>write and run tests for widely-used functions, and</li>
<li>make sure we know what “correct” actually means.</li>
</ul>
<p>The good news is, doing these things will speed up our programming, not slow it down.</p>
<p>The first step toward getting the right answers from our programs is to assume that mistakes <em>will</em> happen and to guard against them. This is called <a href="reference.html#defensive-programming">defensive programming</a>, and the most common way to do it is to add <a href="reference.html#assertion">assertions</a> to our code so that it checks itself as it runs. An assertion is simply a statement that something must be true at a certain point in a program. When MATLAB sees one, it checks the assertion’s condition. If it’s true, MATLAB does nothing, but if it’s false, MATLAB halts the program immediately and prints an error message. For example, this piece of code halts as soon as the loop encounters a value that isn’t positive:</p>
<pre class="sourceCode matlab"><code class="sourceCode matlab">numbers = [<span class="fl">1.5</span>, <span class="fl">2.3</span>, <span class="fl">0.7</span>, -<span class="fl">0.001</span>, <span class="fl">4.4</span>];
total = <span class="fl">0</span>;
for n = numbers
assert(n >= <span class="fl">0</span>);
total += n;
end</code></pre>
<pre class="output"><code>error: assert (n >= 0) failed</code></pre>
<p>Programs like the Firefox browser are full of assertions: 10-20% of the code they contain are there to check that the other 80-90% are working correctly. Broadly speaking, assertions fall into three categories:</p>
<ul>
<li>A <a href="reference.html#precondition">precondition</a> is something that must be true at the start of a function in order for it to work correctly.</li>
<li>A postcondition is something that the function guarantees is true when it finishes.</li>
<li>An invariant is something that is always true at a particular point inside a piece of code.</li>
</ul>
<p>For example, suppose we are representing rectangles using an array of four coordinates <code>(x0, y0, x1, x1)</code>. In order to do some calculations, we need to normalize the rectangle so that it is at the origin and 1.0 units on its longest axis. Here is a function that does that, but checks that its input is correctly formatted and that its result makes sense:</p>
<pre class="sourceCode matlab"><code class="sourceCode matlab">function normalized = normalize_rectangle(rect)
<span class="co">% Normalizes a rectangle so that it is at the origin</span>
<span class="co">% and 1.0 units long on its longest axis:</span>
x0 = rect(<span class="fl">1</span>);
y0 = rect(<span class="fl">2</span>);
x1 = rect(<span class="fl">3</span>);
y1 = rect(<span class="fl">4</span>);
assert(length(rect) == <span class="fl">4</span>, <span class="st">'Rectangle must contain 4 coordinates'</span>);
assert(x0 < x1, <span class="st">'Invalid X coordinates'</span>);
assert(y0 < y1, <span class="st">'Invalid Y coordinates'</span>);
dx = x1 - x0;
dy = y1 - y0;
if dx > dy
scaled = dx/dy;
upper_x = scaled;
upper_y = <span class="fl">1.0</span>;
else
scaled = dx/dy;
upper_x = scaled;
upper_y = <span class="fl">1.0</span>;
end
assert ((<span class="fl">0</span> < upper_x) && (upper_x <= <span class="fl">1.0</span>), <span class="st">'Calculated upper X coordinate invalid'</span>);
assert ((<span class="fl">0</span> < upper_y) && (upper_y <= <span class="fl">1.0</span>), <span class="st">'Calculated upper Y coordinate invalid'</span>);
normalized = [<span class="fl">0</span>, <span class="fl">0</span>, upper_x, upper_y];
end</code></pre>
<p>The first three preconditions catch invalid inputs.</p>
<pre class="sourceCode matlab"><code class="sourceCode matlab">normalize_rectangle([<span class="fl">0</span>, <span class="fl">0</span>, <span class="fl">1</span>])</code></pre>
<pre class="error"><code>error: assert (1 == 2) failed</code></pre>
<pre class="sourceCode matlab"><code class="sourceCode matlab">normalize_rectangle([<span class="fl">1</span>, <span class="fl">0</span>, <span class="fl">0</span>, <span class="fl">2</span>]);</code></pre>
<pre class="error"><code>error: Invalid X coordinates</code></pre>
<p>The post-conditions help us catch bugs by telling us when our calculations cannot have been correct. For example, if we normalize a rectangle that is taller than it is wide, everything seems OK:</p>
<pre class="sourceCode matlab"><code class="sourceCode matlab">normalize_rectangle([<span class="fl">0</span>, <span class="fl">0</span>, <span class="fl">1.0</span>, <span class="fl">5.0</span>]);</code></pre>
<pre class="output"><code>0.00000 0.00000 0.20000 1.00000</code></pre>
<p>but if we normalize one that’s wider than it is tall, the assertion is triggered:</p>
<pre class="sourceCode matlab"><code class="sourceCode matlab">normalize_rectangle([<span class="fl">0.0</span>, <span class="fl">0.0</span>, <span class="fl">5.0</span>, <span class="fl">1.0</span>])</code></pre>
<pre class="output"><code>error: Calculated upper X coordinate invalid</code></pre>
<p>Re-reading our function, we realize that line 19 should should divide <code>dy</code> by <code>dx</code>. If we had left out the assertion at the end of the function, we would have created and returned something that had the right shape as a valid answer, but wasn’t. Detecting and debugging <em>that</em> would almost certainly have taken more time in the long run than writing the assertion.</p>
<p>But assertions aren’t just about catching errors: they also help people understand programs. Each assertion gives the person reading the program a chance to check (consciously or otherwise) that their understanding matches what the code is doing.</p>
<p>Most good programmers follow two rules when adding assertions to their code. The first is, <em>fail early, fail often</em>. The greater the distance between when and where an error occurs and when it’s noticed, the harder the error will be to debug, so good code catches mistakes as early as possible.</p>
<p>The second rule is, <em>turn bugs into assertions or tests</em>. If you made a mistake in a piece of code, the odds are good that you have made other mistakes nearby, or will make the same mistake (or a related one) the next time you change it. Writing assertions to check that you haven’t <a href="reference.html#regression">regressed</a> (i.e., haven’t re-introduced an old problem) can save a lot of time in the long run, and helps to warn people who are reading the code (including your future self) that this bit is tricky.</p>
<div id="finding-conditions" class="challenge panel panel-success">
<div class="panel-heading">
<h2><span class="glyphicon glyphicon-pencil"></span>Finding conditions</h2>
</div>
<div class="panel-body">
<ol style="list-style-type: decimal">
<li>Suppose you are writing a function called <code>average</code> that calculates the average of the numbers in a list. What pre-conditions and post-conditions would you write for it? Compare your answer to your neighbor’s: can you think of a function that will past your tests but not hers or vice versa?</li>
</ol>
</div>
</div>
<p>An assertion checks that something is true at a particular point in the program. The next step is to check the overall behavior of a piece of code, i.e., to make sure that it produces the right output when it’s given a particular input. For example, suppose we need to find where two or more time series overlap. The range of each time series is represented as a pair of numbers, which are the time the interval started and ended. The output is the largest range that they all include:</p>
<p><img src="fig/matlab-overlapping-ranges.png" alt="Overlapping Ranges" /></p>
<p>Most novice programmers would solve this problem like this:</p>
<ol style="list-style-type: decimal">
<li>Write a function <code>range_overlap</code>.</li>
<li>Call it interactively on two or three different inputs.</li>
<li>If it produces the wrong answer, fix the function and re-run that test.</li>
</ol>
<p>This clearly works–after all, thousands of scientists are doing it right now–but there’s a better way:</p>
<ol style="list-style-type: decimal">
<li>Write a short function for each test.</li>
<li>Write a <code>range_overlap</code> function that should pass those tests.</li>
<li>If <code>range_overlap</code> produces any wrong answers, fix it and re-run the test functions.</li>
</ol>
<p>Writing the tests <em>before</em> writing the function they exercise is called <a href="reference.html#test-driven-development">test-driven development</a> (TDD). Its advocates believe it produces better code faster because:</p>
<ol style="list-style-type: decimal">
<li><p>If people write tests after writing the thing to be tested, they are subject to confirmation bias, i.e., they subconsciously write tests to show that their code is correct, rather than to find errors.</p></li>
<li><p>Writing tests helps programmers figure out what the function is actually supposed to do.</p></li>
</ol>
<p>Here are three test functions for <code>range_overlap</code>:</p>
<pre class="sourceCode matlab"><code class="sourceCode matlab"><span class="co">%script test_range_overlap.m</span>
assert(range_overlap([<span class="fl">0</span>, <span class="fl">1.0</span>]) == (<span class="fl">0</span>, <span class="fl">1.0</span>));
assert(range_overlap([<span class="fl">2.0</span>, <span class="fl">3.0</span>], [<span class="fl">2.0</span>, <span class="fl">4.0</span>]) == [<span class="fl">2.0</span>, <span class="fl">3.0</span>]);
assert(range_overlap([<span class="fl">0.0</span>, <span class="fl">1.0</span>], [<span class="fl">0.0</span>, <span class="fl">2.0</span>], [-<span class="fl">1.0</span>, <span class="fl">1.0</span>]) == [<span class="fl">0.0</span>, <span class="fl">1.0</span>]);</code></pre>
<pre class="sourceCode matlab"><code class="sourceCode matlab">test_range_overlap</code></pre>
<pre class="error"><code>Undefined function 'range_overlap' for input arguments of type 'double'.
Error in test_range_overlap (line 6)
assert(range_overlap([0, 1.0]) == [0, 1.0]);</code></pre>
<p>The error is actually reassuring: we haven’t written <code>range_overlap</code> yet, so if the tests passed, it would be a sign that someone else had and that we were accidentally using their function.</p>
<p>And as a bonus of writing these tests, we’ve implicitly defined what our input and output look like: we expect a list of pairs as input, and produce a single pair as output.</p>
<p>Let’s write <code>range_overlap</code>:</p>
<pre class="sourceCode matlab"><code class="sourceCode matlab">function output_range = range_overlap(varargin)
<span class="co">% Return common overlap among a set of [low, high] ranges.</span>
lowest = -inf;
highest = +inf;
for i = <span class="fl">1</span>:nargin
range = varargin{i};
low = range(<span class="fl">1</span>);
high = range(<span class="fl">2</span>);
lowest = max(lowest, low);
highest = min(highest, high);
end
output_range = [lowest, highest];
end</code></pre>
<p>And now when we run the tests:</p>
<pre class="sourceCode matlab"><code class="sourceCode matlab">test_range_overlap</code></pre>
<p>we shouldn’t see an error.</p>
<p>Something important is missing, though. We don’t have any tests for the case where the ranges don’t overlap at all, or for the case where the ranges overlap at a point. We’ll leave this as a final assignment.</p>
<div id="fixing-code-through-testing" class="challenge panel panel-success">
<div class="panel-heading">
<h2><span class="glyphicon glyphicon-pencil"></span>Fixing Code through Testing</h2>
</div>
<div class="panel-body">
<p>Fix <code>range_overlap</code>. Uncomment the two commented lines in <code>test_range_overlap</code>. You’ll see that our objective is to return a special value: <code>NaN</code> (Not a Number), for the following cases:</p>
<ul>
<li>The ranges don’t overlap.</li>
<li>The ranges overlap at their endpoints.</li>
</ul>
<p>As you make change to the code, run <code>test_range_overlap</code> regularly to make sure you aren’t breaking anything. Once you’re done, running <code>test_range_overlap</code> shouldn’t raise any errors.</p>
</div>
</div>
<p>Once testing has uncovered problems, the next step is to fix them. Many novices do this by making more-or-less random changes to their code until it seems to produce the right answer, but that’s very inefficient (and the result is usually only correct for the one case they’re testing). The more experienced a programmer is, the more systematically they debug, and most follow some variation on the rules explained below.</p>
<p>The first step in debugging something is to <em>know what it’s supposed to do</em>. “My program doesn’t work” isn’t good enough: in order to diagnose and fix problems, we need to be able to tell correct output from incorrect. If we can write a test case for the failing case—i.e., if we can assert that with <em>these</em> inputs, the function should produce <em>that</em> result— then we’re ready to start debugging. If we can’t, then we need to figure out how we’re going to know when we’ve fixed things.</p>
<p>But writing test cases for scientific software is frequently harder than writing test cases for commercial applications, because if we knew what the output of the scientific code was supposed to be, we wouldn’t be running the software: we’d be writing up our results and moving on to the next program. In practice, scientists tend to do the following:</p>
<ol style="list-style-type: decimal">
<li><p><em>Test with simplified data.</em> Before doing statistics on a real data set, we should try calculating statistics for a single record, for two identical records, for two records whose values are one step apart, or for some other case where we can calculate the right answer by hand.</p></li>
<li><p><em>Test a simplified case.</em> If our program is supposed to simulate magnetic eddies in rapidly-rotating blobs of supercooled helium, our first test should be a blob of helium that isn’t rotating, and isn’t being subjected to any external electromagnetic fields. Similarly, if we’re looking at the effects of climate change on speciation, our first test should hold temperature, precipitation, and other factors constant.</p></li>
<li><p><em>Compare to an oracle.</em> A <a href="reference.html#test-oracle">test oracle</a> is something—experimental data, an older program whose results are trusted, or even a human expert—against which we can compare the results of our new program. If we have a test oracle, we should store its output for particular cases so that we can compare it with our new results as often as we like without re-running that program.</p></li>
<li><p><em>Check conservation laws.</em> Mass, energy, and other quantitites are conserved in physical systems, so they should be in programs as well. Similarly, if we are analyzing patient data, the number of records should either stay the same or decrease as we move from one analysis to the next (since we might throw away outliers or records with missing values). If “new” patients start appearing out of nowhere as we move through our pipeline, it’s probably a sign that something is wrong.</p></li>
<li><p><em>Visualize.</em> Data analysts frequently use simple visualizations to check both the science they’re doing and the correctness of their code (just as we did in the <a href="01-intro.html">opening lesson</a> of this tutorial). This should only be used for debugging as a last resort, though, since it’s very hard to compare two visualizations automatically.</p></li>
</ol>
<p>We can only debug something when it fails, so the second step is always to find a test case that <em>make it fail every time</em>. The “every time” part is important because few things are more frustrating than debugging an intermittent problem: if we have to call a function a dozen times to get a single failure, the odds are good that we’ll scroll past the failure when it actually occurs.</p>
<p>As part of this, it’s always important to check that our code is “plugged in”, i.e., that we’re actually exercising the problem that we think we are. Every programmer has spent hours chasing a bug, only to realize that they were actually calling their code on the wrong data set or with the wrong configuration parameters, or are using the wrong version of the software entirely. Mistakes like these are particularly likely to happen when we’re tired, frustrated, and up against a deadline, which is one of the reasons late-night (or overnight) coding sessions are almost never worthwhile.</p>
<p>If it takes 20 minutes for the bug to surface, we can only do three experiments an hour. That doesn’t must mean we’ll get less data in more time: we’re also more likely to be distracted by other things as we wait for our program to fail, which means the time we <em>are</em> spending on the problem is less focused. It’s therefore critical to <em>make it fail fast</em>.</p>
<p>As well as making the program fail fast in time, we want to make it fail fast in space, i.e., we want to localize the failure to the smallest possible region of code:</p>
<ol style="list-style-type: decimal">
<li><p>The smaller the gap between cause and effect, the easier the connection is to find. Many programmers therefore use a divide and conquer strategy to find bugs, i.e., if the output of a function is wrong, they check whether things are OK in the middle, then concentrate on either the first or second half, and so on.</p></li>
<li><p>N things can interact in N<sup>2/2</sup> different ways, so every line of code that <em>isn’t</em> run as part of a test means more than one thing we don’t need to worry about.</p></li>
</ol>
<p>Replacing random chunks of code is unlikely to do much good. (After all, if you got it wrong the first time, you’ll probably get it wrong the second and third as well.) Good programmers therefore <em>change one thing at a time</em>, for a reason. They are either trying to gather more information (“is the bug still there if we change the order of the loops?”) or test a fix (“can we make the bug go away by sorting our data before processing it?”).</p>
<p>Every time we make a change, however small, we should re-run our tests immediately, because the more things we change at once, the harder it is to know what’s responsible for what (those N<sup>2</sup> interactions again). And we should re-run <em>all</em> of our tests: more than half of fixes made to code introduce (or re-introduce) bugs, so re-running all of our tests tells us whether we have <a href="reference.html#regression">regressed</a>.</p>
<p>Good scientists keep track of what they’ve done so that they can reproduce their work, and so that they don’t waste time repeating the same experiments or running ones whose results won’t be interesting. Similarly, debugging works best when we <em>keep track of what we’ve done</em> and how well it worked. If we find ourselves asking, “Did left followed by right with an odd number of lines cause the crash? Or was it right followed by left? Or was I using an even number of lines?” then it’s time to step away from the computer, take a deep breath, and start working more systematically.</p>
<p>Records are particularly useful when the time comes to ask for help. People are more likely to listen to us when we can explain clearly what we did, and we’re better able to give them the information they need to be useful.</p>
<p>Version control is often used to reset software to a known state during debugging, and to explore recent changes to code that might be responsible for bugs. In particular, most version control systems have a <code>blame</code> command that will show who last changed particular lines of code…</p>
<p>And speaking of help: if we can’t find a bug in 10 minutes, we should <em>be humble</em> and ask for help. Just explaining the problem aloud is often useful, since hearing what we’re thinking helps us spot inconsistencies and hidden assumptions.</p>
<p>Asking for help also helps alleviate confirmation bias. If we have just spent an hour writing a complicated program, we want it to work, so we’re likely to keep telling ourselves why it should, rather than searching for the reason it doesn’t. People who aren’t emotionally invested in the code can be more objective, which is why they’re often able to spot the simple mistakes we have overlooked.</p>
<p>Part of being humble is learning from our mistakes. Programmers tend to get the same things wrong over and over: either they don’t understand the language and libraries they’re working with, or their model of how things work is wrong. In either case, taking note of why the error occurred and checking for it next time quickly turns into not making the mistake at all.</p>
<p>And that is what makes us most productive in the long run. As the saying goes, <em>A week of hard work can sometimes save you an hour of thought</em>. If we train ourselves to avoid making some kinds of mistakes, to break our code into modular, testable chunks, and to turn every assumption (or mistake) into an assertion, it will actually take us <em>less</em> time to produce working programs, not more.</p>
</div>
</div>
</article>
<div class="footer">
<a class="label swc-blue-bg" href="http://software-carpentry.org">Software Carpentry</a>
<a class="label swc-blue-bg" href="https://github.com/swcarpentry/matlab-novice-inflammation">Source</a>
<a class="label swc-blue-bg" href="mailto:[email protected]">Contact</a>
<a class="label swc-blue-bg" href="LICENSE.html">License</a>
</div>
</div>
<!-- Javascript placed at the end of the document so the pages load faster -->
<script src="http://software-carpentry.org/v5/js/jquery-1.9.1.min.js"></script>
<script src="css/bootstrap/bootstrap-js/bootstrap.js"></script>
</body>
</html>