{{ message }}
-
Notifications
You must be signed in to change notification settings - Fork 62
Expand file tree
/
Copy pathtesting.html
More file actions
474 lines (446 loc) · 31.7 KB
/
Copy pathtesting.html
File metadata and controls
474 lines (446 loc) · 31.7 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Developer’s tips for testing — Matplotlib 2.1.0 documentation</title>
<link rel="stylesheet" href="../_static/mpl.css"
type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css"
type="text/css" />
<link rel="stylesheet" href="../_static/gallery.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '../',
VERSION: '2.1.0',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true,
SOURCELINK_SUFFIX: '.txt'
};
</script>
<script type="text/javascript" src="../_static/jquery.js"></script>
<script type="text/javascript" src="../_static/underscore.js"></script>
<script type="text/javascript" src="../_static/doctools.js"></script>
<link rel="search" type="application/opensearchdescription+xml"
title="Search within Matplotlib 2.1.0 documentation"
href="../_static/opensearch.xml"/>
<link rel="shortcut icon" href="../_static/favicon.ico"/>
<link rel="index" title="Index" href="../genindex.html"
/>
<link rel="search" title="Search" href="../search.html"
/>
<link rel="top" title="Matplotlib 2.1.0 documentation" href="../index.html" />
<link rel="up" title="The Matplotlib Developers’ Guide" href="index.html" />
<link rel="next" title="Developer’s tips for documenting matplotlib" href="documenting_mpl.html" />
<link rel="prev" title="Contributing" href="contributing.html" />
<link rel="canonical" href="https://matplotlib.org/stable/devel/testing.html" />
<script data-domain="matplotlib.org" defer="defer" src="https://views.scientific-python.org/js/script.js"></script>
</head>
<body>
<div id="unreleased-message"> You are reading an old version of the documentation (v2.1.0). For the latest version see <a href="https://matplotlib.org/stable/devel/testing.html">https://matplotlib.org/stable/devel/testing.html</a></div>
<div style="background-color: white; text-align: left; padding: 10px 10px 15px 15px; position: relative;">
<a href="../index.html"><img src="../_static/logo2.svg" width="540px" border="0" alt="matplotlib"/></a>
<!-- The "Fork me on github" ribbon -->
<div id="forkongithub"><a href="https://github.com/matplotlib/matplotlib">Fork me on GitHub</a></div>
</div>
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
<a href="../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="right" >
<a href="documenting_mpl.html" title="Developer’s tips for documenting matplotlib"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="contributing.html" title="Contributing"
accesskey="P">previous</a> |</li>
<li><a href="../index.html">home</a>| </li>
<li><a href="../gallery/index.html">examples</a>| </li>
<li><a href="../tutorials/index.html">tutorials</a>| </li>
<li><a href="../api/pyplot_summary.html">pyplot</a>| </li>
<li><a href="../contents.html">docs</a> »</li>
<li><a href="index.html" accesskey="U">The Matplotlib Developers’ Guide</a> »</li>
</ul>
</div>
<div class="sphinxsidebar">
<div class="sphinxsidebarwrapper"><a href="https://www.numfocus.org">
<image src="https://img.shields.io/badge/powered%20by-NumFOCUS-orange.svg?style=flat&colorA=E1523D&colorB=007D8A">
</a>
<br/>
<a href="http://depsy.org/package/python/matplotlib">
<img src="../_static/depsy_badge.svg">
</a>
<br/>
Travis-CI: <a href="https://travis-ci.org/matplotlib/matplotlib">
<img src="https://travis-ci.org/matplotlib/matplotlib.svg?branch=master"/>
</a>
<br/>
<h3><a href="../contents.html">Table Of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">Developer’s tips for testing</a><ul>
<li><a class="reference internal" href="#requirements">Requirements</a></li>
<li><a class="reference internal" href="#building-matplotlib-for-image-comparison-tests">Building matplotlib for image comparison tests</a></li>
<li><a class="reference internal" href="#running-the-tests">Running the tests</a></li>
<li><a class="reference internal" href="#writing-a-simple-test">Writing a simple test</a></li>
<li><a class="reference internal" href="#random-data-in-tests">Random data in tests</a></li>
<li><a class="reference internal" href="#writing-an-image-comparison-test">Writing an image comparison test</a></li>
<li><a class="reference internal" href="#known-failing-tests">Known failing tests</a></li>
<li><a class="reference internal" href="#creating-a-new-module-in-matplotlib-tests">Creating a new module in matplotlib.tests</a></li>
<li><a class="reference internal" href="#using-travis-ci">Using Travis CI</a></li>
<li><a class="reference internal" href="#using-tox">Using tox</a></li>
</ul>
</li>
</ul>
<div class="relations">
<h3>Related Topics</h3>
<ul>
<li><a href="../contents.html">Documentation overview</a><ul>
<li><a href="index.html">The Matplotlib Developers’ Guide</a><ul>
<li>Previous: <a href="contributing.html" title="previous chapter">Contributing</a></li>
<li>Next: <a href="documenting_mpl.html" title="next chapter">Developer’s tips for documenting matplotlib</a></li>
</ul></li>
</ul></li>
</ul>
</div>
<div role="note" aria-label="source link">
<h3>This Page</h3>
<ul class="this-page-menu">
<li><a href="../_sources/devel/testing.rst.txt"
rel="nofollow">Show Source</a></li>
</ul>
</div>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
<form class="search" action="../search.html" method="get">
<div><input type="text" name="q" /></div>
<div><input type="submit" value="Go" /></div>
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body">
<div class="section" id="developer-s-tips-for-testing">
<span id="testing"></span><h1>Developer’s tips for testing<a class="headerlink" href="#developer-s-tips-for-testing" title="Permalink to this headline">¶</a></h1>
<p>Matplotlib’s testing infrastructure depends on <a class="reference external" href="http://doc.pytest.org/en/latest/">pytest</a>. The tests are in
<code class="file docutils literal"><span class="pre">lib/matplotlib/tests</span></code>, and customizations to the pytest testing
infrastructure are in <code class="xref py py-mod docutils literal"><span class="pre">matplotlib.testing</span></code>.</p>
<div class="section" id="requirements">
<h2>Requirements<a class="headerlink" href="#requirements" title="Permalink to this headline">¶</a></h2>
<p>The following software is required to run the tests:</p>
<blockquote>
<div><ul class="simple">
<li><a class="reference external" href="http://doc.pytest.org/en/latest/">pytest</a>, version 3.0.0 or later</li>
<li><a class="reference external" href="https://docs.python.org/3/library/unittest.mock.html>">mock</a>, when running Python versions < 3.3</li>
<li><a class="reference external" href="https://www.ghostscript.com/">Ghostscript</a> (to render PDF files)</li>
<li><a class="reference external" href="https://inkscape.org">Inkscape</a> (to render SVG files)</li>
</ul>
</div></blockquote>
<p>Optionally you can install:</p>
<blockquote>
<div><ul class="simple">
<li><a class="reference external" href="https://pytest-cov.readthedocs.io/en/latest/">pytest-cov</a> to collect coverage information</li>
<li><a class="reference external" href="https://pypi.python.org/pypi/pytest-pep8">pytest-pep8</a> to test coding standards</li>
<li><a class="reference external" href="https://pypi.python.org/pypi/pytest-timeout">pytest-timeout</a> to limit runtime in case of stuck tests</li>
<li><a class="reference external" href="https://pypi.python.org/pypi/pytest-xdist">pytest-xdist</a> to run tests in parallel</li>
</ul>
</div></blockquote>
</div>
<div class="section" id="building-matplotlib-for-image-comparison-tests">
<h2>Building matplotlib for image comparison tests<a class="headerlink" href="#building-matplotlib-for-image-comparison-tests" title="Permalink to this headline">¶</a></h2>
<p>matplotlib’s test suite makes heavy use of image comparison tests,
meaning the result of a plot is compared against a known good result.
Unfortunately, different versions of FreeType produce differently
formed characters, causing these image comparisons to fail. To make
them reproducible, matplotlib can be built with a special local copy
of FreeType. This is recommended for all matplotlib developers.</p>
<p>Add the following content to a <code class="docutils literal"><span class="pre">setup.cfg</span></code> file at the root of the
matplotlib source directory:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="p">[</span><span class="n">test</span><span class="p">]</span>
<span class="n">local_freetype</span> <span class="o">=</span> <span class="kc">True</span>
<span class="n">tests</span> <span class="o">=</span> <span class="kc">True</span>
</pre></div>
</div>
<p>or by setting the <code class="docutils literal"><span class="pre">MPLLOCALFREETYPE</span></code> environmental variable to any true
value.</p>
</div>
<div class="section" id="running-the-tests">
<h2>Running the tests<a class="headerlink" href="#running-the-tests" title="Permalink to this headline">¶</a></h2>
<p>Running the tests is simple. Make sure you have pytest installed and run:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">py</span><span class="o">.</span><span class="n">test</span>
</pre></div>
</div>
<p>or:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">python</span> <span class="n">tests</span><span class="o">.</span><span class="n">py</span>
</pre></div>
</div>
<p>in the root directory of the distribution. The script takes a set of
commands, such as:</p>
<table border="1" class="docutils">
<colgroup>
<col width="36%" />
<col width="64%" />
</colgroup>
<tbody valign="top">
<tr class="row-odd"><td><code class="docutils literal"><span class="pre">--pep8</span></code></td>
<td>Perform pep8 checks (requires <a class="reference external" href="https://pypi.python.org/pypi/pytest-pep8">pytest-pep8</a>)</td>
</tr>
<tr class="row-even"><td><code class="docutils literal"><span class="pre">-m</span> <span class="pre">"not</span> <span class="pre">network"</span></code></td>
<td>Disable tests that require network access</td>
</tr>
</tbody>
</table>
<p>Additional arguments are passed on to pytest. See the pytest documentation for
<a class="reference external" href="http://doc.pytest.org/en/latest/usage.html">supported arguments</a>. Some of the more important ones are given here:</p>
<table border="1" class="docutils">
<colgroup>
<col width="43%" />
<col width="57%" />
</colgroup>
<tbody valign="top">
<tr class="row-odd"><td><code class="docutils literal"><span class="pre">--verbose</span></code></td>
<td>Be more verbose</td>
</tr>
<tr class="row-even"><td><code class="docutils literal"><span class="pre">--n</span> <span class="pre">NUM</span></code></td>
<td>Run tests in parallel over NUM
processes (requires <a class="reference external" href="https://pypi.python.org/pypi/pytest-xdist">pytest-xdist</a>)</td>
</tr>
<tr class="row-odd"><td><code class="docutils literal"><span class="pre">--timeout=SECONDS</span></code></td>
<td>Set timeout for results from each test
process (requires <a class="reference external" href="https://pypi.python.org/pypi/pytest-timeout">pytest-timeout</a>)</td>
</tr>
<tr class="row-even"><td><code class="docutils literal"><span class="pre">--capture=no</span></code> or <code class="docutils literal"><span class="pre">-s</span></code></td>
<td>Do not capture stdout</td>
</tr>
</tbody>
</table>
<p>To run a single test from the command line, you can provide a file path,
optionally followed by the function separated by two colons, e.g., (tests do
not need to be installed, but Matplotlib should be):</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="n">lib</span><span class="o">/</span><span class="n">matplotlib</span><span class="o">/</span><span class="n">tests</span><span class="o">/</span><span class="n">test_simplification</span><span class="o">.</span><span class="n">py</span><span class="p">::</span><span class="n">test_clipping</span>
</pre></div>
</div>
<p>or, if tests are installed, a dot-separated path to the module, optionally
followed by the function separated by two colons, such as:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">--</span><span class="n">pyargs</span> <span class="n">matplotlib</span><span class="o">.</span><span class="n">tests</span><span class="o">.</span><span class="n">test_simplification</span><span class="p">::</span><span class="n">test_clipping</span>
</pre></div>
</div>
<p>If you want to run the full test suite, but want to save wall time try
running the tests in parallel:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">--</span><span class="n">verbose</span> <span class="o">-</span><span class="n">n</span> <span class="mi">5</span>
</pre></div>
</div>
<p>Depending on your version of Python and pytest-xdist, you may need to set
<code class="docutils literal"><span class="pre">PYTHONHASHSEED</span></code> to a fixed value when running in parallel:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">PYTHONHASHSEED</span><span class="o">=</span><span class="mi">0</span> <span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">--</span><span class="n">verbose</span> <span class="o">-</span><span class="n">n</span> <span class="mi">5</span>
</pre></div>
</div>
<p>An alternative implementation that does not look at command line arguments
and works from within Python is to run the tests from the Matplotlib library
function <code class="xref py py-func docutils literal"><span class="pre">matplotlib.test()</span></code>:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">matplotlib</span>
<span class="n">matplotlib</span><span class="o">.</span><span class="n">test</span><span class="p">()</span>
</pre></div>
</div>
</div>
<div class="section" id="writing-a-simple-test">
<h2>Writing a simple test<a class="headerlink" href="#writing-a-simple-test" title="Permalink to this headline">¶</a></h2>
<p>Many elements of Matplotlib can be tested using standard tests. For
example, here is a test from <code class="xref py py-mod docutils literal"><span class="pre">matplotlib.tests.test_basic</span></code>:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">test_simple</span><span class="p">():</span>
<span class="sd">"""</span>
<span class="sd"> very simple example test</span>
<span class="sd"> """</span>
<span class="k">assert</span> <span class="mi">1</span> <span class="o">+</span> <span class="mi">1</span> <span class="o">==</span> <span class="mi">2</span>
</pre></div>
</div>
<p>Pytest determines which functions are tests by searching for files whose names
begin with <code class="docutils literal"><span class="pre">"test_"</span></code> and then within those files for functions beginning with
<code class="docutils literal"><span class="pre">"test"</span></code> or classes beginning with <code class="docutils literal"><span class="pre">"Test"</span></code>.</p>
<p>Some tests have internal side effects that need to be cleaned up after their
execution (such as created figures or modified rc params). The pytest fixture
<code class="xref py py-func docutils literal"><span class="pre">mpl_test_settings()</span></code> will automatically clean
these up; there is no need to do anything further.</p>
</div>
<div class="section" id="random-data-in-tests">
<h2>Random data in tests<a class="headerlink" href="#random-data-in-tests" title="Permalink to this headline">¶</a></h2>
<p>Random data can is a very convenient way to generate data for examples,
however the randomness is problematic for testing (as the tests
must be deterministic!). To work around this set the seed in each test.
For numpy use:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">seed</span><span class="p">(</span><span class="mi">19680801</span><span class="p">)</span>
</pre></div>
</div>
<p>and Python’s random number generator:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">random</span>
<span class="n">random</span><span class="o">.</span><span class="n">seed</span><span class="p">(</span><span class="mi">19680801</span><span class="p">)</span>
</pre></div>
</div>
<p>The seed is John Hunter’s birthday.</p>
</div>
<div class="section" id="writing-an-image-comparison-test">
<h2>Writing an image comparison test<a class="headerlink" href="#writing-an-image-comparison-test" title="Permalink to this headline">¶</a></h2>
<p>Writing an image based test is only slightly more difficult than a
simple test. The main consideration is that you must specify the
“baseline”, or expected, images in the
<code class="xref py py-func docutils literal"><span class="pre">image_comparison()</span></code> decorator. For
example, this test generates a single image and automatically tests
it:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="kn">import</span> <span class="nn">matplotlib</span>
<span class="kn">from</span> <span class="nn">matplotlib.testing.decorators</span> <span class="k">import</span> <span class="n">image_comparison</span>
<span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span>
<span class="nd">@image_comparison</span><span class="p">(</span><span class="n">baseline_images</span><span class="o">=</span><span class="p">[</span><span class="s1">'spines_axes_positions'</span><span class="p">],</span>
<span class="n">extensions</span><span class="o">=</span><span class="p">[</span><span class="s1">'png'</span><span class="p">])</span>
<span class="k">def</span> <span class="nf">test_spines_axes_positions</span><span class="p">():</span>
<span class="c1"># SF bug 2852168</span>
<span class="n">fig</span> <span class="o">=</span> <span class="n">plt</span><span class="o">.</span><span class="n">figure</span><span class="p">()</span>
<span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">linspace</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span><span class="mi">2</span><span class="o">*</span><span class="n">np</span><span class="o">.</span><span class="n">pi</span><span class="p">,</span><span class="mi">100</span><span class="p">)</span>
<span class="n">y</span> <span class="o">=</span> <span class="mi">2</span><span class="o">*</span><span class="n">np</span><span class="o">.</span><span class="n">sin</span><span class="p">(</span><span class="n">x</span><span class="p">)</span>
<span class="n">ax</span> <span class="o">=</span> <span class="n">fig</span><span class="o">.</span><span class="n">add_subplot</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span><span class="mi">1</span><span class="p">,</span><span class="mi">1</span><span class="p">)</span>
<span class="n">ax</span><span class="o">.</span><span class="n">set_title</span><span class="p">(</span><span class="s1">'centered spines'</span><span class="p">)</span>
<span class="n">ax</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">x</span><span class="p">,</span><span class="n">y</span><span class="p">)</span>
<span class="n">ax</span><span class="o">.</span><span class="n">spines</span><span class="p">[</span><span class="s1">'right'</span><span class="p">]</span><span class="o">.</span><span class="n">set_position</span><span class="p">((</span><span class="s1">'axes'</span><span class="p">,</span><span class="mf">0.1</span><span class="p">))</span>
<span class="n">ax</span><span class="o">.</span><span class="n">yaxis</span><span class="o">.</span><span class="n">set_ticks_position</span><span class="p">(</span><span class="s1">'right'</span><span class="p">)</span>
<span class="n">ax</span><span class="o">.</span><span class="n">spines</span><span class="p">[</span><span class="s1">'top'</span><span class="p">]</span><span class="o">.</span><span class="n">set_position</span><span class="p">((</span><span class="s1">'axes'</span><span class="p">,</span><span class="mf">0.25</span><span class="p">))</span>
<span class="n">ax</span><span class="o">.</span><span class="n">xaxis</span><span class="o">.</span><span class="n">set_ticks_position</span><span class="p">(</span><span class="s1">'top'</span><span class="p">)</span>
<span class="n">ax</span><span class="o">.</span><span class="n">spines</span><span class="p">[</span><span class="s1">'left'</span><span class="p">]</span><span class="o">.</span><span class="n">set_color</span><span class="p">(</span><span class="s1">'none'</span><span class="p">)</span>
<span class="n">ax</span><span class="o">.</span><span class="n">spines</span><span class="p">[</span><span class="s1">'bottom'</span><span class="p">]</span><span class="o">.</span><span class="n">set_color</span><span class="p">(</span><span class="s1">'none'</span><span class="p">)</span>
</pre></div>
</div>
<p>The first time this test is run, there will be no baseline image to
compare against, so the test will fail. Copy the output images (in
this case <code class="xref py py-obj docutils literal"><span class="pre">result_images/test_category/spines_axes_positions.png</span></code>) to
the correct subdirectory of <code class="xref py py-obj docutils literal"><span class="pre">baseline_images</span></code> tree in the source
directory (in this case
<code class="xref py py-obj docutils literal"><span class="pre">lib/matplotlib/tests/baseline_images/test_category</span></code>). Put this new
file under source code revision control (with <code class="xref py py-obj docutils literal"><span class="pre">git</span> <span class="pre">add</span></code>). When
rerunning the tests, they should now pass.</p>
<p>The <code class="xref py py-func docutils literal"><span class="pre">image_comparison()</span></code> decorator
defaults to generating <code class="docutils literal"><span class="pre">png</span></code>, <code class="docutils literal"><span class="pre">pdf</span></code> and <code class="docutils literal"><span class="pre">svg</span></code> output, but in
interest of keeping the size of the library from ballooning we should only
include the <code class="docutils literal"><span class="pre">svg</span></code> or <code class="docutils literal"><span class="pre">pdf</span></code> outputs if the test is explicitly exercising
a feature dependent on that backend.</p>
<p>There are two optional keyword arguments to the <code class="xref py py-obj docutils literal"><span class="pre">image_comparison</span></code>
decorator:</p>
<blockquote>
<div><ul class="simple">
<li><code class="xref py py-obj docutils literal"><span class="pre">extensions</span></code>: If you only wish to test additional image formats
(rather than just <code class="xref py py-obj docutils literal"><span class="pre">png</span></code>), pass any additional file types in the
list of the extensions to test. When copying the new
baseline files be sure to only copy the output files, not their
conversions to <code class="docutils literal"><span class="pre">png</span></code>. For example only copy the files
ending in <code class="docutils literal"><span class="pre">pdf</span></code>, not in <code class="docutils literal"><span class="pre">_pdf.png</span></code>.</li>
<li><code class="xref py py-obj docutils literal"><span class="pre">tol</span></code>: This is the image matching tolerance, the default <code class="xref py py-obj docutils literal"><span class="pre">1e-3</span></code>.
If some variation is expected in the image between runs, this
value may be adjusted.</li>
</ul>
</div></blockquote>
</div>
<div class="section" id="known-failing-tests">
<h2>Known failing tests<a class="headerlink" href="#known-failing-tests" title="Permalink to this headline">¶</a></h2>
<p>If you’re writing a test, you may mark it as a known failing test with the
<code class="xref py py-func docutils literal"><span class="pre">pytest.mark.xfail()</span></code> decorator. This allows the test to be added to the
test suite and run on the buildbots without causing undue alarm. For example,
although the following test will fail, it is an expected failure:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">pytest</span>
<span class="nd">@pytest</span><span class="o">.</span><span class="n">mark</span><span class="o">.</span><span class="n">xfail</span>
<span class="k">def</span> <span class="nf">test_simple_fail</span><span class="p">():</span>
<span class="sd">'''very simple example test that should fail'''</span>
<span class="k">assert</span> <span class="mi">1</span> <span class="o">+</span> <span class="mi">1</span> <span class="o">==</span> <span class="mi">3</span>
</pre></div>
</div>
<p>Note that the first argument to the <code class="xref py py-func docutils literal"><span class="pre">xfail()</span></code> decorator is a
fail condition, which can be a value such as True, False, or may be a
dynamically evaluated expression. If a condition is supplied, then a reason
must also be supplied with the <code class="docutils literal"><span class="pre">reason='message'</span></code> keyword argument.</p>
</div>
<div class="section" id="creating-a-new-module-in-matplotlib-tests">
<h2>Creating a new module in matplotlib.tests<a class="headerlink" href="#creating-a-new-module-in-matplotlib-tests" title="Permalink to this headline">¶</a></h2>
<p>We try to keep the tests categorized by the primary module they are
testing. For example, the tests related to the <code class="docutils literal"><span class="pre">mathtext.py</span></code> module
are in <code class="docutils literal"><span class="pre">test_mathtext.py</span></code>.</p>
</div>
<div class="section" id="using-travis-ci">
<h2>Using Travis CI<a class="headerlink" href="#using-travis-ci" title="Permalink to this headline">¶</a></h2>
<p><a class="reference external" href="https://travis-ci.org/">Travis CI</a> is a hosted CI system “in the
cloud”.</p>
<p>Travis is configured to receive notifications of new commits to GitHub
repos (via GitHub “service hooks”) and to run builds or tests when it
sees these new commits. It looks for a YAML file called
<code class="docutils literal"><span class="pre">.travis.yml</span></code> in the root of the repository to see how to test the
project.</p>
<p>Travis CI is already enabled for the <a class="reference external" href="https://github.com/matplotlib/matplotlib/">main matplotlib GitHub
repository</a> – for
example, see <a class="reference external" href="https://travis-ci.org/matplotlib/matplotlib">its Travis page</a>.</p>
<p>If you want to enable Travis CI for your personal matplotlib GitHub
repo, simply enable the repo to use Travis CI in either the Travis CI
UI or the GitHub UI (Admin | Service Hooks). For details, see <a class="reference external" href="https://docs.travis-ci.com/user/getting-started/">the
Travis CI Getting Started page</a>. This
generally isn’t necessary, since any pull request submitted against
the main matplotlib repository will be tested.</p>
<p>Once this is configured, you can see the Travis CI results at
<a class="reference external" href="https://travis-ci.org/your_GitHub_user_name/matplotlib">https://travis-ci.org/your_GitHub_user_name/matplotlib</a> – here’s <a class="reference external" href="https://travis-ci.org/msabramo/matplotlib">an
example</a>.</p>
</div>
<div class="section" id="using-tox">
<h2>Using tox<a class="headerlink" href="#using-tox" title="Permalink to this headline">¶</a></h2>
<p><a class="reference external" href="https://tox.readthedocs.io/en/latest/">Tox</a> is a tool for running
tests against
multiple Python environments, including multiple versions of Python
(e.g., 2.7, 3.4, 3.5) and even different Python implementations
altogether (e.g., CPython, PyPy, Jython, etc.)</p>
<p>Testing all versions of Python (2.6, 2.7, 3.*) requires
having multiple versions of Python installed on your system and on the
PATH. Depending on your operating system, you may want to use your
package manager (such as apt-get, yum or MacPorts) to do this.</p>
<p>tox makes it easy to determine if your working copy introduced any
regressions before submitting a pull request. Here’s how to use it:</p>
<div class="highlight-bash"><div class="highlight"><pre><span></span>$ pip install tox
$ tox
</pre></div>
</div>
<p>You can also run tox on a subset of environments:</p>
<div class="highlight-bash"><div class="highlight"><pre><span></span>$ tox -e py26,py27
</pre></div>
</div>
<p>Tox processes everything serially so it can take a long time to test
several environments. To speed it up, you might try using a new,
parallelized version of tox called <code class="docutils literal"><span class="pre">detox</span></code>. Give this a try:</p>
<div class="highlight-bash"><div class="highlight"><pre><span></span>$ pip install -U -i http://pypi.testrun.org detox
$ detox
</pre></div>
</div>
<p>Tox is configured using a file called <code class="docutils literal"><span class="pre">tox.ini</span></code>. You may need to
edit this file if you want to add new environments to test (e.g.,
<code class="docutils literal"><span class="pre">py33</span></code>) or if you want to tweak the dependencies or the way the
tests are run. For more info on the <code class="docutils literal"><span class="pre">tox.ini</span></code> file, see the <a class="reference external" href="https://tox.readthedocs.io/en/latest/config.html">Tox
Configuration Specification</a>.</p>
</div>
</div>
</div>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="footer">
© Copyright 2002 - 2012 John Hunter, Darren Dale, Eric Firing, Michael Droettboom and the Matplotlib development team; 2012 - 2017 The Matplotlib development team.
Last updated on Oct 13, 2017.
Created using <a
href="http://sphinx-doc.org/">Sphinx</a> 1.6.3.
</div>
</body>
<footer>
</footer>
</html>
You can’t perform that action at this time.