{{ message }}
-
Notifications
You must be signed in to change notification settings - Fork 62
Expand file tree
/
Copy pathMEP19.html
More file actions
348 lines (322 loc) · 16.3 KB
/
Copy pathMEP19.html
File metadata and controls
348 lines (322 loc) · 16.3 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>MEP19: Continuous Integration — Matplotlib 2.1.1 documentation</title>
<link rel="stylesheet" href="../../_static/mpl.css"
type="text/css" />
<link rel="stylesheet" href="../../_static/pygments.css"
type="text/css" />
<link rel="stylesheet" href="../../_static/gallery.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '../../',
VERSION: '2.1.1',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true,
SOURCELINK_SUFFIX: '.txt'
};
</script>
<script type="text/javascript" src="../../_static/jquery.js"></script>
<script type="text/javascript" src="../../_static/underscore.js"></script>
<script type="text/javascript" src="../../_static/doctools.js"></script>
<link rel="search" type="application/opensearchdescription+xml"
title="Search within Matplotlib 2.1.1 documentation"
href="../../_static/opensearch.xml"/>
<link rel="shortcut icon" href="../../_static/favicon.ico"/>
<link rel="index" title="Index" href="../../genindex.html"
/>
<link rel="search" title="Search" href="../../search.html"
/>
<link rel="top" title="Matplotlib 2.1.1 documentation" href="../../index.html" />
<link rel="up" title="Matplotlib Enhancement Proposals" href="index.html" />
<link rel="next" title="MEP21: color and cm refactor" href="MEP21.html" />
<link rel="prev" title="MEP15 - Fix axis autoscaling when limits are specified for one axis only" href="MEP15.html" />
<link rel="canonical" href="https://matplotlib.org/stable/devel/MEP/MEP19.html" />
<script data-domain="matplotlib.org" defer="defer" src="https://views.scientific-python.org/js/script.js"></script>
</head>
<body>
<div id="unreleased-message"> You are reading an old version of the documentation (v2.1.1). For the latest version see <a href="https://matplotlib.org/stable/devel/MEP/MEP19.html">https://matplotlib.org/stable/devel/MEP/MEP19.html</a></div>
<div style="background-color: white; text-align: left; padding: 10px 10px 15px 15px; position: relative;">
<a href="../../index.html"><img src="../../_static/logo2.svg" width="540px" border="0" alt="matplotlib"/></a>
<!-- The "Fork me on github" ribbon -->
<div id="forkongithub"><a href="https://github.com/matplotlib/matplotlib">Fork me on GitHub</a></div>
</div>
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../../genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
<a href="../../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="right" >
<a href="MEP21.html" title="MEP21: color and cm refactor"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="MEP15.html" title="MEP15 - Fix axis autoscaling when limits are specified for one axis only"
accesskey="P">previous</a> |</li>
<li><a href="../../index.html">home</a>| </li>
<li><a href="../../gallery/index.html">examples</a>| </li>
<li><a href="../../tutorials/index.html">tutorials</a>| </li>
<li><a href="../../api/pyplot_summary.html">pyplot</a>| </li>
<li><a href="../../contents.html">docs</a> »</li>
<li><a href="../index.html" >The Matplotlib Developers’ Guide</a> »</li>
<li><a href="index.html" accesskey="U">Matplotlib Enhancement Proposals</a> »</li>
</ul>
</div>
<div class="sphinxsidebar">
<div class="sphinxsidebarwrapper"><a href="https://www.numfocus.org">
<image src="https://img.shields.io/badge/powered%20by-NumFOCUS-orange.svg?style=flat&colorA=E1523D&colorB=007D8A">
</a>
<br/>
<a href="http://depsy.org/package/python/matplotlib">
<img src="../../_static/depsy_badge.svg">
</a>
<br/>
Travis-CI: <a href="https://travis-ci.org/matplotlib/matplotlib">
<img src="https://travis-ci.org/matplotlib/matplotlib.svg?branch=master"/>
</a>
<br/>
<h3><a href="../../contents.html">Table Of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">MEP19: Continuous Integration</a><ul>
<li><a class="reference internal" href="#status">Status</a></li>
<li><a class="reference internal" href="#branches-and-pull-requests">Branches and Pull requests</a></li>
<li><a class="reference internal" href="#abstract">Abstract</a></li>
<li><a class="reference internal" href="#detailed-description">Detailed description</a><ul>
<li><a class="reference internal" href="#current-state-of-the-art">Current state-of-the-art</a></li>
<li><a class="reference internal" href="#peculiarities-of-matplotlib">Peculiarities of matplotlib</a></li>
<li><a class="reference internal" href="#requirements">Requirements</a></li>
</ul>
</li>
<li><a class="reference internal" href="#implementation">Implementation</a></li>
<li><a class="reference internal" href="#backward-compatibility">Backward compatibility</a></li>
<li><a class="reference internal" href="#alternatives">Alternatives</a></li>
<li><a class="reference internal" href="#hangout-notes">Hangout Notes</a><ul>
<li><a class="reference internal" href="#ci-infrastructure">CI Infrastructure</a></li>
<li><a class="reference internal" href="#the-test-framework-itself">The test framework itself</a></li>
</ul>
</li>
</ul>
</li>
</ul>
<div class="relations">
<h3>Related Topics</h3>
<ul>
<li><a href="../../contents.html">Documentation overview</a><ul>
<li><a href="../index.html">The Matplotlib Developers’ Guide</a><ul>
<li><a href="index.html">Matplotlib Enhancement Proposals</a><ul>
<li>Previous: <a href="MEP15.html" title="previous chapter">MEP15 - Fix axis autoscaling when limits are specified for one axis only</a></li>
<li>Next: <a href="MEP21.html" title="next chapter">MEP21: color and cm refactor</a></li>
</ul></li>
</ul></li>
</ul></li>
</ul>
</div>
<div role="note" aria-label="source link">
<h3>This Page</h3>
<ul class="this-page-menu">
<li><a href="../../_sources/devel/MEP/MEP19.rst.txt"
rel="nofollow">Show Source</a></li>
</ul>
</div>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
<form class="search" action="../../search.html" method="get">
<div><input type="text" name="q" /></div>
<div><input type="submit" value="Go" /></div>
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body">
<div class="section" id="mep19-continuous-integration">
<h1>MEP19: Continuous Integration<a class="headerlink" href="#mep19-continuous-integration" title="Permalink to this headline">¶</a></h1>
<div class="section" id="status">
<h2>Status<a class="headerlink" href="#status" title="Permalink to this headline">¶</a></h2>
<p><strong>Discussion</strong></p>
</div>
<div class="section" id="branches-and-pull-requests">
<h2>Branches and Pull requests<a class="headerlink" href="#branches-and-pull-requests" title="Permalink to this headline">¶</a></h2>
</div>
<div class="section" id="abstract">
<h2>Abstract<a class="headerlink" href="#abstract" title="Permalink to this headline">¶</a></h2>
<p>matplotlib could benefit from better and more reliable continuous
integration, both for testing and building installers and
documentation.</p>
</div>
<div class="section" id="detailed-description">
<h2>Detailed description<a class="headerlink" href="#detailed-description" title="Permalink to this headline">¶</a></h2>
<div class="section" id="current-state-of-the-art">
<h3>Current state-of-the-art<a class="headerlink" href="#current-state-of-the-art" title="Permalink to this headline">¶</a></h3>
<p><strong>Testing</strong></p>
<p>matplotlib currently uses Travis-CI for automated tests. While
Travis-CI should be praised for how much it does as a free service, it
has a number of shortcomings:</p>
<ul class="simple">
<li>It often fails due to network timeouts when installing dependencies.</li>
<li>It often fails for inexplicable reasons.</li>
<li>build or test products can only be saved from build off of branches
on the main repo, not pull requsts, so it is often difficult to
“post mortem” analyse what went wrong. This is particularly
frustrating when the failure can not be subsequently reproduced
locally.</li>
<li>It is not extremely fast. matplotlib’s cpu and memory requirements
for testing are much higher than the average Python project.</li>
<li>It only tests on Ubuntu Linux, and we have only minimal control over
the specifics of the platform. It can be upgraded at any time
outside of our control, causing unexpected delays at times that may
not be convenient in our release schedule.</li>
</ul>
<p>On the plus side, Travis-CI’s integration with github – automatically
testing all pending pull requests – is exceptional.</p>
<p><strong>Builds</strong></p>
<p>There is no centralized effort for automated binary builds for
matplotlib. However, the following disparate things are being done
[If the authors mentioned here could fill in detail, that would be
great!]:</p>
<ul class="simple">
<li>@sandrotosi: builds Debian packages</li>
<li>@takluyver: Has automated Ubuntu builds on Launchpad</li>
<li>@cgohlke: Makes Windows builds (don’t know how automated that is)</li>
<li>@r-owen: Makes OS-X builds (don’t know how automated that is)</li>
</ul>
<p><strong>Documentation</strong></p>
<p>Documentation of master is now built by travis and uploaded to <a class="reference external" href="http://matplotlib.org/devdocs/index.html">http://matplotlib.org/devdocs/index.html</a></p>
<p>@NelleV, I believe, generates the docs automatically and posts them on
the web to chart MEP10 progress.</p>
</div>
<div class="section" id="peculiarities-of-matplotlib">
<h3>Peculiarities of matplotlib<a class="headerlink" href="#peculiarities-of-matplotlib" title="Permalink to this headline">¶</a></h3>
<p>matplotlib has complex requirements that make testing and building
more taxing than many other Python projects.</p>
<ul class="simple">
<li>The CPU time to run the tests is quite high. It puts us beyond the
free accounts of many CI services (e.g. ShiningPanda)</li>
<li>It has a large number of dependencies, and testing the full matrix
of all combinations is impractical. We need to be clever about what
space we test and guarantee to support.</li>
</ul>
</div>
<div class="section" id="requirements">
<h3>Requirements<a class="headerlink" href="#requirements" title="Permalink to this headline">¶</a></h3>
<p>This section outlines the requirements that we would like to have.</p>
<ol class="arabic simple">
<li>Testing all pull requests by hooking into the Github API, as
Travis-CI does</li>
<li>Testing on all major platforms: Linux, Mac OS-X, MS Windows (in
that order of priority, based on user survey)</li>
<li>Retain the last n days worth of build and test products, to aid in
post-mortem debugging.</li>
<li>Automated nightly binary builds, so that users can test the
bleeding edge without installing a complete compilation
environment.</li>
<li>Automated benchmarking. It would be nice to have a standard
benchmark suite (separate from the tests) whose performance could
be tracked over time, in different backends and platforms. While
this is separate from building and testing, ideally it would run on
the same infrastructure.</li>
<li>Automated nightly building and publishing of documentation (or as
part of testing, to ensure PRs don’t introduce documentation bugs).
(This would not replace the static documentation for stable
releases as a default).</li>
<li>The test systems should be managable by multiple developers, so
that no single person becomes a bottleneck. (Travis-CI’s design
does this well – storing build configuration in the git
repository, rather than elsewhere, is a very good design.)</li>
<li>Make it easy to test a large but sparse matrix of different
versions of matplotlib’s dependencies. The matplotlib user survey
provides some good data as to where to focus our efforts:
<a class="reference external" href="https://docs.google.com/spreadsheet/ccc?key=0AjrPjlTMRTwTdHpQS25pcTZIRWdqX0pNckNSU01sMHc#gid=0">https://docs.google.com/spreadsheet/ccc?key=0AjrPjlTMRTwTdHpQS25pcTZIRWdqX0pNckNSU01sMHc#gid=0</a></li>
<li>Nice to have: A decentralized design so that those with more
obscure platforms can publish build results to a central dashboard.</li>
</ol>
</div>
</div>
<div class="section" id="implementation">
<h2>Implementation<a class="headerlink" href="#implementation" title="Permalink to this headline">¶</a></h2>
<p>This part is yet-to-be-written.</p>
<p>However, ideally, the implementation would be a third-party service,
to avoid adding system administration to our already stretched time.
As we have some donated funds, this service may be a paid one if it
offers significant time-saving advantages over free offerings.</p>
</div>
<div class="section" id="backward-compatibility">
<h2>Backward compatibility<a class="headerlink" href="#backward-compatibility" title="Permalink to this headline">¶</a></h2>
<p>Backward compatibility is not a major concern for this MEP. We will
replace current tools and procedures with something better and throw
out the old.</p>
</div>
<div class="section" id="alternatives">
<h2>Alternatives<a class="headerlink" href="#alternatives" title="Permalink to this headline">¶</a></h2>
</div>
<div class="section" id="hangout-notes">
<h2>Hangout Notes<a class="headerlink" href="#hangout-notes" title="Permalink to this headline">¶</a></h2>
<div class="section" id="ci-infrastructure">
<h3>CI Infrastructure<a class="headerlink" href="#ci-infrastructure" title="Permalink to this headline">¶</a></h3>
<ul class="simple">
<li>We like Travis and it will probably remain part of our arsenal in
any event. The reliability issues are being looked into.</li>
<li>Enable Amazon S3 uploads of testing products on Travis. This will
help with post-mortem of failures (@mdboom is looking into this
now).</li>
<li>We want Mac coverage. The best bet is probably to push Travis to
enable it for our project by paying them for a Pro account (since
they don’t otherwise allow testing on both Linux and Mac).</li>
<li>We want Windows coverage. Shining Panda is an option there.</li>
<li>Investigate finding or building a tool that would collect and
synthesize test results from a number of sources and post it to
Github using the Github API. This may be of general use to the
Scipy community.</li>
<li>For both Windows and Mac, we should document (or better yet, script)
the process of setting up the machine for a build, and how to build
binaries and installers. This may require getting information from
Russel Owen and Christoph Gohlke. This is a necessary step for
doing automated builds, but would also be valuable for a number of
other reasons.</li>
</ul>
</div>
<div class="section" id="the-test-framework-itself">
<h3>The test framework itself<a class="headerlink" href="#the-test-framework-itself" title="Permalink to this headline">¶</a></h3>
<ul>
<li><p class="first">We should investigate ways to make it take less time</p>
<blockquote>
<div><ul class="simple">
<li>Eliminating redundant tests, if possible</li>
<li>General performance improvements to matplotlib will help</li>
</ul>
</div></blockquote>
</li>
<li><p class="first">We should be covering more things, particularly more backends</p>
</li>
<li><p class="first">We should have more unit tests, fewer integration tests, if possible</p>
</li>
</ul>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="footer">
© Copyright 2002 - 2012 John Hunter, Darren Dale, Eric Firing, Michael Droettboom and the Matplotlib development team; 2012 - 2017 The Matplotlib development team.
Last updated on Dec 11, 2017.
Created using <a
href="http://sphinx-doc.org/">Sphinx</a> 1.6.5.
</div>
</body>
<footer>
</footer>
</html>
You can’t perform that action at this time.