Skip to content
Navigation Menu
{{ message }}
forked from apache/cassandra-python-driver
-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathperformance.html
More file actions
171 lines (141 loc) · 10.2 KB
/
Copy pathperformance.html
File metadata and controls
171 lines (141 loc) · 10.2 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta http-equiv="refresh" content="0; URL=https://docs.datastax.com/en/developer/python-driver/latest/performance/">
<link rel="canonical" href="https://docs.datastax.com/en/developer/python-driver/latest/performance/">
<title>Performance Notes — Cassandra Driver 3.13.0 documentation</title>
<link rel="stylesheet" href="_static/custom.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: './',
VERSION: '3.13.0',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true,
SOURCELINK_SUFFIX: '.txt'
};
</script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="Paging Large Queries" href="query_paging.html" />
<link rel="prev" title="Execution Profiles" href="execution_profiles.html" />
<link rel="stylesheet" href="_static/custom.css" type="text/css" />
<meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
</head>
<body>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<div class="section" id="performance-notes">
<h1>Performance Notes<a class="headerlink" href="#performance-notes" title="Permalink to this headline">¶</a></h1>
<p>The Python driver for Cassandra offers several methods for executing queries.
You can synchronously block for queries to complete using
<a class="reference internal" href="api/cassandra/cluster.html#cassandra.cluster.Session.execute" title="cassandra.cluster.Session.execute"><code class="xref py py-meth docutils literal"><span class="pre">Session.execute()</span></code></a>, you can obtain asynchronous request futures through
<a class="reference internal" href="api/cassandra/cluster.html#cassandra.cluster.Session.execute_async" title="cassandra.cluster.Session.execute_async"><code class="xref py py-meth docutils literal"><span class="pre">Session.execute_async()</span></code></a>, and you can attach a callback to the future
with <a class="reference internal" href="api/cassandra/cluster.html#cassandra.cluster.ResponseFuture.add_callback" title="cassandra.cluster.ResponseFuture.add_callback"><code class="xref py py-meth docutils literal"><span class="pre">ResponseFuture.add_callback()</span></code></a>.</p>
<p>Examples of multiple request patterns can be found in the benchmark scripts included in the driver project.</p>
<p>The choice of execution pattern will depend on the application context. For applications dealing with multiple
requests in a given context, the recommended pattern is to use concurrent asynchronous
requests with callbacks. For many use cases, you don’t need to implement this pattern yourself.
<a class="reference internal" href="api/cassandra/concurrent.html#cassandra.concurrent.execute_concurrent" title="cassandra.concurrent.execute_concurrent"><code class="xref py py-meth docutils literal"><span class="pre">cassandra.concurrent.execute_concurrent()</span></code></a> and <a class="reference internal" href="api/cassandra/concurrent.html#cassandra.concurrent.execute_concurrent_with_args" title="cassandra.concurrent.execute_concurrent_with_args"><code class="xref py py-meth docutils literal"><span class="pre">cassandra.concurrent.execute_concurrent_with_args()</span></code></a>
provide this pattern with a synchronous API and tunable concurrency.</p>
<p>Due to the GIL and limited concurrency, the driver can become CPU-bound pretty quickly. The sections below
discuss further runtime and design considerations for mitigating this limitation.</p>
<div class="section" id="pypy">
<h2>PyPy<a class="headerlink" href="#pypy" title="Permalink to this headline">¶</a></h2>
<p><a class="reference external" href="http://pypy.org">PyPy</a> is an alternative Python runtime which uses a JIT compiler to
reduce CPU consumption. This leads to a huge improvement in the driver performance,
more than doubling throughput for many workloads.</p>
</div>
<div class="section" id="cython-extensions">
<h2>Cython Extensions<a class="headerlink" href="#cython-extensions" title="Permalink to this headline">¶</a></h2>
<p><a class="reference external" href="http://cython.org/">Cython</a> is an optimizing compiler and language that can be used to compile the core files and
optional extensions for the driver. Cython is not a strict dependency, but the extensions will be built by default.</p>
<p>See <a class="reference internal" href="installation.html"><span class="doc">Installation</span></a> for details on controlling this build.</p>
</div>
<div class="section" id="multiprocessing">
<h2>multiprocessing<a class="headerlink" href="#multiprocessing" title="Permalink to this headline">¶</a></h2>
<p>All of the patterns discussed above may be used over multiple processes using the
<a class="reference external" href="http://docs.python.org/2/library/multiprocessing.html">multiprocessing</a>
module. Multiple processes will scale better than multiple threads, so if high throughput is your goal,
consider this option.</p>
<p>Be sure to <strong>never share any</strong> <a class="reference internal" href="api/cassandra/cluster.html#cassandra.cluster.Cluster" title="cassandra.cluster.Cluster"><code class="xref py py-class docutils literal"><span class="pre">Cluster</span></code></a>, <a class="reference internal" href="api/cassandra/cluster.html#cassandra.cluster.Session" title="cassandra.cluster.Session"><code class="xref py py-class docutils literal"><span class="pre">Session</span></code></a>,
<strong>or</strong> <a class="reference internal" href="api/cassandra/cluster.html#cassandra.cluster.ResponseFuture" title="cassandra.cluster.ResponseFuture"><code class="xref py py-class docutils literal"><span class="pre">ResponseFuture</span></code></a> <strong>objects across multiple processes</strong>. These
objects should all be created after forking the process, not before.</p>
<p>For further discussion and simple examples using the driver with <code class="docutils literal"><span class="pre">multiprocessing</span></code>,
see <a class="reference external" href="http://www.datastax.com/dev/blog/datastax-python-driver-multiprocessing-example-for-improved-bulk-data-throughput">this blog post</a>.</p>
</div>
</div>
</div>
</div>
</div>
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper">
<h1 class="logo"><a href="index.html">Cassandra Driver</a></h1>
<p class="blurb">Python driver for Cassandra</p>
<p>
<iframe src="https://ghbtns.com/github-btn.html?user=datastax&repo=python-driver&type=star&count=true&size=large&v=2"
allowtransparency="true" frameborder="0" scrolling="0" width="200px" height="35px"></iframe>
</p>
<h3>Navigation</h3>
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="api/index.html">API Documentation</a></li>
<li class="toctree-l1"><a class="reference internal" href="installation.html">Installation</a></li>
<li class="toctree-l1"><a class="reference internal" href="getting_started.html">Getting Started</a></li>
<li class="toctree-l1"><a class="reference internal" href="upgrading.html">Upgrading</a></li>
<li class="toctree-l1"><a class="reference internal" href="execution_profiles.html">Execution Profiles</a></li>
<li class="toctree-l1 current"><a class="current reference internal" href="#">Performance Notes</a><ul>
<li class="toctree-l2"><a class="reference internal" href="#pypy">PyPy</a></li>
<li class="toctree-l2"><a class="reference internal" href="#cython-extensions">Cython Extensions</a></li>
<li class="toctree-l2"><a class="reference internal" href="#multiprocessing">multiprocessing</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="query_paging.html">Paging Large Queries</a></li>
<li class="toctree-l1"><a class="reference internal" href="lwt.html">Lightweight Transactions (Compare-and-set)</a></li>
<li class="toctree-l1"><a class="reference internal" href="security.html">Security</a></li>
<li class="toctree-l1"><a class="reference internal" href="user_defined_types.html">User Defined Types</a></li>
<li class="toctree-l1"><a class="reference internal" href="object_mapper.html">Object Mapper</a></li>
<li class="toctree-l1"><a class="reference internal" href="dates_and_times.html">Working with Dates and Times</a></li>
<li class="toctree-l1"><a class="reference internal" href="faq.html">Frequently Asked Questions</a></li>
</ul>
<div class="relations">
<h3>Related Topics</h3>
<ul>
<li><a href="index.html">Documentation overview</a><ul>
<li>Previous: <a href="execution_profiles.html" title="previous chapter">Execution Profiles</a></li>
<li>Next: <a href="query_paging.html" title="next chapter">Paging Large Queries</a></li>
</ul></li>
</ul>
</div>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
<form class="search" action="search.html" method="get">
<div><input type="text" name="q" /></div>
<div><input type="submit" value="Go" /></div>
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="footer">
©2013-2017 DataStax.
|
Powered by <a href="http://sphinx-doc.org/">Sphinx 1.6.6</a>
& <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.10</a>
|
<a href="_sources/performance.rst.txt"
rel="nofollow">Page source</a>
</div>
</body>
</html>
You can’t perform that action at this time.