gh-148072: Cache pickle.dumps/loads per interpreter in XIData · python/cpython@1d378e7 · GitHub
Skip to content

Commit 1d378e7

Browse files
committed
gh-148072: Cache pickle.dumps/loads per interpreter in XIData
Store references to pickle.dumps and pickle.loads in _PyXI_state_t so they are looked up only once per interpreter lifetime, avoiding repeated PyImport_ImportModuleAttrString calls on every cross-interpreter data transfer via pickle fallback. Benchmarks show 1.7x-3.3x speedup for InterpreterPoolExecutor when transferring mutable types (list, dict) through XIData.
1 parent 7e275d4 commit 1d378e7

3 files changed

Lines changed: 66 additions & 6 deletions

File tree

Include/internal/pycore_crossinterp.h

Lines changed: 6 additions & 0 deletions
Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
Cache ``pickle.dumps`` and ``pickle.loads`` per interpreter in the XIData
2+
framework, avoiding repeated module lookups on every cross-interpreter data
3+
transfer. This speeds up :class:`~concurrent.futures.InterpreterPoolExecutor`
4+
for mutable types (``list``, ``dict``) by 1.7x--3.3x.

Python/crossinterp.c

Lines changed: 56 additions & 6 deletions

0 commit comments

Comments
 (0)