Close #16742: Fix misuse of memory allocations in PyOS_Readline() · python/cpython@2fe9bac · GitHub
Skip to content

Commit 2fe9bac

Browse files
committed
Close #16742: Fix misuse of memory allocations in PyOS_Readline()
The GIL must be held to call PyMem_Malloc(), whereas PyOS_Readline() releases the GIL to read input. The result of the C callback PyOS_ReadlineFunctionPointer must now be a string allocated by PyMem_RawMalloc() or PyMem_RawRealloc() (or NULL if an error occurred), instead of a string allocated by PyMem_Malloc() or PyMem_Realloc(). Fixing this issue was required to setup a hook on PyMem_Malloc(), for example using the tracemalloc module. PyOS_Readline() copies the result of PyOS_ReadlineFunctionPointer() into a new buffer allocated by PyMem_Malloc(). So the public API of PyOS_Readline() does not change.
1 parent 6cf185d commit 2fe9bac

5 files changed

Lines changed: 41 additions & 8 deletions

File tree

Doc/c-api/veryhigh.rst

Lines changed: 8 additions & 0 deletions

Doc/whatsnew/3.4.rst

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -587,3 +587,9 @@ that may require changes to your code.
587587
attribute in the chain referring to the innermost function. Introspection
588588
libraries that assumed the previous behaviour was intentional can use
589589
:func:`inspect.unwrap` to gain equivalent behaviour.
590+
591+
* (C API) The result of the :c:var:`PyOS_ReadlineFunctionPointer` callback must
592+
now be a string allocated by :c:func:`PyMem_RawMalloc` or
593+
:c:func:`PyMem_RawRealloc`, or *NULL* if an error occurred, instead of a
594+
string allocated by :c:func:`PyMem_Malloc` or :c:func:`PyMem_Realloc`.
595+

Misc/NEWS

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,11 @@ Projected release date: 2013-10-20
1010
Core and Builtins
1111
-----------------
1212

13+
- Issue #16742: The result of the C callback PyOS_ReadlineFunctionPointer must
14+
now be a string allocated by PyMem_RawMalloc() or PyMem_RawRealloc() (or NULL
15+
if an error occurred), instead of a string allocated by PyMem_Malloc() or
16+
PyMem_Realloc().
17+
1318
- Issue #19199: Remove ``PyThreadState.tick_counter`` field
1419

1520
- Fix macro expansion of _PyErr_OCCURRED(), and make sure to use it in at

Modules/readline.c

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1176,7 +1176,7 @@ call_readline(FILE *sys_stdin, FILE *sys_stdout, char *prompt)
11761176

11771177
/* We got an EOF, return a empty string. */
11781178
if (p == NULL) {
1179-
p = PyMem_Malloc(1);
1179+
p = PyMem_RawMalloc(1);
11801180
if (p != NULL)
11811181
*p = '\0';
11821182
RESTORE_LOCALE(saved_locale)
@@ -1204,7 +1204,7 @@ call_readline(FILE *sys_stdin, FILE *sys_stdout, char *prompt)
12041204
/* Copy the malloc'ed buffer into a PyMem_Malloc'ed one and
12051205
release the original. */
12061206
q = p;
1207-
p = PyMem_Malloc(n+2);
1207+
p = PyMem_RawMalloc(n+2);
12081208
if (p != NULL) {
12091209
strncpy(p, q, n);
12101210
p[n] = '\n';

Parser/myreadline.c

Lines changed: 20 additions & 6 deletions

0 commit comments

Comments
 (0)