+++++++++++

Python News

+++++++++++

What's New in Python 3.7.0 alpha 1?

===================================

*Release date: XXXX-XX-XX*

Core and Builtins

-----------------

- bpo-30736: The internal unicodedata database has been upgraded to Unicode

10.0.

- bpo-30604: Move co_extra_freefuncs from per-thread to per-interpreter to

avoid crashes.

- bpo-30597: ``print`` now shows expected input in custom error message when

used as a Python 2 statement. Patch by Sanyam Khurana.

- bpo-30682: Removed a too-strict assertion that failed for certain f-strings,

such as eval("f'\\\n'") and eval("f'\\\r'").

- bpo-30501: The compiler now produces more optimal code for complex condition

expressions in the "if", "while" and "assert" statement, the "if" expression,

and generator expressions and comprehensions.

- bpo-28180: Implement PEP 538 (legacy C locale coercion). This means that when

a suitable coercion target locale is available, both the core interpreter and

locale-aware C extensions will assume the use of UTF-8 as the default text

encoding, rather than ASCII.

- bpo-30486: Allows setting cell values for __closure__. Patch by Lisa Roach.

- bpo-30537: itertools.islice now accepts integer-like objects (having

an __index__ method) as start, stop, and slice arguments

- bpo-25324: Tokens needed for parsing in Python moved to C. ``COMMENT``,

``NL`` and ``ENCODING``. This way the tokens and tok_names in the token

module don't get changed when you import the tokenize module.

- bpo-29104: Fixed parsing backslashes in f-strings.

- bpo-27945: Fixed various segfaults with dict when input collections are

mutated during searching, inserting or comparing. Based on patches by

Duane Griffin and Tim Mitchell.

- bpo-25794: Fixed type.__setattr__() and type.__delattr__() for

non-interned attribute names. Based on patch by Eryk Sun.

- bpo-30039: If a KeyboardInterrupt happens when the interpreter is in

the middle of resuming a chain of nested 'yield from' or 'await'

calls, it's now correctly delivered to the innermost frame.

- bpo-28974: ``object.__format__(x, '')`` is now equivalent to ``str(x)``

rather than ``format(str(self), '')``.

- bpo-30024: Circular imports involving absolute imports with binding

a submodule to a name are now supported.

- bpo-12414: sys.getsizeof() on a code object now returns the sizes

which includes the code struct and sizes of objects which it references.

Patch by Dong-hee Na.

- bpo-29839: len() now raises ValueError rather than OverflowError if

__len__() returned a large negative integer.

- bpo-11913: README.rst is now included in the list of distutils standard

READMEs and therefore included in source distributions.

- bpo-29914: Fixed default implementations of __reduce__ and __reduce_ex__().

object.__reduce__() no longer takes arguments, object.__reduce_ex__() now

requires one argument.

- bpo-29949: Fix memory usage regression of set and frozenset object.

- bpo-29935: Fixed error messages in the index() method of tuple, list and deque

when pass indices of wrong type.

- bpo-29816: Shift operation now has less opportunity to raise OverflowError.

ValueError always is raised rather than OverflowError for negative counts.

Shifting zero with non-negative count always returns zero.

- bpo-24821: Fixed the slowing down to 25 times in the searching of some

unlucky Unicode characters.

- bpo-29102: Add a unique ID to PyInterpreterState. This makes it easier

to identify each subinterpreter.

- bpo-29894: The deprecation warning is emitted if __complex__ returns an

instance of a strict subclass of complex. In a future versions of Python

this can be an error.

- bpo-29859: Show correct error messages when any of the pthread_* calls in

thread_pthread.h fails.

- bpo-29849: Fix a memory leak when an ImportError is raised during from import.

- bpo-28856: Fix an oversight that %b format for bytes should support objects

follow the buffer protocol.

- bpo-29723: The ``sys.path[0]`` initialization change for bpo-29139 caused a

regression by revealing an inconsistency in how sys.path is initialized when

executing ``__main__`` from a zipfile, directory, or other import location.

The interpreter now consistently avoids ever adding the import location's

parent directory to ``sys.path``, and ensures no other ``sys.path`` entries

are inadvertently modified when inserting the import location named on the

command line.

- bpo-29568: Escaped percent "%%" in the format string for classic string

formatting no longer allows any characters between two percents.

- bpo-29714: Fix a regression that bytes format may fail when containing zero

bytes inside.

- bpo-29695: bool(), float(), list() and tuple() no longer take keyword arguments.

The first argument of int() can now be passes only as positional argument.

- bpo-28893: Set correct __cause__ for errors about invalid awaitables

returned from __aiter__ and __anext__.

- bpo-28876: ``bool(range)`` works even if ``len(range)``

raises :exc:`OverflowError`.

- bpo-29683: Fixes to memory allocation in _PyCode_SetExtra. Patch by

Brian Coleman.

- bpo-29684: Fix minor regression of PyEval_CallObjectWithKeywords.

It should raise TypeError when kwargs is not a dict. But it might

cause segv when args=NULL and kwargs is not a dict.

- bpo-28598: Support __rmod__ for subclasses of str being called before

str.__mod__. Patch by Martijn Pieters.

- bpo-29607: Fix stack_effect computation for CALL_FUNCTION_EX.

Patch by Matthieu Dartiailh.

- bpo-29602: Fix incorrect handling of signed zeros in complex constructor for

complex subclasses and for inputs having a __complex__ method. Patch

by Serhiy Storchaka.

- bpo-29347: Fixed possibly dereferencing undefined pointers

when creating weakref objects.

- bpo-29463: Add ``docstring`` field to Module, ClassDef, FunctionDef,

and AsyncFunctionDef ast nodes. docstring is not first stmt in their body

anymore. It affects ``co_firstlineno`` and ``co_lnotab`` of code object

for module and class.

- bpo-29438: Fixed use-after-free problem in key sharing dict.

- bpo-29546: Set the 'path' and 'name' attribute on ImportError for ``from ... import ...``.

- bpo-29546: Improve from-import error message with location

- bpo-29478: If max_line_length=None is specified while using the Compat32 policy,

it is no longer ignored. Patch by Mircea Cosbuc.

- Issue #29319: Prevent RunMainFromImporter overwriting sys.path[0].

- Issue #29337: Fixed possible BytesWarning when compare the code objects.

Warnings could be emitted at compile time.

- Issue #29327: Fixed a crash when pass the iterable keyword argument to

sorted().

- Issue #29034: Fix memory leak and use-after-free in os module (path_converter).

- Issue #29159: Fix regression in bytes(x) when x.__index__() raises Exception.

- Issue #29049: Call _PyObject_GC_TRACK() lazily when calling Python function.

Calling function is up to 5% faster.

- Issue #28927: bytes.fromhex() and bytearray.fromhex() now ignore all ASCII

whitespace, not only spaces. Patch by Robert Xiao.

- Issue #28932: Do not include <sys/random.h> if it does not exist.

- Issue #25677: Correct the positioning of the syntax error caret for

indented blocks. Based on patch by Michael Layzell.

- Issue #29000: Fixed bytes formatting of octals with zero padding in alternate

form.

- Issue #18896: Python function can now have more than 255 parameters.

collections.namedtuple() now supports tuples with more than 255 elements.

- Issue #28596: The preferred encoding is UTF-8 on Android. Patch written by

Chi Hsuan Yen.

- bpo-22257: Clean up interpreter startup (see PEP 432).

- Issue #26919: On Android, operating system data is now always encoded/decoded

to/from UTF-8, instead of the locale encoding to avoid inconsistencies with

os.fsencode() and os.fsdecode() which are already using UTF-8.

- Issue #28991: functools.lru_cache() was susceptible to an obscure reentrancy

bug triggerable by a monkey-patched len() function.

- Issue #28147: Fix a memory leak in split-table dictionaries: setattr()

must not convert combined table into split table. Patch written by INADA

Naoki.

- Issue #28739: f-string expressions are no longer accepted as docstrings and

by ast.literal_eval() even if they do not include expressions.

- Issue #28512: Fixed setting the offset attribute of SyntaxError by

PyErr_SyntaxLocationEx() and PyErr_SyntaxLocationObject().

- Issue #28918: Fix the cross compilation of xxlimited when Python has been

built with Py_DEBUG defined.

- Issue #23722: Rather than silently producing a class that doesn't support

zero-argument ``super()`` in methods, failing to pass the new

``__classcell__`` namespace entry up to ``type.__new__`` now results in a

``DeprecationWarning`` and a class that supports zero-argument ``super()``.

- Issue #28797: Modifying the class __dict__ inside the __set_name__ method of

a descriptor that is used inside that class no longer prevents calling the

__set_name__ method of other descriptors.

- Issue #28799: Remove the ``PyEval_GetCallStats()`` function and deprecate

the untested and undocumented ``sys.callstats()`` function. Remove the

``CALL_PROFILE`` special build: use the :func:`sys.setprofile` function,

:mod:`cProfile` or :mod:`profile` to profile function calls.

- Issue #12844: More than 255 arguments can now be passed to a function.

- Issue #28782: Fix a bug in the implementation ``yield from`` when checking

if the next instruction is YIELD_FROM. Regression introduced by WORDCODE

(issue #26647).

- Issue #28774: Fix error position of the unicode error in ASCII and Latin1

encoders when a string returned by the error handler contains multiple

non-encodable characters (non-ASCII for the ASCII codec, characters out

of the U+0000-U+00FF range for Latin1).

- Issue #28731: Optimize _PyDict_NewPresized() to create correct size dict.

Improve speed of dict literal with constant keys up to 30%.

- Issue #28532: Show sys.version when -V option is supplied twice.

- Issue #27100: The with-statement now checks for __enter__ before it

checks for __exit__. This gives less confusing error messages when

both methods are missing. Patch by Jonathan Ellington.

- Issue #28746: Fix the set_inheritable() file descriptor method on platforms

that do not have the ioctl FIOCLEX and FIONCLEX commands.

- Issue #26920: Fix not getting the locale's charset upon initializing the

interpreter, on platforms that do not have langinfo.

- Issue #28648: Fixed crash in Py_DecodeLocale() in debug build on Mac OS X

when decode astral characters. Patch by Xiang Zhang.

- Issue #28665: Improve speed of the STORE_DEREF opcode by 40%.

- Issue #19398: Extra slash no longer added to sys.path components in case of

empty compile-time PYTHONPATH components.

- Issue #28621: Sped up converting int to float by reusing faster bits counting

implementation. Patch by Adrian Wielgosik.

- Issue #28580: Optimize iterating split table values.

Patch by Xiang Zhang.

- Issue #28583: PyDict_SetDefault didn't combine split table when needed.

Patch by Xiang Zhang.

- Issue #28128: Deprecation warning for invalid str and byte escape

sequences now prints better information about where the error

occurs. Patch by Serhiy Storchaka and Eric Smith.

- Issue #28509: dict.update() no longer allocate unnecessary large memory.

- Issue #28426: Fixed potential crash in PyUnicode_AsDecodedObject() in debug

build.

- Issue #28517: Fixed of-by-one error in the peephole optimizer that caused

keeping unreachable code.

- Issue #28214: Improved exception reporting for problematic __set_name__

attributes.

- Issue #23782: Fixed possible memory leak in _PyTraceback_Add() and exception

loss in PyTraceBack_Here().

- Issue #28183: Optimize and cleanup dict iteration.

- Issue #26081: Added C implementation of asyncio.Future.

Original patch by Yury Selivanov.

- Issue #28379: Added sanity checks and tests for PyUnicode_CopyCharacters().

Patch by Xiang Zhang.

- Issue #28376: The type of long range iterator is now registered as Iterator.

Patch by Oren Milman.

- Issue #28376: Creating instances of range_iterator by calling range_iterator

type now is disallowed. Calling iter() on range instance is the only way.

Patch by Oren Milman.

- Issue #26906: Resolving special methods of uninitialized type now causes

implicit initialization of the type instead of a fail.

- Issue #18287: PyType_Ready() now checks that tp_name is not NULL.

Original patch by Niklas Koep.

- Issue #24098: Fixed possible crash when AST is changed in process of

compiling it.

- Issue #28201: Dict reduces possibility of 2nd conflict in hash table when

hashes have same lower bits.

- Issue #28350: String constants with null character no longer interned.

- Issue #26617: Fix crash when GC runs during weakref callbacks.

- Issue #27942: String constants now interned recursively in tuples and frozensets.

- Issue #28289: ImportError.__init__ now resets not specified attributes.

- Issue #21578: Fixed misleading error message when ImportError called with

invalid keyword args.

- Issue #28203: Fix incorrect type in complex(1.0, {2:3}) error message.

Patch by Soumya Sharma.

- Issue #28086: Single var-positional argument of tuple subtype was passed

unscathed to the C-defined function. Now it is converted to exact tuple.

- Issue #28214: Now __set_name__ is looked up on the class instead of the

instance.

- Issue #27955: Fallback on reading /dev/urandom device when the getrandom()

syscall fails with EPERM, for example when blocked by SECCOMP.

- Issue #28192: Don't import readline in isolated mode.

- Issue #27441: Remove some redundant assignments to ob_size in longobject.c.

Thanks Oren Milman.

- Issue #27222: Clean up redundant code in long_rshift function. Thanks

Oren Milman.

- Upgrade internal unicode databases to Unicode version 9.0.0.

- Issue #28131: Fix a regression in zipimport's compile_source(). zipimport

should use the same optimization level as the interpreter.

- Issue #28126: Replace Py_MEMCPY with memcpy(). Visual Studio can properly

optimize memcpy().

- Issue #28120: Fix dict.pop() for splitted dictionary when trying to remove a

"pending key" (Not yet inserted in split-table). Patch by Xiang Zhang.

- Issue #26182: Raise DeprecationWarning when async and await keywords are

used as variable/attribute/class/function name.

- Issue #26182: Fix a refleak in code that raises DeprecationWarning.

- Issue #28721: Fix asynchronous generators aclose() and athrow() to

handle StopAsyncIteration propagation properly.

- Issue #26110: Speed-up method calls: add LOAD_METHOD and CALL_METHOD

opcodes.

Extension Modules

-----------------

- Issue #29169: Update zlib to 1.2.11.

Library

-------

- bpo-29212: Fix concurrent.futures.thread.ThreadPoolExecutor threads to have

a non repr() based thread name by default when no thread_name_prefix is

supplied. They will now identify themselves as "ThreadPoolExecutor-y_n".

- [Security] bpo-30694: Upgrade expat copy from 2.2.0 to 2.2.1 to get fixes

of multiple security vulnerabilities including: CVE-2017-9233 (External

entity infinite loop DoS), CVE-2016-9063 (Integer overflow, re-fix),

CVE-2016-0718 (Fix regression bugs from 2.2.0's fix to CVE-2016-0718)

and CVE-2012-0876 (Counter hash flooding with SipHash).

Note: the CVE-2016-5300 (Use os-specific entropy sources like getrandom)

doesn't impact Python, since Python already gets entropy from the OS to set

the expat secret using ``XML_SetHashSalt()``.

- bpo-29755: Fixed the lgettext() family of functions in the gettext module.

They now always return bytes.

- [Security] bpo-30500: Fix urllib.parse.splithost() to correctly parse

fragments. For example, ``splithost('//127.0.0.1#@evil.com/')`` now

correctly returns the ``127.0.0.1`` host, instead of treating ``@evil.com``

as the host in an authentification (``login@host``).

- bpo-30616: Functional API of enum allows to create empty enums.

Patched by Dong-hee Na

- bpo-30038: Fix race condition between signal delivery and wakeup file

descriptor. Patch by Nathaniel Smith.

- bpo-23894: lib2to3 now recognizes ``rb'...'`` and ``f'...'`` strings.

- [Security] bpo-29591: Update expat copy from 2.1.1 to 2.2.0 to get fixes

of CVE-2016-0718 and CVE-2016-4472. See

https://sourceforge.net/p/expat/bugs/537/ for more information.

- bpo-24744: pkgutil.walk_packages function now raises ValueError if *path*

is a string. Patch by Sanyam Khurana.

- bpo-24484: Avoid race condition in multiprocessing cleanup.

- bpo-30589: Fix multiprocessing.Process.exitcode to return the opposite

of the signal number when the process is killed by a signal (instead

of 255) when using the "forkserver" method.

- bpo-28994: The traceback no longer displayed for SystemExit raised in

a callback registered by atexit.

- bpo-30508: Don't log exceptions if Task/Future "cancel()" method was

called.

- bpo-11822: The dis.dis() function now is able to disassemble nested

code objects.

- bpo-30624: selectors does not take KeyboardInterrupt and SystemExit into

account, leaving a fd in a bad state in case of error. Patch by Giampaolo

Rodola'.

- bpo-30595: multiprocessing.Queue.get() with a timeout now polls its reader in

non-blocking mode if it succeeded to aquire the lock but the acquire took

longer than the timeout.

- bpo-28556: Updates to typing module: Add generic AsyncContextManager, add

support for ContextManager on all versions. Original PRs by Jelle Zijlstra

and Ivan Levkivskyi

- bpo-30605: re.compile() no longer raises a BytesWarning when compiling a

bytes instance with misplaced inline modifier. Patch by Roy Williams.

- bpo-29870: Fix ssl sockets leaks when connection is aborted in asyncio/ssl

implementation. Patch by Michaël Sghaïer.

- bpo-29743: Closing transport during handshake process leaks open socket.

Patch by Nikolay Kim

- bpo-27585: Fix waiter cancellation in asyncio.Lock.

Patch by Mathieu Sornay.

- bpo-30014: modify() method of poll(), epoll() and devpoll() based classes of

selectors module is around 10% faster. Patch by Giampaolo Rodola'.

- bpo-30418: On Windows, subprocess.Popen.communicate() now also ignore EINVAL

on stdin.write() if the child process is still running but closed the pipe.

- bpo-30463: Addded empty __slots__ to abc.ABC. This allows subclassers

to deny __dict__ and __weakref__ creation. Patch by Aaron Hall.

- bpo-30520: Loggers are now pickleable.

- bpo-30557: faulthandler now correctly filters and displays exception codes

on Windows

- bpo-30526: Add TextIOWrapper.reconfigure() and a TextIOWrapper.write_through

attribute.

- bpo-30245: Fix possible overflow when organize struct.pack_into

error message. Patch by Yuan Liu.

- bpo-30378: Fix the problem that logging.handlers.SysLogHandler cannot

handle IPv6 addresses.

- bpo-16500: Allow registering at-fork handlers.

- bpo-30470: Deprecate invalid ctypes call protection on Windows. Patch by

Mariatta Wijaya.

- bpo-30414: multiprocessing.Queue._feed background running

thread do not break from main loop on exception.

- bpo-30003: Fix handling escape characters in HZ codec. Based on patch

by Ma Lin.

- bpo-30149: inspect.signature() now supports callables with

variable-argument parameters wrapped with partialmethod.

Patch by Dong-hee Na.

- bpo-30436: importlib.find_spec() raises ModuleNotFoundError instead of

AttributeError if the specified parent module is not a package

(i.e. lacks a __path__ attribute).

- bpo-30301: Fix AttributeError when using SimpleQueue.empty() under

*spawn* and *forkserver* start methods.

- bpo-30375: Warnings emitted when compile a regular expression now always

point to the line in the user code. Previously they could point into inners

of the re module if emitted from inside of groups or conditionals.

- bpo-30329: imaplib and poplib now catch the Windows socket WSAEINVAL error

(code 10022) on shutdown(SHUT_RDWR): An invalid operation was attempted.

This error occurs sometimes on SSL connections.

- bpo-29196: Removed previously deprecated in Python 2.4 classes Plist, Dict

and _InternalDict in the plistlib module. Dict values in the result of

functions readPlist() and readPlistFromBytes() are now normal dicts. You

no longer can use attribute access to access items of these dictionaries.

- bpo-9850: The :mod:`macpath` is now deprecated and will be removed

in Python 3.8.

- bpo-30299: Compiling regular expression in debug mode on CPython now displays

the compiled bytecode in human readable form.

- bpo-30048: Fixed ``Task.cancel()`` can be ignored when the task is

running coroutine and the coroutine returned without any more ``await``.

- bpo-30266: contextlib.AbstractContextManager now supports anti-registration

by setting __enter__ = None or __exit__ = None, following the pattern

introduced in bpo-25958. Patch by Jelle Zijlstra.

- bpo-30340: Enhanced regular expressions optimization. This increased

the performance of matching some patterns up to 25 times.

- bpo-30298: Weaken the condition of deprecation warnings for inline modifiers.

Now allowed several subsequential inline modifiers at the start of the

pattern (e.g. ``'(?i)(?s)...'``). In verbose mode whitespaces and comments

now are allowed before and between inline modifiers (e.g.

``'(?x) (?i) (?s)...'``).

- bpo-30285: Optimized case-insensitive matching and searching of regular

expressions.

- bpo-29990: Fix range checking in GB18030 decoder. Original patch by Ma Lin.

- bpo-29979: rewrite cgi.parse_multipart, reusing the FieldStorage class and

making its results consistent with those of FieldStorage for

multipart/form-data requests. Patch by Pierre Quentel.

- bpo-30243: Removed the __init__ methods of _json's scanner and encoder.

Misusing them could cause memory leaks or crashes. Now scanner and encoder

objects are completely initialized in the __new__ methods.

- bpo-30215: Compiled regular expression objects with the re.LOCALE flag no

longer depend on the locale at compile time. Only the locale at matching

time affects the result of matching.

- bpo-30185: Avoid KeyboardInterrupt tracebacks in forkserver helper process

when Ctrl-C is received.

- bpo-30103: binascii.b2a_uu() and uu.encode() now support using ``'`'``

as zero instead of space.

- bpo-28556: Various updates to typing module: add typing.NoReturn type, use

WrapperDescriptorType, minor bug-fixes. Original PRs by

Jim Fasarakis-Hilliard and Ivan Levkivskyi.

- bpo-30205: Fix getsockname() for unbound AF_UNIX sockets on Linux.

- bpo-30228: The seek() and tell() methods of io.FileIO now set the internal

seekable attribute to avoid one syscall on open() (in buffered or text mode).

- bpo-30190: unittest's assertAlmostEqual and assertNotAlmostEqual provide a

better message in case of failure which includes the difference between

left and right arguments. (patch by Giampaolo Rodola')

- bpo-30101: Add support for curses.A_ITALIC.

- bpo-29822: inspect.isabstract() now works during __init_subclass__. Patch

by Nate Soares.

- bpo-29960: Preserve generator state when _random.Random.setstate()

raises an exception. Patch by Bryan Olson.

- bpo-30070: Fixed leaks and crashes in errors handling in the parser module.

- bpo-22352: Column widths in the output of dis.dis() are now adjusted for

large line numbers and instruction offsets.

- bpo-30061: Fixed crashes in IOBase methods __next__() and readlines() when

readline() or __next__() respectively return non-sizeable object.

Fixed possible other errors caused by not checking results of PyObject_Size(),

PySequence_Size(), or PyMapping_Size().

- bpo-30218: Fix PathLike support for shutil.unpack_archive. Patch by Jelle

Zijlstra.

- bpo-10076: Compiled regular expression and match objects in the re module

now support copy.copy() and copy.deepcopy() (they are considered atomic).

- bpo-30068: _io._IOBase.readlines will check if it's closed first when

hint is present.

- bpo-29694: Fixed race condition in pathlib mkdir with flags

parents=True. Patch by Armin Rigo.

- bpo-29692: Fixed arbitrary unchaining of RuntimeError exceptions in

contextlib.contextmanager. Patch by Siddharth Velankar.

- bpo-26187: Test that sqlite3 trace callback is not called multiple

times when schema is changing. Indirectly fixed by switching to

use sqlite3_prepare_v2() in bpo-9303. Patch by Aviv Palivoda.

- bpo-30017: Allowed calling the close() method of the zip entry writer object

multiple times. Writing to a closed writer now always produces a ValueError.

- bpo-29998: Pickling and copying ImportError now preserves name and path

attributes.

- bpo-29995: re.escape() now escapes only regex special characters.

- bpo-29962: Add math.remainder operation, implementing remainder

as specified in IEEE 754.

- bpo-29649: Improve struct.pack_into() exception messages for problems

with the buffer size and offset. Patch by Andrew Nester.

- bpo-29654: Support If-Modified-Since HTTP header (browser cache). Patch

by Pierre Quentel.

- bpo-29931: Fixed comparison check for ipaddress.ip_interface objects.

Patch by Sanjay Sundaresan.

- bpo-29953: Fixed memory leaks in the replace() method of datetime and time

objects when pass out of bound fold argument.

- bpo-29942: Fix a crash in itertools.chain.from_iterable when encountering

long runs of empty iterables.

- bpo-10030: Sped up reading encrypted ZIP files by 2 times.

- bpo-29204: Element.getiterator() and the html parameter of XMLParser() were

deprecated only in the documentation (since Python 3.2 and 3.4 correspondintly).

Now using them emits a deprecation warning.

- bpo-27863: Fixed multiple crashes in ElementTree caused by race conditions

and wrong types.

- bpo-25996: Added support of file descriptors in os.scandir() on Unix.

os.fwalk() is sped up by 2 times by using os.scandir().

- bpo-28699: Fixed a bug in pools in multiprocessing.pool that raising an

exception at the very first of an iterable may swallow the exception or

make the program hang. Patch by Davin Potts and Xiang Zhang.

- bpo-23890: unittest.TestCase.assertRaises() now manually breaks a reference

cycle to not keep objects alive longer than expected.

- bpo-29901: The zipapp module now supports general path-like objects, not

just pathlib.Path.

- bpo-25803: Avoid incorrect errors raised by Path.mkdir(exist_ok=True)

when the OS gives priority to errors such as EACCES over EEXIST.

- bpo-29861: Release references to tasks, their arguments and their results

as soon as they are finished in multiprocessing.Pool.

- bpo-19930: The mode argument of os.makedirs() no longer affects the file

permission bits of newly-created intermediate-level directories.

- bpo-29884: faulthandler: Restore the old sigaltstack during teardown.

Patch by Christophe Zeitouny.

- bpo-25455: Fixed crashes in repr of recursive buffered file-like objects.

- bpo-29800: Fix crashes in partial.__repr__ if the keys of partial.keywords

are not strings. Patch by Michael Seifert.

- bpo-8256: Fixed possible failing or crashing input() if attributes "encoding"

or "errors" of sys.stdin or sys.stdout are not set or are not strings.

- bpo-28692: Using non-integer value for selecting a plural form in gettext is

now deprecated.

- bpo-26121: Use C library implementation for math functions erf() and erfc().

- bpo-29619: os.stat() and os.DirEntry.inode() now convert inode (st_ino) using

unsigned integers.

- bpo-28298: Fix a bug that prevented array 'Q', 'L' and 'I' from accepting big

intables (objects that have __int__) as elements.

- bpo-29645: Speed up importing the webbrowser module. webbrowser.register()

is now thread-safe.

- bpo-28231: The zipfile module now accepts path-like objects for external

paths.

- bpo-26915: index() and count() methods of collections.abc.Sequence now

check identity before checking equality when do comparisons.

- bpo-28682: Added support for bytes paths in os.fwalk().

- bpo-29728: Add new :data:`socket.TCP_NOTSENT_LOWAT` (Linux 3.12) constant.

Patch by Nathaniel J. Smith.

- bpo-29623: Allow use of path-like object as a single argument in

ConfigParser.read(). Patch by David Ellis.

- bpo-9303: Migrate sqlite3 module to _v2 API. Patch by Aviv Palivoda.

- bpo-28963: Fix out of bound iteration in asyncio.Future.remove_done_callback

implemented in C.

- bpo-29704: asyncio.subprocess.SubprocessStreamProtocol no longer closes before

all pipes are closed.

- bpo-29271: Fix Task.current_task and Task.all_tasks implemented in C

to accept None argument as their pure Python implementation.

- bpo-29703: Fix asyncio to support instantiation of new event loops

in child processes.

- bpo-29615: SimpleXMLRPCDispatcher no longer chains KeyError (or any other

exception) to exception(s) raised in the dispatched methods.

Patch by Petr Motejlek.

- bpo-7769: Method register_function() of xmlrpc.server.SimpleXMLRPCDispatcher

and its subclasses can now be used as a decorator.

- bpo-29376: Fix assertion error in threading._DummyThread.is_alive().

- bpo-28624: Add a test that checks that cwd parameter of Popen() accepts

PathLike objects. Patch by Sayan Chowdhury.

- bpo-28518: Start a transaction implicitly before a DML statement.

Patch by Aviv Palivoda.

- bpo-29742: get_extra_info() raises exception if get called on closed ssl transport.

Patch by Nikolay Kim.

- Issue #16285: urrlib.parse.quote is now based on RFC 3986 and hence includes

'~' in the set of characters that is not quoted by default. Patch by

Christian Theune and Ratnadeep Debnath.

- bpo-29532: Altering a kwarg dictionary passed to functools.partial()

no longer affects a partial object after creation.

- bpo-29110: Fix file object leak in aifc.open() when file is given as a

filesystem path and is not in valid AIFF format. Patch by Anthony Zhang.

- bpo-22807: Add uuid.SafeUUID and uuid.UUID.is_safe to relay information from

the platform about whether generated UUIDs are generated with a

multiprocessing safe method.

- bpo-29576: Improve some deprecations in importlib. Some deprecated methods

now emit DeprecationWarnings and have better descriptive messages.

- bpo-29534: Fixed different behaviour of Decimal.from_float()

for _decimal and _pydecimal. Thanks Andrew Nester.

- bpo-10379: locale.format_string now supports the 'monetary' keyword argument,

and locale.format is deprecated.

- bpo-29851: importlib.reload() now raises ModuleNotFoundError if the

module lacks a spec.

- Issue #28556: Various updates to typing module: typing.Counter, typing.ChainMap,

improved ABC caching, etc. Original PRs by Jelle Zijlstra, Ivan Levkivskyi,

Manuel Krebber, and Łukasz Langa.

- Issue #29100: Fix datetime.fromtimestamp() regression introduced in Python

3.6.0: check minimum and maximum years.

- Issue #29416: Prevent infinite loop in pathlib.Path.mkdir

- Issue #29444: Fixed out-of-bounds buffer access in the group() method of

the match object. Based on patch by WGH.

- Issue #29377: Add WrapperDescriptorType, MethodWrapperType, and

MethodDescriptorType built-in types to types module.

Original patch by Manuel Krebber.

- Issue #29218: Unused install_misc command is now removed. It has been

documented as unused since 2000. Patch by Eric N. Vander Weele.

- Issue #29368: The extend() method is now called instead of the append()

method when unpickle collections.deque and other list-like objects.

This can speed up unpickling to 2 times.

- Issue #29338: The help of a builtin or extension class now includes the

constructor signature if __text_signature__ is provided for the class.

- Issue #29335: Fix subprocess.Popen.wait() when the child process has

exited to a stopped instead of terminated state (ex: when under ptrace).

- Issue #29290: Fix a regression in argparse that help messages would wrap at

non-breaking spaces.

- Issue #28735: Fixed the comparison of mock.MagickMock with mock.ANY.

- Issue #29197: Removed deprecated function ntpath.splitunc().

- Issue #29210: Removed support of deprecated argument "exclude" in

tarfile.TarFile.add().

- Issue #29219: Fixed infinite recursion in the repr of uninitialized

ctypes.CDLL instances.

- Issue #29192: Removed deprecated features in the http.cookies module.

- Issue #29193: A format string argument for string.Formatter.format()

is now positional-only.

- Issue #29195: Removed support of deprecated undocumented keyword arguments

in methods of regular expression objects.

- Issue #28969: Fixed race condition in C implementation of functools.lru_cache.

KeyError could be raised when cached function with full cache was

simultaneously called from differen threads with the same uncached arguments.

- Issue #20804: The unittest.mock.sentinel attributes now preserve their

identity when they are copied or pickled.

- Issue #29142: In urllib.request, suffixes in no_proxy environment variable with

leading dots could match related hostnames again (e.g. .b.c matches a.b.c).

Patch by Milan Oberkirch.

- Issue #28961: Fix unittest.mock._Call helper: don't ignore the name parameter

anymore. Patch written by Jiajun Huang.

- Issue #15812: inspect.getframeinfo() now correctly shows the first line of

a context. Patch by Sam Breese.

- Issue #28985: Update authorizer constants in sqlite3 module.

Patch by Dingyuan Wang.

- Issue #29079: Prevent infinite loop in pathlib.resolve() on Windows

- Issue #13051: Fixed recursion errors in large or resized

curses.textpad.Textbox. Based on patch by Tycho Andersen.

- Issue #9770: curses.ascii predicates now work correctly with negative

integers.

- Issue #28427: old keys should not remove new values from

WeakValueDictionary when collecting from another thread.

- Issue #28923: Remove editor artifacts from Tix.py.

- Issue #28871: Fixed a crash when deallocate deep ElementTree.

- Issue #19542: Fix bugs in WeakValueDictionary.setdefault() and

WeakValueDictionary.pop() when a GC collection happens in another

thread.

- Issue #20191: Fixed a crash in resource.prlimit() when passing a sequence that

doesn't own its elements as limits.

- Issue #16255: subprocess.Popen uses /system/bin/sh on Android as the shell,

instead of /bin/sh.

- Issue #28779: multiprocessing.set_forkserver_preload() would crash the

forkserver process if a preloaded module instantiated some

multiprocessing objects such as locks.

- Issue #26937: The chown() method of the tarfile.TarFile class does not fail

now when the grp module cannot be imported, as for example on Android

platforms.

- Issue #28847: dbm.dumb now supports reading read-only files and no longer

writes the index file when it is not changed. A deprecation warning is now

emitted if the index file is missed and recreated in the 'r' and 'w' modes

(will be an error in future Python releases).

- Issue #27030: Unknown escapes consisting of ``'\'`` and an ASCII letter in

re.sub() replacement templates regular expressions now are errors.

- Issue #28835: Fix a regression introduced in warnings.catch_warnings():

call warnings.showwarning() if it was overriden inside the context manager.

- Issue #27172: To assist with upgrades from 2.7, the previously documented

deprecation of ``inspect.getfullargspec()`` has been reversed. This decision

may be revisited again after the Python 2.7 branch is no longer officially

supported.

- Issue #28740: Add sys.getandroidapilevel(): return the build time API version

of Android as an integer. Function only available on Android.

- Issue #26273: Add new :data:`socket.TCP_CONGESTION` (Linux 2.6.13) and

:data:`socket.TCP_USER_TIMEOUT` (Linux 2.6.37) constants. Patch written by

Omar Sandoval.

- Issue #28752: Restored the __reduce__() methods of datetime objects.

- Issue #28727: Regular expression patterns, _sre.SRE_Pattern objects created

by re.compile(), become comparable (only x==y and x!=y operators). This

change should fix the issue #18383: don't duplicate warning filters when the

warnings module is reloaded (thing usually only done in unit tests).

- Issue #20572: Remove the subprocess.Popen.wait endtime parameter. It was

deprecated in 3.4 and undocumented prior to that.

- Issue #25659: In ctypes, prevent a crash calling the from_buffer() and

from_buffer_copy() methods on abstract classes like Array.

- Issue #28548: In the "http.server" module, parse the protocol version if

possible, to avoid using HTTP 0.9 in some error responses.

- Issue #19717: Makes Path.resolve() succeed on paths that do not exist.

Patch by Vajrasky Kok

- Issue #28563: Fixed possible DoS and arbitrary code execution when handle

plural form selections in the gettext module. The expression parser now

supports exact syntax supported by GNU gettext.

- Issue #28387: Fixed possible crash in _io.TextIOWrapper deallocator when

the garbage collector is invoked in other thread. Based on patch by

Sebastian Cufre.

- Issue #27517: LZMA compressor and decompressor no longer raise exceptions if

given empty data twice. Patch by Benjamin Fogle.

- Issue #28549: Fixed segfault in curses's addch() with ncurses6.

- Issue #28449: tarfile.open() with mode "r" or "r:" now tries to open a tar

file with compression before trying to open it without compression. Otherwise

it had 50% chance failed with ignore_zeros=True.

- Issue #23262: The webbrowser module now supports Firefox 36+ and derived

browsers. Based on patch by Oleg Broytman.

- Issue #24241: The webbrowser in an X environment now prefers using the

default browser directly. Also, the webbrowser register() function now has

a documented 'preferred' argument, to specify browsers to be returned by

get() with no arguments. Patch by David Steele

- Issue #27939: Fixed bugs in tkinter.ttk.LabeledScale and tkinter.Scale caused

by representing the scale as float value internally in Tk. tkinter.IntVar

now works if float value is set to underlying Tk variable.

- Issue #28255: calendar.TextCalendar.prweek() no longer prints a space after

a weeks's calendar. calendar.TextCalendar.pryear() no longer prints redundant

newline after a year's calendar. Based on patch by Xiang Zhang.

- Issue #28255: calendar.TextCalendar.prmonth() no longer prints a space

at the start of new line after printing a month's calendar. Patch by

Xiang Zhang.

- Issue #20491: The textwrap.TextWrapper class now honors non-breaking spaces.

Based on patch by Kaarle Ritvanen.

- Issue #28353: os.fwalk() no longer fails on broken links.

- Issue #28430: Fix iterator of C implemented asyncio.Future doesn't accept

non-None value is passed to it.send(val).

- Issue #27025: Generated names for Tkinter widgets now start by the "!" prefix

for readability.

- Issue #25464: Fixed HList.header_exists() in tkinter.tix module by addin

a workaround to Tix library bug.

- Issue #28488: shutil.make_archive() no longer adds entry "./" to ZIP archive.

- Issue #25953: re.sub() now raises an error for invalid numerical group

reference in replacement template even if the pattern is not found in

the string. Error message for invalid group reference now includes the

group index and the position of the reference.

Based on patch by SilentGhost.

- Issue #28469: timeit now uses the sequence 1, 2, 5, 10, 20, 50,... instead

of 1, 10, 100,... for autoranging.

- Issue #28115: Command-line interface of the zipfile module now uses argparse.

Added support of long options.

- Issue #18219: Optimize csv.DictWriter for large number of columns.

Patch by Mariatta Wijaya.

- Issue #28448: Fix C implemented asyncio.Future didn't work on Windows.

- Issue #23214: In the "io" module, the argument to BufferedReader and

BytesIO's read1() methods is now optional and can be -1, matching the

BufferedIOBase specification.

- Issue #28480: Fix error building socket module when multithreading is

disabled.

- Issue #28240: timeit: remove ``-c/--clock`` and ``-t/--time`` command line

options which were deprecated since Python 3.3.

- Issue #28240: timeit now repeats the benchmarks 5 times instead of only 3

to make benchmarks more reliable.

- Issue #28240: timeit autorange now uses a single loop iteration if the

benchmark takes less than 10 seconds, instead of 10 iterations.

"python3 -m timeit -s 'import time' 'time.sleep(1)'" now takes 4 seconds

instead of 40 seconds.

- Distutils.sdist now looks for README and setup.py files with case

sensitivity. This behavior matches that found in Setuptools 6.0 and

later. See `setuptools 100

<https://github.com/pypa/setuptools/issues/100>`_ for rationale.

- Issue #24452: Make webbrowser support Chrome on Mac OS X. Patch by

Ned Batchelder.

- Issue #20766: Fix references leaked by pdb in the handling of SIGINT