The every dict has a keys "object", of type PyDictKeysObject. While it isn't actually a Python object, it does have a refcount, which is used to know when to free it. PyDictKeysObject (and the helpers, dictkeys_incref() and dictkeys_decref()) was not updated to be immortal when the other singletons were. When it comes to interpreter isolation, that's a problem for empty dicts.
Every empty dict shares a global, statically allocated singleton for its keys: Py_EMPTY_KEYS (AKA static PyDictKeysObject empty_keys_struct). This singleton is defined and used internally in dictobject.c, so we don't have the same ABI compatibility concerns that we have with object ref counts generally,
One way or another, we need to isolate Py_EMPTY_KEYS. Otherwise we end up with races on the refcount.
cc @eduardo-elizondo @markshannon
Possible solutions:
- update the code in dictobject.c to make
Py_EMPTY_KEYS immortal
- move
Py_EMPTY_KEYS to PyInterpreterState
The first one seems simpler.
Linked PRs
The every dict has a keys "object", of type
PyDictKeysObject. While it isn't actually a Python object, it does have a refcount, which is used to know when to free it.PyDictKeysObject(and the helpers,dictkeys_incref()anddictkeys_decref()) was not updated to be immortal when the other singletons were. When it comes to interpreter isolation, that's a problem for empty dicts.Every empty dict shares a global, statically allocated singleton for its keys:
Py_EMPTY_KEYS(AKAstatic PyDictKeysObject empty_keys_struct). This singleton is defined and used internally in dictobject.c, so we don't have the same ABI compatibility concerns that we have with object ref counts generally,One way or another, we need to isolate
Py_EMPTY_KEYS. Otherwise we end up with races on the refcount.cc @eduardo-elizondo @markshannon
Possible solutions:
Py_EMPTY_KEYSimmortalPy_EMPTY_KEYStoPyInterpreterStateThe first one seems simpler.
Linked PRs