bpo-36356: Destroy the GIL at exit by vstinner · Pull Request #12453 · python/cpython · GitHub
Skip to content

bpo-36356: Destroy the GIL at exit - #12453

Merged
vstinner merged 1 commit into
python:masterfrom
vstinner:pyeval_finithreads2
Apr 29, 2019
Merged

bpo-36356: Destroy the GIL at exit#12453
vstinner merged 1 commit into
python:masterfrom
vstinner:pyeval_finithreads2

Conversation

@vstinner

@vstinner vstinner commented Mar 20, 2019 •
edited by bedevere-bot
Loading

Copy link
Copy Markdown
Member
  • Add _PyEval_FiniThreads2(). _PyEval_FiniThreads() now only clears
    the pending lock, whereas _PyEval_FiniThreads2() destroys the GIL.
  • pymain_free() now calls _PyEval_FiniThreads2().
  • Py_FinalizeEx() now calls _PyEval_FiniThreads().

https://bugs.python.org/issue36356

@vstinner

vstinner commented Mar 20, 2019

Copy link
Copy Markdown
Member Author

@vstinner

vstinner commented Apr 26, 2019

Copy link
Copy Markdown
Member Author

PR #12667 might help to make Python more reliable with this change.

@vstinner vstinner mentioned this pull request Apr 26, 2019
Merged
@vstinner

vstinner commented Apr 26, 2019

Copy link
Copy Markdown
Member Author

With this change, test_tracemalloc fails because pymain_exit_error() tries to cleanup too many things, even for errors. I wrote PR #12968 to leave the runtime alive on this case, it fix test_tracemalloc.

@vstinner

vstinner commented Apr 26, 2019

Copy link
Copy Markdown
Member Author

I merged my PR #12968 and rebased this PR on top of it. It should fix test_tracemalloc.

@vstinner

vstinner commented Apr 26, 2019

Copy link
Copy Markdown
Member Author

The Ubuntu PR Tests Job of Azure Pipelines PR failed with:

FAIL: test_daemon_threads_shutdown_stderr_deadlock (test.test_io.CMiscIOTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vsts/work/1/s/Lib/test/test_io.py", line 4189, in test_daemon_threads_shutdown_stderr_deadlock
    self.check_daemon_threads_shutdown_deadlock('stderr')
  File "/home/vsts/work/1/s/Lib/test/test_io.py", line 4177, in check_daemon_threads_shutdown_deadlock
    self.assertIn("Fatal Python error: could not acquire lock "
AssertionError: "Fatal Python error: could not acquire lock for <_io.BufferedWriter name='<stderr>'> at interpreter shutdown, possibly due to daemon threads" not found in '...
(...)
.......................!.Fatal Python error: PyCOND_FINI(_PyRuntime.ceval.gil.cond) failed'

test test_io failed

@vstinner vstinner changed the title bpo-36356: Destroy the GIL at exit [WIP] bpo-36356: Destroy the GIL at exit Apr 26, 2019
@vstinner

vstinner commented Apr 26, 2019

Copy link
Copy Markdown
Member Author

I don't understand test_io failuire. I prefer to wait until PR #12667 is merged to see if it does fix this issue. In the meanwhile, I put this issue again in WIP state.

@vstinner
bpo-36356: Destroy the GIL at exit
26dcd26 
* Add _PyEval_FiniThreads2(). _PyEval_FiniThreads() now only clears
  the pending lock, whereas _PyEval_FiniThreads2() destroys the GIL.
* pymain_free() now calls _PyEval_FiniThreads2().
* Py_FinalizeEx() now calls _PyEval_FiniThreads().
@vstinner

vstinner commented Apr 29, 2019

Copy link
Copy Markdown
Member Author

I merged PR #12667 and then rebased this PR on top of it.

@vstinner vstinner changed the title [WIP] bpo-36356: Destroy the GIL at exit bpo-36356: Destroy the GIL at exit Apr 29, 2019
@vstinner
vstinner merged commit b36e5d6 into python:master Apr 29, 2019
@vstinner
vstinner deleted the pyeval_finithreads2 branch April 29, 2019 09:16
@bedevere-bot

bedevere-bot commented Apr 29, 2019

Copy link
Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x RHEL 3.x has failed when building commit b36e5d6.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/21/builds/2780) and take a look at the build logs.
  4. Check if the failure is related to this commit (b36e5d6) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/21/builds/2780

Click to see traceback logs 
Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z/build/Lib/test/test_io.py", line 4186, in test_daemon_threads_shutdown_stdout_deadlock
    self.check_daemon_threads_shutdown_deadlock('stdout')
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z/build/Lib/test/test_io.py", line 4177, in check_daemon_threads_shutdown_deadlock
    self.assertIn("Fatal Python error: could not acquire lock "
AssertionError: "Fatal Python error: could not acquire lock for <_io.BufferedWriter name='<stdout>'> at interpreter shutdown, possibly due to daemon threads" not found in 'Fatal Python error: PyCOND_FINI(_PyRuntime.ceval.gil.cond) failed'

----------------------------------------------------------------------

Ran 606 tests in 35.026s

FAILED (failures=1, skipped=2)


Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z/build/Lib/test/test_io.py", line 4186, in test_daemon_threads_shutdown_stdout_deadlock
    self.check_daemon_threads_shutdown_deadlock('stdout')
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z/build/Lib/test/test_io.py", line 4177, in check_daemon_threads_shutdown_deadlock
    self.assertIn("Fatal Python error: could not acquire lock "
AssertionError: "Fatal Python error: could not acquire lock for <_io.BufferedWriter name='<stdout>'> at interpreter shutdown, possibly due to daemon threads" not found in 'Fatal Python error: PyCOND_FINI(_PyRuntime.ceval.gil.cond) failed'

----------------------------------------------------------------------

Ran 606 tests in 36.979s

FAILED (failures=1, skipped=2)

@bedevere-bot

bedevere-bot commented Apr 29, 2019

Copy link
Copy Markdown

vstinner added a commit that referenced this pull request Apr 29, 2019
@vstinner
Revert "bpo-36356: Destroy the GIL at exit (GH-12453)" (GH613006)
99fcc61 
This reverts commit b36e5d6.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ericsnowcurrently ericsnowcurrently Awaiting requested review from ericsnowcurrently

Assignees

No one assigned

Labels

skip news

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@vstinner @bedevere-bot @the-knights-who-say-ni