gh-140936: Fix JIT assertion crash at finalization if some generator is alive by efimov-mikhail · Pull Request #140969 · python/cpython · GitHub
Skip to content

gh-140936: Fix JIT assertion crash at finalization if some generator is alive#140969

Merged
Fidget-Spinner merged 8 commits intopython:mainfrom
efimov-mikhail:issue-140936-jit-crash-at-finalization
Nov 12, 2025
Merged

gh-140936: Fix JIT assertion crash at finalization if some generator is alive#140969
Fidget-Spinner merged 8 commits intopython:mainfrom
efimov-mikhail:issue-140936-jit-crash-at-finalization

Conversation

@efimov-mikhail
Copy link
Copy Markdown
Member

@efimov-mikhail efimov-mikhail commented Nov 3, 2025 •
edited by bedevere-app Bot
Loading

I've removed setting interp->jit to false at finalization.
Because in rare case it may be a problem when some optimized code is used after finalize_modules.

New test needs subprocess since we have assertion crash only at the finalization stage.
On release builds there was no real problem, and test passed with this fix and without it.

@bedevere-app bedevere-app Bot mentioned this pull request Nov 3, 2025
Closed
@Fidget-Spinner
Copy link
Copy Markdown
Member

Fidget-Spinner commented Nov 3, 2025

@efimov-mikhail
Copy link
Copy Markdown
Member Author

efimov-mikhail commented Nov 3, 2025

CC @brandtbucher as author of #129194

@efimov-mikhail
Copy link
Copy Markdown
Member Author

efimov-mikhail commented Nov 3, 2025

The real fix should be to check if interp->jit is false in _PyOptimizer_Optimize and immediately return if not.

Ok, I've provided this fix.

@Fidget-Spinner
Copy link
Copy Markdown
Member

Fidget-Spinner commented Nov 3, 2025

@efimov-mikhail thanks. Just checking: does it pass the test on your system? I didn't test it but I think it should.

@efimov-mikhail
Copy link
Copy Markdown
Member Author

efimov-mikhail commented Nov 3, 2025

On my system new test passes with this fix and fails without it on debug build.
And all others tests also pass.

Fidget-Spinner
Fidget-Spinner reviewed Nov 3, 2025
View reviewed changes
Comment thread Lib/test/test_capi/test_opt.py Outdated
ZeroIntensity
ZeroIntensity reviewed Nov 4, 2025
View reviewed changes
Comment thread Lib/test/test_capi/test_opt.py Outdated
sergey-miryanov
sergey-miryanov reviewed Nov 4, 2025
View reviewed changes
Comment thread Lib/test/test_capi/test_opt.py
sergey-miryanov
sergey-miryanov approved these changes Nov 4, 2025
View reviewed changes
Copy link
Copy Markdown
Contributor

@sergey-miryanov sergey-miryanov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Fidget-Spinner
Fidget-Spinner reviewed Nov 6, 2025
View reviewed changes
Comment thread Lib/test/test_capi/test_opt.py Outdated
@efimov-mikhail
Copy link
Copy Markdown
Member Author

efimov-mikhail commented Nov 12, 2025

Friendly ping, @Fidget-Spinner, @ZeroIntensity .
Can we merge this or there's something else that I have to address?

Fidget-Spinner
Fidget-Spinner reviewed Nov 12, 2025
View reviewed changes
Comment thread Python/optimizer.c
@Fidget-Spinner Fidget-Spinner merged commit 35ed3e4 into python:main Nov 12, 2025
62 checks passed
@miss-islington-app
Copy link
Copy Markdown

miss-islington-app Bot commented Nov 12, 2025

Thanks @efimov-mikhail for the PR, and @Fidget-Spinner for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

@miss-islington-app
Copy link
Copy Markdown

miss-islington-app Bot commented Nov 12, 2025

Sorry, @efimov-mikhail and @Fidget-Spinner, I could not cleanly backport this to 3.14 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 35ed3e4cedc8aef3936da81a6b64e90374532b13 3.14

@efimov-mikhail
Copy link
Copy Markdown
Member Author

efimov-mikhail commented Nov 12, 2025

Thanks for the merge, @Fidget-Spinner. I'll provide manual backport for 3.14.

efimov-mikhail added a commit to efimov-mikhail/cpython that referenced this pull request Nov 13, 2025
@efimov-mikhail
pythongh-140936: Fix JIT assertion crash at finalization if some gene…
e996564 
…rator is alive (pythonGH-140969)
@bedevere-app
Copy link
Copy Markdown

bedevere-app Bot commented Nov 13, 2025

@bedevere-app bedevere-app Bot removed the needs backport to 3.14 bugs and security fixes label Nov 13, 2025
Fidget-Spinner pushed a commit that referenced this pull request Nov 13, 2025
@efimov-mikhail
[3.14] gh-140936: Fix JIT assertion crash at finalization if some gen…
48b7d75 
…erator is alive (GH-140969) (GH-141494)

gh-140936: Fix JIT assertion crash at finalization if some generator is alive (GH-140969)
CuriousLearner added a commit to CuriousLearner/cpython that referenced this pull request Nov 13, 2025
@CuriousLearner
Merge branch 'main' into fix-issue21914
b9f473c 
* main: (463 commits)
  pythongh-140601: Add ResourceWarning to iterparse when not closed (pythonGH-140603)
  pythongh-137969: Fix double evaluation of `ForwardRef`s which rely on globals (python#140974)
  pythongh-139109: A new tracing JIT compiler frontend for CPython (pythonGH-140310)
  pythongh-141004: Document `PyErr_RangedSyntaxLocationObject` (python#141521)
  pythongh-140873: Add support of non-descriptor callables in functools.singledispatchmethod() (pythonGH-140884)
  pythongh-139653: Add PyUnstable_ThreadState_SetStackProtection() (python#139668)
  pythongh-141004: Document `PyCode_Optimize` (pythonGH-141378)
  pythongh-141004: Document C APIs for dictionary keys, values, and items (pythonGH-141009)
  pythongh-137959: Fix `TIER1_TO_TIER2` macro name in JIT InternalDocs (pythonGH-141496)
  pythongh-139871: Add `bytearray.take_bytes([n])` to efficiently extract `bytes` (pythonGH-140128)
  pythongh-140601: Refactor ElementTree.iterparse() tests (pythonGH-141499)
  pythongh-135801: Add the module parameter to compile() etc (pythonGH-139652)
  pythongh-140260: fix data race in `_struct` module initialization with subinterpreters (python#140909)
  pythongh-137109: refactor warning about threads when forking (python#141438)
  pythongh-141004: Document `PyRun_InteractiveOneObject` (pythonGH-141405)
  pythongh-124111: Fix TCL 9 thread detection (pythonGH-128103)
  pythongh-141442: Add escaping to iOS testbed arguments (python#141443)
  pythongh-140936: Fix JIT assertion crash at finalization if some generator is alive (pythonGH-140969)
  Add details about JIT build infrastructure and updating dependencies to `Tools/jit` (python#141167)
  pythongh-141412: Use reliable target URL for urllib example (pythonGH-141428)
  ...
CuriousLearner added a commit to CuriousLearner/cpython that referenced this pull request Nov 14, 2025
@CuriousLearner
Merge branch 'main' of github.com:python/cpython into fix-Issue18387
c46ecc5 
* 'main' of github.com:python/cpython: (464 commits)
  pythongh-140601: Add ResourceWarning to iterparse when not closed (pythonGH-140603)
  pythongh-137969: Fix double evaluation of `ForwardRef`s which rely on globals (python#140974)
  pythongh-139109: A new tracing JIT compiler frontend for CPython (pythonGH-140310)
  pythongh-141004: Document `PyErr_RangedSyntaxLocationObject` (python#141521)
  pythongh-140873: Add support of non-descriptor callables in functools.singledispatchmethod() (pythonGH-140884)
  pythongh-139653: Add PyUnstable_ThreadState_SetStackProtection() (python#139668)
  pythongh-141004: Document `PyCode_Optimize` (pythonGH-141378)
  pythongh-141004: Document C APIs for dictionary keys, values, and items (pythonGH-141009)
  pythongh-137959: Fix `TIER1_TO_TIER2` macro name in JIT InternalDocs (pythonGH-141496)
  pythongh-139871: Add `bytearray.take_bytes([n])` to efficiently extract `bytes` (pythonGH-140128)
  pythongh-140601: Refactor ElementTree.iterparse() tests (pythonGH-141499)
  pythongh-135801: Add the module parameter to compile() etc (pythonGH-139652)
  pythongh-140260: fix data race in `_struct` module initialization with subinterpreters (python#140909)
  pythongh-137109: refactor warning about threads when forking (python#141438)
  pythongh-141004: Document `PyRun_InteractiveOneObject` (pythonGH-141405)
  pythongh-124111: Fix TCL 9 thread detection (pythonGH-128103)
  pythongh-141442: Add escaping to iOS testbed arguments (python#141443)
  pythongh-140936: Fix JIT assertion crash at finalization if some generator is alive (pythonGH-140969)
  Add details about JIT build infrastructure and updating dependencies to `Tools/jit` (python#141167)
  pythongh-141412: Use reliable target URL for urllib example (pythonGH-141428)
  ...
CuriousLearner added a commit to CuriousLearner/cpython that referenced this pull request Nov 14, 2025
@CuriousLearner
Merge branch 'main' of github.com:python/cpython into fix-issue24255
c240289 
* 'main' of github.com:python/cpython: (464 commits)
  pythongh-140601: Add ResourceWarning to iterparse when not closed (pythonGH-140603)
  pythongh-137969: Fix double evaluation of `ForwardRef`s which rely on globals (python#140974)
  pythongh-139109: A new tracing JIT compiler frontend for CPython (pythonGH-140310)
  pythongh-141004: Document `PyErr_RangedSyntaxLocationObject` (python#141521)
  pythongh-140873: Add support of non-descriptor callables in functools.singledispatchmethod() (pythonGH-140884)
  pythongh-139653: Add PyUnstable_ThreadState_SetStackProtection() (python#139668)
  pythongh-141004: Document `PyCode_Optimize` (pythonGH-141378)
  pythongh-141004: Document C APIs for dictionary keys, values, and items (pythonGH-141009)
  pythongh-137959: Fix `TIER1_TO_TIER2` macro name in JIT InternalDocs (pythonGH-141496)
  pythongh-139871: Add `bytearray.take_bytes([n])` to efficiently extract `bytes` (pythonGH-140128)
  pythongh-140601: Refactor ElementTree.iterparse() tests (pythonGH-141499)
  pythongh-135801: Add the module parameter to compile() etc (pythonGH-139652)
  pythongh-140260: fix data race in `_struct` module initialization with subinterpreters (python#140909)
  pythongh-137109: refactor warning about threads when forking (python#141438)
  pythongh-141004: Document `PyRun_InteractiveOneObject` (pythonGH-141405)
  pythongh-124111: Fix TCL 9 thread detection (pythonGH-128103)
  pythongh-141442: Add escaping to iOS testbed arguments (python#141443)
  pythongh-140936: Fix JIT assertion crash at finalization if some generator is alive (pythonGH-140969)
  Add details about JIT build infrastructure and updating dependencies to `Tools/jit` (python#141167)
  pythongh-141412: Use reliable target URL for urllib example (pythonGH-141428)
  ...
StanFromIreland pushed a commit to StanFromIreland/cpython that referenced this pull request Dec 6, 2025
@efimov-mikhail @StanFromIreland
pythongh-140936: Fix JIT assertion crash at finalization if some gene…
3ff68f7 
…rator is alive (pythonGH-140969)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Fidget-Spinner Fidget-Spinner Fidget-Spinner approved these changes

@sergey-miryanov sergey-miryanov sergey-miryanov approved these changes

@ericsnowcurrently ericsnowcurrently Awaiting requested review from ericsnowcurrently

@markshannon markshannon Awaiting requested review from markshannon markshannon is a code owner

@ZeroIntensity ZeroIntensity Awaiting requested review from ZeroIntensity

Assignees

@Fidget-Spinner Fidget-Spinner

Labels

skip news

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@efimov-mikhail @Fidget-Spinner @sergey-miryanov @ZeroIntensity