iframe-proxy

mpage · 2025-02-28T22:00:43Z

This PR eliminates most reference counting overhead for references pushed onto the operand stack using LOAD_FAST{_LOAD_FAST} when we can be sure that the reference in the frame outlives the reference that is pushed onto the operand stack. Instructions that meet this criteria are replaced with new variants (LOAD_FAST_BORROW{_LOAD_FAST_BORROW}) that push appropriately tagged borrowed references.

Performance on the benchmark suite looks good:

A ~3% improvement on the free-threaded build. I think this might actually be higher and will continue investigating, but I wanted to put this up for review.
A ~2.7% improvement on the default build.

This approach looks like its quite effective at optimization too, at least on the benchmark suite. Roughly 97% of LOAD_FAST{_LOAD_FAST} instructions are optimized according to pystats. Note that these stats were collected using fastbench, so may not match those collected using pyperformance exactly.

The main pieces of the PR are:

New bytecodes

This adds two new bytecode instructions: LOAD_FAST_BORROW and its superinstruction form, LOAD_FAST_BORROW_LOAD_FAST_BORROW.

A new optimization pass

This adds a new optimization pass, optimize_load_fast, to the bytecode compiler that identifies and optimizes eligible instructions. Please read the detailed comment in flowgraph.c for a description of how it works.

Runtime support changes

A new function, PyStackRef_Borrow, was added to the stackref API. It creates a new stackref from an existing stackref without incrementing the reference count on the underlying object.

There are a few places in the runtime where we need to convert borrowed references into owned references:

When a frame escapes into the heap (i.e. when it is copied into a generator or when its materialized and unwound).
When a reference flows up the call stack (i.e. in RETURN_VALUE or YIELD_VALUE).
When someone destroys a reference to the frame "out of band" by poking at f_locals. We place the old reference into a tuple owned by the frame object.

The default build also required:

Adding support for stackrefs in the GC.
Removing reuse of stackrefs in PyFloat_FromDoubleConsumeInputs.

Issue: Optimize reference counting overhead of LOAD_FAST variants #130704

📚 Documentation preview 📚: https://cpython-previews--130708.org.readthedocs.build/

derp

Ref will be 2 if borrowed

…r frame

Otherwise, it ends up being loaded using `LOAD_FAST_CHECK`, which increfs and causes the refcount check to fail when it uses `LOAD_FAST_BORROW`.

These need to be tagged appropriately, not just increfed, so that they are decrefed when the frame is destroyed.

This may be 1 if the `LOAD_FAST` is optimized to a `LOAD_FAST_BORROW`. It's not clear that this is testing anything useful, so remove it.

The initial value will differ depending on whether a owned or borrowed reference is loaded onto the operand stack.

These don't push enough values on the stack.

…unconditional_jump_threading` Make sure we have a statically known stack depth

…mized to borrowed variants

PyStackRef_AsPyObjectSteal creates a new reference if the stackref is deferred. This reference is leaked if we deopt before the corresponding decref.

These may provide support for borrowed references contained in frames closer to the top of the call stack. Add them to a list attached to the frame when they are overwritten, to be destroyed when the frame is destroyed.

`STORE_FAST_LOAD_FAST` and `LOAD_FAST_AND_CLEAR` both need to kill the local.

This ensures we hit all the blocks

mpage · 2025-03-24T18:19:43Z

colesbury

LGTM

markshannon

Missing one hint for the cases generator, otherwise looks good.

We could make the analysis more robust by using the cases generator, but that's for a later PR.

markshannon · 2025-03-26T15:30:59Z

                _PyInterpreterFrame *gen_frame = &gen->gi_iframe;
                STACK_SHRINK(1);
-                _PyFrame_StackPush(gen_frame, v);
+                _PyFrame_StackPush(gen_frame, PyStackRef_MakeHeapSafe(v));


I think you need a DEAD(v); here

markshannon · 2025-03-26T15:51:05Z

+                    break;
+                }
+
+                // We treat opcodes that do not consume all of their inputs on


This approach seems fine for now, but the code generator knows exactly how many values are popped and consumed, as opposed to peek at.

We should add a _PyOpcode_num_peeked function, the we'd have consumed = _PyOpcode_num_popped() - _PyOpcode_num_peeked() which would be more robust.

bedevere-bot · 2025-03-31T18:43:26Z

- Header files have moved around. - Reference counting has changed. It appears to be python/cpython#130708 that's eliding some reference counting within functions and caused us to need to lower our expected reference count in a few places. NOTE: I'm not 100% sure this is the case; dis.dis is broken and won't show the function bodies so I can't confirm the new opcodes are being used.

- Header files have moved around. - Reference counting has changed. It appears to be python/cpython#130708 that's eliding some reference counting within functions and caused us to need to lower our expected reference count in a few places. NOTE: I'm not 100% sure this is the case; but `dis.dis` shows the new opcode being used for the variables we're testing the refcount of.

@jerome

Running that test on py3.14 fails: ____________________ test_strings_refcount _____________________ def test_strings_refcount(): # buffer with string data - not bytes nor unicode so that when builtin # string types are patched there is no case where bytes is created from the # same bytes, or unicode is created from the same unicode - only increasing # refcount of original object. data = bytearray([ord('a'), ord('b'), ord('c'), ord('4')]) # first verify our logic on std type obj = bytes(data); assert type(obj) is bytes > gc.collect(); assert sys.getrefcount(obj) == 1+1 # +1 due to obj passed to getrefcount call ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ E AssertionError: assert 1 == (1 + 1) E + where 1 = <built-in function getrefcount>(b'abc4') E + where <built-in function getrefcount> = sys.getrefcount golang/golang_str_test.py:301: AssertionError That happens because on py3.14 the interpreter now does not add a reference when passing on-stack argument to a function if it can prove that is safe to do so. Please see python/cpython#130708 and python/cpython@f2379535 for details. -> Adjust the test to account for that on py3.14 . /reviewed-by @jerome /reviewed-on https://lab.nexedi.com/nexedi/pygolang/-/merge_requests/39 + https://lab.nexedi.com/nexedi/slapos/-/merge_requests/1863#note_260064

mpage added 30 commits February 28, 2025 11:41

Experiment with borrowing load_fast

d716faa

Checkpoint poc

3736923

Fix pyframe copy

7a14254

Strengthen refs when frame is copied

b1607aa

Cleanup

e765735

Consider all instructions when computing mutations

291ace9

derp

Add a super instruction

17d6dd6

Don't optimize during quickening

0a74052

Use abstract interpretation

afbfd88

Fix test_generators

696c630

Ref will be 2 if borrowed

Optimize returns

483ac7a

Remove unused arg

259d5db

Make sure we convert borrowed refs on frame

aeafa98

Don't test with malformed bytecode

85f9a64

Make sure we convert borrowed refs to func/code when copying generato…

b6ab2f7

…r frame

Add support for disassembling LOAD_FAST_BORROW_LOAD_FAST_BORROW

fd1ad3d

Make sure exc_obj is always defined

eee2195

Otherwise, it ends up being loaded using `LOAD_FAST_CHECK`, which increfs and causes the refcount check to fail when it uses `LOAD_FAST_BORROW`.

Make sure we store new stackrefs for frame executable/funcobj

d75ec9a

These need to be tagged appropriately, not just increfed, so that they are decrefed when the frame is destroyed.

Remove refcount check

66f5351

This may be 1 if the `LOAD_FAST` is optimized to a `LOAD_FAST_BORROW`. It's not clear that this is testing anything useful, so remove it.

Don't hardcode initial refcount in refcount tests

7ef6a0b

The initial value will differ depending on whether a owned or borrowed reference is loaded onto the operand stack.

Remove invalid bytecode from test_peepholer

2af2bbc

These don't push enough values on the stack.

Fix invalid bytecode in `test_peepholer.DirectCfgOptimizerTests.test_…

bf19b7d

…unconditional_jump_threading` Make sure we have a statically known stack depth

Fix tests that checked for LOAD_FAST instructions that are now opti…

a9bca03

…mized to borrowed variants

Update disassembly in test_dis to match new bytecode

293c317

Fix refleak in _BINARY_OP_INPLACE_ADD_UNICODE

a12ccd9

PyStackRef_AsPyObjectSteal creates a new reference if the stackref is deferred. This reference is leaked if we deopt before the corresponding decref.

Create new references to fast locals overwritten via f_locals

1ef26c5

These may provide support for borrowed references contained in frames closer to the top of the call stack. Add them to a list attached to the frame when they are overwritten, to be destroyed when the frame is destroyed.

Implement two missing opcodes in the static analysis

1eb9226

`STORE_FAST_LOAD_FAST` and `LOAD_FAST_AND_CLEAR` both need to kill the local.

Use g_block_list when resetting stack depth

7291c49

This ensures we hit all the blocks

Avoid reallocating state for each basic block

90bf8df

Generators

9bfa922

mpage added 2 commits March 23, 2025 14:38

Avoid having stackref only visible from the c stack

80fc5aa

Merge branch 'main' into load-fast-borrow-absinterp

492cce1

mpage requested review from colesbury, iritkatriel and markshannon March 24, 2025 18:19

colesbury approved these changes Mar 24, 2025

View reviewed changes

bedevere-app Bot added awaiting merge and removed awaiting change review labels Mar 24, 2025

markshannon approved these changes Mar 26, 2025

View reviewed changes

Merge branch 'main' into load-fast-borrow-absinterp

2e38f0d

mpage added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Mar 31, 2025

bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Mar 31, 2025

Merge branch 'main' into load-fast-borrow-absinterp

2c55722

mpage merged commit 053c285 into python:main Apr 1, 2025

bedevere-app Bot removed the awaiting merge label Apr 1, 2025

mpage mentioned this pull request Apr 1, 2025

gh-131987: Bump the magic number #131991

Merged

wjakob mentioned this pull request Apr 10, 2025

[BUG]: Python 3.14a7 breaks reference call policy test test50_call_policy() wjakob/nanobind#1006

Closed

jamadden mentioned this pull request Apr 11, 2025

Add initial support for Python 3.14a7. python-greenlet/greenlet#442

Merged

mpage mentioned this pull request Apr 28, 2025

BUG: Python 3.14 may not increment refcount pandas-dev/pandas#61368

Closed

3 tasks

efimov-mikhail mentioned this pull request May 4, 2025

segfault when deleting an object after returning it #133371

Closed

efimov-mikhail mentioned this pull request Sep 29, 2025

Use after free (stackref borrow after close) in BINARY_OP_INPLACE_ADD_UNICODE #130786

Closed

tyler-yankee mentioned this pull request Nov 3, 2025

[setup] Move macOS to Python 3.14 RobotLocomotion/drake#23655

Merged

Fidget-Spinner mentioned this pull request Jan 3, 2026

BINARY_OP_INPLACE_ADD_UNICODE has wrong reference count management #143401

Closed

Sunbelt Computer Software

PL/B Language Development and Support

Navigation Menu

Uh oh!

Uh oh!

gh-130704: Strength reduce `LOAD_FAST{_LOAD_FAST}` - #130708

gh-130704: Strength reduce `LOAD_FAST{_LOAD_FAST}`#130708
mpage merged 81 commits into
python:mainfrom
mpage:load-fast-borrow-absinterp

mpage commented Feb 28, 2025 •

edited

Loading

Uh oh!

mpage commented Mar 24, 2025

Uh oh!

colesbury left a comment

Uh oh!

markshannon left a comment

Uh oh!

markshannon Mar 26, 2025

Uh oh!

Uh oh!

markshannon Mar 26, 2025 •

edited

Loading

Uh oh!

bedevere-bot commented Mar 31, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Sunbelt Computer Software

PL/B Language Development and Support

Uh oh!

Uh oh!

Conversation

mpage commented Feb 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mpage commented Mar 24, 2025

Uh oh!

colesbury left a comment

Choose a reason for hiding this comment

Uh oh!

markshannon left a comment

Choose a reason for hiding this comment

Uh oh!

markshannon Mar 26, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

markshannon Mar 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bedevere-bot commented Mar 31, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

mpage commented Feb 28, 2025 •

edited

Loading

markshannon Mar 26, 2025 •

edited

Loading