GH-108362: Incremental GC implementation by markshannon · Pull Request #116206 · python/cpython · GitHub
Skip to content

GH-108362: Incremental GC implementation - #116206

Merged
markshannon merged 16 commits into
python:mainfrom
faster-cpython:incremental-gc-3
Mar 20, 2024
Merged

GH-108362: Incremental GC implementation#116206
markshannon merged 16 commits into
python:mainfrom
faster-cpython:incremental-gc-3

Conversation

@markshannon

@markshannon markshannon commented Mar 1, 2024 •
edited by github-actions Bot
Loading

Copy link
Copy Markdown
Member

Implements incremental cyclic GC.
Instead of traversing one generation on each collection, we traverse the young generation and the oldest part of the old generation. By traversing the old generation a chunk at a time, we keep pause times down a lot.

See faster-cpython/ideas#613 for the idea and algorithm.

📚 Documentation preview 📚: https://cpython-previews--116206.org.readthedocs.build/

@bedevere-app bedevere-app Bot mentioned this pull request Mar 1, 2024
Closed
@markshannon

markshannon commented Mar 4, 2024 •
edited
Loading

Copy link
Copy Markdown
Member Author

@markshannon markshannon added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Mar 5, 2024
@bedevere-bot

bedevere-bot commented Mar 5, 2024

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @markshannon for commit 912ebc1 🤖

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Mar 5, 2024
@markshannon

markshannon commented Mar 6, 2024

Copy link
Copy Markdown
Member Author

!buildbot bigmem

@bedevere-bot

bedevere-bot commented Mar 6, 2024

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @markshannon for commit 912ebc1 🤖

The command will test the builders whose names match following regular expression: bigmem

The builders matched are:

  • AMD64 Windows11 Bigmem PR

@markshannon markshannon added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Mar 7, 2024
@bedevere-bot

bedevere-bot commented Mar 7, 2024

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @markshannon for commit 3c1ef42 🤖

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Mar 7, 2024
@markshannon

markshannon commented Mar 7, 2024

Copy link
Copy Markdown
Member Author

!buildbot bigmem

@bedevere-bot

bedevere-bot commented Mar 7, 2024

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @markshannon for commit 3c1ef42 🤖

The command will test the builders whose names match following regular expression: bigmem

The builders matched are:

  • AMD64 Windows11 Bigmem PR

@markshannon
markshannon marked this pull request as ready for review March 8, 2024 10:07
@markshannon
markshannon requested review from DinoV and removed request for kumaraditya303 March 11, 2024 09:28
@markshannon

markshannon commented Mar 19, 2024

Copy link
Copy Markdown
Member Author

!buildbot bigmem

@bedevere-bot

bedevere-bot commented Mar 19, 2024

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @markshannon for commit f4f04d6 🤖

The command will test the builders whose names match following regular expression: bigmem

The builders matched are:

  • AMD64 Windows11 Bigmem PR

DinoV
DinoV approved these changes Mar 19, 2024
View reviewed changes

@DinoV DinoV left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM now!

@markshannon

markshannon commented Mar 20, 2024

Copy link
Copy Markdown
Member Author

Note on the test failures:

  • JIT / aarch64-unknown-linux-gnu/gcc (Debug) (pull_request) passes ~450 out of 460 tests then is timed out after an hour. I'm considering that a pass.
  • Tests / CIFuzz (memory) (pull_request) is just broken and fails on everything at the moment.

@markshannon
markshannon merged commit 1530932 into python:main Mar 20, 2024
vstinner pushed a commit to vstinner/cpython that referenced this pull request Mar 20, 2024
@markshannon @vstinner
pythonGH-108362: Incremental Cycle GC (pythonGH-116206)
f4b2af9
@hauntsaninja hauntsaninja mentioned this pull request Mar 21, 2024
Closed
This was referenced Mar 21, 2024
Closed
Closed
Merged
Merged
@ericsnowcurrently

ericsnowcurrently commented Mar 22, 2024

Copy link
Copy Markdown
Member

@mdboom mdboom mentioned this pull request Mar 22, 2024
Open
gvanrossum added a commit to gvanrossum/cpython that referenced this pull request Mar 23, 2024
@gvanrossum
Revert "pythonGH-108362: Incremental Cycle GC (pythonGH-116206)"
e6480f7 
Second half of the revert.

This reverts commit 1530932.
adorilson pushed a commit to adorilson/cpython that referenced this pull request Mar 25, 2024
@markshannon @adorilson
pythonGH-108362: Incremental Cycle GC (pythonGH-116206)
41b15e9
diegorusso pushed a commit to diegorusso/cpython that referenced this pull request Apr 17, 2024
@markshannon @diegorusso
pythonGH-108362: Incremental Cycle GC (pythonGH-116206)
56ad354
abhinav-upadhyay added a commit to abhinav-upadhyay/cpython that referenced this pull request May 16, 2024
@abhinav-upadhyay
Move the comment to the right place where it used to be
fa21e29 
In pythonGH-116206, the comment about moving reachable objects to next generation got moved from its original place to a place where there is no code below it. This PR puts it back where the actual movement of rechable objects happen to the next generation.
@abhinav-upadhyay abhinav-upadhyay mentioned this pull request May 16, 2024
Merged
@user202729 user202729 mentioned this pull request Jul 26, 2024
Closed
@markshannon
markshannon deleted the incremental-gc-3 branch August 6, 2024 10:18
Yhg1s added a commit to Yhg1s/cpython that referenced this pull request Sep 29, 2024
@Yhg1s
Revert "pythonGH-108362: Incremental Cycle GC (pythonGH-116206)"
4be9e85 
This reverts commit 1530932.
Yhg1s added a commit to Yhg1s/cpython that referenced this pull request Sep 29, 2024
@Yhg1s
Revert "pythonGH-108362: Incremental Cycle GC (pythonGH-116206)"
1388aff 
This reverts commit 1530932.
Yhg1s added a commit to Yhg1s/cpython that referenced this pull request Sep 29, 2024
@Yhg1s
Revert "pythonGH-108362: Incremental Cycle GC (pythonGH-116206)"
d15eace 
This reverts commit 1530932.
@Yhg1s Yhg1s mentioned this pull request Sep 29, 2024
Merged
nascheme pushed a commit that referenced this pull request Aug 7, 2025
@abhinav-upadhyay
GH-119085: Move comment in Python/gc.c to correct place.
e3ad900 
In GH-116206, the comment about moving reachable objects to next generation
got moved from its original place to a place where there is no code below
it. Put the comment back to where the actual movement of reachable objects
happens.
Agent-Hellboy pushed a commit to Agent-Hellboy/cpython that referenced this pull request Aug 19, 2025
@abhinav-upadhyay @Agent-Hellboy
pythonGH-119085: Move comment in Python/gc.c to correct place.
7bde5d6 
In pythonGH-116206, the comment about moving reachable objects to next generation
got moved from its original place to a place where there is no code below
it. Put the comment back to where the actual movement of reachable objects
happens.
@pgdr pgdr mentioned this pull request Oct 20, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gvanrossum gvanrossum gvanrossum left review comments
@DinoV DinoV DinoV approved these changes
@pablogsal pablogsal Awaiting requested review from pablogsal pablogsal is a code owner
@ericsnowcurrently ericsnowcurrently Awaiting requested review from ericsnowcurrently ericsnowcurrently is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@markshannon @bedevere-bot @DinoV @ericsnowcurrently @gvanrossum