You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The goal here is to the improve import time of pdb, one of the slowest stdlib modules to import.
Before: 52 ms
main on macOS with optimised build:
After: 9 ms
We can lazily import several dependencies, and also do the same in its dependencies bdb and code to make them lighter:
pdb.set_trace() (or breakpoint() with defaults) is commonly used for this module, and will reify 6 of the 18 lazy imports here, and the other 12 remain deferred.
And pdb is often imported without running set_trace(), so we get the benefit of all 18 lazy imports.
This also speeds up every pytest invocation, not just pytest --pdb: pytest unconditionally imports pdb at startup via its default debugging plugin. The pytest project found deferring import of pdb saved ~7% of pytest startup on Python 3.14 (pytest-dev/pytest#14874), but the change was declined in favour of improving import time in CPython itself.
I think making some import lazy definitely makes sense. Making import pdb faster benefits more than pdb itself - pytest is a great example. But how did we choose the libraries to lazy import? Is it quite arbitrary? Is there a rule to follow in stdlib - like which are the libraries we should lazy import? Or did we leave the ones that will "definitely" be used? I'm not against the current list, but it might be clearer for future developers if we have a guidance to follow.
This gives the blue diagram to visualise the import times.
Then looked at the slowest imports, and made them lazy one by one.
Some would gett reified right away, like bdb (due to class Pdb(bdb.Bdb, cmd.Cmd)) and code (due to class _PdbInteractiveConsole(code.InteractiveConsole)), so I left those as eager imports but also made some of their own imports lazy in the same way to make then lighter.
Others like typing also get reified immediately (due to @typing.override), so that was also left as eager. We've tackled making typing faster before.
The tests ensure that a lazy import doesn't end up getting immediately reified due to some other change. If that happens down the line, we can either do something to ensure it remains lazy, or make a conscious decision to allow it be eager, and update the test.
We also have some deferred imports, like runpy, which are imported inside functions. This was the pre-lazy way of doing it, so we can now hoist them up as explicit lazy imports.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The goal here is to the improve import time of
pdb, one of the slowest stdlib modules to import.Before: 52 ms
mainon macOS with optimised build:After: 9 ms
We can lazily import several dependencies, and also do the same in its dependencies
bdbandcodeto make them lighter:pdb.set_trace()(orbreakpoint()with defaults) is commonly used for this module, and will reify 6 of the 18 lazy imports here, and the other 12 remain deferred.And
pdbis often imported without runningset_trace(), so we get the benefit of all 18 lazy imports.This also speeds up every pytest invocation, not just
pytest --pdb: pytest unconditionally importspdbat startup via its default debugging plugin. The pytest project found deferring import ofpdbsaved ~7% of pytest startup on Python 3.14 (pytest-dev/pytest#14874), but the change was declined in favour of improving import time in CPython itself.