gh-130167: Improve speed of `inspect.formatannotation` by replacing `re` by donbarbos · Pull Request #130242 · python/cpython · GitHub
Skip to content

gh-130167: Improve speed of inspect.formatannotation by replacing re - #130242

Open
donbarbos wants to merge 8 commits into
python:mainfrom
donbarbos:improve-inspect-speed
Open

donbarbos wants to merge 8 commits into
python:mainfrom
donbarbos:improve-inspect-speed

Conversation

@donbarbos

@donbarbos donbarbos commented Feb 18, 2025

Copy link
Copy Markdown
Contributor

timeit benchmark with my script:

inspect_bench.py:

import re
import timeit
from typing import List, Dict, Any, Union

def old_format_annotation(annotation):
    if getattr(annotation, "__module__", None) == "typing":
        def repl(match):
            text = match.group()
            return text.removeprefix("typing.")
        return re.sub(r"[\w\.]+", repl, repr(annotation))
    return repr(annotation)

def new_format_annotation(annotation):
    if getattr(annotation, "__module__", None) == "typing":
        return (repr(annotation)
                .replace(".typing.", "@TYPING@")
                .replace("typing.", "")
                .replace("@TYPING@", ".typing."))
    return repr(annotation)

old_time = timeit.timeit(lambda: old_format_annotation(Union[List[str], Dict[str, Any]]), number=100_000)
new_time = timeit.timeit(lambda: new_format_annotation(Union[List[str], Dict[str, Any]]), number=100_000)

print(f"Old version (re.sub): {old_time:.6f}s")
print(f"New version (.replace()): {new_time:.6f}s")
print(f"Difference: {old_time / new_time:.2f}x")

Result: 2.0s -> 1.18s = x1.74 as fast

$ ./python -B inspect_bench.py
Old version (re.sub): 2.043900s
New version (.replace()): 1.175867s
Difference: 1.74x

✔️ This was also the only use of re in module so i got rid of the import

@donbarbos donbarbos changed the title Improve speed of inspect.formatannotation by replacing re gh-130167: Improve speed of inspect.formatannotation by replacing re Feb 18, 2025
Comment thread Misc/NEWS.d/next/Library/2025-02-18-05-04-13.gh-issue-130167.aOgAz_.rst Outdated
donbarbos and others added 2 commits March 28, 2025 01:58
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>

@AA-Turner AA-Turner left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm less sure this is worth it, the replacement isn't clearly better maintenence-wise.

What do the benchmarks look like if you extract repl to a module-level _formatannotation_repl and use .sub() on a pre-compiled pattern?

A

@donbarbos

donbarbos commented May 1, 2025

Copy link
Copy Markdown
Contributor Author

@Engelbarts

Copy link
Copy Markdown

Out of curiosity, why use the little replace dance in the refactored code instead of just return repr(annotation).removeprefix("typing.")?

@donbarbos
donbarbos requested a review from AA-Turner October 22, 2025 13:49
@charmander

charmander commented Nov 27, 2025

Copy link
Copy Markdown

Out of curiosity, why use the little replace dance in the refactored code instead of just return repr(annotation).removeprefix("typing.")?

- Union[typing.List[str], typing.Dict[str, typing.Any]]
+ Union[List[str], Dict[str, Any]]

The replace dance is pretty messy, though, and doesn’t preserve behaviour for footyping.Bar – not that applying a regex to repr output is perfectly reliable to begin with (def foo() -> Literal["typing."]: ...).

For what it’s worth:

pat = re.compile(r"\b(?<!\.)typing\.")
pat.sub("", repr(annotation))

@github-actions

Copy link
Copy Markdown

@github-actions github-actions Bot added the stale Stale PR or inactive for long period of time. label Apr 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

awaiting review stale Stale PR or inactive for long period of time.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants