{{ message }}
fix(perf): avoid repeated scan of entire venv via packages_distributions() at import time#16579
Merged
daniel-sanche merged 1 commit intoApr 8, 2026
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request optimizes the _get_pypi_package_name function by replacing the slow packages_distributions() scan with targeted metadata.distribution() lookups. It also introduces lazy resolution for package labels to improve performance on supported Python versions. However, the top-level import of importlib.metadata breaks compatibility with Python 3.7. Feedback was provided to handle this as an optional import and to ensure _get_pypi_package_name handles cases where the metadata module is unavailable.
packages_distributions() scans every installed package in the environment to build a complete module-to-distribution mapping. In large venvs (500+ packages, common with many google-cloud-* libs), this causes multi-second import delays for google.api_core and every library that depends on it. Two changes: - Wrap packages_distributions() with functools.cache so the expensive O(n) scan happens at most once per process. - Defer the package label resolution in check_python_version() so it only runs when a warning is actually emitted, not on the common happy path of a supported Python version.
bonauer-pf
force-pushed
the
fix/avoid-packages-distributions-scan
branch
from
e63555e to
6425506
Compare
bonauer-pf
changed the title
Contributor
1 task
vchudnov-g
added
kokoro:force-run
vchudnov-g
approved these changes
Apr 8, 2026
daniel-sanche
changed the title
daniel-sanche
approved these changes
Apr 8, 2026
daniel-sanche
left a comment
daniel-sanche
left a comment
Contributor
There was a problem hiding this comment.
LGTM, thanks for putting this together!
This was referenced Apr 9, 2026
daniel-sanche
pushed a commit
that referenced
this pull request
Apr 9, 2026
PR created by the Librarian CLI to initialize a release. Merging this PR will auto trigger a release. Librarian Version: v0.0.0-20260407211006-7cd74ea13ba9 Language Image: us-central1-docker.pkg.dev/cloud-sdk-librarian-prod/images-prod/python-librarian-generator@sha256:234b9d1f2ddb057ed7ac6a38db0bf8163d839c65c6cf88ade52530cddebce59e <details><summary>bigquery-magics: v0.13.0</summary> ## [v0.13.0](bigquery-magics-v0.12.2...bigquery-magics-v0.13.0) (2026-04-09) ### Features * drop support for Python 3.9 (#16587) ([961eacd](961eacd0)) * deprecate `--engine=bigframes`, run `%load_ext bigframes` and use `%%bqsql` magics instead (#16573) ([dbaf8f5](dbaf8f54)) </details> <details><summary>google-api-core: v2.30.3</summary> ## [v2.30.3](google-api-core-v2.30.2...google-api-core-v2.30.3) (2026-04-09) ### Bug Fixes * avoid repeated scan of entire venv via packages_distributions() at import time (#16579) ([c5728b2](c5728b24)) </details> <details><summary>google-apps-chat: v0.8.0</summary> ## [v0.8.0](google-apps-chat-v0.7.0...google-apps-chat-v0.8.0) (2026-04-09) ### Features * Addition of Section and SectionItem APIs (PiperOrigin-RevId: 893488928) ([56ccbd8](56ccbd86)) * Support app authentication with admin-consent scopes for Chat API ListMessages, GetMessage, GetSpaceEvent and ListSpaceEvents (PiperOrigin-RevId: 893619899) ([56ccbd8](56ccbd86)) ### Documentation * Update reference documentation for Chat API ListMessages, GetMessage, GetSpaceEvent and ListSpaceEvents. (PiperOrigin-RevId: 893619899) ([56ccbd8](56ccbd86)) </details> <details><summary>google-auth: v2.49.2</summary> ## [v2.49.2](google-auth-v2.49.1...google-auth-v2.49.2) (2026-04-09) ### Bug Fixes * use requests transport for GCE MDS (#16480) ([614a3d0](614a3d04)) </details> <details><summary>google-cloud-appoptimize: v0.1.0</summary> ## [v0.1.0](google-cloud-appoptimize-v0.0.0...google-cloud-appoptimize-v0.1.0) (2026-04-09) ### Features * onboard a new library (PiperOrigin-RevId: 895558469) ([8e92bae](8e92bae8)) </details> <details><summary>google-cloud-dlp: v3.36.0</summary> ## [v3.36.0](google-cloud-dlp-v3.35.0...google-cloud-dlp-v3.36.0) (2026-04-09) ### Features * added support for detecting key-value pairs in client provided metadata (PiperOrigin-RevId: 895098649) ([56ccbd8](56ccbd86)) </details> <details><summary>google-cloud-iamconnectorcredentials: v0.1.0</summary> ## [v0.1.0](google-cloud-iamconnectorcredentials-v0.0.0...google-cloud-iamconnectorcredentials-v0.1.0) (2026-04-09) ### Features * onboard a new library (PiperOrigin-RevId: 889980347) ([2a3458c](2a3458c8)) </details> <details><summary>google-cloud-pubsub: v2.37.0</summary> ## [v2.37.0](google-cloud-pubsub-v2.36.0...google-cloud-pubsub-v2.37.0) (2026-04-09) ### Documentation * Fix documentation URL AIInference MessageTransform service_account_email field (PiperOrigin-RevId: 895485801) ([56ccbd8](56ccbd86)) </details> <details><summary>google-cloud-translate: v3.26.0</summary> ## [v3.26.0](google-cloud-translate-v3.25.0...google-cloud-translate-v3.26.0) (2026-04-09) ### Features * A new field `mime_type` is added to message `.google.cloud.translation.v3.AdaptiveMtTranslateRequest` (PiperOrigin-RevId: 895422613) ([56ccbd8](56ccbd86)) </details> <details><summary>pandas-gbq: v0.35.0</summary> ## [v0.35.0](pandas-gbq-v0.34.1...pandas-gbq-v0.35.0) (2026-04-09) ### Features * drop support for Python 3.9 (#16476) ([f38cd44](f38cd44b)) </details>
1 task
potiuk
pushed a commit
to apache/airflow
that referenced
this pull request
May 30, 2026
) google-api-core 2.28.1 introduced an import-time performance regression: a Python-version check added to google/api_core/__init__.py calls importlib.metadata.packages_distributions(), which scans every installed package on each import. In a large Airflow venv this adds several seconds to importing google.api_core and anything built on it, pushing the import of operators like DataprocCreateBatchOperator and BigQueryInsertJobOperator past the Dag file import timeout. Fixed upstream in google-api-core 2.30.3 (googleapis/google-cloud-python#16579); a maintainer confirmed on the issue that upgrading to 2.30.3 cuts import time from 30-50s to 2-5s. This raises the google provider floor to google-api-core>=2.30.3, excluding the regressed 2.28.1-2.30.2 range. The previous !=2.16.0,!=2.18.0 exclusions are subsumed by the higher floor and dropped. uv.lock already resolves to 2.30.3, so only the recorded specifier changes. closes: #62373
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
packages_distributions() scans every installed package in the environment to build a complete module-to-distribution mapping. In large venvs (500+ packages, common with many google-cloud-* libs), this causes multi-second import delays for google.api_core and every library that depends on it.
This PR contains 2 changes:
packages_distributions()with functools.cache so the expensive O(n) scan happens at most once per process.check_python_version()so it only runs when a warning is actually emitted, not on the common happy path of a supported Python version.Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:
Fixes #15015 and #16552.