feat(experimental): integrate writes strategy and appendable object writer by Pulkit0110 · Pull Request #1695 · googleapis/python-storage · GitHub
Skip to content
This repository was archived by the owner on Mar 31, 2026. It is now read-only.

feat(experimental): integrate writes strategy and appendable object writer#1695

Merged
Pulkit0110 merged 20 commits into
googleapis:mainfrom
Pulkit0110:bidi-writes-retry-integrate
Jan 28, 2026
Merged

feat(experimental): integrate writes strategy and appendable object writer#1695
Pulkit0110 merged 20 commits into
googleapis:mainfrom
Pulkit0110:bidi-writes-retry-integrate

Conversation

@Pulkit0110

Copy link
Copy Markdown
Contributor

Integrate all the components required for bidi writes retries

@product-auto-label product-auto-label Bot added size: xl Pull request size is extra large. api: storage Issues related to the googleapis/python-storage API. labels Dec 24, 2025
@gemini-code-assist

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a significant and well-designed refactoring to integrate retry strategies for bidirectional reads and writes, enhancing the robustness of experimental asyncio features. The abstraction of retry logic into a _BidiStreamRetryManager and specific _WriteResumptionStrategy and _ReadResumptionStrategy classes is a solid architectural choice. The accompanying unit and conformance tests are comprehensive and provide good coverage for the new functionality. My review comments focus on improving error handling by replacing broad except Exception blocks with more specific ones and avoiding print() statements in library code.

Comment thread google/cloud/storage/_experimental/asyncio/async_appendable_object_writer.py Outdated
Comment thread google/cloud/storage/_experimental/asyncio/async_multi_range_downloader.py Outdated
Comment thread google/cloud/storage/_experimental/asyncio/async_multi_range_downloader.py Outdated
Comment thread google/cloud/storage/_experimental/asyncio/retry/reads_resumption_strategy.py Outdated
Comment thread google/cloud/storage/_experimental/asyncio/retry/writes_resumption_strategy.py Outdated
@Pulkit0110 Pulkit0110 force-pushed the bidi-writes-retry-integrate branch from c3ba20d to 36a02d7 Compare January 2, 2026 10:57
@product-auto-label product-auto-label Bot added size: l Pull request size is large. and removed size: xl Pull request size is extra large. labels Jan 2, 2026
@Pulkit0110 Pulkit0110 force-pushed the bidi-writes-retry-integrate branch from 36a02d7 to 5d39296 Compare January 13, 2026 06:53
@product-auto-label product-auto-label Bot added size: xl Pull request size is extra large. and removed size: l Pull request size is large. labels Jan 13, 2026
@Pulkit0110 Pulkit0110 marked this pull request as ready for review January 13, 2026 06:59
@Pulkit0110 Pulkit0110 requested review from a team January 13, 2026 06:59
@Pulkit0110 Pulkit0110 requested a review from a team as a code owner January 13, 2026 06:59
Comment thread google/cloud/storage/_experimental/asyncio/async_appendable_object_writer.py Outdated
Comment thread google/cloud/storage/_experimental/asyncio/async_appendable_object_writer.py Outdated
Comment thread google/cloud/storage/_experimental/asyncio/async_appendable_object_writer.py Outdated
Comment thread google/cloud/storage/_experimental/asyncio/retry/_helpers.py
Comment thread google/cloud/storage/_experimental/asyncio/async_appendable_object_writer.py Outdated
Comment thread google/cloud/storage/_experimental/asyncio/async_appendable_object_writer.py Outdated
@Pulkit0110 Pulkit0110 force-pushed the bidi-writes-retry-integrate branch from fb9c976 to 8347af2 Compare January 21, 2026 11:35
@Pulkit0110 Pulkit0110 force-pushed the bidi-writes-retry-integrate branch from 637019e to 359a23c Compare January 23, 2026 06:30
@Pulkit0110

Copy link
Copy Markdown
Contributor Author

/gcbrun

Comment thread google/cloud/storage/_experimental/asyncio/async_appendable_object_writer.py Outdated
Comment thread google/cloud/storage/_experimental/asyncio/async_write_object_stream.py Outdated
Comment thread google/cloud/storage/_experimental/asyncio/async_write_object_stream.py Outdated
Comment thread google/cloud/storage/_experimental/asyncio/retry/writes_resumption_strategy.py Outdated
Comment thread google/cloud/storage/_experimental/asyncio/retry/writes_resumption_strategy.py Outdated
Comment thread google/cloud/storage/_experimental/asyncio/retry/writes_resumption_strategy.py Outdated
Comment thread tests/unit/asyncio/test_async_appendable_object_writer.py Outdated
Comment thread tests/unit/asyncio/test_async_appendable_object_writer.py Outdated
chandra-siri
chandra-siri previously approved these changes Jan 27, 2026
Comment thread tests/unit/asyncio/test_async_appendable_object_writer.py Outdated
@Pulkit0110

Copy link
Copy Markdown
Contributor Author

/gcbrun

@Pulkit0110 Pulkit0110 added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jan 28, 2026
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jan 28, 2026
@Pulkit0110

Copy link
Copy Markdown
Contributor Author

@Pulkit0110 Pulkit0110 merged commit dbd162b into googleapis:main Jan 28, 2026
18 of 19 checks passed
chandra-siri added a commit that referenced this pull request Feb 2, 2026
PR created by the Librarian CLI to initialize a release. Merging this PR
will auto trigger a release.

Librarian Version: v1.0.2-0.20251119154421-36c3e21ad3ac
Language Image:
us-central1-docker.pkg.dev/cloud-sdk-librarian-prod/images-prod/python-librarian-generator@sha256:8e2c32496077054105bd06c54a59d6a6694287bc053588e24debe6da6920ad91
<details><summary>google-cloud-storage: 3.9.0</summary>

##
[3.9.0](v3.8.0...v3.9.0)
(2026-02-02)

### Features

* update generation for MRD (#1730)
([08bc708](08bc7082))

* add get_object method for async grpc client (#1735)
([0e5ec29](0e5ec29b))

* Add micro-benchmarks for reads comparing standard (regional) vs rapid
(zonal) buckets. (#1697)
([1917649](1917649f))

* Add support for opening via `write_handle` and fix `write_handle` type
(#1715)
([2bc15fa](2bc15fa5))

* add samples for appendable objects writes and reads
([2e1a1eb](2e1a1eb5))

* add samples for appendable objects writes and reads (#1705)
([2e1a1eb](2e1a1eb5))

* add context manager to mrd (#1724)
([5ac2808](5ac2808a))

* Move Zonal Buckets features of `_experimental` (#1728)
([74c9ecc](74c9ecc5))

* add default user agent for grpc (#1726)
([7b31946](7b319469))

* expose finalized_time in blob.py applicable for GET_OBJECT in ZB
(#1719)
([8e21a7f](8e21a7fe))

* expose `DELETE_OBJECT` in `AsyncGrpcClient` (#1718)
([c8dd7a0](c8dd7a0b))

* send `user_agent` to grpc channel (#1712)
([cdb2486](cdb2486b))

* integrate writes strategy and appendable object writer (#1695)
([dbd162b](dbd162b3))

* Add micro-benchmarks for writes comparing standard (regional) vs rapid
(zonal) buckets. (#1707)
([dbe9d8b](dbe9d8b8))

* add support for `generation=0` to avoid overwriting existing objects
and add `is_stream_open` support (#1709)
([ea0f5bf](ea0f5bf8))

* add support for `generation=0` to prevent overwriting existing objects
([ea0f5bf](ea0f5bf8))

* add `is_stream_open` property to AsyncAppendableObjectWriter for
stream status check
([ea0f5bf](ea0f5bf8))

### Bug Fixes

* receive eof while closing reads stream (#1733)
([2ef6339](2ef63396))

* update write handle on every recv() (#1716)
([5d9fafe](5d9fafe1))

* implement requests_done method to signal end of requests in async
streams. Gracefully close streams. (#1700)
([6c16079](6c160794))

* implement requests_done method to signal end of requests in async
streams. Gracefully close streams.
([6c16079](6c160794))

* instance grpc client once per process in benchmarks (#1725)
([721ea2d](721ea2dd))

* Fix formatting in setup.py dependencies list (#1713)
([cc4831d](cc4831d7))

* Change contructors of MRD and AAOW AsyncGrpcClient.grpc_client to
AsyncGrpcClient (#1727)
([e730bf5](e730bf50))

</details>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

api: storage Issues related to the googleapis/python-storage API. size: xl Pull request size is extra large.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants