feat: Add iterate methods for paginated collections by Pijukatel · Pull Request #771 · apify/apify-client-python · GitHub
Skip to content

feat: Add iterate methods for paginated collections#771

Merged
vdusek merged 4 commits into
masterfrom
add-iterate-helpers
May 7, 2026
Merged

feat: Add iterate methods for paginated collections#771
vdusek merged 4 commits into
masterfrom
add-iterate-helpers

Conversation

@Pijukatel

@Pijukatel Pijukatel commented Apr 30, 2026 •
edited by vdusek
Loading

Copy link
Copy Markdown
Contributor

Description

Support more convenient iteration through paginated endpoints of collection clients.

  • Added iterate methods to following clients(same for async clients):
    • ActorCollectionClient
    • BuildCollectionClient
    • RunCollectionClient
    • ScheduleCollectionClient
    • TaskCollectionClient
    • WebhookCollectionClient
    • WebhookDispatchCollectionClient
    • DatasetCollectionClient
    • KeyValueStoreCollectionClient
    • RequestQueueCollectionClient
    • StoreCollectionClient
    • ActorEnvVarCollectionClient
    • ActorVersionCollectionClient

Example usage

...
# Sync
datasets_client = ApifyClient(token='...').datasets()

# Same as before
list_page = datasets_client.list(...)

# New functionality
individual_items = [item for item in datasets_client.iterate(...)]

...
# Async
datasets_client = ApifyClientAsync(token='...').datasets()

# Same as before
list_page = await datasets_client.list(...)

# New functionality
individual_items = [item async for item in datasets_client.iterate(...)]

Testing

  • Unit tests
  • Manual API tests

Checklist

  • CI passed

Issues

Closes: #539

@github-actions github-actions Bot added this to the 139th sprint - Tooling team milestone Apr 30, 2026
@github-actions github-actions Bot added the t-tooling Issues with this label are in the ownership of the tooling team. label Apr 30, 2026
@Pijukatel Pijukatel force-pushed the add-iterate-helpers branch from 809e66d to a6d7d51 Compare April 30, 2026 08:49
@github-actions github-actions Bot added the tested Temporary label used only programatically for some analytics. label Apr 30, 2026
@codecov

codecov Bot commented Apr 30, 2026 •
edited
Loading

Copy link
Copy Markdown

@Pijukatel Pijukatel force-pushed the add-iterate-helpers branch from a6d7d51 to 9c38df4 Compare April 30, 2026 09:16
This was referenced Apr 30, 2026
Merged
Closed
vdusek pushed a commit that referenced this pull request Apr 30, 2026
@Pijukatel
ci: pass secrets to unit tests reusable workflow (#772)
63b866a 
## Summary

Adds `secrets: inherit` to the `unit_tests` job in
`.github/workflows/_tests.yaml` so `CODECOV_TOKEN` is forwarded to the
reusable workflow at
`apify/workflows/.github/workflows/python_unit_tests.yaml@main`.

## Why

The reusable unit-tests workflow declares `CODECOV_TOKEN` as a
`workflow_call` secret and gates its upload step on `if:
env.CODECOV_TOKEN != ''`. Without `secrets: inherit` (or an explicit
`secrets:` map) on the caller, the secret is empty inside the called
workflow, so the upload step is silently skipped on every commit and the
`unit` flag is never sent to Codecov — Codecov shows it as
carried-forward.

This is why PRs (e.g. #771) only see the `integration` flag in the
Codecov comment and patch coverage numbers reflect only integration
tests. `secrets: inherit` on the grandparent (`on_pull_request.yaml`)
does not auto-propagate through an intermediate caller — it must be set
on each `uses:` invocation.

`apify/apify-sdk-python` already has `secrets: inherit` set on the same
reusable workflow call.
@vdusek vdusek changed the title Add iterate helpers feat: Add iterate methods for paginated collections May 5, 2026
@vdusek vdusek marked this pull request as ready for review May 5, 2026 07:32
@vdusek vdusek force-pushed the add-iterate-helpers branch from fb54c8e to 1259abb Compare May 5, 2026 07:44
@vdusek vdusek requested review from janbuchar and vdusek May 5, 2026 07:44
@vdusek vdusek force-pushed the add-iterate-helpers branch from 4685d7f to e8e4dfc Compare May 6, 2026 11:40
vdusek
vdusek approved these changes May 6, 2026
View reviewed changes

@vdusek vdusek left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did a few updates, LGTM from my side now. @janbuchar, could you take a look at this as well?

janbuchar
janbuchar approved these changes May 7, 2026
View reviewed changes

@janbuchar janbuchar left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@vdusek vdusek merged commit 3f3129c into master May 7, 2026
26 checks passed
@vdusek vdusek deleted the add-iterate-helpers branch May 7, 2026 11:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@janbuchar janbuchar janbuchar approved these changes
@vdusek vdusek vdusek approved these changes

Assignees

@Pijukatel Pijukatel

Labels

t-tooling Issues with this label are in the ownership of the tooling team. tested Temporary label used only programatically for some analytics.

Projects

None yet

Milestone

139th sprint - Tooling team

Development

Successfully merging this pull request may close these issues.

Add better support for pagination

4 participants

@Pijukatel @janbuchar @vdusek @apify-service-account