Feat/pagination helper by LifeJiggy · Pull Request #78 · digitalocean/gradient-python · GitHub
Skip to content
This repository was archived by the owner on Aug 17, 2026. It is now read-only.

Feat/pagination helper - #78

Open
LifeJiggy wants to merge 1 commit into
digitalocean:mainfrom
LifeJiggy:feat/pagination-helper
Open

Feat/pagination helper#78
LifeJiggy wants to merge 1 commit into
digitalocean:mainfrom
LifeJiggy:feat/pagination-helper

Conversation

@LifeJiggy

@LifeJiggy LifeJiggy commented Oct 29, 2025

Copy link
Copy Markdown

Summary

This PR adds a PaginationHelper utility class that simplifies handling paginated API responses, enabling developers to easily fetch all items across multiple pages.

Problem

Many Gradient API endpoints return paginated results, requiring developers to manually handle pagination logic. This leads to:

  • Repetitive pagination code across different endpoints
  • Difficulty managing page limits and error handling
  • Manual tracking of pagination state
  • Complex logic for handling different response formats

Solution

Add PaginationHelper class with:

  • Automatic pagination through all pages
  • Configurable page size and maximum pages
  • Support for different response formats (data/items/results)
  • Async pagination support
  • Error handling for pagination end conditions

Key Features

  • Automatic Pagination: Fetches all pages until no more data
  • Configurable Limits: Set page size and maximum pages
  • Format Agnostic: Handles different response structures
  • Async Support: Full async/await compatibility
  • Error Handling: Graceful handling of pagination boundaries
  • Standard Library: No external dependencies

Benefits

  • Eliminates repetitive pagination code
  • Simplifies large dataset handling
  • Improves code maintainability
  • Handles edge cases automatically

Testing

  • Added comprehensive test suite covering:
  • Basic pagination functionality
  • Max pages limits
  • Different response formats
  • Empty responses and edge cases
  • Item extraction from various structures
  • Error handling scenarios

All tests pass with full coverage of pagination functionality.

Usage Examples

from gradient._utils import PaginationHelper

helper = PaginationHelper(page_size=50, max_pages=10)

# Paginate through all models
all_models = helper.paginate(client.models.list)

# Paginate with custom parameters
all_agents = helper.paginate(
    client.agents.list,
    workspace_id="workspace-123"
)

# Async pagination
import asyncio

async def get_all_data():
    all_data = await helper.paginate_async(client.data.list)
    return all_data

# Limit maximum pages to prevent runaway requests
limited_helper = PaginationHelper(page_size=20, max_pages=5)
results = limited_helper.paginate(client.large_dataset.list)

@bbatha

bbatha commented Nov 25, 2025

Copy link
Copy Markdown
Collaborator

@LifeJiggy

LifeJiggy commented Nov 26, 2025

Copy link
Copy Markdown
Author

Thanks for the feedback, @bbatha! 🙌
Spot on—the key validator and CLI are out of scope here.
I'll remove them fully and keep this PR solely on the PaginationHelper + tests.
Pushing the cleaned-up version in the next 24–48 hours.
Thrilled for your take on the pagination logic once it's streamlined!

@LifeJiggy
LifeJiggy force-pushed the feat/pagination-helper branch from 985ec36 to e733095 Compare November 27, 2025 11:37
@LifeJiggy

LifeJiggy commented Nov 27, 2025

Copy link
Copy Markdown
Author

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@LifeJiggy @bbatha