fix: respect provider Retry-After before retrying model requests by rudycelekli · Pull Request #49 · mohit67890/imajev · GitHub
Skip to content

fix: respect provider Retry-After before retrying model requests - #49

Open
rudycelekli wants to merge 1 commit into
mohit67890:mainfrom
rudycelekli:fix/imajev-provider-retry-after-20261006
Open

rudycelekli wants to merge 1 commit into
mohit67890:mainfrom
rudycelekli:fix/imajev-provider-retry-after-20261006

Conversation

@rudycelekli

Copy link
Copy Markdown
Contributor

Summary

Honor a retryable hosted model endpoint's Retry-After header before the existing fallback backoff. A provider asking for 45 seconds currently gets a 15-second retry on HTTP 429 or a one-second retry on HTTP 503, which can keep a benchmark run inside the same rate-limit window.

Accept delta seconds and HTTP dates, clamp past dates to zero, and retain the existing fallback for malformed or overflowing values. Final attempts and nonretryable responses retain their existing error behavior. This is provider request pacing; dataset download retry handling in #33 is a separate caller.

Reproduction and validation

  • A real local HTTP endpoint returns 429 or 503, then a valid model response. Calling the public OpenAI provider on unchanged main reproduces the wrong captured retry delay. No paid endpoint or model weights are needed.
  • Eight regression cases cover both statuses, delta seconds, HTTP dates, malformed headers and an overflowing integer. The actual request payload is unchanged between attempts; the wait callback is recorded without sleeping for 45 seconds.
  • Focused triage tests: 21 passed.
  • Full suite at the signed head: 721 passed, 136 skipped, the same four baseline failures, 35 baseline warnings. Unchanged main: 713 passed, 136 skipped, four failures. Two require unavailable optional mlx_vlm; two import the missing test_p3_labeltrain fixture. No new full-suite failures.
  • git diff --check passes. No hosted checks are currently configured/reported; this is not a remote-green claim.

AI assistance

Prepared with AI assistance. The native reproduction, source changes and exact-head tests were independently reviewed before submission.

Signed-off-by: Rudy Celekli <47457359+rudycelekli@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant