BasicCrawler asks the request manager for work through is_empty() / is_finished(), and it finds a ThrottlingRequestManager with isinstance checks in six places (_basic_crawler.py lines 720, 741, 1457, 1651, 1733, 1802). Any wrapper hides the throttler, which is the root cause of #2238. A manager also has no way to report "requests exist, but none is ready until T" other than the throttler-specific get_stall_reason().

JS v4 replaced the two booleans and the capability probe with members of the request manager interface. Port the same shape.

Proposed solution

• check_readiness() on RequestLoader and RequestManager replaces is_empty() and is_finished(). It returns one of ready, waiting (with an optional ready_at), stalled (with a reason, managers only) or finished. RequestManagerTandem combines its two sources with the precedence ready > stalled > waiting > finished.
• record_pacing_signal(signal) -> bool on RequestManager. The crawler reports 429/503 responses (with Retry-After), robots.txt crawl-delay and a same-domain delay floor as signals. Wrappers such as RequestManagerTandem forward signals without looking inside them. The return value tells the crawler whether anything consumed the signal, which replaces the "not using ThrottlingRequestManager" warnings.
• get_pending_count() on RequestLoader. RequestManagerTandem.get_total_count() uses it; today it adds the loader total to the manager total, so every transferred request is counted twice.
• The inner argument of ThrottlingRequestManager becomes optional and defaults to the default request queue.

Storage backends are not affected: RequestQueueClient keeps is_empty() / is_finished() (#1985), as in JS.

Acceptance criteria

• No isinstance(..., ThrottlingRequestManager) left in the crawlers.
• A throttler behind RequestManagerTandem applies 429 backoff and crawl-delay.
• RequestManagerTandem.get_total_count() counts each request once.
• Upgrade notes for custom RequestLoader / RequestManager implementations.

JS references

• Implementation: refactor!: Add checkReadiness() and recordPacingSignal() to IRequestManager to fix leaky abstraction of ThrottlingRequestManager crawlee#4061
• Issues it closes: Capabilities of a ThrottlingRequestManager are lost when it is wrapped in another manager crawlee#3999, Make the inner option of ThrottlingRequestManager optional crawlee#4027; related: A rate-limited domain is hammered when its requests come from the manager ThrottlingRequestManager wraps crawlee#4039, fix(core): honour domain backoff for requests the wrapped manager holds crawlee#4040
• Tandem counters: v4: RequestManagerTandem request counters are off crawlee#3787
• Still open in JS, without effect on the interface: Reconsider RequestManagerTandem naming crawlee#3722 (tandem naming), Optimize the request transfer in RequestManagerTandem crawlee#3723 (transfer optimization)

This will be a breaking change.

✍️ Drafted by Claude Code