Request mixes the stored record with crawler state. Crawler fields such as crawl_depth, session_id, max_retries, state, skip_navigation and enqueue_strategy live in user_data['__crawlee'] (CrawleeRequestData in _request.py). That is why user_data does not belong to the user alone (#1261). RequestOptions also accepts id, which only a storage should assign.

JS v4 split the two roles. Request is the stored record that storages read and write, and CrawlingRequest is what the crawler and handlers work with. Port the same split and use it to settle #1261.

Proposed solution

• Request holds only what a storage persists. CrawlingRequest adds the crawler-facing accessors, and the crawler builds one from each fetched request.
• Request construction keeps accepting the crawler options, so Request.from_url(url, label=..., max_retries=...) and request lists are unchanged; only the read side moves.
• Remove id from RequestOptions; requests coming from a storage go through a dedicated constructor.
• Decide whether crawler data stays under user_data['__crawlee'] for wire compatibility with JS and the Apify platform, or moves out of user_data (The user_data doesn't belong exclusively to the user #1261).

JS references

• Issue: Improve separation of concerns in Request and RequestOptions crawlee#3086
• Implementation: refactor!: keep crawler vocabulary out of @crawlee/core's Request crawlee#4148

This will be a breaking change.

✍️ Drafted by Claude Code