Hi @Mantisus and thanks for the PR! I did a bunch more research in the JS port and figured out a different way around this. It involves changes to the request manager, so it will presumably take more time to pass code review.

Just letting you know so that you don't end up thinking I forgot about this 🙂

@Mantisus feel free to check apify/crawlee#4061 now — in its current state, it should be pretty clear what direction it's headed.

However, it's unlikely that it can be replicated here without breaking changes. @vdusek do you think it makes sense to wait for v2, or should we try and think of a backwards-compatible fix?

@Mantisus feel free to check apify/crawlee#4061 now — in its current state, it should be pretty clear what direction it's headed.

However, it's unlikely that it can be replicated here without breaking changes. @vdusek do you think it makes sense to wait for v2, or should we try and think of a backwards-compatible fix?

If we manage to make it backward-compatible in a reasonable way, I'd go for it. But if that requires introducing some weird quirks/workarounds, I would wait for next major. Especially considering that, nobody has actually reported this issue, if I'm correct.

I updated the PR. That resolves the issue with retry_on_blocked only. This does not require any breaking changes or workarounds.

@janbuchar do you want to check this as well?

Merging, this will close #2126, and I opened a follow-up issue for the 2nd case - #2238