Defer cross-context actions on idle actors instead of aborting by jqmmes · Pull Request #7857 · cloudflare/workerd · GitHub
Skip to content

Defer cross-context actions on idle actors instead of aborting - #7857

Draft
jqmmes wants to merge 2 commits into
mainfrom
joaquim/fix-io-context-run
Draft

jqmmes wants to merge 2 commits into
mainfrom
joaquim/fix-io-context-run

Conversation

@jqmmes

@jqmmes jqmmes commented Oct 20, 2026

Copy link
Copy Markdown
Contributor

An actor's IoContext outlives its requests. When another IoContext settles one of its promises, or forwards an AbortSignal abort, the action is queued, and the delete queue signal task runs it with IoContext::run(). An idle actor, such as one holding hibernatable WebSockets, has no current IncomingRequest, so the run failed and the signal task aborted the IoContext, breaking the actor for all its clients.

This branch changes the behavior to ensure IoContext::run() fails with a JavaScript Error carrying NO_CURRENT_INCOMING_REQUEST_DETAIL_ID when there is no current request, instead of an untyped internal error.

An actor's IoContext outlives its requests, so IoContext::run() can
start while the actor is idle, or lose its request while it waits
for a lock. run() needs a current IncomingRequest, so it then failed
with an untyped error: thrown or rejected depending on where, and
logged as an internal error if it reached JavaScript. Callers that
legitimately race with the end of a request, such as facet startup,
couldn't tell it from a bug.

Now run() checks for a request when called and after acquiring each
lock. Without one, the callback doesn't run and run() fails with a
JavaScript Error carrying NO_CURRENT_INCOMING_REQUEST_DETAIL_ID. As
before, it throws if there's no request when it's called, and the
promise rejects if the request ends while it waits for a lock.

Such a run could never succeed, but its failure is now reported as
a user error rather than an internal one: JavaScript sees the
message instead of "internal error", and the runtime doesn't log it
as an internal error. To keep a trail for the cases that are bugs,
run() logs a NOSENTRY warning at most once an hour, naming where the
context's last request was delivered, with a stack trace.

The error is not marked transient, since whether a retry can succeed
depends on the context receiving another request, which the caller
has no way to know. Its wording follows the existing error for a
context that has ended.

The delete queue signal task still aborts the IoContext on this
error; the next commit fixes that.
Other IoContexts can queue actions on an IoContext, e.g. to settle
one of its promises, and its delete queue signal task runs them.
Actions run JavaScript, so they need a current IncomingRequest. An
idle actor, such as one holding hibernatable WebSockets, has none,
so the run failed and the signal task aborted the IoContext,
breaking the actor for all its clients. The same happened if the
request ended while the run waited for a lock.

Instead:

- Without a current request, leave the actions queued rather than
  calling run(). Also skip a wake that finds the queue empty,
  because an earlier run already took the actions.
- If the run loses its request while it waits for a lock, which
  run() reports with NO_CURRENT_INCOMING_REQUEST_DETAIL_ID before
  its callback starts, keep the actions queued and leave the
  IoContext alone. Any other failure still aborts the IoContext,
  including an action's own, whatever details its exception carries.
- Have IncomingRequest::delivered() signal the queue if actions are
  waiting, so the next request runs them. Actions are only left
  waiting while no request is in progress, so only a request
  delivered then needs to check.
- Re-arm the signal before running the actions, so an action queued
  during the run still wakes the context.
- Take the actions out of the queue before running them, so one that
  schedules another can't deadlock on the queue's lock. Nothing does
  that today.

A deferred action runs as part of the request it runs under: it
counts toward that request's limits, appears in its logs and traces,
and has no ordering guarantee relative to the request's own handler.
Queued actions keep the objects they act on, such as a settled
promise and its value, alive on the isolate's heap, so its memory
limit bounds the queue.

handle_cross_request_promise_resolution only guarantees that a
promise settles in its owning IoContext, not when, so deferring
keeps that contract. Document the deferral, and that queued actions
are dropped if the IoContext ends first, which is logged as a
NOSENTRY warning at most once an hour. AbortSignal aborts forwarded
to an idle actor are deferred the same way.

The old behavior broke the actor, so we accept the risk of changing
it without a compatibility flag.
@jqmmes
jqmmes force-pushed the joaquim/fix-io-context-run branch from 36e4e9f to a743d7f Compare October 20, 2026 20:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant