Answer a parked approval in one call, now that the decision carries the run forward by sarve-shreyas · Pull Request #445 · browserstack/mcp-server · GitHub
Skip to content

Answer a parked approval in one call, now that the decision carries the run forward - #445

Merged
gaurav-singh-9227 merged 11 commits into
mainfrom
askrelay/central-mcp
Sep 30, 2026
Merged

gaurav-singh-9227 merged 11 commits into
mainfrom
askrelay/central-mcp

Conversation

@sarve-shreyas

@sarve-shreyas sarve-shreyas commented Sep 27, 2026 •

Copy link
Copy Markdown
Collaborator

Pairs with agent-platform#952 (AIMCP-230 / AIMCP-231). Ship together, and publish
this FIRST — see Rollout.

Why

Atlas's approval gate has changed shape. An ask used to be a paused coroutine on the
far end of our socket; it now parks in Atlas's LangGraph checkpoint. That makes the
decision request the one that carries the run forward, and its reply is either the
next ask or the finished run — the same thing the START replies with.

Against that server this client breaks in four places, three silently and one
dangerously:

  1. runResumed required a 204. Atlas answers 200 with the continuation, so
    every resume would have been reported as "the decision could not be delivered…
    nothing was changed"
    while the write actually went through — inviting exactly
    the retry that files the thing twice.
  2. It then reattached to GET /agent/{run_id}/stream, which no longer exists.
  3. The decision body carried no product, which Atlas now requires — a 400 before
    any of the above was reached.
  4. The elicitation path POSTed mid-stream and kept reading, expecting the run to
    continue on the same socket. The stream ends at the park now.

What changes — it gets smaller

Because a decision reply and a start reply are now the same shape:

  • runResumed is one request where it was two. POST the decision (with product);
    the reply is the next ask or the result.
  • DecisionTransport / fetchDecisionTransport deleted — one transport posts and
    reads park-or-result, and one place knows how to tell them apart.
  • resumeUrl and the null-body GET branch deleted, with the reattach route.
  • runStreamed becomes stream-then-decide-loop. The start stays streamed (a long
    turn needs the heartbeats), then each ask is elicited, POSTed, and the reply either
    parks again or finishes. Elicitation and deferred now differ only in who supplies
    the decision.
  • resumeFailed(error) factored out of incompleteResume, so a 404 — no such
    parked run: expired, already answered, or not ours — says which. A denial we could
    not deliver is not a denial.

Also included, from earlier in this branch: the deferred two-call flow itself, the
next_step payload that tells the model to put the ask to a human (it names no tool
and no client, deliberately), and ASK_BROWSERSTACK_RELAY_TRANSPORT so the deferred
path can be exercised from a local stdio install.

One behaviour change worth calling out

A decision Atlas refuses no longer leaves a "completed" run. That used to be
shruggable because the result arrived on the stream regardless; the refused request is
now the missing continuation, so reporting ok would claim an answer nobody produced.
Still never re-sent — a retry could land an approval on a step the run has moved past.

Testing

tsc --noEmit clean, 692 tests across 56 files, main merged in (70 commits, no
conflicts).

Verified live, not just in the harness: this client driven over stdio against a local
Atlas with a real Postgres checkpointer and a real prod-minted JWT —

call 1  -> status=needs_approval, run_id + perm_id + next_step,
           "A HUMAN MUST APPROVE THIS BEFORE IT CAN CONTINUE…", nothing written
call 2  -> decision=allow -> status=ok, project created, exactly one write
deny    -> "wasn't created because the approval prompt wasn't confirmed",
           and the server's own count confirms zero

That live run is also what caught the bugs fixed in agent-platform#952 — including one
where this client silently discarded Atlas's ask because the perm_id did not match
PERM_ID_PATTERN, and the park read as an ordinary result with no question in it.

Rollout — the ordering is no longer load-bearing

It was, in both directions, and both directions are now closed:

  • Old client, new Atlas — agent-platform#952 checks permission_relay.contract.
    A pre-1.6 client gets reads as normal and an honest refusal naming the upgrade on
    anything needing approval, instead of ok: true for a run that did nothing.
  • New client, old Atlas — this client answers each ask as it arrives on the
    stream and reads the reply to tell which server it is talking to (a bare 204
    continues on the stream; anything else carries the run). It used to wait for the
    stream to end first, which deadlocked against a server still holding it open, until
    that server's five-minute gate denied an ask nobody had seen.

So the steps can run in any order, and each is safe on its own:

  1. Merge and publish this. npm-publish.yml is dispatched by hand.
  2. Merge #(the 1.6.0 bump PR) — deliberately separate, and after this, or it
    ships a version identical to 1.5.1.
  3. Refresh the lockfile in remote-mcp-server (^1.5.1 already covers 1.6.0) → new
    image.
  4. Deploy Atlas agent-platform#952.

Set ASK_BROWSERSTACK_ALLOW_REMOTE_RELAY=true — that flag is deleted by this PR.
It guarded switching on elicitation in the hosted deployment, which is what pinned an
McpServer per session (the OOMs of September: ai-common-infra-ops#119, reverted in
#120/#121). The hosted path defers now and holds nothing open, so the hazard it
guarded is gone. Two kill switches remain: ASK_BROWSERSTACK_DISABLED, read per call
and needing no pod roll, and Atlas's own delegation.permission_relay, which turns
the relay off for every client at once.

Still open, deliberately (review thread): with that flag gone, the hosted two-call
flow switches on as soon as remote-mcp-server refreshes its lockfile, and hosted
clients that do declare elicitation get the two-call flow rather than the
fail-closed one. That is a rollout/design decision, not a bug fix, so it is not
changed here.

🤖 Generated with Claude Code

sarve-shreyas and others added 6 commits September 21, 2026 13:50
… state

The counterpart to the Atlas `deferred` transport. Elicitation is what makes
this process stateful — a suspended `elicitation/create` pins an McpServer in
one pod's heap until the answer arrives, which is why the hosted deployment
needs sessions, ingress affinity keyed on the caller's credentials, and a cap on
live sessions. Deferred removes the suspension instead of routing around it.

    call 1  askBrowserStackAI(product, query)
            -> status "needs_approval", run_id, perm_id, the pending ask
    call 2  askBrowserStackAI(run_id, perm_id, decision)
            -> POST the decision, reattach, return the remainder

Nothing is held between the two, so either call can be served by any pod.

`relayMode()` now answers per DEPLOYMENT rather than per client:

  REMOTE_MCP + opt-in  -> "deferred"
  REMOTE_MCP, no opt-in-> "remote_mode"  (unchanged: read-only)
  stdio                -> elicitation, exactly as before

stdio deliberately keeps eliciting. One process, nothing to pin, and the CLIENT
renders the prompt so the model cannot fabricate the answer. A local install
would pay the two-call cost for nothing.

THE TRADE, stated because it is real and inverts an existing test. On the
deferred path approval rides the client's own permission prompt on call 2, not
an elicitation, so client elicitation support stops being the gate. Claude
Desktop declares none but does prompt before a tool call, and gains writes it
could never have had. The same fact the other way: a client configured not to
prompt (`--permission-mode auto`) now approves with no human, where elicitation
failed closed by construction. The test that asserted "the opt-in does NOT force
the relay onto a client that cannot be asked" is inverted on purpose, and says
so at length rather than being quietly deleted.

`perm_id` travels to the caller and back rather than being remembered here. The
first sketch kept a `Map<runId, permId>` in the module, which would have made
call 2 depend on reaching the same pod — reintroducing, in this file, the exact
statefulness the change exists to remove. It also names WHICH ask is being
answered, which matters as soon as a task needs more than one: a resumed run
that hits a second gate parks again rather than answering it for the user.

Ordering inside the resume is a safety property: the decision is POSTed BEFORE
the reattach, so a failed reattach cannot leave the gate waiting on an answer we
already have. A lost reattach costs the caller the answer; a lost decision would
cost them the run. An undelivered decision (0 / 404 / 409) is reported as
undelivered and never as an approval, and does not reattach.

A read-only task still costs ONE call: with no ask, the ordinary result comes
back through the same `buildResult` the streaming path uses, so an entitlement
refusal, a 401 and a plain failure read identically whichever transport produced
them.

653 tests pass; lint, format and tsc clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The parked result's `answer` described the plumbing — "this deployment cannot
hold a prompt open mid-run, so it returned the pending approval instead" —
which is true and useless. It told the model how the transport works, not that
a person has to answer. Measured consequence: the model read the ask, judged it
matched what the user had originally asked for, and sent `allow` itself in the
same turn without putting it to anyone.

So the answer now opens with the obligation, and the actionable half moves out
of prose into a `next_step` object carrying the exact params to send back. "I
did not know what to send" stops being a reason to improvise.

Two things it deliberately does not do. It names no tool and no client when it
points at a structured confirmation affordance — that differs on every client,
and a name here would be wrong everywhere else and would rot as clients change,
so it describes the act and lets the model pick the means. And it says
"BrowserStack AI", not "BrowserStack", because what paused is this agent, not
the company.

None of this can MAKE a model ask; nothing in the protocol can, which remains
the standing argument for elicitation. It removes the excuse.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`relayMode` derived the transport from `REMOTE_MCP`, which conflates two
different questions: "am I the hosted process" and "can I hold a connection
open across a human answer". Only the second decides the transport, and they
come apart the moment anyone wants to exercise the deferred two-call flow from
a local stdio install: `REMOTE_MCP=true` cannot do it, because this entrypoint
exits on that flag by design (the hosted deployment runs remote-mcp-server
instead). Setting it to test the flow killed the server with CONNECTION_CLOSED.

`ASK_BROWSERSTACK_RELAY_TRANSPORT` selects it directly — "deferred" or
"elicit", anything else ignored — so the deferred path can be driven from a
local install and watched end to end, and "elicit" forces the other direction
for the same reason. Absent the variable, behaviour is unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ward

Atlas's converged gate changed the contract under this client, and it broke in
four places — three of which would have been silent and one dangerous:

  * `runResumed` required a 204. Atlas answers 200 WITH the run's continuation,
    so every resume would have been reported as "the decision could not be
    delivered… nothing was changed" while the write actually went through —
    inviting a retry that files the thing twice.
  * it then reattached to `GET /agent/{run_id}/stream`, which no longer exists.
  * the decision body carried no `product`, which Atlas now requires, so the
    POST was a 400 before any of that was reached.
  * the elicit path POSTed mid-stream and kept reading, expecting the run to
    continue on the same socket. The stream ends at the park now.

THE SHAPE, and why it is simpler. An ask parks in Atlas's checkpoint rather than
in a coroutine on the far end of a socket, so the decision request is the one
that carries the run forward and its reply is either the next ask or the
finished run. That is the same thing the START replies with, so:

  * `DecisionTransport` and `fetchDecisionTransport` are gone — one transport
    posts and reads a park-or-result, and one place knows how to tell them apart.
  * `resumeUrl` and the null-body GET branch are gone with the reattach route.
  * `runResumed` is one request where it was two.
  * `runStreamed` becomes stream-then-decide-loop: the start stays streamed
    (a long turn needs the heartbeats), then each ask is elicited, POSTed, and
    the reply either parks again or finishes. Elicitation and deferred now differ
    only in WHO supplies the decision.

`resumeFailed(error)` is factored out of `incompleteResume` so a 404 — no such
parked run: expired, already answered, or not ours — says which, rather than
being folded into a generic failure. It is the one status worth its own words,
because a denial we could not deliver is not a denial.

One behaviour deliberately changed: a decision Atlas refuses no longer leaves a
"completed" run. It used to be shruggable because the result arrived on the
stream regardless; now the refused request IS the missing continuation, so
reporting `ok` would claim an answer nobody produced. Still never re-sent — a
retry could land an approval on a step the run has moved past.

652 tests pass, tsc clean. The E2E stub was repointed rather than rewritten: it
no longer holds each ask open until answered, because the real server cannot do
that any more.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`ApprovalRecord` was only referenced by the old two-request `runResumed`, which
built a failure result inline before `resumeFailed` existed. Lint caught it; the
build is what surfaced it, since tsc does not fail on an unused import.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Sep 27, 2026 •

Copy link
Copy Markdown

sarve-shreyas and others added 2 commits September 27, 2026 20:56
`ASK_BROWSERSTACK_ALLOW_REMOTE_RELAY` gated the relay on the hosted deployment
while relaying meant ELICITATION — it was a guard against switching on the thing
that pins an McpServer per session, which is what forced sessions, ingress
affinity and a session cap onto that deployment and then OOMed it. Deferred
holds no sessions, so the hazard the guard existed for is the one this branch
deletes. Keeping it meant a manual, forgettable deploy step standing between
users and a feature that is safe by construction.

It was also the weakest switch available. Read at boot, so disabling needed a
pod roll — while `ASK_BROWSERSTACK_DISABLED`, defined ten lines away, is read
PER CALL for exactly that reason, and Atlas's own `delegation.permission_relay`
turns the relay off for every client at once from a config change. Two better
kill switches, neither of which this duplicated.

`remote_mode` goes with it, because it can no longer happen. Its message told
the reader "this hosted server has no way to put an approval prompt in front of
you… mid-run approval works when the server runs locally over stdio" — advice to
go and run the server locally to get something they now already have.

Tests: the `REMOTE_MCP — the relay is a stdio-only feature` block is removed
whole; it pinned that sentence. The property that replaced it was already
covered — remote mode selecting `deferred`, over request/response, with the
stream never opened — and that test no longer needs the opt-in to reach it. One
assertion there had gone vacuous with the mode's removal (`reason` not being a
value that no longer exists) and now pins what is actually true: `used: true`,
no reason at all.

686 tests, tsc clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both survived the previous commit because neither failed: one passed
`"remote_mode"` as a literal to `buildResult` in a test asserting that
`not_entitled` wins over it, and the other still exported
`ASK_BROWSERSTACK_ALLOW_REMOTE_RELAY=true` into an env nothing reads. Dead
either way, and the kind of reference that reads as intent later.

The precedence test now loops over the modes that still exist, which is what it
was really about: a 403 says the ACCOUNT was refused, and how the relay was
configured stops mattering at that point.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@sarve-shreyas

Copy link
Copy Markdown
Collaborator Author

Dropped ASK_BROWSERSTACK_ALLOW_REMOTE_RELAY — the rollout no longer has a manual enable step.

It gated the relay on the hosted deployment while relaying meant elicitation: a guard against switching on the thing that pins an McpServer per session, which is what forced sessions, ingress affinity and a session cap onto that deployment and then OOMed it in September. Deferred holds no sessions, so the hazard the guard existed for is the one this PR deletes.

It was also the weakest switch available — read at boot, so disabling needed a pod roll. Two better ones remain, neither of which it duplicated:

  • ASK_BROWSERSTACK_DISABLED, defined ten lines away and read per call, precisely so it works without a restart;
  • Atlas's delegation.permission_relay, which turns the relay off for every client at once from a config change.

remote_mode goes with it, since it can no longer happen. Its message told the reader "this hosted server has no way to put an approval prompt in front of you… mid-run approval works when the server runs locally over stdio" — advice to go and run the server locally to get something they now already have.

Consequence worth naming: the hosted server starts asking for approvals the moment the new image rolls. There is no staged enable any more. If a canary is wanted, the flag was the mechanism for it and this removes that option — say so and I'll put it back.

Rollout is now: publish 1.6.0 → refresh remote-mcp's lockfile → deploy Atlas agent-platform#952.

@sarve-shreyas

Copy link
Copy Markdown
Collaborator Author

Cross-reference: agent-platform#952 now carries a measured compatibility finding that bears on this PR's rollout section.

Short version — a user still on the published 1.5.1 who approves an action against the new Atlas gets ok: true, status: "ok" back for a run that did nothing. Nothing is created (the decision is refused with 400 product is required), but 1.5.1 logs that 400 to stderr and then surfaces the parked result frame as if it were a finished run.

So the "publish this first" ordering in the description is necessary but not sufficient: it protects remote-mcp, whose image we control, and not stdio users pinned to an older version.

Options are laid out on agent-platform#952. The one that would land here rather than in Atlas is a wire version marker on the decision body, so Atlas can tell a 1.6 caller from a legacy one without sniffing product.

Nothing decided yet — flagging so this PR is not merged on the assumption that ordering alone covers it.

`permission_relay.contract: 2` on both paths. Atlas reads an absent or lower
value as a build that cannot complete an approval and refuses to park, rather
than handing back an ask nobody can answer.

DECLARED, not inferred, because the two are identical on the way in: the stdio
path here sends `{"mode": "stream"}` exactly as 1.5.1 does, so there is nothing
for Atlas to tell them apart by. Bump it only when a change leaves an older
client unable to COMPLETE the exchange — not merely unaware of a new field.

Why it is worth a field at all: measured against a converged Atlas, 1.5.1
answers a decision without `product`, takes the 400, logs it to stderr and then
reports `ok: true` for a run that created nothing. With the marker it gets an
honest refusal naming the upgrade instead.

Also removes two things the flag removal left behind and lint caught — an empty
`finally` and a `buildRemoteServer` helper orphaned when the `remote_mode` tests
went. Both failed the BUILD, not the tests, so `dist/` had been silently stale:
the "new" client was running without the contract marker and behaving like an
old one until this was fixed.

686 tests, tsc clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@ruturaj-browserstack ruturaj-browserstack left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the detailed write-up. Requesting changes, mainly for two correctness issues:

  1. A timeout after an allow is reported as "never reached" (stream.ts). The rest of the agent run now happens inside a single 120s POST, so a slow run can time out after the write went through, and the result then tells the caller nothing was sent.
  2. Following next_step literally fails validation, because query is still required (deferred.ts).

Two more need a rollout/design discussion, because both deploy orders currently break someone:

  1. The hosted server now uses the two-call flow with no opt-in and no kill switch, and the model can supply allow itself. Rollout step 4 in the PR body refers to a flag this PR deletes.
  2. stdio against a pre-#952 Atlas waits until the 300s gate denies, because phase 1 now waits for the stream to end before answering.

The remaining comments are about the tool-description length budget and stale comments.

Not run locally. CI on this PR runs only CodeQL, so lint, tests and tsc aren't checked there.

Comment thread src/tools/ask-browserstack/stream.ts Outdated
body: AgentRequest | DecisionBody,
) => Promise<{ status: number; body: unknown }>;

export function fetchDeferredTransport(timeoutMs = 120_000): DeferredTransport {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[High] A successful approval can be reported as "never reached".

After the human allows, the rest of the agent run (the write plus everything after it) now happens inside this single POST: 120s timeout, no heartbeats. The stream it replaced allowed WHOLE_RUN_TIMEOUT_MS (30 min). On a timeout or proxy idle cutoff this throws AskError("could not be reached"). The outer catch then builds errorResult, which reports permission_relay.reason: "not_reached" ("The request never left this process"), even though the allow landed and the write may have gone through. That's the same "retry files it twice" risk the PR description says it fixes.

Suggestion: if the transport throws after a decision was sent, report it as a separate outcome: "decision sent, outcome unknown — do not retry". Also give the decision POST at least the whole-run timeout, or stream its reply. Worth a test: transport throws after an allow → result must not say not_reached.

product,
run_id: runId,
perm_id: ask.perm_id,
decision: "allow | deny — the user's answer, not your own judgement",

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[High] Following next_step exactly fails schema validation.

The instruction says to call again "with exactly these params" (product, run_id, perm_id, decision), but query is still a required z.string() in register.ts. A model that follows the instruction literally gets InvalidParams.

Suggestion: make query optional when run_id is present (.optional() plus a refine/superRefine), or add query to next_step.params. Add a test that calls the tool with exactly next_step.params.

// own `ASK_BROWSERSTACK_DISABLED`, which is read per call and so does not need a
// pod roll, and Atlas's `delegation.permission_relay`, which turns the relay off
// for every client at once from a config change.
return "deferred";

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[High / needs discussion] The hosted two-call flow removes both the human gate and the kill switch.

  • In the two-call flow the model supplies decision: "allow" itself. Once a user picks "always allow askBrowserStackAI" (common in most clients), every later approval goes through with no human.
  • This picks the two-call flow even for hosted clients that do declare elicitation, so they lose the fail-closed behaviour they had.
  • ASK_BROWSERSTACK_ALLOW_REMOTE_RELAY is deleted, so hosted writes turn on as soon as remote-mcp-server refreshes its lockfile. That's Rollout step 3, before Atlas#952 is deployed in step 5. Rollout step 4 in the PR body still says to set this flag, which no longer exists.

Suggestion: keep a default-off opt-in for the hosted two-call flow until #952 is live, prefer elicitation whenever the client declares it, and update the rollout steps.

Comment thread src/tools/ask-browserstack/register.ts Outdated
);
continue;
}
pending = ask;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[High] stdio against a pre-#952 Atlas hangs, then denies.

Phase 1 now waits for the stream to end before answering anything. An older Atlas keeps the stream open until it gets a decision, so both sides wait until the 300s gate denies. Then phase 2 POSTs a stale decision and gets a 404 or 409. Because the plan is "publish first", every stdio user with elicitation is broken until Atlas deploys. A 409 isn't handled explicitly either; it falls through to buildResult. And if a stream carries more than one permission frame, pending is overwritten, so only the last ask gets answered.

Suggestion: answer an ask as soon as it arrives on the stream (the old behaviour) and use the new decision-reply handling only once the stream has ended at a park. Alternatively, have Atlas echo contract back so the client can tell which version it's talking to.

"The `perm_id` the previous call returned, naming WHICH approval this " +
"answers. Send it with `run_id` and `decision`.",
),
decision: z

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Medium] .describe() text is over the budget.

The run_id, perm_id and decision descriptions run about 105–165 characters. The limit in CLAUDE.md / rules/tool-design.md is 60, with a hard ceiling of 120. Every client loads these at connect time. e.g. "run_id from a needs_approval result.".

Also, DESCRIPTION still says "If your client cannot show you a prompt, the run is read-only", which is no longer true in the hosted two-call flow.

Comment thread src/tools/ask-browserstack/register.ts Outdated
*
* When refused, Atlas runs read-only — a supported path that already works — and
* `permission_relay.reason` says `remote_mode` so nobody mistakes it for a human's no.
* `permission_relay.reason` names the reason so nobody mistakes it for a human's no.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Low] The doc block this line sits in (starting at line 259, ("HOSTED ONLY WHEN ITS OPERATOR OPTS IN…") no longer attaches to any function and describes the removed opt-in. Suggest deleting it or folding what's still true into relayMode.

Comment thread src/tools/ask-browserstack/stream.ts Outdated
url: string,
headers: Record<string, string>,
/**
* `null` means REATTACH: `GET /agent/{run_id}/stream`, which carries no body.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Low] This new "null means REATTACH" note contradicts the removal of reattach in this same PR, and the type doesn't allow null. Also:

  • RELAY_CONTRACT (line 68) was inserted between PERM_ID_PATTERN and its doc comment.
  • DeferredTransport has two stacked JSDoc blocks, and the old "Posts one decision" comment is still there.
  • There are no unit tests for fetchDeferredTransport (status/body pass-through, throw → AskError), though a test comment says they exist.

Comment thread src/tools/ask-browserstack/types.ts Outdated
@@ -146,9 +156,34 @@ export type RelayMode =
/** The client declares no `elicitation` capability, so nobody could be prompted. */
| "no_human"
/** This process is the hosted multi-tenant server, which cannot prompt a human. */

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Low] The old remote_mode doc comment is still here, directly above the new deferred JSDoc. More generally, the "X IS GONE" comments across src and tests (e.g. config.ts:54, relay.ts, stream.ts) describe history. That belongs in commit messages; the code comments should explain why the current code is the way it is.

Review on #445. Four correctness fixes and the comment cleanups; the hosted
opt-in question is left for a separate decision.

WORKS AGAINST BOTH SERVERS. Answering only after the stream closed assumed the
converged Atlas, which emits its asks as the stream ends. An Atlas that predates
it holds the stream OPEN waiting for the decision, so both sides waited until
the server's own five-minute gate denied an ask nobody had seen — and the user
got a refusal no human made. Each ask is now answered as it arrives, and the
reply says which server this is: a bare 204 carries nothing and the run
continues on the stream, anything else IS the continuation and the stream is
over. Both loops go through one `answerAsk`, so the two paths cannot drift.

That removes the deploy ordering constraint in both directions. With the
contract-2 guard already on the Atlas side, neither half now has to ship first.

A LOST REPLY IS NOT "NOTHING HAPPENED". The decision request carries the whole
continuation, so a timeout spans everything from "it never arrived" to "the
write landed and the answer was lost". Both paths reported `not_reached` —
"NOTHING WAS ASKED AND NOTHING WAS REFUSED" — which is the claim that gets the
same change made twice. There is a distinct `outcome_unknown` for it now, which
says not to retry. The deferred transport was also still bounded at 120s, sized
for a request that returned as soon as the run parked; it gets the whole-run
bound the streaming path has.

`next_step` CAN NOW BE FOLLOWED LITERALLY. It says to call again "with exactly
these params" — product, run_id, perm_id, decision — while `query` was a
required string, so a model doing exactly what it was told got InvalidParams
before the handler ran. `query` is optional in the schema and required by the
handler for a start, which is the only path that has anything for it to say.

ALSO: the three resume parameter descriptions ran 117–167 characters against a
60/120 budget, and every client loads them at connect time; they are 32–49 now,
with what they explained at length left to `next_step`, which arrives with the
values. The tool description no longer claims a client that cannot prompt makes
the run read-only, which stopped being true of the hosted path. An orphaned doc
block describing the removed opt-in is gone, along with four "X IS GONE" notes
and a `null` means REATTACH comment for a type that never allowed null.

686 tests before, 694 after; lint, prettier and tsc clean. The compatibility
test was checked against the previous code first — it times out there, which is
the deadlock it exists to catch.
@sarve-shreyas

Copy link
Copy Markdown
Collaborator Author

Thanks — this was a good catch list. All four correctness issues and the comment cleanups are fixed in 03cf2d1. #3 is left alone deliberately; reasons below.

1. A timeout after an allow reported as "never reached" — fixed

Both halves were wrong, not just the reporting:

  • fetchDeferredTransport was still bounded at 120s, sized back when the request returned as soon as the run parked. Since the converged gate the decision request carries the whole continuation, so it now gets WHOLE_RUN_TIMEOUT_MS, the same bound the streaming path has.
  • A throw is caught at the decision POST on both paths and returns a distinct outcome_unknown instead of falling through to not_reached. It says the approval was delivered, the outcome is unknown, and do not retry, with applied_before_stop: null — nobody measured it, which is not the same as measuring nothing.

Test as you suggested: transport throws after an allow → asserts reason is not not_reached, and that the detail never contains "nothing was changed", since that is the sentence that invites the retry.

2. next_step failing validation — fixed

query is .optional() in the schema and required by the handler for a start, which is the only path that has anything for it to say. A resume never carries one — the task is in Atlas's checkpoint. Starting with no query now gets a sentence it can act on rather than an SDK InvalidParams.

Two tests: one calls the tool with exactly next_step.params, one starts with no query and asserts the worded refusal.

4. stdio against a pre-#952 Atlas — fixed, and this is the important one

You were right about the mechanism, and the fix is the one you proposed. Each ask is answered as it arrives on the stream, and the reply says which server this is: a bare 204 carries nothing so the run continues on the stream; anything else is the continuation and the stream is already over. Both loops now go through one answerAsk, so the two paths cannot drift apart in how an approval is asked for or recorded.

I checked the new test against the previous code before trusting it — it times out at 10s there, which is exactly the deadlock you described, and passes with the fix.

This changes the rollout, and for the better. agent-platform#952 already handles old-client/new-Atlas via the contract check. With new-client/old-Atlas closed too, neither half has to ship first — the ordering constraint is gone in both directions. PR body updated to say so, and step 4's reference to the deleted flag is struck with an explanation of what replaced it.

Two small corrections on that comment, for the record:

  • "only the last ask gets answered" — phase 1 did overwrite pending, but the phase-2 loop re-derives the next ask from each decision reply, so earlier asks come back on the following round trip rather than being lost.
  • "a 409 isn't handled explicitly" — 404 is handled explicitly, and POST /agent/{run_id}/permission never returns 409; every 409 in Atlas is on workflow or browser routes.

Medium / low — fixed

  • Descriptions were 117–167 chars; now 32–49. What they explained at length is in next_step, which arrives with the values, where these are loaded by every client at connect time on every session.
  • DESCRIPTION no longer claims a client that cannot prompt makes the run read-only.
  • Gone: the orphaned opt-in doc block, the null means REATTACH note on a type that never allowed null, RELAY_CONTRACT sitting between PERM_ID_PATTERN and its doc, the stacked DeferredTransport JSDoc, the stale remote_mode line, and four "X IS GONE" notes.
  • fetchDeferredTransport now has the unit tests the comment claimed: status/body pass-through, raise_error: false, the timeout bound, and throw → AskError that names nothing internal.

3. The hosted flow — not changed, deliberately

Your facts are right: relayMode returns deferred unconditionally for REMOTE_MCP including clients that declare elicitation, and with the flag gone the hosted path switches on when remote-mcp-server refreshes its lockfile.

Leaving it as a decision rather than folding it into a fix, because re-adding an opt-in reverses a deliberate call and the "model supplies the decision" property is inherent to the two-call design rather than an accident of it. It is stated in RelayMode's own JSDoc: approval rides the client's own permission prompt, so a client configured not to prompt approves without a human; elicitation fails closed by construction, this fails closed only if the client asks. I have flagged it in the PR body as explicitly open. Happy to add a default-off gate if that is the call.


686 tests before, 694 after. tsc, eslint and prettier clean, npm run build exits 0. You are right that CI here only runs CodeQL — the test job shows skipping.

@ruturaj-browserstack ruturaj-browserstack left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed at 03cf2d1. Approving. Thanks for the thorough follow-up.

Verified locally: tsc, eslint and prettier are clean, and vitest passes 694/694.

Checked against both Atlas versions, for both deployment modes:

client Atlas main (today) Atlas #952
local stdio (elicitation) ✅ each ask is answered as it arrives, Atlas replies 204, and the run continues on the stream. parse_permission_relay checks only mode, and parse_decision ignores extra keys, so contract and product are harmless ✅ the decision reply carries the rest of the run
remote (REMOTE_MCP, two-call flow) ✅ safe: mode: "deferred" isn't "stream", so Atlas runs read-only and returns plain JSON. No hang, no writes ✅ needs_approval, then the resume finishes the run. It needs no sessions, so the stateless remote default is fine

So the "no deploy ordering" claim holds for both modes. No code changes are needed in remote-mcp-server: it references none of the changed names, and the optional query doesn't affect its tool schema changes.

Status of the earlier findings: #2, #4 and the cleanups are fixed. #1 is fixed for thrown errors, but one gap remains (inline, register.ts:369): a proxy 504 on the decision request still reports "nothing was changed". Please address it in a follow-up if not in this PR.

#3 (hosted gating): noted as a deliberate call. For whoever runs the rollout: hosted writes actually start when #952 is deployed with delegation_permission_relay="stream", not at the remote lockfile refresh. That knob is the staged switch, but it's global (it also stops stdio writes). ASK_BROWSERSTACK_DISABLED is the only hosted-only switch, and it removes the tool entirely.

}
// 204 MEANS AN OLDER SERVER: it acknowledged the decision and is continuing on the
// stream, so there is no continuation to read here and nothing to supersede.
if (response.status === 204) return {};

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Medium] A proxy timeout on the decision request still reads as "nothing was changed".

The new catch covers errors that are thrown. But if an ingress or proxy cuts this request (it now carries the whole rest of the run with no heartbeats, and a common proxy idle timeout is 60s), the client gets an HTTP status, e.g. 504 with an HTML body. That falls through to carried, then buildResult, where neverReachedAgent() sees a body that isn't a run result and reports not_reached: "no step ran… nothing was changed… run it again". Same retry risk as before, just arriving as a status instead of an exception.

Suggestion: on the decision paths, treat any response other than 404 that isn't a run result (!looksLikeDelegationResult(response.body)) as decisionUnknown(), and add a test where the decision reply is a 504 with an HTML body. Longer term, having Atlas stream the decision reply would give it heartbeats, the same reason phase 1 stays streamed.

Also, for the record: Atlas main does map decisions to 409 (DECIDE_ALREADY) and 403 (DECIDE_NOT_OWNER); #952 turns both into 404. Since this commit explicitly supports the pre-#952 Atlas, those statuses land here too, and a 403 would read as not_entitled. It's unlikely in practice, because each ask is answered once by the same user.

}
// Finished. Through the SAME builder the start path uses, so an entitlement refusal,
// a 401 and a plain failure read identically whichever call produced them.
return buildResult(response, [], mode, product);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same gap as the comment on register.ts:369: a 504 or other non-result reply from a proxy on the resume POST ends up here and is reported as not_reached ("nothing was changed"), not as decisionUnknown().

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Low] approvals: [] here (and in resumeFailed on the stdio 404 path) drops the record of what the human already approved in this call. On stdio runStreamed holds that trail, so passing approvals through would keep the audit record complete when the outcome is unknown.

sarve-shreyas added a commit that referenced this pull request Sep 30, 2026
…server.json

Re-pointed from 1.6.0. `main` reached 2.0.0 while this sat open (#447), so the
old target would have been a DOWNGRADE — and that is what the conflict on this
PR was: `package.json` 2.0.0 -> 1.6.0 and `server.json`'s server version
2.0.0 -> 1.0.0.

2.1.0 rather than 2.0.1 or 3.0.0: the change this ships (#445) adds optional
`run_id` / `perm_id` / `decision` arguments and loosens `query` from required to
optional. Nothing is removed and nothing that worked stops working, so it is a
minor.

Both `server.json` versions move together, which is how #447 left them.
@gaurav-singh-9227
gaurav-singh-9227 merged commit 8014f43 into main Sep 30, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants