Resume Max Tokens replies outside the mounted message UI

When a response reaches the token limit in a background thread, this effect does not exist because ContinueMessageBarForLastMessage is mounted only for the currently rendered thread. This is especially harmful for the prompt queue, which explicitly continues advancing background threads: later queued prompts make the truncated reply no longer isLast, so returning to the thread cannot trigger this effect and the reply is never finished. Automatic continuation needs to run from the thread/runtime completion path rather than solely from this visible message component.

Useful? React with 👍 / 👎.

Turning this one down. It describes a limitation that predates the PR, and the remedy is an architectural preference rather than a bug.

The gates the automatic path uses are literally the gates the manual bar already used. On origin/main, ContinueMessageBar bails at thread.tsx:6448 on !isLast before it subscribes to anything, and the body gate at 6500 repeats it. The PR only lifted those into resumable. So a truncated reply in a background queued thread had no Continue affordance on main either, and once later queued prompts land it stops being isLast and never gets one. That is the documented "Newest turn only: appending to an older one would strand the replies after it" design, not something this change introduces.

Which means nothing is worse after the PR than before it. No reply is lost: the truncated text is still in the transcript, still branchable, still retryable, exactly as it was. The PR strictly adds automatic resumption to the case that was already served, and takes nothing away from the case that was not.

I checked isLast is per selected branch path rather than per thread (@assistant-ui/core@0.1.17, thread-runtime.js:248, isLast: messages.at(-1)?.id === message.id), and that the prompt queue deliberately subscribes to runningByThreadId instead of aui.thread() so queues keep advancing in the background (the module-scope comment at thread.tsx:320-323). So I understand the shape you are pointing at.

Moving continuation into the thread or runtime completion path is a reasonable thing to want, and it would extend the feature to background threads. But it is a new capability on top of a newest-turn-only design, not a correctness fix, and it is a much larger surface than this PR. Worth its own change if background auto-continuation is wanted; not a defect here.

Make automatic continuation idempotent under StrictMode

When an already-truncated message first mounts—for example after loading a saved thread—autoContinuing is true on the initial render, and the app's <StrictMode> in src/main.tsx replays this effect. Both setups retain the same autoContinuing value, so each records a round and calls startRun; the module-level map is not rechecked inside the effect. The local assistant runtime does not reject a second startRun while one is active, so development builds can issue two concurrent continuations, consume two budget slots, and potentially incur duplicate provider requests.

Useful? React with 👍 / 👎.

Confirmed. The deps are [autoContinuing, parentId, startContinuation], all identical across StrictMode's replay, and nothing inside re-checked, so both passes recorded a round and called startRun.

Worth noting your point about the module-level map: rechecking it inside the effect would not have fixed this, because one recorded round still leaves the limit of three unspent. A ref keyed on the message id is what actually makes it idempotent, and refs survive StrictMode's simulated remount on the same fiber.

Fixed that way, and the run is skipped when the ref already names this message. Typecheck is clean.

Track continued messages across component remounts

The new ref prevents the StrictMode replay only while this fiber survives; it does not make a message idempotent across real unmounts. For example, after A1 automatically creates a completed sibling, select the still-truncated A1 branch, leave the chat, and return: the ref is reset while the parent still has budget remaining, so this effect automatically continues A1 again, creating another sibling and another potentially paid provider request. Remembering all automatically continued message IDs for the session, rather than one ID in a component-local ref, would prevent revisiting a truncated branch from rerunning it.

Useful? React with 👍 / 👎.

Correct, and your reproduction is exact: the ref dies with the component, so the claim only covered the StrictMode replay it was written for, not a real unmount. Select the still-truncated branch, leave, come back, and the parent still has budget, so it fires again.

claimAutoContinue(messageId) now records in module scope and returns false for every later caller. It is deliberately separate from spent: that counts rounds per logical turn and is what bounds a runaway loop, whereas this answers whether THIS message has already been resumed. Conflating them would have been wrong in the other direction, since one recorded round still leaves the limit of three unspent, which is exactly why rechecking the budget inside the effect would not have fixed the StrictMode case either.

resetAutoContinue() clears both, so the existing test seam is unchanged.

Five tests: claimed exactly once, survives a simulated remount, tracked per message, a missing id refused rather than claimed, and cleared by a full reset. 4102 frontend tests pass and typecheck is clean.

Stop showing the spinner when the message is already claimed

When a previously auto-continued truncated branch is selected again or remounted, shouldAutoContinue still makes autoContinuing true while the new module-level claimAutoContinue(messageId) correctly returns false, so no run starts. This condition nevertheless renders “Continuing automatically” indefinitely and hides the manual Continue action. The fresh evidence versus the earlier duplicate-request report is the new persistent claim: it prevents the request but is not reflected in the render condition.

Useful? React with 👍 / 👎.

Right, and the reason is exactly the one you name: the claim is per message, the render is not. Fixed in 4cd046b.

autoContinuing is computed only from shouldAutoContinue, whose last line is (spent.get(key) ?? 0) < limit keyed on the PARENT. After one round spent is 1 against AUTO_CONTINUE_LIMIT of 3, so the turn still says yes while claimAutoContinue(messageId) says no. The effect bails, the render at if (autoContinuing) does not, and nothing in the component ever re-evaluates, so the spinner is terminal and the handleContinue bar underneath is unreachable.

Reachable without a reload, and I checked this against the pinned runtime rather than assuming it. The continuation runs as a sibling and @assistant-ui/core@0.1.17 then switches to it (local-thread-runtime-core.js:298, resetHead with the comment "Switch to the new message branch right after adding it for the first time"), so the truncated partial becomes a non-selected branch that BranchPicker reaches on every assistant message. Clicking back calls switchToBranch, head becomes the truncated message again, isLast is true again (thread-runtime.js:248), every other resumable gate holds, and you get the permanent spinner. Leaving the chat and returning is a real remount and lands the same way.

One detail that ruled out a cheaper fix: ThreadPrimitive.Messages keys children by index and MessageByIndex memoises on index, so a branch switch does not unmount the component, it just hands the same instance a different message id. Any per-mount ref marker would be stale across the switch and would keep the spinner stuck. The gate has to be keyed on the message id.

So there is now shouldAutoContinueMessage(messageId, ...), which returns false for an already-claimed message and otherwise defers to shouldAutoContinue, and the render reads that. Effect and render agree, and a claimed message falls through to the ordinary Continue bar. No flash on the mount that does fire: startRun reaches resetHead plus _notifySubscribers synchronously before its first await, so by the next render this message is no longer isLast.

Three cases added to tests/chat-continuation.test.ts, including that a claim on one message does not silence another. Guard check: with the wasAutoContinued check removed the new test fails (actual true, expected false). typecheck clean, npm test 4105 passed.

Coordinate automatic claims across browser tabs

When the same saved thread is loaded in two browser tabs while its last reply has reason: "length", each tab has its own module instance of continued (and spent), so both tabs successfully claim the same message and call startRun. The runtime's running state is also per-tab, so it cannot prevent this race; the result is duplicate continuation branches and potentially multiple paid provider requests. Store the claim in a cross-tab/backend-coordinated lease rather than only in module memory.

Useful? React with 👍 / 👎.

Confirmed. The effect fires on a fresh mount, not only on the tick a stream ends, and the comment above it says so from an earlier round: leaving the chat with a truncated branch selected and returning fired it again and created another sibling and another paid request. A second tab is exactly that fresh mount with an empty continued set, so both tabs claim the same message and both call startContinuation.

Single-tab reload is not affected, since a continued message is no longer the newest turn and resumable gates on that. The two-tab case races before either continuation exists, which is what module memory cannot see.

This PR is already merged, so the cross-tab lease is going out as a follow-up PR rather than more commits here, and I will link it back to this thread.

Follow-up PR: #9425

The claim is now backed by a localStorage lease, a random token written then read back so two tabs that both saw the slot free leave exactly one winner, with a 2 minute TTL so a tab that dies mid-run cannot wedge the message. BroadcastChannel was rejected: it does not replay for a tab that opens later, it needs an async handshake the synchronous render decision cannot make, and it dies with its tab so it cannot express an expiring claim. Storage that is absent or throws falls back to today's module-only claim.

Verified independently: the 11 new cases all fail against the previous continuation.ts and pass with the lease, and the whole frontend suite is green.

One thing that PR deliberately does not do: the round budget is still per tab, so budgets are not merged across tabs. The lease keeps a second tab off the same message, which is the duplicate-request bug here.