Chat: stop a keystroke in the composer costing one pass over the thread - #9054
Conversation
Typing one character does work proportional to the whole thread, and none of it
is rendering. assistant-ui gives the client one notification manager, useAuiState
subscribes every caller to it through useSyncExternalStore, and the selector IS
the getSnapshot, so one store write runs every selector in the tree. The composer
is controlled off store state, so every character is a store write.
Instrumented at the notification manager on the heavy-thread fixture:
20 messages 955 subscriptions 1,020 selector runs per keystroke
80 messages 3,726 subscriptions 3,791 selector runs per keystroke
220 messages 10,193 subscriptions 10,258 selector runs per keystroke
Layout and style recalculation are flat over the same range (1.8ms to 2.1ms
layout, 1.7ms to 1.6ms style), so the fan-out, not the DOM, is what grows.
Three of those subscriptions were minted more often than the question they ask:
* the render_html presence scan lived in the markdown BLOCK component, so one
subscription per block, 800 of the 10,193 at 220 messages, each re-scanning
message.parts per keystroke. It is a property of the message: asked once per
message part now and read from context by the blocks.
* the continue bar mounts under every assistant message and returns null unless
that message is the newest, but its ten subscriptions all ran first. It now
subscribes to isLast alone and delegates the rest to the newest message.
* the composer's "has this thread used research" gate walked every message and
read metadata through the state proxy on every keystroke, to return the same
boolean. Cached per message array, which is the revision the answer depends
on, in the shape research-reply-owners.ts uses for the sibling question.
Measured with tests/studio/playwright_heavy_thread.py's fixture at 220 messages:
10,193 subscriptions to 8,523 and 10,258 selector runs per keystroke to 8,588,
both exactly reproducible. The wall-clock keystroke stall does not resolve that
on a shared host: over four paired interleaved rounds at 300K characters the
round-to-round spread is larger than the change on all three engines, so this
lands as less work per keystroke rather than as a stall number.
Behaviour is unchanged: the same blocks render the same artifact cards, the
newest message keeps the whole continue bar, and the research gate answers what
the scan answered, including that only a string run id counts.
The header cited tests/studio/playwright_heavy_thread.py and thread-delete-render-budget.test.ts, neither of which is on main or on this branch, so a reader could not run either. Describe the heavy-thread fixture inline and cite the two sibling budget tests that do exist. Also say plainly that the subscription counts in the header are the motivation rather than an assertion, since the file pins source shape and a regression in the counts alone would not fail it, and note the two cases where the research presence cache simply misses.
|
@codex review |
|
Codex Review: Didn't find any major issues. Delightful! Reviewed commit: ℹ️ About Codex in GitHubCodex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback". |
|
@codex review |
|
Codex Review: Didn't find any major issues. Keep it up! Reviewed commit: ℹ️ About Codex in GitHubCodex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback". |



What is slow, and why it is not the paint
Typing in the composer gets slower as the thread grows: PR #9016's harness reads the keystroke's longest stall at 32 ms with 25K characters of thread content and 157 ms at 300K, 4.9x, and the same curve appears on WebKit (4.1x) and Firefox (6.7x). Keystroke-to-paint does not move at all: it sits on the two-vsync floor at every size, so the character still paints on the next frame. What grows is the work that follows the paint.
It is not the DOM either. Chromium's own counters over the same range are flat: layout 1.8 ms to 2.1 ms, style recalculation 1.7 ms to 1.6 ms, while the keystroke's total task time goes 88 ms to 249 ms for five keystrokes.
It is the store fan-out. assistant-ui gives the client ONE notification manager.
useAuiStatesubscribes every caller to it throughuseSyncExternalStore, and the selector IS the getSnapshot, so a single store write runs every selector in the tree. The composer is controlled off store state, so every character typed is a store write. Instrumented at the notification manager, on the heavy-thread fixture:That is about 46 subscriptions per message, and it is linear in thread length by construction: each message mounts its own set. 61% of them are this repo's components and 39% are assistant-ui's own primitives (
MessagePrimitive.Root,MessageIf,ThreadPrimitiveViewportSlack,useMessagePartTextand friends).What this changes
Three of those subscriptions were minted more often than the question they ask. None of this changes what renders.
message.parts.some(isRenderableRenderHtmlToolPart)is a property of the MESSAGE, but the component asking it is mounted once per markdown block: 800 of the 10,193 subscriptions at 220 messages, each re-scanning the parts array on every keystroke. It is now asked once per message part, above the blocks, and read from context by them. There is exactly oneBlockComponentin the tree and the provider always wraps it.useAuiStatecalls ran first, and one of them walks the thread looking for audio input. It now subscribes toisLastalone and delegates to the full bar on the newest message. The component has no state and no effects, so the gate is the same condition asked before the work instead of after it.thread.messages.some(...)reading metadata through the state proxy, returning the same boolean, once per character. Cached on the message array, which is the revision the answer depends on, in the shaperesearch-reply-owners.tsuses for the sibling per-message question.Measured
Deterministic, at 220 messages: 10,193 subscriptions to 8,523 (-16.4%) and 10,258 selector runs per keystroke to 8,588 (-16.3%), reproduced exactly across runs.
Wall clock does not resolve that on this shared host. Four paired, interleaved rounds of PR #9016's harness, five repetitions per cell, 25K and 300K, Chromium plus WebKit plus Firefox: the keystroke stall medians at 300K move 239 to 238 ms (Chromium), 173 to 136 ms (WebKit), 268 to 242 ms (Firefox), and the round-to-round spread within each arm is larger than any of those differences (Chromium base alone reads 239, 245, 239, 240; Firefox base 216, 293, 258, 278). 16% off a term that is itself roughly half of the growth is about 8%, which is under the noise here. So this lands as less work per keystroke, demonstrated by counting it, not as a stall number. Delete and menu are unchanged, as expected: nothing here is on those paths.
Honest limit
This does not make typing flat, and nothing in this repo can. That is worth stating precisely, because it bounds what this PR can be asked to do. In the installed
@assistant-ui/store@0.2.9,useAuiStateisuseSyncExternalStore(aui.subscribe, () => selector(proxiedState), ...)with no scope argument, andNotificationManager.notifySubscribers()is a barefor (const cb of subscribers)over one flatSet. So a store write runs every selector in the tree, and no change on this side of the boundary can make a single notification cheap on a large thread. Upstream assistant-ui #3952 does not scope the notification either; it removes per-message top-anchor subscriptions and aparseCssLengthforced reflow, which reduces the same fan-out linearly. Notification scoping is a 0.15.x concern and a separate decision. The fan-out is one subscription per message by construction and 39% of them belong to assistant-ui's own primitives, so the slope can be reduced but not removed without either notification scoping upstream or not mounting off-screen messages at all.Tests
studio/frontend/tests/thread-research-presence.test.tscovers the gate's semantics first and the cache second, because a memoization bug there silently changes whether deep research is offered.studio/frontend/tests/composer-keystroke-subscription-budget.test.tspins the two wiring seams, which are invisible in the rendered output, the waychat-autoscroll-frame-budget.test.tsanddrag-costs-no-render.test.tsdo.Each test was run against a deliberately broken tree:
npm run typecheck,npm test(2,946 pass),npm run buildall green. Repo tests (CPU) fails on the stale prompt-queue contract that PR #9026 repairs, unrelated to this change.Relation to the open perf PRs
PR #9014 (part components at module scope) and PR #9042 (message slot plus
researchReplyOwners) both cut per-render work, and neither touches this: a keystroke re-renders nothing in the thread, so the fan-out is paid whole either way. PR #9042 does already fixuseOwnsResearchMessage, the other per-message export scan, with the same per-revision cache shape used here, so that one is deliberately not duplicated in this PR.