Chat: stop a keystroke in the composer costing one pass over the thread by danielhanchen · Pull Request #9054 · unslothai/unsloth · GitHub
Skip to content

Chat: stop a keystroke in the composer costing one pass over the thread - #9054

Merged
danielhanchen merged 5 commits into
mainfrom
perf-keystroke-fanout
Aug 17, 2026
Merged

Chat: stop a keystroke in the composer costing one pass over the thread#9054
danielhanchen merged 5 commits into
mainfrom
perf-keystroke-fanout

Conversation

@danielhanchen

@danielhanchen danielhanchen commented Aug 17, 2026

Copy link
Copy Markdown
Member

What is slow, and why it is not the paint

Typing in the composer gets slower as the thread grows: PR #9016's harness reads the keystroke's longest stall at 32 ms with 25K characters of thread content and 157 ms at 300K, 4.9x, and the same curve appears on WebKit (4.1x) and Firefox (6.7x). Keystroke-to-paint does not move at all: it sits on the two-vsync floor at every size, so the character still paints on the next frame. What grows is the work that follows the paint.

It is not the DOM either. Chromium's own counters over the same range are flat: layout 1.8 ms to 2.1 ms, style recalculation 1.7 ms to 1.6 ms, while the keystroke's total task time goes 88 ms to 249 ms for five keystrokes.

It is the store fan-out. assistant-ui gives the client ONE notification manager. useAuiState subscribes every caller to it through useSyncExternalStore, and the selector IS the getSnapshot, so a single store write runs every selector in the tree. The composer is controlled off store state, so every character typed is a store write. Instrumented at the notification manager, on the heavy-thread fixture:

messages store subscriptions selector runs per keystroke time inside notifySubscribers
20 955 1,020 1.8 ms
80 3,726 3,791 7.2 ms
220 10,193 10,258 19.6 ms

That is about 46 subscriptions per message, and it is linear in thread length by construction: each message mounts its own set. 61% of them are this repo's components and 39% are assistant-ui's own primitives (MessagePrimitive.Root, MessageIf, ThreadPrimitiveViewportSlack, useMessagePartText and friends).

What this changes

Three of those subscriptions were minted more often than the question they ask. None of this changes what renders.

  • The render_html presence scan lived in the markdown BLOCK component. message.parts.some(isRenderableRenderHtmlToolPart) is a property of the MESSAGE, but the component asking it is mounted once per markdown block: 800 of the 10,193 subscriptions at 220 messages, each re-scanning the parts array on every keystroke. It is now asked once per message part, above the blocks, and read from context by them. There is exactly one BlockComponent in the tree and the provider always wraps it.
  • The continue bar subscribed ten times to find out it should not render. It mounts under every assistant message and returns null unless that message is the newest, but the ten useAuiState calls ran first, and one of them walks the thread looking for audio input. It now subscribes to isLast alone and delegates to the full bar on the newest message. The component has no state and no effects, so the gate is the same condition asked before the work instead of after it.
  • The composer's research gate walked every message per keystroke. thread.messages.some(...) reading metadata through the state proxy, returning the same boolean, once per character. Cached on the message array, which is the revision the answer depends on, in the shape research-reply-owners.ts uses for the sibling per-message question.

Measured

Deterministic, at 220 messages: 10,193 subscriptions to 8,523 (-16.4%) and 10,258 selector runs per keystroke to 8,588 (-16.3%), reproduced exactly across runs.

Wall clock does not resolve that on this shared host. Four paired, interleaved rounds of PR #9016's harness, five repetitions per cell, 25K and 300K, Chromium plus WebKit plus Firefox: the keystroke stall medians at 300K move 239 to 238 ms (Chromium), 173 to 136 ms (WebKit), 268 to 242 ms (Firefox), and the round-to-round spread within each arm is larger than any of those differences (Chromium base alone reads 239, 245, 239, 240; Firefox base 216, 293, 258, 278). 16% off a term that is itself roughly half of the growth is about 8%, which is under the noise here. So this lands as less work per keystroke, demonstrated by counting it, not as a stall number. Delete and menu are unchanged, as expected: nothing here is on those paths.

Honest limit

This does not make typing flat, and nothing in this repo can. That is worth stating precisely, because it bounds what this PR can be asked to do. In the installed @assistant-ui/store@0.2.9, useAuiState is useSyncExternalStore(aui.subscribe, () => selector(proxiedState), ...) with no scope argument, and NotificationManager.notifySubscribers() is a bare for (const cb of subscribers) over one flat Set. So a store write runs every selector in the tree, and no change on this side of the boundary can make a single notification cheap on a large thread. Upstream assistant-ui #3952 does not scope the notification either; it removes per-message top-anchor subscriptions and a parseCssLength forced reflow, which reduces the same fan-out linearly. Notification scoping is a 0.15.x concern and a separate decision. The fan-out is one subscription per message by construction and 39% of them belong to assistant-ui's own primitives, so the slope can be reduced but not removed without either notification scoping upstream or not mounting off-screen messages at all.

Tests

studio/frontend/tests/thread-research-presence.test.ts covers the gate's semantics first and the cache second, because a memoization bug there silently changes whether deep research is offered. studio/frontend/tests/composer-keystroke-subscription-budget.test.ts pins the two wiring seams, which are invisible in the rendered output, the way chat-autoscroll-frame-budget.test.ts and drag-costs-no-render.test.ts do.

Each test was run against a deliberately broken tree:

break tests that fail
block subscribes to the store again markdown block reads from context
provider removed scan happens once per message part
continue bar gate removed continue bar subscribes once; newest message gets the whole bar
bar keeps its gate but loses the audio-input check newest message gets the whole continue bar
composer scans inline again composer asks through the cache
cache lookup removed answer is cached on the message array
cache keyed globally instead of per array four of the five presence tests
truthy run id instead of a string one only a string run id counts

npm run typecheck, npm test (2,946 pass), npm run build all green. Repo tests (CPU) fails on the stale prompt-queue contract that PR #9026 repairs, unrelated to this change.

Relation to the open perf PRs

PR #9014 (part components at module scope) and PR #9042 (message slot plus researchReplyOwners) both cut per-render work, and neither touches this: a keystroke re-renders nothing in the thread, so the fan-out is paid whole either way. PR #9042 does already fix useOwnsResearchMessage, the other per-message export scan, with the same per-revision cache shape used here, so that one is deliberately not duplicated in this PR.

danielhanchen and others added 3 commits August 17, 2026 02:15
Typing one character does work proportional to the whole thread, and none of it
is rendering. assistant-ui gives the client one notification manager, useAuiState
subscribes every caller to it through useSyncExternalStore, and the selector IS
the getSnapshot, so one store write runs every selector in the tree. The composer
is controlled off store state, so every character is a store write.

Instrumented at the notification manager on the heavy-thread fixture:

  20 messages     955 subscriptions   1,020 selector runs per keystroke
  80 messages   3,726 subscriptions   3,791 selector runs per keystroke
  220 messages 10,193 subscriptions  10,258 selector runs per keystroke

Layout and style recalculation are flat over the same range (1.8ms to 2.1ms
layout, 1.7ms to 1.6ms style), so the fan-out, not the DOM, is what grows.

Three of those subscriptions were minted more often than the question they ask:

  * the render_html presence scan lived in the markdown BLOCK component, so one
    subscription per block, 800 of the 10,193 at 220 messages, each re-scanning
    message.parts per keystroke. It is a property of the message: asked once per
    message part now and read from context by the blocks.
  * the continue bar mounts under every assistant message and returns null unless
    that message is the newest, but its ten subscriptions all ran first. It now
    subscribes to isLast alone and delegates the rest to the newest message.
  * the composer's "has this thread used research" gate walked every message and
    read metadata through the state proxy on every keystroke, to return the same
    boolean. Cached per message array, which is the revision the answer depends
    on, in the shape research-reply-owners.ts uses for the sibling question.

Measured with tests/studio/playwright_heavy_thread.py's fixture at 220 messages:
10,193 subscriptions to 8,523 and 10,258 selector runs per keystroke to 8,588,
both exactly reproducible. The wall-clock keystroke stall does not resolve that
on a shared host: over four paired interleaved rounds at 300K characters the
round-to-round spread is larger than the change on all three engines, so this
lands as less work per keystroke rather than as a stall number.

Behaviour is unchanged: the same blocks render the same artifact cards, the
newest message keeps the whole continue bar, and the research gate answers what
the scan answered, including that only a string run id counts.
The header cited tests/studio/playwright_heavy_thread.py and
thread-delete-render-budget.test.ts, neither of which is on main or on this
branch, so a reader could not run either. Describe the heavy-thread fixture
inline and cite the two sibling budget tests that do exist.

Also say plainly that the subscription counts in the header are the motivation
rather than an assertion, since the file pins source shape and a regression in
the counts alone would not fail it, and note the two cases where the research
presence cache simply misses.
@danielhanchen

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Delightful!

Reviewed commit: def7decea8

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@danielhanchen

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep it up!

Reviewed commit: eed39d6585

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

danielhanchen pushed a commit to danielhanchen/unsloth-staging-2 that referenced this pull request Aug 17, 2026
danielhanchen pushed a commit to danielhanchen/unsloth-staging-2 that referenced this pull request Aug 17, 2026
@danielhanchen

Copy link
Copy Markdown
Member Author

@danielhanchen
danielhanchen merged commit 0f62511 into main Aug 17, 2026
38 of 39 checks passed
@danielhanchen
danielhanchen deleted the perf-keystroke-fanout branch August 17, 2026 10:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant