Chat: stop a message delete from re-rendering the whole thread by danielhanchen · Pull Request #9042 · unslothai/unsloth · GitHub
Skip to content

Chat: stop a message delete from re-rendering the whole thread - #9042

Merged
danielhanchen merged 3 commits into
mainfrom
perf-thread-delete
Aug 17, 2026
Merged

Chat: stop a message delete from re-rendering the whole thread#9042
danielhanchen merged 3 commits into
mainfrom
perf-thread-delete

Conversation

@danielhanchen

Copy link
Copy Markdown
Member

What this is

Bottleneck 3 from the heavy-thread profiling in #9016: deleting a message from a long chat. It cost 98ms at 25K characters of thread content and 472ms at 300K, growing 4.8x across that range, of which 503ms of the 505ms measured there was script and 1.9ms was style.

The attribution was wrong, and here is the profile

The cost was attributed to the export / rebuild / import round trip in delete-thread-message.ts: export the thread, rebuild it without the message, import it back. That is not where the time goes.

Sampled with the CDP profiler on Chromium at 300K characters (220 messages, 43,422 DOM nodes), inclusive time for one delete of 498ms wall:

deleteThreadMessage                    1.8 ms
syncExportedRepositoryToBackend        1.3 ms
exportedItemToRecord                   0.9 ms

performSyncWorkOnRoot                414.1 ms
  renderRootSync                     299.5 ms
  commitRoot                         114.5 ms
  flushPassiveEffects                 83.4 ms

The whole round trip is under 2ms. The rest is one synchronous React render, in a microtask after the click handler returns, and it is that big because deleting one message re-rendered every remaining message in the thread. Two things caused that, both in thread.tsx.

1. The components map defeats assistant-ui's bail-out

ThreadPrimitive.Messages rebuilds its whole element array whenever the message COUNT changes, so every message's wrapper re-renders on a delete. assistant-ui absorbs that in RenderChildrenWithAccessor: when the render prop returns an element with no props, the same element object is handed back on every render, and React skips reconciling that subtree.

The components={{ UserMessage, EditComposer, AssistantMessage }} form never reaches it. What the render prop returns there is <ThreadMessageComponent components={...} />, whose props object is freshly allocated every render, so the memo can never hit and the whole message body, action bar and tooltip tree is reconciled again for all 219 remaining messages.

This PR renders through the children form and returns one shared propless element. The wrapper still re-renders; what is under it does not. Sharing the element rather than building a fresh propless one also makes the bail-out React's own (current.memoizedProps === pendingProps), so it does not depend on a library detail staying put.

The role and edit-state selectors are exactly the ones assistant-ui's ThreadMessageComponent uses, and threadMessageKind reproduces its getComponent fallback chain for the three components this thread supplies, across all six role x editing combinations. An editing message of any role goes to EditComposer; a system message that is not being edited renders nothing, as it always has.

2. useOwnsResearchMessage re-rendered every action bar, and exported per message

const messages = useAuiState(({ thread }) => thread.messages);
...
return aui.thread().export().messages.some(...)

Selecting the array subscribed every user message's action bar to every thread change, so a delete re-rendered all of them along with their copy, edit, fork and delete controls and their tooltips, whether or not the answer had moved. Behind that, each one exported the whole repository, which is quadratic in thread length and is paid on every token of a generation as well.

It now selects the answer, so nothing re-renders unless the answer changes, and the export is shared across the messages at one revision. The export itself stays: the question is about the whole repository, since a research reply can sit on a branch the view is not showing, and it cannot be read off the visible message list. What is removed is the per-message factor, not the export.

Measured

Harness is tests/studio/playwright_heavy_thread.py from #9016, unchanged, driven against this branch and against its merge base. Paired and interleaved (base, fix, base, fix, ...), because this host's load drifts over tens of minutes and two back-to-back blocks would charge that drift to the change. Every cell is itself the median of 3 repetitions; the round-level number is the median across rounds. Ratios are the headline, absolute milliseconds are context: this is a loaded shared host and the harness runs against a vite dev server, so React is unminified and unbundled.

Chromium, 3 interleaved rounds:

thread content before after ratio
25K chars (20 messages) 98.3 ms 25.9 ms 3.80x
100K chars (80 messages) 206.5 ms 34.7 ms 5.95x
300K chars (220 messages) 472.3 ms 63.5 ms 7.44x

Growth across the range: 4.80x before, 2.45x after.

WebKit, 2 interleaved rounds:

thread content before after ratio
25K chars 61.5 ms 30.0 ms 2.05x
100K chars 186.0 ms 38.5 ms 4.83x
300K chars 472.5 ms 56.5 ms 8.36x

Growth: 7.68x before, 1.88x after.

Firefox, 1 round:

thread content before after ratio
25K chars 95 ms 32 ms 2.97x
100K chars 262 ms 50 ms 5.24x
300K chars 615 ms 58 ms 10.60x

Growth: 6.47x before, 1.81x after.

Two things to read these against. The harness clocks this action across a double rAF, so roughly 33ms of every number above is the vsync floor rather than work; subtracting it, the Chromium 300K cell goes from about 439ms to about 30ms. And one base cell (Chromium, 25K, round 3) timed out on page load and is excluded; its column is the median of the other two rounds.

Nothing else in the harness moved: keystroke, scroll and jump are flat, and menu and reopen are unchanged within run-to-run noise. Those two are separate bottlenecks and are not touched here.

Same output, not just faster: DOM node counts are identical at every size (4,048 / 15,887 / 43,422 before and after), message counts before and after the delete are identical, and both trees report zero console warnings and zero stray API requests.

Tests

Six new tests over the two extracted modules, plus four source pins, all under npm test.

tests/thread-message-slot.test.ts covers the component choice across all six role x editing combinations, and the two properties the bail-out needs: the render prop returns the same element object every time, and that element carries no props.

tests/research-reply-owners.test.ts covers what the shared answer contains, that a rootless reply names no owner, that one export serves every message at a revision, and that a new revision is exported again and sees the change.

tests/thread-delete-render-budget.test.ts pins the three seams that are silently load-bearing, following the existing research-render-budget.test.ts idiom. Undo any one of them and the thread still renders correctly, the unit tests above still pass, and the delete goes back to being linear in thread length: the render-prop form rather than the map, the render prop being hoisted to module scope, and the revision key being the array the store hands out rather than a copy of it.

Each of the 13 tests was run against a deliberately broken tree and shown to fail. Per test, the break it catches:

test broken tree it catches
editing wins over role, for every role role checked before editing
a message that is not being edited goes to its role's component user and assistant swapped
a system message that is not being edited renders nothing system falls through to the assistant body
the slot hands back one shared element rather than a new one per render slot builds a fresh element per call
the slot's element carries no props slot puts a prop on the element
collects the parents of research replies and nothing else every reply counted, research or not
a rootless research reply names no owner rootless replies counted as owned
one export serves every message at the same revision cache dropped; cache keyed on one key for every revision
a new revision is exported again, and sees the change cache keyed on one key for every revision
the message list is rendered through a render prop, not a components map thread.tsx back on the components map; render prop written inline
the render prop is built once, at module scope hoisted render prop rebuilds its element per call
ThreadMessage sends each kind to the component that names it switch sends user messages to the assistant body
research-reply ownership is selected as an answer, not as the message list revision key copied instead of passed through; ownership back to selecting the message array

tests/studio/test_deep_research_frontend_contract.py pinned the exact expression the ownership question used to be written as. Repointed at the question and its scope, not at the expression that used to compute it.

Verification

From studio/frontend: npm test 2949 passed 0 failed, npm run typecheck clean, npm run build clean, biome and eslint on the changed files produce byte-identical diagnostics to the merge base for thread.tsx and nothing at all for the new files.

pytest tests/studio is 3900 passed, 1 failed. The one failure is test_multi_chat_prompt_queue_contract.py::test_composer_only_queues_behind_the_current_chat, which fails identically on the merge base and is fixed by #9026.

Not done here, deliberately

MessagePrimitive.Parts in the same file is passed an inline components literal whose tools object is reallocated per render, and assistant-ui memoizes each part on that object's identity, so that memo can never hit either. It is the same mistake one level down. It does not affect a delete any more, because the message subtree no longer re-renders, so it belongs with the menu and reopen work rather than here.

Deleting one message from a long chat cost 98ms at 25K characters of thread
content and 472ms at 300K, growing 4.8x across that range. The cost was
attributed to the export / rebuild / import round trip in
delete-thread-message.ts. Profiling says otherwise: sampled on Chromium at 300K
characters (220 messages, 43,422 DOM nodes), deleteThreadMessage is 1.8ms of a
498ms delete, and the whole of syncExportedRepositoryToBackend under it is
1.3ms. The other 400ms is one synchronous React render, and the reason it is
that big is that deleting a message re-rendered every remaining message.

Two things caused that, both here rather than in the repository code.

ThreadPrimitive.Messages rebuilds its element array whenever the message COUNT
changes, so every message's wrapper re-renders on a delete. assistant-ui absorbs
that with a bail-out: an element with no props is memoized and the same object
is handed back, and React skips reconciling it. The `components={{...}}` form
never reaches it, because what the render prop returns there is
<ThreadMessageComponent components={...} />, whose props object is freshly
allocated on every render. Rendering through the children form, returning one
shared propless element, means the wrapper re-renders and the message body,
action bar and tooltips underneath it do not. The role and edit-state selectors
are the ones assistant-ui's own ThreadMessageComponent uses, and the component
choice reproduces its getComponent fallback chain for the three components this
thread supplies.

useOwnsResearchMessage selected `thread.messages`, which subscribed every user
message's action bar to every thread change, and then answered its question by
exporting the whole repository once per message. It now selects the answer, so a
change that does not move the answer re-renders nothing, and the export is
shared across the messages at one revision instead of repeated per message. The
question needs the repository rather than the visible list, since a research
reply can sit on a branch the view is not showing, so the export stays.

Measured with the heavy-thread harness from #9016, chromium, three interleaved
A/B rounds, each cell the median of 3 repetitions:

    25K chars    98.3ms -> 25.9ms
    100K chars  206.5ms -> 34.7ms
    300K chars  472.3ms -> 63.5ms

Growth across the range falls from 4.80x to 2.45x. Firefox at 300K goes 615ms ->
58ms. DOM node counts, message counts before and after the delete, console
warnings and stray requests are identical at every size, so what changed is how
much of the tree React walks, not what it produces.

test_deep_research_frontend_contract pinned the exact expression the ownership
question used to be written as. It is repointed at the question and its scope
rather than at the expression.
rhsCZ pushed a commit to rhsCZ/unsloth that referenced this pull request Aug 16, 2026
The ThreadPrimitive.Messages half of this change is reverted. Hoisting
the map there cannot help, and it collides with unslothai#9042 which fixes that
call properly.

Reading the primitive settles it. Given a components map, assistant-ui
builds:

  children: () => <ThreadMessageComponent components={components} />

so the per-message element always carries a props object and never
reaches the propless bail-out in RenderChildrenWithAccessor. A stable
map only lets the outer memo bail out, which is the cheap part. unslothai#9042
moves the call to the children form returning one shared propless
element, which does reach the bail-out, and measures 7.44x on a delete
at 300K characters. Its test asserts ThreadPrimitive.Messages carries no
components prop at all, which is the exact opposite of what a hoist
looks like, so the two could not both land.

MessagePrimitive.Parts is unaffected and keeps the fix.
MessagePrimitivePartByIndex compares components field by field, checking
components.tools by identity, so the inline tools literal really did
defeat it on every render. That part is unchanged and still measured.

The test is scoped to MessagePrimitive.Parts and carries the reason
ThreadPrimitive.Messages is excluded, so nobody re-adds the hoist there
on the strength of the same reasoning.

Discrimination: re-inlining the Parts literal fails 2 of the 3 tests.
The third pins the upstream comparator and passes on both trees by
design, as before.

npm test 2,939 passed, 0 failed. typecheck clean.
danielhanchen added a commit to danielhanchen/unsloth-staging-2 that referenced this pull request Aug 17, 2026
@danielhanchen

Copy link
Copy Markdown
Member Author

Verification before merge

Measured on a live Studio, merge base c87fe20e3 against head, each with its own UNSLOTH_STUDIO_HOME, Chromium unless stated.

The cost this change removes

The original problem reproduces on the merge base: a delete costs 132.7 ms at 37K characters of thread content and 657.8 ms at 207K.

turn pairs thread chars base, median of 3 head, median of 3
20 37.5K 132.7 ms (125-146) 103.8 ms (85-149)
60 112.7K 449.1 ms (289-758) 359.9 ms (245-366)
110 206.7K 657.8 ms (508-692) 478.9 ms (317-509)

The 20-pair row is not a result: the spreads overlap. Re-running 110 pairs at 5 repetitions gives base 580.0 ms (482-652) against head 370.3 ms (351-452), which do not overlap, a 36 percent reduction, with the longest task falling from 384 ms to 243 ms.

To be precise about what this does and does not do: the re-render half of the cost is removed, the growth with thread length is not. Head still runs 103.8 ms to 478.9 ms across the same range. That is consistent with the original breakdown, which attributed almost all of the time to the export and import round trip in delete-thread-message.ts, and this change deliberately does not touch that file.

Correctness

Delete of the first, last, middle and only message, on Chromium, WebKit and Firefox. For each case: rendered thread before, rendered thread after, after a reload, in a second tab on the same thread, and the persisted server state. All 60 comparisons are identical between base and head. Deletes land on the intended message and cascade correctly.

The research-reply ownership guard is the one user-visible behaviour this changes, since it decides whether a prompt keeps its edit, fork and delete controls. Identical between base and head for a plain thread, a thread whose reply carries a researchRunId, and a thread whose researchRunId is the empty string. The three cases differ from each other as they should, so the check discriminates rather than passing vacuously. The empty-id case still shows the controls, which is the behaviour hasResearchRunId exists to preserve.

The WeakMap in research-reply-owners.ts is keyed on thread.messages, so a repository mutation that reused that array would freeze the answer. It cannot: every mutator on MessageRepository dirties the cached array, and tapResources rebuilds its result whenever those deps change. Confirmed against the real runtime, including a research reply added on a branch the view is not showing. Deliberately breaking the cache key fails exactly that assertion and no other.

The propless bail-out works as described. In @assistant-ui/core@0.1.17, RenderChildrenWithAccessor normalises a zero-prop element's props to a frozen shared object and returns it through a useMemo, so the element is reference-stable and React skips the subtree. The components={{...}} form is also marked deprecated upstream in favour of the children form.

One thing to flag rather than bury

Replacing the assertion that pinned parentId === messageId && Boolean(getResearchRunId(message.metadata)) is a weakening. The replacements are source-text checks that would still pass if researchReplyOwners returned wrong answers. Pinning the question rather than the expression is the right call, but the behavioural coverage now has to come from elsewhere.

Cross platform and remaining gaps

Linux, macOS and Windows plus Playwright on a staging repo. One Windows frontend job fails on 10 clipboard tests; that is pre-existing, is already addressed by #8980, and this branch touches no clipboard file.

Not tested: delete during streaming, which is structurally blocked by disabled={isRunning} on the control and was read rather than exercised. Attachment and image parts did not render in the seeded threads, so those two cells did not cover what was intended. Playwright WebKit is a proxy for the webviews Desktop embeds, not those webviews.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Delightful!

Reviewed commit: 9af5eabb09

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@danielhanchen

Copy link
Copy Markdown
Member Author

Before and after evidence

Two isolated installs, this branch's merge base c87fe20e3 against head 9af5eabb0, each built by install.sh --local from its own tree so the bundle matches the side it is labelled with. The same scene drives both.

This is a render-cost change that must not alter what the user sees, so identical halves are the claim rather than the failure. That makes the picture the weaker half of the evidence: a thread that lost the wrong message still photographs as an ordinary thread. The expectation was therefore written before the run and names the survivors by their exact rendered text.

At rest, all eight messages:

thread at rest, before and after

After deleting PROMPT-3 through its own hover action bar:

thread after deleting the middle prompt

Both sides report the same facts on every key:

fact merge base head
rendered before PROMPT-1, REPLY-1, PROMPT-2, REPLY-2, PROMPT-3, REPLY-3, PROMPT-4, REPLY-4 same
rendered after PROMPT-1, REPLY-1, PROMPT-2, REPLY-2, PROMPT-4, REPLY-4 same
removed PROMPT-3, REPLY-3 same
survivor order kept true true
persisted message count 6 6

REPLY-3 goes because deleting a user message cascades its assistant reply, which is the existing behaviour and is unchanged here. PROMPT-4 and REPLY-4 survive and still sit after PROMPT-2 and REPLY-2, so the tail neither shifted nor vanished. The persisted thread read back from the server agrees with what was rendered, which matters because a delete cannot be undone.

danielhanchen added a commit that referenced this pull request Aug 17, 2026
…9014)

* Studio: stop every message part re-rendering on each streaming chunk

A streaming assistant reply rebuilt all of its already-finished parts on
every chunk, so the per-chunk cost grew with the length of the reply.

MessagePrimitivePartByIndex is memoized, and its comparator checks the
components fields one at a time rather than comparing the object as a
whole:

  prev.components?.Text === next.components?.Text &&
  ...
  prev.components?.tools === next.components?.tools &&

Every field thread.tsx passes is a module-level component, so all of them
compare equal across renders. The exception was tools, an object literal
built inline in the JSX, which meant a fresh identity on every render.
That single mismatch failed the comparator and re-rendered every part of
the message.

Both maps move to module scope. THREAD_MESSAGE_COMPONENTS is declared
after the three components it names, because a module-scope initializer
runs at import time and would otherwise read them in their temporal dead
zone.

The guard test covers the two ways this regresses. It fails on the tree
before this change, and it also pins the upstream assumption: if an
assistant-ui upgrade stops comparing components.tools by identity, the
test fails and says to re-measure rather than leaving behind a hoist and
a comment that no longer describe what the library does.

npm test 2939 passed, 0 failed. typecheck clean. biome adds no new
errors on either file.

* Narrow this to MessagePrimitive.Parts, where hoisting actually works

The ThreadPrimitive.Messages half of this change is reverted. Hoisting
the map there cannot help, and it collides with #9042 which fixes that
call properly.

Reading the primitive settles it. Given a components map, assistant-ui
builds:

  children: () => <ThreadMessageComponent components={components} />

so the per-message element always carries a props object and never
reaches the propless bail-out in RenderChildrenWithAccessor. A stable
map only lets the outer memo bail out, which is the cheap part. #9042
moves the call to the children form returning one shared propless
element, which does reach the bail-out, and measures 7.44x on a delete
at 300K characters. Its test asserts ThreadPrimitive.Messages carries no
components prop at all, which is the exact opposite of what a hoist
looks like, so the two could not both land.

MessagePrimitive.Parts is unaffected and keeps the fix.
MessagePrimitivePartByIndex compares components field by field, checking
components.tools by identity, so the inline tools literal really did
defeat it on every render. That part is unchanged and still measured.

The test is scoped to MessagePrimitive.Parts and carries the reason
ThreadPrimitive.Messages is excluded, so nobody re-adds the hoist there
on the strength of the same reasoning.

Discrimination: re-inlining the Parts literal fails 2 of the 3 tests.
The third pins the upstream comparator and passes on both trees by
design, as before.

npm test 2,939 passed, 0 failed. typecheck clean.

* Tell an install problem apart from a comparator change in the pin test

The upstream comparator is read with readFileSync, so a half-installed or
relocated node_modules surfaces as a bare ENOENT stack inside this test. That
happened once and read as a real regression in the code under test until it was
disbelieved. Wrap the read and say plainly that it is an install problem.

Also make the inline-literal assertion message one template literal, which drops
the useTemplate error the file was adding to biome.

* Tighten the parts memo comments

---------

Co-authored-by: danielhanchen <unslothshared@gmail.com>
@danielhanchen
danielhanchen merged commit 98994b9 into main Aug 17, 2026
35 of 37 checks passed
@danielhanchen
danielhanchen deleted the perf-thread-delete branch August 17, 2026 10:44
rhsCZ pushed a commit to rhsCZ/unsloth that referenced this pull request Aug 17, 2026
One conflict, in thread.tsx, where unslothai#9042 replaced the message list's `components` map with a
propless render prop so that assistant-ui's bail-out can skip a message subtree. This branch
had replaced the same element with ProgressiveMessages.

Resolved by taking both: ProgressiveMessages now takes `renderMessage={renderThreadMessage}`
and maps MessageByIndexProvider over its bounded index range, rendering that one shared
propless element in every row. That keeps unslothai#9042's optimisation exactly, because the bail-out
it relies on is React's own element-identity check, and it is if anything a better fit here:
a widening commit changes the row count every frame, which is precisely when handing React an
identical element for the rows that already existed pays.

ThreadPrimitive.MessageByIndex is not used, because its props carry the `components` object
unslothai#9042 removed. MessageByIndexProvider is the same provider ThreadPrimitive.Messages mounts;
what is dropped is RenderChildrenWithAccessor, whose accessor the thread's slot ignores and
which emits no DOM.

tests/thread-delete-render-budget.test.ts pinned the old JSX by spelling. It now pins the
property instead: the slot reaches the row map, and neither list takes a components object.

npm test 3623 passed, typecheck, build and biome clean.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant