Chat: stop a message delete from re-rendering the whole thread - #9042
Conversation
Deleting one message from a long chat cost 98ms at 25K characters of thread
content and 472ms at 300K, growing 4.8x across that range. The cost was
attributed to the export / rebuild / import round trip in
delete-thread-message.ts. Profiling says otherwise: sampled on Chromium at 300K
characters (220 messages, 43,422 DOM nodes), deleteThreadMessage is 1.8ms of a
498ms delete, and the whole of syncExportedRepositoryToBackend under it is
1.3ms. The other 400ms is one synchronous React render, and the reason it is
that big is that deleting a message re-rendered every remaining message.
Two things caused that, both here rather than in the repository code.
ThreadPrimitive.Messages rebuilds its element array whenever the message COUNT
changes, so every message's wrapper re-renders on a delete. assistant-ui absorbs
that with a bail-out: an element with no props is memoized and the same object
is handed back, and React skips reconciling it. The `components={{...}}` form
never reaches it, because what the render prop returns there is
<ThreadMessageComponent components={...} />, whose props object is freshly
allocated on every render. Rendering through the children form, returning one
shared propless element, means the wrapper re-renders and the message body,
action bar and tooltips underneath it do not. The role and edit-state selectors
are the ones assistant-ui's own ThreadMessageComponent uses, and the component
choice reproduces its getComponent fallback chain for the three components this
thread supplies.
useOwnsResearchMessage selected `thread.messages`, which subscribed every user
message's action bar to every thread change, and then answered its question by
exporting the whole repository once per message. It now selects the answer, so a
change that does not move the answer re-renders nothing, and the export is
shared across the messages at one revision instead of repeated per message. The
question needs the repository rather than the visible list, since a research
reply can sit on a branch the view is not showing, so the export stays.
Measured with the heavy-thread harness from #9016, chromium, three interleaved
A/B rounds, each cell the median of 3 repetitions:
25K chars 98.3ms -> 25.9ms
100K chars 206.5ms -> 34.7ms
300K chars 472.3ms -> 63.5ms
Growth across the range falls from 4.80x to 2.45x. Firefox at 300K goes 615ms ->
58ms. DOM node counts, message counts before and after the delete, console
warnings and stray requests are identical at every size, so what changed is how
much of the tree React walks, not what it produces.
test_deep_research_frontend_contract pinned the exact expression the ownership
question used to be written as. It is repointed at the question and its scope
rather than at the expression.
The ThreadPrimitive.Messages half of this change is reverted. Hoisting the map there cannot help, and it collides with unslothai#9042 which fixes that call properly. Reading the primitive settles it. Given a components map, assistant-ui builds: children: () => <ThreadMessageComponent components={components} /> so the per-message element always carries a props object and never reaches the propless bail-out in RenderChildrenWithAccessor. A stable map only lets the outer memo bail out, which is the cheap part. unslothai#9042 moves the call to the children form returning one shared propless element, which does reach the bail-out, and measures 7.44x on a delete at 300K characters. Its test asserts ThreadPrimitive.Messages carries no components prop at all, which is the exact opposite of what a hoist looks like, so the two could not both land. MessagePrimitive.Parts is unaffected and keeps the fix. MessagePrimitivePartByIndex compares components field by field, checking components.tools by identity, so the inline tools literal really did defeat it on every render. That part is unchanged and still measured. The test is scoped to MessagePrimitive.Parts and carries the reason ThreadPrimitive.Messages is excluded, so nobody re-adds the hoist there on the strength of the same reasoning. Discrimination: re-inlining the Parts literal fails 2 of the 3 tests. The third pins the upstream comparator and passes on both trees by design, as before. npm test 2,939 passed, 0 failed. typecheck clean.
Verification before mergeMeasured on a live Studio, merge base The cost this change removesThe original problem reproduces on the merge base: a delete costs 132.7 ms at 37K characters of thread content and 657.8 ms at 207K.
The 20-pair row is not a result: the spreads overlap. Re-running 110 pairs at 5 repetitions gives base 580.0 ms (482-652) against head 370.3 ms (351-452), which do not overlap, a 36 percent reduction, with the longest task falling from 384 ms to 243 ms. To be precise about what this does and does not do: the re-render half of the cost is removed, the growth with thread length is not. Head still runs 103.8 ms to 478.9 ms across the same range. That is consistent with the original breakdown, which attributed almost all of the time to the export and import round trip in CorrectnessDelete of the first, last, middle and only message, on Chromium, WebKit and Firefox. For each case: rendered thread before, rendered thread after, after a reload, in a second tab on the same thread, and the persisted server state. All 60 comparisons are identical between base and head. Deletes land on the intended message and cascade correctly. The research-reply ownership guard is the one user-visible behaviour this changes, since it decides whether a prompt keeps its edit, fork and delete controls. Identical between base and head for a plain thread, a thread whose reply carries a The The propless bail-out works as described. In One thing to flag rather than buryReplacing the assertion that pinned Cross platform and remaining gapsLinux, macOS and Windows plus Playwright on a staging repo. One Windows frontend job fails on 10 clipboard tests; that is pre-existing, is already addressed by #8980, and this branch touches no clipboard file. Not tested: delete during streaming, which is structurally blocked by |
|
Codex Review: Didn't find any major issues. Delightful! Reviewed commit: ℹ️ About Codex in GitHubCodex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback". |
Before and after evidenceTwo isolated installs, this branch's merge base This is a render-cost change that must not alter what the user sees, so identical halves are the claim rather than the failure. That makes the picture the weaker half of the evidence: a thread that lost the wrong message still photographs as an ordinary thread. The expectation was therefore written before the run and names the survivors by their exact rendered text. At rest, all eight messages: After deleting Both sides report the same facts on every key:
|
…9014) * Studio: stop every message part re-rendering on each streaming chunk A streaming assistant reply rebuilt all of its already-finished parts on every chunk, so the per-chunk cost grew with the length of the reply. MessagePrimitivePartByIndex is memoized, and its comparator checks the components fields one at a time rather than comparing the object as a whole: prev.components?.Text === next.components?.Text && ... prev.components?.tools === next.components?.tools && Every field thread.tsx passes is a module-level component, so all of them compare equal across renders. The exception was tools, an object literal built inline in the JSX, which meant a fresh identity on every render. That single mismatch failed the comparator and re-rendered every part of the message. Both maps move to module scope. THREAD_MESSAGE_COMPONENTS is declared after the three components it names, because a module-scope initializer runs at import time and would otherwise read them in their temporal dead zone. The guard test covers the two ways this regresses. It fails on the tree before this change, and it also pins the upstream assumption: if an assistant-ui upgrade stops comparing components.tools by identity, the test fails and says to re-measure rather than leaving behind a hoist and a comment that no longer describe what the library does. npm test 2939 passed, 0 failed. typecheck clean. biome adds no new errors on either file. * Narrow this to MessagePrimitive.Parts, where hoisting actually works The ThreadPrimitive.Messages half of this change is reverted. Hoisting the map there cannot help, and it collides with #9042 which fixes that call properly. Reading the primitive settles it. Given a components map, assistant-ui builds: children: () => <ThreadMessageComponent components={components} /> so the per-message element always carries a props object and never reaches the propless bail-out in RenderChildrenWithAccessor. A stable map only lets the outer memo bail out, which is the cheap part. #9042 moves the call to the children form returning one shared propless element, which does reach the bail-out, and measures 7.44x on a delete at 300K characters. Its test asserts ThreadPrimitive.Messages carries no components prop at all, which is the exact opposite of what a hoist looks like, so the two could not both land. MessagePrimitive.Parts is unaffected and keeps the fix. MessagePrimitivePartByIndex compares components field by field, checking components.tools by identity, so the inline tools literal really did defeat it on every render. That part is unchanged and still measured. The test is scoped to MessagePrimitive.Parts and carries the reason ThreadPrimitive.Messages is excluded, so nobody re-adds the hoist there on the strength of the same reasoning. Discrimination: re-inlining the Parts literal fails 2 of the 3 tests. The third pins the upstream comparator and passes on both trees by design, as before. npm test 2,939 passed, 0 failed. typecheck clean. * Tell an install problem apart from a comparator change in the pin test The upstream comparator is read with readFileSync, so a half-installed or relocated node_modules surfaces as a bare ENOENT stack inside this test. That happened once and read as a real regression in the code under test until it was disbelieved. Wrap the read and say plainly that it is an install problem. Also make the inline-literal assertion message one template literal, which drops the useTemplate error the file was adding to biome. * Tighten the parts memo comments --------- Co-authored-by: danielhanchen <unslothshared@gmail.com>
One conflict, in thread.tsx, where unslothai#9042 replaced the message list's `components` map with a propless render prop so that assistant-ui's bail-out can skip a message subtree. This branch had replaced the same element with ProgressiveMessages. Resolved by taking both: ProgressiveMessages now takes `renderMessage={renderThreadMessage}` and maps MessageByIndexProvider over its bounded index range, rendering that one shared propless element in every row. That keeps unslothai#9042's optimisation exactly, because the bail-out it relies on is React's own element-identity check, and it is if anything a better fit here: a widening commit changes the row count every frame, which is precisely when handing React an identical element for the rows that already existed pays. ThreadPrimitive.MessageByIndex is not used, because its props carry the `components` object unslothai#9042 removed. MessageByIndexProvider is the same provider ThreadPrimitive.Messages mounts; what is dropped is RenderChildrenWithAccessor, whose accessor the thread's slot ignores and which emits no DOM. tests/thread-delete-render-budget.test.ts pinned the old JSX by spelling. It now pins the property instead: the slot reaches the row map, and neither list takes a components object. npm test 3623 passed, typecheck, build and biome clean.



What this is
Bottleneck 3 from the heavy-thread profiling in #9016: deleting a message from a long chat. It cost 98ms at 25K characters of thread content and 472ms at 300K, growing 4.8x across that range, of which 503ms of the 505ms measured there was script and 1.9ms was style.
The attribution was wrong, and here is the profile
The cost was attributed to the export / rebuild / import round trip in
delete-thread-message.ts: export the thread, rebuild it without the message, import it back. That is not where the time goes.Sampled with the CDP profiler on Chromium at 300K characters (220 messages, 43,422 DOM nodes), inclusive time for one delete of 498ms wall:
The whole round trip is under 2ms. The rest is one synchronous React render, in a microtask after the click handler returns, and it is that big because deleting one message re-rendered every remaining message in the thread. Two things caused that, both in
thread.tsx.1. The
componentsmap defeats assistant-ui's bail-outThreadPrimitive.Messagesrebuilds its whole element array whenever the message COUNT changes, so every message's wrapper re-renders on a delete. assistant-ui absorbs that inRenderChildrenWithAccessor: when the render prop returns an element with no props, the same element object is handed back on every render, and React skips reconciling that subtree.The
components={{ UserMessage, EditComposer, AssistantMessage }}form never reaches it. What the render prop returns there is<ThreadMessageComponent components={...} />, whose props object is freshly allocated every render, so the memo can never hit and the whole message body, action bar and tooltip tree is reconciled again for all 219 remaining messages.This PR renders through the children form and returns one shared propless element. The wrapper still re-renders; what is under it does not. Sharing the element rather than building a fresh propless one also makes the bail-out React's own (
current.memoizedProps === pendingProps), so it does not depend on a library detail staying put.The role and edit-state selectors are exactly the ones assistant-ui's
ThreadMessageComponentuses, andthreadMessageKindreproduces itsgetComponentfallback chain for the three components this thread supplies, across all six role x editing combinations. An editing message of any role goes toEditComposer; a system message that is not being edited renders nothing, as it always has.2.
useOwnsResearchMessagere-rendered every action bar, and exported per messageSelecting the array subscribed every user message's action bar to every thread change, so a delete re-rendered all of them along with their copy, edit, fork and delete controls and their tooltips, whether or not the answer had moved. Behind that, each one exported the whole repository, which is quadratic in thread length and is paid on every token of a generation as well.
It now selects the answer, so nothing re-renders unless the answer changes, and the export is shared across the messages at one revision. The export itself stays: the question is about the whole repository, since a research reply can sit on a branch the view is not showing, and it cannot be read off the visible message list. What is removed is the per-message factor, not the export.
Measured
Harness is
tests/studio/playwright_heavy_thread.pyfrom #9016, unchanged, driven against this branch and against its merge base. Paired and interleaved (base, fix, base, fix, ...), because this host's load drifts over tens of minutes and two back-to-back blocks would charge that drift to the change. Every cell is itself the median of 3 repetitions; the round-level number is the median across rounds. Ratios are the headline, absolute milliseconds are context: this is a loaded shared host and the harness runs against a vite dev server, so React is unminified and unbundled.Chromium, 3 interleaved rounds:
Growth across the range: 4.80x before, 2.45x after.
WebKit, 2 interleaved rounds:
Growth: 7.68x before, 1.88x after.
Firefox, 1 round:
Growth: 6.47x before, 1.81x after.
Two things to read these against. The harness clocks this action across a double rAF, so roughly 33ms of every number above is the vsync floor rather than work; subtracting it, the Chromium 300K cell goes from about 439ms to about 30ms. And one base cell (Chromium, 25K, round 3) timed out on page load and is excluded; its column is the median of the other two rounds.
Nothing else in the harness moved: keystroke, scroll and jump are flat, and menu and reopen are unchanged within run-to-run noise. Those two are separate bottlenecks and are not touched here.
Same output, not just faster: DOM node counts are identical at every size (4,048 / 15,887 / 43,422 before and after), message counts before and after the delete are identical, and both trees report zero console warnings and zero stray API requests.
Tests
Six new tests over the two extracted modules, plus four source pins, all under
npm test.tests/thread-message-slot.test.tscovers the component choice across all six role x editing combinations, and the two properties the bail-out needs: the render prop returns the same element object every time, and that element carries no props.tests/research-reply-owners.test.tscovers what the shared answer contains, that a rootless reply names no owner, that one export serves every message at a revision, and that a new revision is exported again and sees the change.tests/thread-delete-render-budget.test.tspins the three seams that are silently load-bearing, following the existingresearch-render-budget.test.tsidiom. Undo any one of them and the thread still renders correctly, the unit tests above still pass, and the delete goes back to being linear in thread length: the render-prop form rather than the map, the render prop being hoisted to module scope, and the revision key being the array the store hands out rather than a copy of it.Each of the 13 tests was run against a deliberately broken tree and shown to fail. Per test, the break it catches:
tests/studio/test_deep_research_frontend_contract.pypinned the exact expression the ownership question used to be written as. Repointed at the question and its scope, not at the expression that used to compute it.Verification
From
studio/frontend:npm test2949 passed 0 failed,npm run typecheckclean,npm run buildclean, biome and eslint on the changed files produce byte-identical diagnostics to the merge base forthread.tsxand nothing at all for the new files.pytest tests/studiois 3900 passed, 1 failed. The one failure istest_multi_chat_prompt_queue_contract.py::test_composer_only_queues_behind_the_current_chat, which fails identically on the merge base and is fixed by #9026.Not done here, deliberately
MessagePrimitive.Partsin the same file is passed an inlinecomponentsliteral whosetoolsobject is reallocated per render, and assistant-ui memoizes each part on that object's identity, so that memo can never hit either. It is the same mistake one level down. It does not affect a delete any more, because the message subtree no longer re-renders, so it belongs with the menu and reopen work rather than here.