Studio: remember chat parameters per model - #8757
Conversation
Sampling params were a single global set, so switching models handed the
next model the previous one's temperature and system prompt. Presets
were the only way back, and they exist to switch use case within one
model, not between models.
Params are now remembered per checkpoint and replayed on switch, with a
Settings toggle (on by default). A model with nothing remembered keeps
whatever is on screen rather than snapping to defaults.
Storage is a per-model map the server deep merges per key, so tuning one
model never clobbers another's:
{
"rememberParamsPerModel": true,
"inferenceParamsByModel": {
"unsloth/Qwen3.5-9B-GGUF": { "temperature": 0.2 },
"unsloth/Llama-4-8B": { "temperature": 0.9 }
}
}
Replay runs in setParams as well as setCheckpoint. The interactive local
load, the switch people use most, calls setParams with the destination
checkpoint and the backend's recommended params and only reaches
setCheckpoint later, once params.checkpoint already matches, so replay
gated on setCheckpoint alone would never fire for it. Entries hold a
full snapshot rather than the edited keys, since replay overlays the
entry onto the outgoing model's params and any gap would keep that
model's value.
Rules live in features/chat/lib/per-model-params.ts, free of store and
network imports so they stay unit testable.
…e cap Five issues from review, all confirmed against the code. A model was only remembered once it was edited, so the settings it was actually used with were lost on the way out. That hit upgrades hardest: the checkpoint restored at startup predates the map, so switching away and back handed it the other model's settings. Both switch paths now snapshot the model being left. Hydration fenced the whole map behind one version counter. If any edit landed before the settings response, every other model's persisted entry was discarded for the session. The fence is per model id now, matching what inference params already do per key. The auto-load paths call setCheckpoint and then setParams with the load response, which carries the model's context length as maxTokens. The second call saw no checkpoint change, skipped replay, overwrote the remembered budget and then snapshotted the overwritten value, even for background loads passing persist: false. Those calls are marked, replay runs for them, and a non-persisting update no longer rewrites memory. The load-time trim did not bound anything: the server merge keeps every key in its original position, so a trimmed model's later edits were dropped again on every reload and it could never retain settings. Removed, since the bound was unenforceable rather than merely loose. Replaying a model left activePreset and activePresetSource describing the model being left. For Default the settings sheet reads provenance alone, so it reported no unsaved changes over custom values. A replay that changes params now marks provenance modified.
getReplayStatePatch was only wired into setCheckpoint, so the local load path, which replays through setParams, left preset provenance describing the model being left. That matters beyond the settings sheet. Every post-load hook that re-applies model defaults bails when the preset source is not builtin-default: mergeBackendRecommendedInference returns early, and both Qwen thinking-param sites check it before writing. Leaving provenance alone therefore let the status refresh and the Qwen defaults overwrite the values that had just been replayed, which the next switch then snapshotted back over the model's memory. Marking the replay closes all of them through the mechanism already in the codebase. Test drives the real mergeBackendRecommendedInference both ways: with builtin-default the replayed temperature is overwritten, with modified it survives.
|
Correct, and it was the more important half of the earlier provenance comment. Fixed in 772af93. I had wired The consequence is bigger than the settings sheet, and it is what makes this the right lever rather than a cosmetic fix. Every post-load hook that re-applies model defaults is already gated on the preset source:
So leaving provenance alone let those overwrite the values that had just been replayed, and the next switch snapshotted the overwritten values back over the model's memory. Marking the replay closes all of them at once through a mechanism already in the codebase, instead of my chasing individual call sites, which is what I had started doing with Pinned by a test that drives the real Checks: 2317 frontend tests, backend suites, typecheck and build clean. |
A queued external message auto-loads with preserveVisibleSettings, then restores the visible model. Both switches carry trackQueuedSettings off, and restoreVisibleModelState puts params and the runtime fields back but not activePresetSource, so marking those replays left the visible model reading as modified after every background load. That both showed Default as unsaved and stood the backend and Qwen default hooks down, which is the opposite of preserving visible settings. Provenance is now restated only for a switch that is a visible settings change. Every ordinary switch passes no options and stays marked; only the background load and its restore pass the flag off.
|
Correct, and a regression from my own previous fix. Fixed in a8d54dd. Verified the sequence: a queued external message calls Provenance is now restated only for a switch that is a visible settings change. I considered adding Checks: 2317 frontend tests, backend suites, typecheck and build clean. The two branches still combine to a clean build. |
A status response that lands before chat settings marked its checkpoint as locally edited, so hydration kept the recommendation and dropped the model's persisted settings; switching away then wrote the recommendation back. Per-model memory is now only recorded and fenced after hydration, and the model already selected when settings land has its memory replayed, since a resident or restored checkpoint never crosses a transition. Replace the replay provenance mark with a replay at the three sites that re-apply a model's defaults. Marking modified stood the defaults down for every model, including ones with nothing remembered, which is the behaviour this PR set out to fix.
|
The P1 is right, and chasing it showed the provenance mark from the last round was the wrong mechanism. Both are addressed in 14dc706. Status-derived snapshots were fenced as local edits (P1). Confirmed against the real store, not just by reading. With the settings response held open, and it was durable, not just a session loss. Switching away wrote that snapshot back: Per-model memory is now only recorded and fenced after hydration, matching the rule the HTTP write already followed. That alone was not enough: the global params still held the recommendation, so the later switch-away snapshot would have picked it up anyway. A model that is resident at startup, or an external checkpoint restored from storage, never crosses a checkpoint transition, so nothing ever replays its memory. Hydration now does. Same scenario after the fix: Persisting the replayed provenance (P2). The observation about the mark is accurate: it is state only, so an in-flight hydration restores the saved source over it and a reload has no record of it. But persisting it would have made things worse, so I have removed the mark instead. The mark existed to stand down the hooks that re-apply a model's defaults, all of which bail on a source other than There are exactly three places that re-apply a model's defaults: the load response and the Qwen3 thinking params in That also removes the failure from the previous round: nothing writes One thing worth flagging separately. The status race also clobbers the global Also fixed while verifying: Checks: 2321 frontend tests, typecheck, build, lint at parity with Still not verified: none of this has been clicked through in a browser. |
A model's defaults are not settings it was used with. Recording them made the load response become memory, which the Qwen thinking-mode hook then replayed over itself: its minP and presence penalty were cancelled on every fresh load. Params staged for the model about to load were filed against the model still on screen, so its remembered context length became the other model's. Only an edit is recorded now; both cases are still picked up by the snapshot taken when the model is left, which clearCheckpoint now takes too, so an unload or eviction no longer forgets what the model was running with. A replayed output budget is clamped to the context the model just loaded with, and model defaults no longer fence hydration, so an edit made while the settings request is in flight outranks the replay.
|
All five hold, and two of them are regressions from the last round. Fixed in 20ec42d. Model defaults were being recorded as memory (P1). Confirmed against the real store. A fresh Qwen3 load applies the load response, which creates a full entry for the destination, and the thinking-mode hook that runs straight after then replays that entry over its own values: so Staged load params were filed against the outgoing model (P1). Confirmed. Returning to A would then have replayed another model's context length, which is a VRAM change rather than a cosmetic one. That call is now marked as staged: it still applies to the runtime, and it is no longer recorded.
A pre-hydration edit lost to the replay (P2). Correct, and it was the trade I made last round when adding the hydration replay. The fence that protects a user edit from the hydrated global set now protects it from the replay too, per key. That works because model defaults no longer bump those versions: a model's own defaults must not outrank the settings the user saved for it, which is what the hydration response is delivering. A slider moved while the request is in flight now survives, and a key the user did not touch still replays. That also removes the caveat I raised last round about the status race clobbering the global Clamping the replayed budget (P2). Correct. A GGUF reloaded with a smaller context would have had its older, larger Checks: 2328 frontend tests, typecheck, build and lint at parity with Still not verified: none of this has been clicked through in a browser. |
Marking the staging call stopped it recording, but the switch that follows snapshots the params it left behind, so the outgoing model still ended up remembering the incoming model's context length. The snapshot now puts back what staging wrote, for those keys only and only while nothing has changed them since, so an edit made before the load still wins. A stored entry can be partial, and replay lays what it has over the params on screen, so the gaps came from whichever model was selected beforehand. Entries are completed from the saved global set as they hydrate.
|
Both correct. Fixed in e781182. The staged value still reached the outgoing model (P1). Confirmed, and it is the half my previous fix missed: marking the staging call stopped that call recording, but the switch that follows snapshots the params staging left behind, so the outgoing model was recorded with the incoming model's context length anyway: The snapshot now puts the staged values back before recording. Deliberately narrow: only the keys staging wrote, and only while the live value still matches what staging wrote. A context the user sets themselves between staging and the load is theirs, and is recorded as theirs. Both cases are pinned by tests. My first attempt kept the whole pre-staging params object and swapped it in wholesale, which quietly reverted unrelated edits made after staging; an existing test caught it. Worth mentioning since the narrower rule is the reason for the extra bookkeeping. Partial entries borrowed from the previous model (P2). Correct. Entries this app writes are full snapshots, but a stored entry can be partial: an older write, a field that did not survive sanitising, or a hand-written payload, since the backend accepts a partial per-model dict. Replay lays what the entry has over the params on screen, so the gaps were filled by whichever model was selected beforehand, which is the bleed this PR exists to stop. Entries are completed as they hydrate, from the saved global set in the same response. That is the same answer every time and it is what the model would have run with before this PR, rather than snapping unpinned values to library defaults. A key missing from both is still left alone. Checks: 2331 frontend tests, typecheck, build and lint at parity with Still not verified: none of this has been clicked through in a browser. |
A model's context is already kept per model by its load config, and that is the copy the load uses. Keeping a second one here is what let a load stage the next model's context into the model being left, and what let a replay leave the runtime advertising a context the backend never loaded. The memory no longer records or replays it, which also retires the staged-load bookkeeping the two earlier rounds added. A model loaded while the settings request is in flight has no entry to restore its defaults from, so the hydrated global set handed it the sampling of whichever model was used last. Those keys are kept, and still lose to the model's own entry when it has one.
This comment was marked as resolved.
This comment was marked as resolved.
… restores Last round's fix for a model loaded during hydration also covered the model already resident at startup, which is the one the saved global set describes, so an upgraded install had its temperature replaced by the recommendation. The exception now applies only when the defaults replaced another model. A restore after a hidden auto-load steps off the model that load put there, so its snapshot was recorded and persisted as that model's memory even though the load itself was told not to persist. A restore no longer records it; a visible switch still does.
|
Both correct. Fixed in 477e38b. The resident model lost its saved settings (P1). Correct, and it is the earlier finding coming back through last round's fix. Reproduced on an upgraded install, with only The A restore recording the hidden model (P2). Correct. Checks: 2332 frontend tests, typecheck, build and lint errors at parity with Still not verified: none of this has been clicked through in a browser. |
|
Codex Review: Didn't find any major issues. 👍 Reviewed commit: ℹ️ About Codex in GitHubCodex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback". |
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4fce9b8cc2
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| const baseParams = getReplayedParams( | ||
| state.rememberParamsPerModel, | ||
| outgoing ?? state.paramsByModel, | ||
| state.params, | ||
| modelId, |
There was a problem hiding this comment.
Mark replayed preset parameters as modified
When model A has remembered non-default parameters, selecting the Default preset on model B and switching back to A replays A's parameters here without updating the global activePreset or activePresetSource. The settings sheet determines whether Default has unsaved changes solely from activePresetSource (chat-settings-sheet.tsx:659-662), so it now labels A's tuned parameters as an unmodified Default preset, which can cause users to run experiments under settings different from the preset they believe is active. Reconcile the preset metadata when replay changes preset-owned fields, or remember that metadata per model.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Not taking this one, though the mechanism is real: for Default, hasUnsavedPresetChanges reads activePresetSource alone (chat-settings-sheet.tsx:659-662), so a replay that moves params leaves the chip saying unmodified.
What it costs is the chip's hint, not the run. The params replayed are the ones the user set for that model, they are the values the sliders in the same panel are showing, and they are what the request carries. Nothing is sent that the user did not choose for that model.
Both remedies are worse than the symptom. Marking the replay modified was tried on this branch and reverted deliberately: activePresetSource is global, and pinning it stands down mergeBackendRecommendedInference and the Qwen thinking-params hooks for every model, so a model with nothing remembered stops getting its own recommended sampling and inherits whatever the previous one was running. That is the bleed this PR exists to stop, so buying a label with it is the wrong trade. Per-model preset metadata is the other option you name and it is a reasonable design, but it is a new feature with its own persistence and merge semantics rather than a fix, and it belongs in its own PR.
|
@codex review |
…apshot Per-model memory (#8757) landed in setParams, so the two layers are composed rather than stacked: the replay lays a model's remembered settings over its defaults, and the open chat goes on top of that. A chat is more specific than either. A captured sampling edit now stays out of the per-model memory as well as the installation defaults, since both are shared with every other chat, and remembering one chat's temperature against the model leaked it to every new chat opened on that model. Main independently marked the load-time Qwen block and the status poll as model defaults, so those edits are dropped in favour of main's, which also carry maxTokensCap. Toggling Think applies its mode params again. A pinned chat stores every sampling key, so marking the toggle as a model default meant the restore put all of them back and the mode changed while temperature and top-p stayed on the previous mode's values. The user asked for the mode in that chat, so the values land and belong to the chat after. The load-time path applies the same table unasked and stays marked. The in-memory copy of the installation defaults follows what a model default write just persisted. applyThreadScopedSettings falls back to it for a chat with no snapshot, so leaving it stale meant a chat opened after a model load ran the sampling of whichever model was loaded before it, until a reload. top-k accepts -1, which disables it. ChatCompletionRequest allows -1 to 100 and default.yaml falls back to -1, so the old floor of 0 dropped a value whole model families run with and the chat could not keep it.

Splits #8751 in two. This is the chat parameters half; project file sharing is in #8756.
From feedback on Studio chat:
The problem
inferenceParamsis one flat blob andsetCheckpointcarried every sampling value across a model switch, clamping onlymaxTokens. So a model tuned for one job handed its temperature and system prompt to the next model you loaded. Presets were the only workaround and, as the report says, they are built for switching use case within one model.The change
Sampling params are now remembered per checkpoint and replayed on switch, with a Settings toggle (on by default). A model with nothing remembered keeps whatever is on screen rather than snapping to defaults.
Storage is a per-model map that the server deep merges per key, so tuning one model never clobbers another's:
{ "rememberParamsPerModel": true, "inferenceParamsByModel": { "unsloth/Qwen3.5-9B-GGUF": { "temperature": 0.2, "systemPrompt": "Be terse." }, "unsloth/Llama-4-8B": { "temperature": 0.9 } } }The rules live in
features/chat/lib/per-model-params.ts, free of store and network imports so they stay unit testable.Two subtleties worth flagging in review
Replay has to run in
setParams, not only insetCheckpoint. The interactive local load, which is the switch people actually use, callssetParams(mergeBackendRecommendedInference({ ..., checkpoint: modelId }))and only reachessetCheckpointlater, via the post-load status refresh, by which pointparams.checkpointalready equals the new model andcheckpointChangedis false. Gating replay onsetCheckpointalone left the feature dead for local models. Replaying insetParamsalso puts the remembered settings ahead of the backend's recommended params, which is the point of the setting.Entries hold a full snapshot, not the edited keys. Replay overlays a model's entry onto the outgoing model's params, so any key missing from the entry silently keeps the other model's value. With deltas, tuning only model A's temperature and only model B's prompt meant returning to A gave you A's temperature and B's prompt.
Testing
12 unit tests covering the rules, including the load, edit, switch away, switch back sequence and the mixed partial-edit case above. New backend tests for the per-model settings merge and the payload schema. Round-trip verified over HTTP against a running Studio: two models' settings coexist and patching one leaves the other intact.
Frontend tests, typecheck, build and locale parity all pass, and lint matches
main.Note on merging
Touches four files that #8756 also touches (the runtime store, the settings tab, the settings search index and
en.ts), in each case adding adjacent lines at the same anchor. Whichever lands second needs a trivial keep-both rebase. I confirmed the two branches combine to exactly the content of #8751, differing only in the order of the two settings rows.