Studio: remember chat parameters per model by shimmyshimmer · Pull Request #8757 · unslothai/unsloth · GitHub
Skip to content

Studio: remember chat parameters per model - #8757

Merged
danielhanchen merged 27 commits into
mainfrom
studio-per-model-params
Aug 17, 2026
Merged

Studio: remember chat parameters per model#8757
danielhanchen merged 27 commits into
mainfrom
studio-per-model-params

Conversation

@shimmyshimmer

Copy link
Copy Markdown
Member

Splits #8751 in two. This is the chat parameters half; project file sharing is in #8756.

From feedback on Studio chat:

The way chat parameters are handled is awkward; instead of being tied to specific models, there's basically just a single global setting that applies to the chat itself. There is a preset function, but that's designed for switching settings for the same model based on the use case, not for switching between different models.

The problem

inferenceParams is one flat blob and setCheckpoint carried every sampling value across a model switch, clamping only maxTokens. So a model tuned for one job handed its temperature and system prompt to the next model you loaded. Presets were the only workaround and, as the report says, they are built for switching use case within one model.

The change

Sampling params are now remembered per checkpoint and replayed on switch, with a Settings toggle (on by default). A model with nothing remembered keeps whatever is on screen rather than snapping to defaults.

Storage is a per-model map that the server deep merges per key, so tuning one model never clobbers another's:

{
  "rememberParamsPerModel": true,
  "inferenceParamsByModel": {
    "unsloth/Qwen3.5-9B-GGUF": { "temperature": 0.2, "systemPrompt": "Be terse." },
    "unsloth/Llama-4-8B": { "temperature": 0.9 }
  }
}

The rules live in features/chat/lib/per-model-params.ts, free of store and network imports so they stay unit testable.

Two subtleties worth flagging in review

Replay has to run in setParams, not only in setCheckpoint. The interactive local load, which is the switch people actually use, calls setParams(mergeBackendRecommendedInference({ ..., checkpoint: modelId })) and only reaches setCheckpoint later, via the post-load status refresh, by which point params.checkpoint already equals the new model and checkpointChanged is false. Gating replay on setCheckpoint alone left the feature dead for local models. Replaying in setParams also puts the remembered settings ahead of the backend's recommended params, which is the point of the setting.

Entries hold a full snapshot, not the edited keys. Replay overlays a model's entry onto the outgoing model's params, so any key missing from the entry silently keeps the other model's value. With deltas, tuning only model A's temperature and only model B's prompt meant returning to A gave you A's temperature and B's prompt.

Testing

12 unit tests covering the rules, including the load, edit, switch away, switch back sequence and the mixed partial-edit case above. New backend tests for the per-model settings merge and the payload schema. Round-trip verified over HTTP against a running Studio: two models' settings coexist and patching one leaves the other intact.

Frontend tests, typecheck, build and locale parity all pass, and lint matches main.

Note on merging

Touches four files that #8756 also touches (the runtime store, the settings tab, the settings search index and en.ts), in each case adding adjacent lines at the same anchor. Whichever lands second needs a trivial keep-both rebase. I confirmed the two branches combine to exactly the content of #8751, differing only in the order of the two settings rows.

Sampling params were a single global set, so switching models handed the
next model the previous one's temperature and system prompt. Presets
were the only way back, and they exist to switch use case within one
model, not between models.

Params are now remembered per checkpoint and replayed on switch, with a
Settings toggle (on by default). A model with nothing remembered keeps
whatever is on screen rather than snapping to defaults.

Storage is a per-model map the server deep merges per key, so tuning one
model never clobbers another's:

  {
    "rememberParamsPerModel": true,
    "inferenceParamsByModel": {
      "unsloth/Qwen3.5-9B-GGUF": { "temperature": 0.2 },
      "unsloth/Llama-4-8B": { "temperature": 0.9 }
    }
  }

Replay runs in setParams as well as setCheckpoint. The interactive local
load, the switch people use most, calls setParams with the destination
checkpoint and the backend's recommended params and only reaches
setCheckpoint later, once params.checkpoint already matches, so replay
gated on setCheckpoint alone would never fire for it. Entries hold a
full snapshot rather than the edited keys, since replay overlays the
entry onto the outgoing model's params and any gap would keep that
model's value.

Rules live in features/chat/lib/per-model-params.ts, free of store and
network imports so they stay unit testable.
chatgpt-codex-connector[bot]

This comment was marked as resolved.

…e cap

Five issues from review, all confirmed against the code.

A model was only remembered once it was edited, so the settings it was
actually used with were lost on the way out. That hit upgrades hardest:
the checkpoint restored at startup predates the map, so switching away
and back handed it the other model's settings. Both switch paths now
snapshot the model being left.

Hydration fenced the whole map behind one version counter. If any edit
landed before the settings response, every other model's persisted entry
was discarded for the session. The fence is per model id now, matching
what inference params already do per key.

The auto-load paths call setCheckpoint and then setParams with the load
response, which carries the model's context length as maxTokens. The
second call saw no checkpoint change, skipped replay, overwrote the
remembered budget and then snapshotted the overwritten value, even for
background loads passing persist: false. Those calls are marked, replay
runs for them, and a non-persisting update no longer rewrites memory.

The load-time trim did not bound anything: the server merge keeps every
key in its original position, so a trimmed model's later edits were
dropped again on every reload and it could never retain settings.
Removed, since the bound was unenforceable rather than merely loose.

Replaying a model left activePreset and activePresetSource describing
the model being left. For Default the settings sheet reads provenance
alone, so it reported no unsaved changes over custom values. A replay
that changes params now marks provenance modified.
@shimmyshimmer

Copy link
Copy Markdown
Member Author

chatgpt-codex-connector[bot]

This comment was marked as resolved.

getReplayStatePatch was only wired into setCheckpoint, so the local load
path, which replays through setParams, left preset provenance describing
the model being left.

That matters beyond the settings sheet. Every post-load hook that
re-applies model defaults bails when the preset source is not
builtin-default: mergeBackendRecommendedInference returns early, and both
Qwen thinking-param sites check it before writing. Leaving provenance
alone therefore let the status refresh and the Qwen defaults overwrite
the values that had just been replayed, which the next switch then
snapshotted back over the model's memory. Marking the replay closes all
of them through the mechanism already in the codebase.

Test drives the real mergeBackendRecommendedInference both ways: with
builtin-default the replayed temperature is overwritten, with modified it
survives.
@shimmyshimmer

Copy link
Copy Markdown
Member Author

Correct, and it was the more important half of the earlier provenance comment. Fixed in 772af93.

I had wired getReplayStatePatch into setCheckpoint only, so the local load path, which replays through setParams on a checkpoint change, left provenance describing the model being left.

The consequence is bigger than the settings sheet, and it is what makes this the right lever rather than a cosmetic fix. Every post-load hook that re-applies model defaults is already gated on the preset source:

  • mergeBackendRecommendedInference returns early with only checkpoint and trustRemoteCode when presetSource !== "builtin-default", so the status refresh stops replacing sampling values
  • the Qwen thinking params check activePresetSource === "builtin-default" at both sites, the inline one in use-chat-model-runtime.ts and applyQwenThinkingParams

So leaving provenance alone let those overwrite the values that had just been replayed, and the next switch snapshotted the overwritten values back over the model's memory. Marking the replay closes all of them at once through a mechanism already in the codebase, instead of my chasing individual call sites, which is what I had started doing with fromModelLoad. That flag stays for the auto-load's maxTokens, which has no such gate.

Pinned by a test that drives the real mergeBackendRecommendedInference both ways: with builtin-default the replayed temperature is overwritten with the backend's, with modified it survives.

Checks: 2317 frontend tests, backend suites, typecheck and build clean.

chatgpt-codex-connector[bot]

This comment was marked as resolved.

A queued external message auto-loads with preserveVisibleSettings, then
restores the visible model. Both switches carry trackQueuedSettings off,
and restoreVisibleModelState puts params and the runtime fields back but
not activePresetSource, so marking those replays left the visible model
reading as modified after every background load. That both showed Default
as unsaved and stood the backend and Qwen default hooks down, which is the
opposite of preserving visible settings.

Provenance is now restated only for a switch that is a visible settings
change. Every ordinary switch passes no options and stays marked; only
the background load and its restore pass the flag off.
@shimmyshimmer

Copy link
Copy Markdown
Member Author

Correct, and a regression from my own previous fix. Fixed in a8d54dd.

Verified the sequence: a queued external message calls autoLoadSmallestModel({ preserveVisibleSettings: true }), then restoreVisibleModelState switches the checkpoint back and does setState({ ...snapshot.runtime, ...snapshot.settings, params }). activePresetSource is in neither VISIBLE_MODEL_RUNTIME_KEYS nor the queued-settings snapshot, so the mark my previous commit added survived the restore and left the visible model reading as modified after every background load. As you say, that both shows Default as unsaved and stands the backend and Qwen default hooks down, which is precisely backwards for a load that exists to preserve visible settings.

Provenance is now restated only for a switch that is a visible settings change. trackQueuedSettings is the existing signal for this: restoreVisibleModelState passes it false, and both auto-load paths pass !preserveVisibleSettings. I checked every setCheckpoint call site in the tree, and those three in chat-adapter.ts are the only ones that pass the flag at all; chat-page, shared-composer, use-chat-model-runtime, apply-inference-status, hub-page and adopt-inference-status all pass no options, so every user-visible switch stays marked.

I considered adding activePresetSource to the snapshot instead. Suppressing is the better fit: a hidden load has no visible replay to describe, and the snapshot deliberately covers model runtime state rather than preset bookkeeping.

Checks: 2317 frontend tests, backend suites, typecheck and build clean. The two branches still combine to a clean build.

chatgpt-codex-connector[bot]

This comment was marked as resolved.

A status response that lands before chat settings marked its checkpoint as
locally edited, so hydration kept the recommendation and dropped the model's
persisted settings; switching away then wrote the recommendation back.

Per-model memory is now only recorded and fenced after hydration, and the
model already selected when settings land has its memory replayed, since a
resident or restored checkpoint never crosses a transition.

Replace the replay provenance mark with a replay at the three sites that
re-apply a model's defaults. Marking modified stood the defaults down for
every model, including ones with nothing remembered, which is the behaviour
this PR set out to fix.
chatgpt-codex-connector[bot]

This comment was marked as resolved.

@shimmyshimmer

Copy link
Copy Markdown
Member Author

The P1 is right, and chasing it showed the provenance mark from the last round was the wrong mechanism. Both are addressed in 14dc706.

Status-derived snapshots were fenced as local edits (P1). Confirmed against the real store, not just by reading. With the settings response held open, applyActiveModelStatusToStore's setParams recorded the recommendation under the resident checkpoint and added it to locallyRememberedModels, so hydration preferred that transient snapshot over the persisted entry:

pre-hydration  map: { Qwen: { temperature: 0.9, maxTokens: 131072, systemPrompt: "" } }
post-hydration map: { Qwen: { temperature: 0.9, maxTokens: 131072, systemPrompt: "" } }

and it was durable, not just a session loss. Switching away wrote that snapshot back:

PUT inferenceParamsByModel: { Qwen: { temperature: 0.9, maxTokens: 131072, ... } }

Per-model memory is now only recorded and fenced after hydration, matching the rule the HTTP write already followed. That alone was not enough: the global params still held the recommendation, so the later switch-away snapshot would have picked it up anyway. A model that is resident at startup, or an external checkpoint restored from storage, never crosses a checkpoint transition, so nothing ever replays its memory. Hydration now does. Same scenario after the fix:

post-hydration map: { Qwen: { temperature: 0.2, maxTokens: 4096, systemPrompt: "Be terse." } }
PUT on switch away: temperature 0.2, maxTokens 4096

Persisting the replayed provenance (P2). The observation about the mark is accurate: it is state only, so an in-flight hydration restores the saved source over it and a reload has no record of it. But persisting it would have made things worse, so I have removed the mark instead.

The mark existed to stand down the hooks that re-apply a model's defaults, all of which bail on a source other than builtin-default. That flag is global, so pinning it to modified also stops a model with nothing remembered from getting its own recommended sampling, and it hands that model whatever the previous one was running with. Persisting it makes that permanent. That is the behaviour this PR set out to fix, so buying protection with it was the wrong trade.

There are exactly three places that re-apply a model's defaults: the load response and the Qwen3 thinking params in use-chat-model-runtime.ts, and the status merge in apply-inference-status-to-store.ts. Each now asks for the replay directly, so the model's remembered settings go back over its defaults and a model with nothing remembered keeps its recommendation. fromModelLoad is renamed fromModelDefaults to match. A test reads the three call sites so a fourth cannot be added silently.

That also removes the failure from the previous round: nothing writes activePresetSource on a replay any more, so a background load cannot leave the visible model reading as modified.

One thing worth flagging separately. The status race also clobbers the global inferenceParams, because getChangedInferenceParams bumps the hydration versions for whatever the status moved. That predates this PR and I have left it alone; the hydration replay covers the active model, which is the case that matters here.

Also fixed while verifying: tests/per-model-params.test.ts pulled the whole chat UI into the test program through preset-load-config, which put src/speech-recognition.d.ts out of scope and produced five type errors I had reported as clean. PersistedInferenceParams now lives beside InferenceParams in types/runtime.ts, which is what per-model-params.ts claims about itself, and the test tsconfig picks up the ambient declarations.

Checks: 2321 frontend tests, typecheck, build, lint at parity with main on the touched files, and 79 chat settings and history backend tests. The rest of the backend suite fails identically on origin/main in this environment.

Still not verified: none of this has been clicked through in a browser.

chatgpt-codex-connector[bot]

This comment was marked as resolved.

A model's defaults are not settings it was used with. Recording them made the
load response become memory, which the Qwen thinking-mode hook then replayed
over itself: its minP and presence penalty were cancelled on every fresh load.
Params staged for the model about to load were filed against the model still
on screen, so its remembered context length became the other model's.

Only an edit is recorded now; both cases are still picked up by the snapshot
taken when the model is left, which clearCheckpoint now takes too, so an
unload or eviction no longer forgets what the model was running with.

A replayed output budget is clamped to the context the model just loaded with,
and model defaults no longer fence hydration, so an edit made while the
settings request is in flight outranks the replay.
@shimmyshimmer

Copy link
Copy Markdown
Member Author

All five hold, and two of them are regressions from the last round. Fixed in 20ec42d.

Model defaults were being recorded as memory (P1). Confirmed against the real store. A fresh Qwen3 load applies the load response, which creates a full entry for the destination, and the thinking-mode hook that runs straight after then replays that entry over its own values:

map[Qwen] after the load merge: { temperature: 0.7, minP: 0.01, presencePenalty: 0, ... }
qwen thinking params applied:   { temperature: 0.7, minP: 0.01, presencePenalty: 0 }

so minP: 0 and presencePenalty: 1.5 were cancelled on every fresh load. The rule is now that only an edit is recorded. A model's defaults are not settings it was used with, and nothing is lost by not recording them: the snapshot taken when the model is left records what it actually ran with. After the fix the same sequence keeps 0.6 / 0 / 1.5 and leaves no entry for a model that has not been tuned.

Staged load params were filed against the outgoing model (P1). Confirmed. applyPerModelConfigToRuntime sets the target model's maxSeqLength while the previous one is still current, and that landed as a full snapshot of A carrying B's context length:

map after staging: { Llama-4-8B: { maxSeqLength: 32768, ... } }

Returning to A would then have replayed another model's context length, which is a VRAM change rather than a cosmetic one. That call is now marked as staged: it still applies to the runtime, and it is no longer recorded.

clearCheckpoint bypassed the outgoing snapshot (P2). Correct. An unload or eviction leaves a model exactly the way a switch does, and ten call sites reach it. It takes the same snapshot now, so a model that was never edited is still remembered when it is dropped.

A pre-hydration edit lost to the replay (P2). Correct, and it was the trade I made last round when adding the hydration replay. The fence that protects a user edit from the hydrated global set now protects it from the replay too, per key. That works because model defaults no longer bump those versions: a model's own defaults must not outrank the settings the user saved for it, which is what the hydration response is delivering. A slider moved while the request is in flight now survives, and a key the user did not touch still replays.

That also removes the caveat I raised last round about the status race clobbering the global inferenceParams.

Clamping the replayed budget (P2). Correct. A GGUF reloaded with a smaller context would have had its older, larger maxTokens replayed straight into the next request. The three sites that know the context the model actually loaded with now pass it, and the replay clamps to it. Only those sites pass one: for a non-GGUF load the merge's 4096 is an output default rather than a context cap, and clamping to it would lower a remembered budget for no reason.

Checks: 2328 frontend tests, typecheck, build and lint at parity with main on the touched files. Every new test was confirmed to fail against the previous commit. The two branches still merge to a working build (2344 tests).

Still not verified: none of this has been clicked through in a browser.

chatgpt-codex-connector[bot]

This comment was marked as resolved.

Marking the staging call stopped it recording, but the switch that follows
snapshots the params it left behind, so the outgoing model still ended up
remembering the incoming model's context length. The snapshot now puts back
what staging wrote, for those keys only and only while nothing has changed
them since, so an edit made before the load still wins.

A stored entry can be partial, and replay lays what it has over the params on
screen, so the gaps came from whichever model was selected beforehand. Entries
are completed from the saved global set as they hydrate.
@shimmyshimmer

Copy link
Copy Markdown
Member Author

Both correct. Fixed in e781182.

The staged value still reached the outgoing model (P1). Confirmed, and it is the half my previous fix missed: marking the staging call stopped that call recording, but the switch that follows snapshots the params staging left behind, so the outgoing model was recorded with the incoming model's context length anyway:

A on maxSeqLength 4096, stage B's 32768, switch to B
map[A].maxSeqLength = 32768

The snapshot now puts the staged values back before recording. Deliberately narrow: only the keys staging wrote, and only while the live value still matches what staging wrote. A context the user sets themselves between staging and the load is theirs, and is recorded as theirs. Both cases are pinned by tests.

My first attempt kept the whole pre-staging params object and swapped it in wholesale, which quietly reverted unrelated edits made after staging; an existing test caught it. Worth mentioning since the narrower rule is the reason for the extra bookkeeping.

Partial entries borrowed from the previous model (P2). Correct. Entries this app writes are full snapshots, but a stored entry can be partial: an older write, a field that did not survive sanitising, or a hand-written payload, since the backend accepts a partial per-model dict. Replay lays what the entry has over the params on screen, so the gaps were filled by whichever model was selected beforehand, which is the bleed this PR exists to stop.

Entries are completed as they hydrate, from the saved global set in the same response. That is the same answer every time and it is what the model would have run with before this PR, rather than snapping unpinned values to library defaults. A key missing from both is still left alone.

Checks: 2331 frontend tests, typecheck, build and lint at parity with main on the touched files. Both new behaviour tests were confirmed to fail against the previous commit. The two branches still merge to a working build (2349 tests).

Still not verified: none of this has been clicked through in a browser.

chatgpt-codex-connector[bot]

This comment was marked as resolved.

A model's context is already kept per model by its load config, and that is
the copy the load uses. Keeping a second one here is what let a load stage the
next model's context into the model being left, and what let a replay leave the
runtime advertising a context the backend never loaded. The memory no longer
records or replays it, which also retires the staged-load bookkeeping the two
earlier rounds added.

A model loaded while the settings request is in flight has no entry to restore
its defaults from, so the hydrated global set handed it the sampling of
whichever model was used last. Those keys are kept, and still lose to the
model's own entry when it has one.
@shimmyshimmer

This comment was marked as resolved.

chatgpt-codex-connector[bot]

This comment was marked as resolved.

… restores

Last round's fix for a model loaded during hydration also covered the model
already resident at startup, which is the one the saved global set describes,
so an upgraded install had its temperature replaced by the recommendation. The
exception now applies only when the defaults replaced another model.

A restore after a hidden auto-load steps off the model that load put there, so
its snapshot was recorded and persisted as that model's memory even though the
load itself was told not to persist. A restore no longer records it; a visible
switch still does.
@shimmyshimmer

Copy link
Copy Markdown
Member Author

Both correct. Fixed in 477e38b.

The resident model lost its saved settings (P1). Correct, and it is the earlier finding coming back through last round's fix. Reproduced on an upgraded install, with only inferenceParams saved and the status for the already-resident model beating hydration:

checkpoint before the status: ""
temperature after hydration:  0.9   (saved 0.2)

The keepModelDefaults exception is right for a model loaded while the request was in flight and wrong for the model already resident, which is the one the saved global set describes. The signal that separates them is what the defaults replaced: at startup a local checkpoint is not persisted, so the first status publishes it over an empty one, while a model loaded during the request replaces a model that was already named. The exception now applies only in the second case, and both are pinned by tests.

A restore recording the hidden model (P2). Correct. restoreVisibleModelState switches back through setCheckpoint, and the model it steps off is the one the background load put there, so the outgoing snapshot recorded and persisted parameters the user never chose, despite the load itself passing persist: false. setCheckpoint takes the same option now, and the restore passes it. A visible switch still records the model being left; there is a test for each.

Checks: 2332 frontend tests, typecheck, build and lint errors at parity with main on the touched files. Both new behaviour tests were confirmed to fail against the previous commit, including one that only discriminated after I pinned its starting sampling: state left by an earlier test made the status a no-op and it passed for the wrong reason. The two branches still merge to a working build (2353 tests).

Still not verified: none of this has been clicked through in a browser.

chatgpt-codex-connector[bot]

This comment was marked as resolved.

@unslothai unslothai deleted a comment from shimmyshimmer Aug 16, 2026
@unslothai unslothai deleted a comment from shimmyshimmer Aug 16, 2026
@unslothai unslothai deleted a comment from chatgpt-codex-connector Bot Aug 16, 2026
@unslothai unslothai deleted a comment from shimmyshimmer Aug 16, 2026
@unslothai unslothai deleted a comment from chatgpt-codex-connector Bot Aug 16, 2026
@unslothai unslothai deleted a comment from chatgpt-codex-connector Bot Aug 16, 2026
@unslothai unslothai deleted a comment from shimmyshimmer Aug 16, 2026
@unslothai unslothai deleted a comment from shimmyshimmer Aug 16, 2026
@unslothai unslothai deleted a comment from chatgpt-codex-connector Bot Aug 16, 2026
@unslothai unslothai deleted a comment from chatgpt-codex-connector Bot Aug 16, 2026
@unslothai unslothai deleted a comment from chatgpt-codex-connector Bot Aug 16, 2026
@unslothai unslothai deleted a comment from chatgpt-codex-connector Bot Aug 16, 2026
@unslothai unslothai deleted a comment from chatgpt-codex-connector Bot Aug 16, 2026
@unslothai unslothai deleted a comment from chatgpt-codex-connector Bot Aug 16, 2026
@unslothai unslothai deleted a comment from shimmyshimmer Aug 16, 2026
@unslothai unslothai deleted a comment from shimmyshimmer Aug 16, 2026
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 👍

Reviewed commit: 060ae5d806

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@danielhanchen

Copy link
Copy Markdown
Member

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4fce9b8cc2

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +3746 to +3750
const baseParams = getReplayedParams(
state.rememberParamsPerModel,
outgoing ?? state.paramsByModel,
state.params,
modelId,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Mark replayed preset parameters as modified

When model A has remembered non-default parameters, selecting the Default preset on model B and switching back to A replays A's parameters here without updating the global activePreset or activePresetSource. The settings sheet determines whether Default has unsaved changes solely from activePresetSource (chat-settings-sheet.tsx:659-662), so it now labels A's tuned parameters as an unmodified Default preset, which can cause users to run experiments under settings different from the preset they believe is active. Reconcile the preset metadata when replay changes preset-owned fields, or remember that metadata per model.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not taking this one, though the mechanism is real: for Default, hasUnsavedPresetChanges reads activePresetSource alone (chat-settings-sheet.tsx:659-662), so a replay that moves params leaves the chip saying unmodified.

What it costs is the chip's hint, not the run. The params replayed are the ones the user set for that model, they are the values the sliders in the same panel are showing, and they are what the request carries. Nothing is sent that the user did not choose for that model.

Both remedies are worse than the symptom. Marking the replay modified was tried on this branch and reverted deliberately: activePresetSource is global, and pinning it stands down mergeBackendRecommendedInference and the Qwen thinking-params hooks for every model, so a model with nothing remembered stops getting its own recommended sampling and inherits whatever the previous one was running. That is the bleed this PR exists to stop, so buying a label with it is the wrong trade. Per-model preset metadata is the other option you name and it is a reasonable design, but it is a new feature with its own persistence and merge semantics rather than a fix, and it belongs in its own PR.

@danielhanchen

Copy link
Copy Markdown
Member

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

@danielhanchen
danielhanchen merged commit 2819e29 into main Aug 17, 2026
49 of 54 checks passed
@danielhanchen
danielhanchen deleted the studio-per-model-params branch August 17, 2026 03:39
shimmyshimmer pushed a commit that referenced this pull request Aug 17, 2026
…apshot

Per-model memory (#8757) landed in setParams, so the two layers are composed
rather than stacked: the replay lays a model's remembered settings over its
defaults, and the open chat goes on top of that. A chat is more specific than
either. A captured sampling edit now stays out of the per-model memory as well
as the installation defaults, since both are shared with every other chat, and
remembering one chat's temperature against the model leaked it to every new
chat opened on that model. Main independently marked the load-time Qwen block
and the status poll as model defaults, so those edits are dropped in favour of
main's, which also carry maxTokensCap.

Toggling Think applies its mode params again. A pinned chat stores every
sampling key, so marking the toggle as a model default meant the restore put
all of them back and the mode changed while temperature and top-p stayed on
the previous mode's values. The user asked for the mode in that chat, so the
values land and belong to the chat after. The load-time path applies the same
table unasked and stays marked.

The in-memory copy of the installation defaults follows what a model default
write just persisted. applyThreadScopedSettings falls back to it for a chat
with no snapshot, so leaving it stale meant a chat opened after a model load
ran the sampling of whichever model was loaded before it, until a reload.

top-k accepts -1, which disables it. ChatCompletionRequest allows -1 to 100
and default.yaml falls back to -1, so the old floor of 0 dropped a value whole
model families run with and the chat could not keep it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants