Repository navigation
One model does every job in every base #470
Description
A workspace has exactly one chat model and one embedding model (llm_settings, one row per workspace), and every task reads that same pair: extraction, type resolution, adjudication, the governance agent, mapping exploration, ontology suggestions, and Chat itself.
That is one choice serving jobs with opposite economics. Extraction is the high-volume one — every chunk of every document — and wants something cheap and fast. The agent that merges two entities, or answers a person in Chat, wants the careful one. Raising quality for the second means paying for it on the first. And bases differ: a base of Chinese contracts and a base of English papers do not want the same model, and a base under stricter rules may have to stay on a local endpoint while everything else talks to a hosted API. Today none of that is expressible.
Half the machinery is already there: concurrency limits are keyed by base URL + model (Administration → Deployment), so throttling already thinks in terms of several models at once. Only the choosing does not.
What this adds
Named model profiles, at the deployment level. A profile is a name plus an endpoint (base URL, key, model) — the same three fields the models page collects today, only more than once. The current chat and embedding settings migrate into the first two profiles, so an existing deployment keeps working with no action.
A default per task. Extraction, Chat, type resolution and governance, mapping exploration, embedding — each points at a profile. Anything left unset falls back to the general chat profile, which is today's behaviour.
A per-base override. A base may point any of those tasks at a different profile; unset means the deployment default. The lookup is one helper — model_for(kb, task) — so a call site never reads settings directly again.
Embedding is not like the others. A base's vectors are built with one model and carry its dimension; pointing a base at a different embedding profile means re-embedding that base. So that one change is asked for explicitly and enqueues the work, rather than quietly mixing two vector spaces in one index.
Per-model concurrency keeps working unchanged — it is keyed by endpoint and model, which is exactly what a profile is.
Cuts
- Backend: a
model_profilestable and per-task defaults, themodel_for(kb, task)helper, the migration that turns the existing row into two profiles, and tests. Every call site (extraction.rs,chat.rs,type_resolution.rs,adjudication.rs,governance.rs,mappings.rs,ontology_routes.rs) goes through the helper. - Per-base overrides: columns, API, and the fallback chain.
- UI: Administration → Models becomes a table of profiles with a dialog to add one (test connection before saving, like data sources) plus the task defaults; base settings gets the overrides.
- Changing a base's embedding profile: confirm, re-embed, show progress.
Activity
Maya-Kid commentedon Sep 30, 2026
@WaylandYang is #470 open for a contributor? If so, I'd start with cut 1 as written:
model_profileswith per-task defaults, the migration that turns the workspace's existing row into a chat and an embedding profile, andmodel_for(kb, task)with every call site going through it. No per-base overrides or UI yet.Two questions first:
model_forknows be the list in the issue (extraction, chat, type resolution and governance, mapping exploration, embedding), or finer, following the kinds of call The usage log says which call it was #996 separates, for example alignment votes and rule proposals apart from extraction?WaylandYang commentedon Oct 4, 2026
Yes, it is open, and it is yours. Cut 1 as you describe it:
model_profileswith per-task defaults, the migration that turns the workspace's row into a chat and an embedding profile, andmodel_for(kb, task)with every call site going through it. No per-base overrides and no UI yet.extraction(reading a passage, time interpretation),alignment(kind-word and phrase votes, rule proposals),governance(type resolution, entity adjudication, errata, the ontology agent),chat,mappings,embedding. Alignment has to be its own task: it is six tenths of the tokens per document, and 0044 now names a model trained for extraction and alignment as the planned route, which needs somewhere to plug in. The usage log says which call it was #996's call names stay as they are for the usage log.A short ADR first (the next free number is 0067) with the task list and the fallback chain, then the code.
Maya-Kid commentedon Oct 15, 2026
The ADR is up as #1239 (0067): the six tasks as you gave them, with the calls each one covers, and the chain
model_for(kb, task)follows. Then two code PRs: every call site goes throughmodel_forwhile it still reads today's row (no schema change, no change in behaviour), and thenmodel_profilesandmodel_task_defaultswith the migration. Cut 1 adds no endpoints, so until the UI in cut 3 a task gets a profile of its own only through the database, which is enough for the bench.Three points are open in the record:
implication::read_phrasesis not on your list. It fills in the readings for the implication rules that phrase alignment proposes, so I'd put it underalignmentwith them.chatoverride also cover its other unset tasks.