Both entries mirror the 24 GB one at ctx-size 262144 with parallel 1, which is
the largest pool a single conversation can reach on this model since n_ctx_train
is already 262144. Everything that is arch-specific carries over unchanged: the
q8_0 K that also types the QSA indexer cache, ngram-mod as the only speculative
type because there is no MTP head, ctx-checkpoints 8 because a checkpoint here
is the entire 112 MiB recurrent state, no-mmproj-offload because fit commits the
expert split before CLIP allocates, and the byte-identical chat template pin with
reasoning-effort xhigh.

The dual-GPU entry deliberately carries no tensor-split, and it is the only entry
in that file without one. The tier-wide 1,2 split is correct for its four
siblings because they all run fit = off, but fit refuses to run at all once the
user has populated tensor_split on a multi-device host, and that refusal is
downgraded to a warning rather than being fatal. The server then proceeds with
the raw preset values, n-gpu-layers = -1 offloads all 48 blocks, and since the
26.822 GiB hash table and token_embd are pinned to the CPU by construction the
remaining ~63.5 GiB gets distributed one-third to two-thirds. That asked
22911.90 MiB of an 8192 MiB RTX 2060 SUPER and died as "unable to allocate CUDA0
buffer", which reads like a VRAM sizing problem rather than the disabled fit it
actually is. Without the key fit computes the split and the expert overflow
patterns itself, and n-gpu-layers stays at -1 so it is not tripped either.

Both entries use load-mode = mmap rather than the dio the rest of the presets
migrated to. The 26.822 GiB per_layer_token_embd hash table is created with
TENSOR_READ_LAZY, the loader gates that flag on use_mmap, and dio clears it, so
under dio the lazy branch never runs and the table is read from disk and held in
full. Under mmap the tensor aliases the mapping instead and only touched pages
become resident; a token gathers 16 of the table's 320 million rows, so the
working set stays in the hundreds of MiB. The table is well past the 4 GiB
auto-lazy threshold, so this needs no explicit tensor-read-lazy. no-host is part
of the same mechanism rather than an independent choice: without it CUDA_Host is
prepended to the CPU buffer list, the chosen buffer type then fails the
is_default_buft test, the aliasing branch is skipped, and the table goes through
cudaMallocHost even with mmap selected.

The host-memory side of that matters more on these two tiers than it did on the
24 GB one. This box has 128 GiB rather than the 191 GiB the 24 GB entry was
measured on, so dio would have held roughly 88 GiB of the 90.635 GiB file as
anonymous memory alongside cache-ram 32768, against 106 GiB free. Under mmap the
same bytes are reclaimable page cache.

Unverified. The dual-GPU budget is arithmetic only: 8006 plus 14141 MiB free,
minus fit-target 3072 which is broadcast to both devices rather than split
between them, leaves 16003 MiB usable against a fixed cost of roughly 9024 MiB
from 4600 context, 1821 compute and 2603 of non-expert weights. That would leave
about 7000 MiB, or 5 of 48 expert layers at 1.270 GiB each, where the 24 GB card
held ~8.3 and measured 19.87 t/s. If the observed split lands well under that,
the next lever is a per-device fit-target pair, since 3072 currently reserves
38 percent of the 8 GB card. Throughput on a 2060 SUPER paired with a 4070 Ti
SUPER is also uncharacterised, and the tier's usual argument for pinning to one
card does not apply here because nothing fits on either card alone.

The router launch command had grown to 275 characters, past the 260 the
Windows Run dialog accepts, because it carried the host, port, models
directory, preset path and two CUDA driver variables inline. llama.cpp
already reads every server flag from a LLAMA_ARG_* environment variable
(common/arg.cpp, set_env), and the CUDA driver reads CUDA_* from the
environment, so all six values move into a gitignored .env at the repository
root and the command shrinks to 80 characters. A flag on the command line
still overrides its variable, so switching VRAM tiers for one launch stays a
--models-preset away; llama-server prints a warning naming the overridden
variable when that happens.

load_env.ps1 mirrors the parser in windows_manage_large_language_models byte
for byte: split on the first "=", skip blank lines and lines whose key
contains "#", no quoting, no trimming. Values in .env therefore stay bare and
without spaces around "=". The file is limited to server-scoped LLAMA_ARG_*
keys and CUDA_* so it is safe to dot-source before llama-bench or
speed-bench.ps1; unscoped keys such as LLAMA_ARG_CTX_SIZE would reach every
binary that shares the common argument parser. Router mode is pure C++, so
the launch no longer needs conda activate.

CUDA_SCALE_LAUNCH_QUEUES=4x is recorded in docs/presets.md with the numbers
behind it: on the 16 GB + 8 GB layer split with Qwen3.8-27B.IQ4_XS it lifts
prompt processing 10-11.5 percent at every depth measured (pp8192 1025 to
1143 t/s, pp2048 at 32k 690 to 759, at 64k 509 to 565) and does nothing on a
single GPU. Rebuilding with GGML_SCHED_MAX_COPIES=4 was measured at the same
time and is a non-lever on this pair (-2.5 percent alone, +1.5 percent on top
of the variable), so the build default stays at 1 and nobody needs to repeat
that experiment.

… mechanisms into docs/presets.md

The 6363-word model_tuning.md was read in full for any model-entry task. It now lives as
five files under docs/model_tuning/, one per family, so a task reads presets.md plus one
family file: 2.2k-4.3k words instead of 7.5k.

Four mechanisms that model_tuning.md derived per model - no-host / cudaMallocHost, the fit
aborts and blind spots, the swa-full window collapse, and the get_can_shift force-disable -
are cross-model by the files' own contract and now live once in docs/presets.md. Where the
two source sections disagreed on a vendor line ref the newer (2026-08-27) set is kept.

Every backticked identifier and every measured number from the old file survives in the new
set except the superseded line refs, verified by diffing the token sets. AGENTS.md loses its
Presets section, which restated the Reference table, and its Traps and Reference rows point
at the new files. The two skills that route to docs/ are updated.

…V at 131072 context

The entry ran at tensor-split 1,2 and ctx-size 200000 with q5_0/q4_1 KV, which
left the 4070 Ti SUPER - the display GPU - with 28 MiB free. Below roughly
250-300 MiB Windows' video memory manager demotes part of the working set to
system RAM behind the driver's back; cudaMalloc succeeds, memory_breakdown()
reports nominal figures and the log shows nothing, so throughput was the only
symptom. Through the router with MTP + ngram-mod on, identical requests: 617
t/s on a 32k prompt and 48.9 / 43.5 / 31.5 t/s generation (code / reasoning /
after 32k). Switching only the KV type to q4_0 at the same 200000 context
raised the card to 347 MiB free and pp to 884, tg to 52.1 / 58.0 / 41.6 -
llama-bench puts the intrinsic q4_0 effect at under 1 percent pp, so the
gain is the paging going away, not the quant.

tensor-split is the prompt-processing lever on this pair. Layer-mode prefill
is a pipeline whose throughput is set by its slowest stage, and the 2060
SUPER has about a third of the 4070's tensor throughput, so every layer
moved off it raises pp - llama-bench pp2048 1134 / 1375 / 1578 / 1745 t/s at
1/2, 1/3, 1/4, 1/6 - while bandwidth-bound generation moves 3-8 percent. The
ratio is paid in 4070 VRAM: at 1,3 each 1k tokens of context costs that card
about 15 MiB (three quarters of the KV, its compute scratch, and the MTP
draft context's KV, which is pinned to the target n_ctx), so 1,3 cannot
reach 200000 even at q4_0. 150000 loaded with 445 MiB free from a direct
launch but only 238 MiB when the router launched it minutes later - the
compositor's share of the card moves by hundreds of MiB with desktop state -
and 163840 already showed the paging onset (208 MiB, pp -22 percent).
131072 keeps 563 MiB free through the router and measures 1085 pp and
57.6 / 62.4 / 46.4 tg, indistinguishable from 150000. Against the old entry
that is +76 percent prompt processing and +18 / +43 / +47 percent generation.

split-mode tensor (#19378) was measured and rejected: 1,2 at 131072 gives
63.8 / 71.0 / 48.0 tg but 505 pp on the 32k prompt and llama-bench pp2048
falls 1134 to 663, because every layer ends in an allreduce over the 2060's
PCIe 3.0 x4 link and the layer pipeline is gone. It also caps context lower -
the compute buffer is allocated in full on every device and the draft
context allocates a second one - and asserts on q5_0/q4_1 KV. threads 4-24
and ubatch 1024/2048 were measured flat or worse and are left alone. Draft
acceptance is 0.73-0.77 with the embedded Q4_0 MTP head, so the 0 percent
collapse reported for this family does not reproduce on this file.

kv-unified is dropped from the entry; with parallel = 1 it had no effect.

The WDDM margin is recorded in AGENTS.md and docs/presets.md as a
cross-model rule, the tensor-split mechanism in docs/presets.md, and the
full measurement table in docs/model_tuning/qwen.md. Submodule advanced to
b10759, the build every number above was taken on.

The 16 GB tier only fits this model at IQ3_XXS, and the 4070 Ti SUPER + 2060
SUPER pair has 21.9 GiB, so IQ4_XS fits with the dflash drafter alongside.

The headroom levers are the inverse of the Qwen ones on this model, so the
conclusions in docs/model_tuning/qwen.md do not transfer. Only 13 of the 52
layers are full attention and they carry head_count_kv = 2, so the
context-scaling cache is 4368 B/token at q5_0 K and q4_1 V - 1092.00 MiB for
the whole 262144-token pool - against 14860.33 MiB of GPU weights (13488.93
target plus 1371.40 draft; token_embd.weight's 721.42 MiB stays on the host).
Dropping KV to q4_0 would buy 78 MiB where the same move is worth ~1.4 GiB on
Qwen3.8-27B, so KV type and context are not the levers there; tensor-split is,
and it is paid in prompt processing.

tensor-split 1,3 was measured and rejected. Normalised to the heaviest
compositor state seen here (2990 MiB), 1,2 at -ub 256 leaves 1229 MiB free and
1,3 leaves 173 - under the 400 MiB floor where WDDM starts demoting the working
set. 1,3 is worth it on throughput alone: 1256 vs 1044 t/s prompt processing on
a 31047-token prompt, and 14.77 / 14.83 / 12.10 vs 14.20 / 13.96 / 11.45
generation per target pass. It is not worth an invisible 20-45 percent loss the
first time the desktop grows. Compare rows by tg per mean len, never by raw tg;
the drafter's sampling variance moves the raw figures 2.4-3.5 on one prompt.

parallel = 2 with ctx-size 262144 and no kv-unified gives two slots of 131072,
because n_ctx_seq is n_ctx / n_seq_max when the cache is not unified and 131072
is exactly muse-glimmer.context_length - so no override-kv is needed. It costs
~408 MiB and stops auxiliary client traffic from evicting the main
conversation: a long prompt prefilled 7476 tokens in slot 1, two short side
calls landed in slot 0, and the follow-up returned to slot 1 with prompt_n = 14
and 7512 tokens cached, with no prompt cache round-trip. kv-unified would
defeat this - it clears every idle slot on each new task - and a second, smaller
model would be worse still, since models-max is router-scoped and forces a full
unload and reload per call.

ubatch-size = 256 is a workaround for an upstream over-reservation, not a
tuning choice. common_base_params_to_speculative caps the draft context at
n_outputs_max = 32, but llama-context.cpp discards it because
llama_model_has_encoder() is true for LLM_ARCH_DFLASH and substitutes n_batch,
so reserve() sizes the logits tensor at min(n_ubatch, n_batch) rows - 512 x
202048 x 4 = 394.62 MiB, twice over. The draft context's own log line is the
tell: n_outputs_max = 2048 beside n_outputs_max_per_seq = 16. Halving the
microbatch halves it, 803.03 to 401.90 MiB, and costs 1.5 percent prompt
processing with generation per pass flat. Safe for vision here only because
mtmd_decode_use_non_causal() is true for the Gemma projectors alone, so this
model's 1024-token images may span microbatches; verified with a real 1247-token
image request. The same setting would silently degrade images on gemma-4.

Also measured and rejected: spec-draft-device CUDA0, which would move 1.67 GiB
off the display card but aborts because dflash reuses the target's
output.weight; q8_0 KV, since ggml_cuda_get_best_fattn_kernel returns MMA_F16
for head dim 128 on both cc 8.9 and cc 7.5 whenever Q->ne[1] > 2, so prefill
runs the same kernel either way; and cache-reuse, which an mmproj force-disables
at startup and again at runtime.

The CUDA sysmem fallback policy note in docs/presets.md is replaced with a
measured result: set to Prefer No Sysmem Fallback per program, an over-committed
launch now dies with cudaMalloc failed: out of memory at alloc_tensor_range
instead of spilling. Whether it also prevents demotion of an already-committed
working set is still untested. The 250-300 MiB cliff figure is also conservative
for this machine - rows at 253 and 293 MiB free showed no loss at all.

… prompt

The OpenCode plugin pushed "SESSION_ID=<id>" into output.system on every turn
via experimental.chat.system.transform. That put a per-session unique string
into the part of the prompt every request shares, so llama.cpp's prompt cache
found no reusable prefix from one session to the next and re-prefilled the whole
system block each time. Measured against the dual-GPU Muse-Glimmer-30B entry
with the plugin renamed out of the way as the only variable: a second session on
an identical prompt re-prefilled 22705 of 22724 tokens with the hook and 1 token
without it, about 21.6 s at 1050 t/s on every new session.

It matters more than it would elsewhere because that entry loads an mmproj, and
an mmproj force-disables cache-reuse at startup and again at runtime. Exact
longest-common-prefix matching is then the only reuse mechanism there is, so one
changed token early in the prompt costs the entire remainder. Within a session
nothing was wrong - turn-to-turn reuse re-prefilled 17-18 tokens - so this only
ever cost session starts.

The shell.env hook carries the same value at zero prompt cost, and it suits the
actual use case better: the scratch directory is created by a shell command
anyway, so the literal id never has to be in context. The trade is that it no
longer is - AGENTS.md now says to use $env:SESSION_ID inside a shell command, or
to read it once with Write-Output when Write/Edit needs an absolute path, and
records why the two harnesses differ so this does not get "fixed" back.

Claude Code's SessionStart hook in .claude/settings.json is deliberately left
alone. Its stdout is injected as context at the start of the conversation rather
than into the system block, so it does not sit inside the large cached prefix,
and Anthropic's cache is TTL-bound and server-side, which makes cross-session
reuse a much smaller prize. Claude Code also has no per-session equivalent of
shell.env; its env setting is static and cannot carry a session id.