Include the new knobs in coexistence sizing

ValidateModelRequest accepts spec_draft_cache_type and the adjacent ctx_checkpoints, but _guard_chat_load_against_training never forwards either value into _estimate_gguf_required_gb, whose sizing functions also lack these parameters. Consequently, an SWA load requesting checkpoints is still budgeted as zero checkpoints and may be admitted alongside training despite requiring substantially more VRAM; conversely, a quantized draft cache is still priced as f16 and can be incorrectly rejected. Thread both settings through the validation and load-time coexistence estimator.

Useful? React with 👍 / 👎.

Fixed in 3b86c00. ctx_checkpoints is now threaded from the guard through _estimate_gguf_required_gb into _estimate_gguf_kv_gb and on to _estimate_kv_cache_bytes, so an SWA load asking for checkpoints is budgeted for them. Checkpoints are per-slot snapshots that scale with the slot context, and ggml-org/llama.cpp#21690 is an OOM caused by exactly that, so the under-count was real.

On the draft cache dtype: this estimator prices the drafter's weights but never its KV, so there is no f16 charge for a quantized draft cache to correct here. The field stays on the preflight because it decides which inherited draft-cache flags get stripped, which is what keeps the preflight approving the command /load actually runs. Its description now says that instead of claiming a sizing role it does not have.

Reload when the draft cache dtype is cleared

When a DSpark/DFlash model is resident with an explicit draft dtype such as q8_0, selecting the default f16 changes specDraftCacheDtype to null; this predicate then marks the setting as unpinned, so residentRuntimeMatchesConfig adopts the existing server and skips /load. The backend comparator has the same non-null guard, meaning even a request with a blank dtype cannot trigger the reset. Keep this comparison active whenever the selected speculative mode has a separate drafter so clearing the control actually relaunches with the default cache type.

Useful? React with 👍 / 👎.

Fixed in 3b86c00. Correct catch: the guard was copied from spec_draft_n_max, where an unset value means the platform default the loader recomputes, but that is the wrong model here. Both sides record what the load REQUESTED, so a server launched with no dtype holds null and a blank control still agrees with it, which means the guard bought nothing and cost the reset. Comparing unconditionally on both the frontend predicate and _runtime_matches_intent, and the test that pinned the old behaviour now asserts that clearing the dtype relaunches.

Price the effective checkpoint flag before fitting

When a load sets the new ctx_checkpoints field and also supplies an explicit --ctx-checkpoints/--swa-checkpoints in llama_extra_args, the extra argument is appended later and therefore determines what llama-server allocates, but this estimate uses only the first-class field. For example, ctx_checkpoints=0 with llama_extra_args=["--ctx-checkpoints", "256"] budgets no SWA snapshots while launching 256 per slot, so automatic fitting can admit a context that exhausts VRAM. Parse the last-wins effective value from the emitted arguments before passing it to the KV estimator.

Useful? React with 👍 / 👎.

Fixed in fcab813. Confirmed: both strip sites in the route act on inherited or stored args only, so extras sent for this load survive and last-win at launch, while the fit read the field. Added parse_ctx_checkpoints_override / resolve_ctx_checkpoints in llama_server_args.py, alongside parse_ctx_override and _extra_args_n_ubatch, and both the loader's _effective_ctx_checkpoints and _estimate_gguf_kv_gb now resolve through it, so the coexistence guard sizes the same number the child allocates.

Preserve explicit load-mode overrides

When both the Mmap/Mlock control and an explicit --load-mode (or deprecated alias) are supplied while Model Memory is off, this call removes the explicit argument even though llama_extra_args is documented and diagnosed in the UI as being appended last and winning. Thus a power user selecting mmap in the control and typing --load-mode dio silently runs mmap; distinguish inherited extras from arguments supplied for this load and strip only the inherited copy.

Useful? React with 👍 / 👎.

Fixed in fcab813. Right, and it contradicted the panel: CONTROL_OWNED_FLAGS maps --load-mode to Mmap/Mlock and the diagnostics say the typed flag is passed after the controls and wins. apply_load_mode_policy now strips nothing and only emits before the extras, and the inherited copy is dropped by the route when load_mode is in model_fields_set (aliases included, since a trailing one resets the whole mode), which is the same split the batch pair uses. apply_model_memory_policy keeps its own strip, because there the trailing alias would drop the keep-resident lock and the panel reports that flag as removed rather than winning.

Validate the draft cache dtype before launch

Reject or normalize this field to the supported KV-cache dtype set. A direct /load caller can currently supply an arbitrary or differently-cased value such as fp16 or Q8_0; when a separate drafter is attached, the value is appended verbatim to both draft-cache flags, so llama-server exits during argument parsing after the resident model has already been torn down. The main cache dtype path avoids this by lowercasing and checking its allowlist before emission.

Useful? React with 👍 / 👎.

Fixed in fcab813. The main cache dtype normalizes and checks its allowlist before emitting, and this one did not, so a direct caller could take the server down after the resident model was already unloaded. It now lowercases, checks the shared _VALID_KV_CACHE_TYPES, and drops an unmapped value with a warning rather than passing it through, so the load still comes up. The MTP draft-KV reserve prices the normalized value too, and the override store already dropped non-allowlisted dtypes.

Fall back to the legacy checkpoint flag

On llama-server builds that expose only the older --swa-checkpoints spelling, the capability probe checks only --ctx-checkpoints, so a value selected in the new Checkpoints control is silently skipped. The same patch explicitly classifies --swa-checkpoints as upstream's older spelling in llama_server_args.py:591-594; probe and retain the supported alias, as is already done for the draft-cache flags, so the control works with those builds.

Useful? React with 👍 / 👎.

Correct, and #9416 fixes it. #9410 had already merged when this landed, so the fix is a follow-up rather than another commit here.

Confirmed the alias direction: upstream renamed --swa-checkpoints to --ctx-checkpoints and kept the old name, so both parse on current builds and only the old one exists on a build predating the rename. probe_server_capabilities now walks ("--ctx-checkpoints", "--swa-checkpoints") and records which name this build has, the same handling the draft cache flags already use, and both emission sites use the recorded name, including the Windows full-offload path whose --ctx-checkpoints 0 was skipped for the same reason.

Strip inherited --mlock with the selected load mode

When /load omits llama_extra_args, inherits a resident --mlock, and supplies load_mode, these toggles remove --load-mode and most deprecated aliases but not --mlock, which belongs to the separate _MLOCK_FLAGS group. Because inherited extras are appended after the managed mode, selecting mmap or dio can still launch with locked weights while status records the selected value; enable strip_mlock whenever a first-class load mode owns inherited flags.

Useful? React with 👍 / 👎.

Strip saved load-mode flags before auto-switching

When an OpenAI auto-switch override contains the new load_mode together with carried llama_extra_args, this loop forwards the field but the shadow-stripping call below never enables strip_load_mode, its aliases, or strip_mlock. The extras are appended last, so an entry such as load_mode="mmap" with stale --load-mode dio launches DirectIO while the saved setting and requested-status echo say mmap; include the load-mode groups in the same stripping pass as the other newly forwarded controls.

Useful? React with 👍 / 👎.

Let the draft-cache control override environment defaults

For a separate-drafter load where LLAMA_ARG_SPEC_DRAFT_CACHE_TYPE_K or _V is set, _extra_args_draft_cache_types returns that environment value and these or assignments preserve it even when spec_draft_cache_type is supplied. The generated command-line cache flags override the environment at launch, so the fitter prices a different dtype from the one that runs; for example, a q4 environment default with a q8/f16 request can underbudget draft KV and select a context that exhausts VRAM. Only explicit extra-argument values should outrank the first-class control.

Useful? React with 👍 / 👎.