Overview

The existing sleep_preserve_parameter_names API preserves frozen Engram table weights and scales across level-2 sleep, without retransmitting them during target weight updates (#57891). Extend the same API to draft-private weights for target-only updates; both use cases are documented. Related: #52479.

Claims

• Select draft-private parameters with --sleep-preserve-parameter-names 'draft.*', or individual draft.<name> patterns.
• Reuse existing CPU snapshots and in-place restoration; exclude target-shared Parameter identities.
• Allow an empty draft.* selection (no draft, or no private parameters). Other unmatched patterns still raise.
• No new switch, runner hook, RPC, inference-loop work, or graph recapture. Core diff: two files, 17 additions / 3 deletions.
• Document both the original frozen Engram table use case and draft.* in the existing Sleep Mode section; preserve the distinction between retention and Engram inference-time CPU offload.

Validation

Changed-file pre-commit passes. Focused CPU regression command:

.venv/bin/python -m pytest --confcutdir=tests/v1/worker \
  tests/v1/worker/test_gpu_worker.py -k sleep_preservation -v

The tests check shared ownership, in-place restoration, an absent/all-shared draft, and unmatched target names: baseline 3 failed, simplified pinned candidate 3 passed, main-based candidate 3 passed. Current-main base: 3c062d391d15cfa41d3f321cad24f3030c892a97. Tests used the experimental container's dependencies with current-main Python source; no kernels were exercised by these CPU tests.

H200 model evaluation on the earlier equivalent nonempty-selection implementation, based on 43b4aaea with the composed Vime patches:

Six requests per phase; temperature 1, top-p 0.95, context 65,536, eager execution; model revision 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0. This is not an otherwise-unpatched current-main GPU result. No throughput improvement or ROCm/CPU/XPU/all-drafter validation is claimed.

The graph-enabled eight-GPU Agent training/rollout E2E also passes on the earlier 2ec58c52 implementation in the installed experimental image: two training steps, maximum observed MTP acceptance 87.6%, mean acceptance length up to 3.63. This does not substitute for a clean current-main GPU test.

Details

Level-2 sleep discards target and draft weights. Reloading only the target leaves draft-private storage undefined: verification still works, but speculative acceptance can disappear. This opt-in extension saves the current draft before sleep and restores it at weight wake, using the existing narrow no-grad restore helper. Draft-buffer handling is unchanged.

This is distinct from #52487 (restore a configured disk checkpoint during reload_weights) and #60139 (explicitly load a new draft checkpoint). Neither retains the current draft across target-only IPC/NCCL updates. Those disk APIs remain useful and unchanged.

The measured snapshot is approximately 203 MiB per rank for this model; CPU storage and copies occur only at sleep/wake. Retained parameters must not also be supplied by the weight-update sender, as with the existing retention API.

AI assistance was used for implementation, investigation, and validation. Submitted at the human maintainer's explicit request; kept as a draft for human line-by-line review, which is not asserted complete.

BEFORE SUBMITTING, PLEASE READ https://docs.vllm.ai/en/latest/contributing