iframe-proxy

njhill · 2026-09-23T16:18:35Z

A one-token prompt tail over existing context runs exactly like a decode. This covers a full prefix-cache hit, a P/D KV import on the decode node, and a final one-token chunk. The scheduler may also pad such a tail with placeholder drafts to the K+1 spec-decode shape. The Mamba, GDN and KDA metadata builders already classify these rows as decodes.

Since #51865, any prefilling row made MRV2 reject the FULL decode graph for the whole batch. In P/D serving, every newly arriving request therefore pushed its step's decode batch down to PIECEWISE.

Classify these rows once, from request state, in gather_batch_req_state: prefill_runs_as_decode_np marks prefilling rows that schedule exactly one new prompt token (not counting drafts) over existing context. A batch is eligible for the FULL decode graph if every prefilling row is in this mask. Fresh prompts and multi-token chunks are still excluded, so #49918 stays fixed. PCP batches keep the old behaviour.

Add prefill_runs_as_decode_np to BatchReqState/InputBatch. It is sliced for ubatches and None for PCP-local batches.
Add decode_graph_eligible to BatchReqState/InputBatch, derived from the mask. get_uniform_decode_token_count now takes decode_graph_eligible in place of has_prefill.
The autoregressive and multi-module MTP drafters use the same flag, so they match the target model's graph mode.
MambaHybridModelState now clears is_prefilling for masked rows, replacing its own prompt-tail check from [Bugfix][MRV2] Treat padded prompt tails as spec-decode rows for hybrid models #58434. The is_prefilling it passes to the builders is then the single source of truth for both prefill/decode splitting and spec-decode draft counts, as suggested in review of [Bugfix][MRV2] Treat padded prompt tails as spec-decode rows for hybrid models #58434. This also covers one-token mid-prompt chunks, which the Mamba builder already ran as decodes.
Compute the per-request draft counts once in gather_batch_req_state and reuse them in prepare_inputs.

This is an alternative / more general fix to #58374, and a follow-up to #58434 (merged), which fixed the hybrid-model correctness issue for padded prompt tails.

Test Plan

pytest tests/v1/worker/test_mamba_hybrid_model_state.py tests/v1/worker/test_gpu_batch_ordering.py \
  tests/v1/worker/test_gpu_ubatch_slicing.py tests/v1/worker/test_gpu_batch_shard.py \
  tests/v1/worker/test_gpu_pcp_manager.py tests/models/test_deepseek_v41_replay_start.py \
  tests/v1/attention/test_gdn_metadata_builder.py tests/v1/attention/test_mamba_update_block_table.py \
  tests/v1/spec_decode/test_dynamic_sd_cug.py

173 passed. New tests cover the mask classification (padded tail, one-token mid-prompt chunk, fresh prompt, multi-token chunk) and the hybrid model state passing the adjusted is_prefilling to the GDN builder.

Used Opus 5.5 to help with this.

claude

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

njhill · 2026-09-23T20:42:42Z

Pre-PR vs Post-PR

Before = 8033bd08c1, after = this PR. Both arms include the hybrid padded-tail fix from njhill:mrv2-hybrid-padded-prompt-tail and run with VLLM_USE_V2_MODEL_RUNNER=1. NixlConnector 1P+1D on GB200, random dataset with --ignore-eos, 60s per point (vllm-bench). non-FULL steps = share of decode-side steps not dispatched as FULL CUDA graph (--cudagraph-metrics).

Qwen3-0.6B, P/D TP1/TP1 (mean of 2 reps)

ISL/OSL	req/s	non-FULL steps	mean TPOT (ms)	p99 TPOT (ms)	p99 ITL (ms)	median TTFT (ms)
1024/256	8	1.62% → 0.00%	2.08 → 1.91 (-8.2%)	2.43 → 2.04 (-16.1%)	8.39 → 2.65 (-68.4%)	33.44 → 26.69 (-20.2%)
1024/256	16	3.72% → 0.00%	2.46 → 2.18 (-11.3%)	3.08 → 2.44 (-20.9%)	8.93 → 5.98 (-33.0%)	43.53 → 34.56 (-20.6%)
1024/256	32	11.11% → 0.00%	3.93 → 2.92 (-25.7%)	5.18 → 3.47 (-33.0%)	13.68 → 10.04 (-26.6%)	65.51 → 44.93 (-31.4%)
4096/1024	4	0.84% → 0.00%	2.52 → 2.55 (+1.4%)	3.06 → 3.12 (+1.8%)	23.21 → 21.71 (-6.5%)	79.54 → 71.33 (-10.3%)
4096/1024	8*	2.29% → 0.00%	7.96 → 16.05 (+101.7%)	11.23 → 28.69 (+155.5%)	824.09 → 290.27 (-64.8%)	1.32s → 4.18s (+217.5%)
4096/1024	16*	8.31% → 1.63%	40.52 → 42.45 (+4.7%)	72.17 → 64.86 (-10.1%)	89.12 → 83.87 (-5.9%)	33.36s → 29.66s (-11.1%)

Llama-3-8B-Instruct, P/D TP1/TP1 (mean of 3 reps)

ISL/OSL	req/s	non-FULL steps	mean TPOT (ms)	p99 TPOT (ms)	p99 ITL (ms)	median TTFT (ms)
1024/256	8	2.98% → 0.00%	3.80 → 3.73 (-1.9%)	4.05 → 3.94 (-2.7%)	9.45 → 7.54 (-20.3%)	48.49 → 44.63 (-8.0%)
1024/256	16	6.60% → 0.00%	4.29 → 4.16 (-3.0%)	5.34 → 4.56 (-14.7%)	10.33 → 9.94 (-3.8%)	57.68 → 53.20 (-7.8%)
1024/256	32	16.62% → 0.00%	5.86 → 5.34 (-8.9%)	7.12 → 6.10 (-14.4%)	16.02 → 16.70 (+4.3%)	74.52 → 67.41 (-9.5%)
4096/1024	4	2.24% → 0.00%	6.42 → 6.28 (-2.0%)	7.21 → 6.96 (-3.4%)	30.48 → 28.86 (-5.3%)	125.13 → 129.47 (+3.5%)
4096/1024	8*	8.36% → 0.00%	14.63 → 14.53 (-0.7%)	17.82 → 18.89 (+6.0%)	44.22 → 62.64 (+41.6%)	201.41 → 283.57 (+40.8%)
4096/1024	16*	15.35% → 2.47%	26.57 → 26.87 (+1.1%)	31.54 → 31.68 (+0.4%)	88.60 → 103.38 (+16.7%)	22.62s → 23.85s (+5.4%)

* Offered load exceeds achieved throughput (queueing on the prefill side), so these points are noisy between reps.

MRV1 vs MRV2, Qwen3-0.6B, P/D TP1/TP1 (Δ vs MRV2 before)

ISL/OSL	req/s	arm	non-FULL steps	mean TPOT (ms)	p99 TPOT (ms)	p99 ITL (ms)	median TTFT (ms)
1024/256	8	MRV1	0.00%	1.82 (-12.5%)	2.04 (-16.1%)	2.53 (-69.9%)	25.64 (-23.3%)
1024/256	8	MRV2 before	1.62%	2.08	2.43	8.39	33.44
1024/256	8	MRV2 after	0.00%	1.91 (-8.2%)	2.04 (-16.1%)	2.65 (-68.4%)	26.69 (-20.2%)

gsm8k (5-shot, strict / flexible)

model	MRV1	MRV2 before	MRV2 after
Qwen3-0.6B, P/D TP1/TP1	0.4223 / 0.4177	0.4223 / 0.4177	0.4223 / 0.4177
Llama-3-8B-Instruct, P/D TP1/TP1	–	0.7612 / 0.7574	0.7627 / 0.7574
Qwen3-Next-80B-A3B-Instruct (hybrid GDN), P/D TP2/TP2	–	0.8158 / 0.8567	0.8097 / 0.8560

njhill · 2026-09-23T20:52:24Z

/ci run

github-actions · 2026-09-23T20:52:45Z

✅ Triggered Buildkite CI #90770 for commit 967cce3a94e8.

njhill · 2026-09-23T23:50:53Z

/ci retry

github-actions · 2026-09-23T23:51:12Z

✅ Triggered Buildkite CI #90806 for commit 135d447e7ada, running 7 failed step(s) from Buildkite CI #90770.

njhill · 2026-09-24T15:22:11Z

/ci retry

github-actions · 2026-09-24T15:22:32Z

✅ Triggered Buildkite CI #90961 for commit 0af8d3df07cc, running 2 failed step(s) from Buildkite CI #90806.

seongyun1104 · 2026-09-25T01:02:40Z

The mechanism here is clearer than the description suggests, and I think it is worth
stating explicitly because it also marks where the gain stops.

Why the current behaviour is a hard floor, not a heuristic. Before this change a batch
with has_prefill got uniform_tok_count = None, and _is_compatible rejects every
descriptor that carries a concrete tier:

# vllm/v1/worker/gpu/cudagraph_utils.py:116-120
    return (
        (
            desc.uniform_token_count is None
            or desc.uniform_token_count == uniform_token_count
        )

In the separate-decode-routine, non-varlen configuration (:312) every FULL decode
descriptor is built with uniform_token_count=decode_query_len (:328), so a None from
the caller could not match any of them — the batch went to PIECEWISE unconditionally, not
because a graph was missing but because it was unaddressable. Passing the real tier is
what makes those graphs reachable at all. That is a different and stronger claim than
"one-token tails behave like decodes", and it is consistent with the 1.62% → 0.00%
non-FULL share you measured.

For the varlen configuration the situation is the mirror image: those descriptors leave
uniform_token_count unset and constrain max_query_len instead (:301-307), which is
the field #58436 is separately restricting. The two paths are mutually exclusive
(capture_varlen_decode vs separate_decode_routine and not self.varlen_decode), so the
two PRs do not overlap — I checked, since they look adjacent.

Where the newly eligible batches land. decode_query_lens is plural only under dynamic
SD, where it is derived from the schedule (:284-289); otherwise it is a single value
(:291). When it is plural, one descriptor per tier is appended to the same
descs_by_mode[decode_mode] list (:333-334), and the candidate ladder then consumes the
range once across all of them, grouped only on num_tokens:

# :377-385
                current_range_start = 0
                for num_tokens, group in groupby(lora_descs, lambda d: d.num_tokens):
                    matching = list(group)
                    for i in range(current_range_start, num_tokens + 1):
                        key = (i, num_active_loras)
                        self._candidates.setdefault(key, []).extend(matching)
                    current_range_start = num_tokens + 1

So for a batch size whose next captured num_tokens belongs to a different tier, the
candidates at that key carry the wrong uniform_token_count, _is_compatible rejects them
on the clause quoted above, and dispatch falls through to CUDAGraphMode.NONE (:522).
This PR does not create that behaviour, but it does route a new class of batches into it:
one-token-tail batches now arrive with a concrete tier where they previously arrived with
None.

Why the existing test does not show this. test_dynamic_sd_full_cudagraph_covers_all_uniform_decode_shapes
sweeps every num_reqs in [1, 512] and asserts FULL each time, so it reads as though the
ladder is already gapless. It passes because the fixture makes it gapless by construction —
cudagraph_capture_sizes=list(range(1, max_capture_tokens + 1)), every integer up to
max_num_seqs * max_decode_query_len, with the helper's own docstring saying it "uses an
exact capture-size grid so that every valid uniform decode shape has a directly matching
FULL graph candidate". With no gaps current_range_start advances by one per group and
never spans a tier boundary. On the default capture ladder the gaps are where the tiers
interleave, and that case has no coverage in this file.

I am not suggesting the test is wrong — its grid is deliberate, and it is testing the
candidate construction rather than the ladder. But if you want the benchmark shape you
measured to hold under a dynamic-SD schedule with several query lengths, that is the case
worth adding, and it is cheap: the same sweep with the inferred default capture sizes
rather than the exhaustive grid.

Disclosure. I opened #56136 for the ladder behaviour above, so I have an interest in it
being real; treat the reasoning rather than the source. It is a dispatch-coverage argument
only — I have no hardware and measured no recovered graphs, so I am claiming reachability,
not throughput. Nothing here blocks this PR and I am not asking you to act on that one; the
part that bears on yours is only that the population entering the ladder grows, so if you
benchmark with a multi-tier schedule and see the non-FULL share stop short of 0.00%, this
is the reason rather than something in your change.

njhill · 2026-09-25T16:22:41Z

/ci run

github-actions · 2026-09-25T16:23:00Z

✅ Triggered Buildkite CI #91237 for commit 1cbb155a5d44.

njhill · 2026-09-25T18:30:13Z

/ci run

github-actions · 2026-09-25T18:30:58Z

❌ This PR is 1 commit behind upstream main. Your branch must contain every commit currently on upstream main. No new CI build was started. Merge or rebase onto the latest main, then rerun /ci run. To test this branch at your own risk, use /ci run --allow-stale.

njhill · 2026-09-25T21:00:27Z

/ci run

njhill · 2026-09-25T22:18:52Z

/ci run

github-actions · 2026-09-25T22:19:11Z

* [Bugfix][XPU] store the pointer raw bit pattern instead of its numeric value (#54514) Signed-off-by: Lai, Yejing <yejing.lai@intel.com> Co-authored-by: Kunshang Ji <kunshang.ji@intel.com> * [Attention][CPU] Run Zen CPU encoder attention on zentorch SDPA (#54508) Signed-off-by: priyansh jain <priyansh.jain2@amd.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [MRV2] Validate MRV2 entrypoint logits processors (#57728) Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com> * [Bugfix][Rust Frontend] Prevent MM timing from enabling debug tracing (#58378) Co-authored-by: Bugen Zhao <i@bugenzhao.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: reidliu41 <reid201711@gmail.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com> * [XPU][UT] Align HF and vLLM inputs for Qwen2 embedding test by preventing Sentence Transformers from applying chat template (#58117) Signed-off-by: RyanMa29 <ziyang.ma@intel.com> * [CPU] Gate the AVX10.2 paths on compiler support (#58133) Signed-off-by: R <Ganesh.R@amd.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Li, Jiang <jiang1.li@intel.com> * [Perf][Frontend] Offload streaming derender detokenization (#57528) Signed-off-by: Shrey Gajjar <shreygajjar007@gmail.com> * [Multimodal] Reuse the supplied tokenizer in the MiniMax-M3 VL processor (#58460) Signed-off-by: Zijing Liu <liuzijing2014@gmail.com> * [XPU] enable XPU GRAPH by default (#51600) Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com> * [ROCm][DSv4.1][Perf] Emit MXFP8 from the sparse decode reduce and run wo_a as a grouped FP8 GEMM (#58456) Signed-off-by: fai <fangzhouai@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * Remove `.gemini/` and `CLAUDE.md` (#58541) Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * [Bugfix][Pooling] Fix JinaVL label configuration and restore multimodal tests (#57347) Signed-off-by: Linze-Shi <linzeshi0@gmail.com> * [Chore] Use Transformers v5 names and drop redundant processor `use_fast` (#58550) Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * [Refactor] Remove dead or duplicate tests (#58446) Signed-off-by: yewentao256 <zhyanwentao@126.com> * [Perf][Attention] Bound FlashInfer prefill dequantization scratch (#57918) Signed-off-by: Yueh-Ting Chen <yueh.ting.chen@gmail.com> * fix(config): apply presence_penalty/frequency_penalty from override-generation-config (#50769) Signed-off-by: Chenglun Hu <chenglunhu@gmail.com> Signed-off-by: hclsys <chenglunhu@gmail.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> * [Bugfix] Resolve the Hub revision once per repo (#56092) Signed-off-by: Wauplin <lucainp@gmail.com> Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [Frontend] Remove the slow tokenizer mode (#58545) Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Revert "[DSpark] Support pipeline-parallel targets in aggregated serving (#56956)" (#58484) * [transformer] RMSNorm matching for alternative rsqrt (#54461) Signed-off-by: Thomas Ortner <boh@zurich.ibm.com> Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> * [Bugfix] Count unsplit Idefics3 image patches (#48760) Signed-off-by: nightcityblade <nightcityblade@gmail.com> Co-authored-by: nightcityblade <nightcityblade@gmail.com> Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk> * [Bugfix] Keep JIT warmup under enforce-eager when fault tolerance is on (#58593) Signed-off-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: Kimi <noreply@moonshot.cn> * [Core] Skip JIT monitor when JIT warmup is disabled (#58590) Signed-off-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: Kimi Code <noreply@moonshot.cn> * [Fast Start] Wait for weight cache daemon readiness (#58370) * [Bugfix][Quantization] Add Humming to the W4A8 (INT4xFP8) MoE oracle (#58427) Signed-off-by: mgoin <mgoin64@gmail.com> * [Refactor] Move auxiliary files out of the repository root (#58572) Signed-off-by: mgoin <mgoin64@gmail.com> * [Cleanup] Remove online quantization support in `fp8.py` in favor of online shorthands (#53585) Signed-off-by: Felix Marty <Felix.Marty@amd.com> Signed-off-by: mgoin <mgoin64@gmail.com> Co-authored-by: mgoin <mgoin64@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [ROCm] Fix misrouting race-condition in multi-decode P/D disagg with mori-io (#51681) Signed-off-by: Vincent Cave <vincent.cave@amd.com> Signed-off-by: Shiksha Patel <shikpate@amd.com> Co-authored-by: Shiksha Patel <shikpate@amd.com> Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [Perf] DiffusionGemma: constrained reads over the request's logprob_token_ids (#58216) Signed-off-by: Matt Mastracci <matthew@mastracci.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Misha Goin <mgoin64@gmail.com> * [Bugfix] Pass quant_config to DiffusionGemma's ParallelLMHead (#48521) Signed-off-by: Aaron Kang <aaron.h.kang@icloud.com> Co-authored-by: Misha Goin <mgoin64@gmail.com> * [ROCm][CI] skip the ROCm MRV1 default where MRV1 cannot serve the config (#58535) Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [DFlash] Capture the context K/V precompute in the draft CUDA graph (#57632) Signed-off-by: Juntian Liu <Juntianl777@gmail.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * [Bugfix][Outlines] Fix EOS termination and unconstrained masks after rejected drafts (#58612) * [Bugfix][KV Cache] Fix incremental multimodal block hashing (#51694) Signed-off-by: Jellow <49915976+CZT0@users.noreply.github.com> Signed-off-by: Jellow <dvdx@foxmail.com> * [XPU][CI] enable prompt embeds tests on XPU (#58283) Signed-off-by: Lin, Fanli <fanli.lin@intel.com> Signed-off-by: Fanli Lin <fanli.lin@intel.com> Co-authored-by: Kunshang Ji <kunshang.ji@intel.com> * [CI] Report to CRCR after all jobs finish, gated on the build's long pole (#58628) * [PD][PushConnector] Record last activity of remotes on the D side (#52245) Signed-off-by: Sunita Nadampalli <nadampal@amazon.com> Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai> * [BUGFIX] fix ovis2_5 multimodal tokens (#52623) Signed-off-by: Milosz Grunwald <milosz.grunwald@intel.com> * [Bugfix][Core] Keep every multimodal feature in the partial-block KV event (#58288) Signed-off-by: haosenwang1018 <haosenwang1018@users.noreply.github.com> Co-authored-by: haosenwang1018 <haosenwang1018@users.noreply.github.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> * [ROCm][CI] Mirror the DSv4-Flash disaggregated DP EP group on MI355 (#58558) Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [Bugfix][Quantization] Give LM heads standard linear metadata (#58444) Signed-off-by: mgoin <mgoin64@gmail.com> * [Bugfix][Mamba] Restore prompt-tail prefix-cache hits with MTP (#58368) Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: Benjamin Chislett <bchislett@nvidia.com> * [Perf] Parallelize registered CUDA Triton kernel warmup at startup (#58582) Signed-off-by: mgoin <mgoin64@gmail.com> Co-authored-by: Codex <noreply@openai.com> * [KV Connector] Fix DecodeBench fp8 fill values and add a startup fill mode (#58472) Signed-off-by: Zijing Liu <liuzijing2014@gmail.com> Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai> Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Release prompt_embeds tensor when its InputBatch slot is freed (#57988) Signed-off-by: khushali9 <khushali.desai9@gmail.com> * [Bugfix][KVConnector] Finalize saves on steps without a forward (#57775) Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai> Co-authored-by: OpenAI Codex <noreply@openai.com> Co-authored-by: Kimi Code <noreply@moonshot.ai> * [Bugfix][Frontend] Count reasoning tokens for Harmony, DeepSeek-V3 and Step3 parsers (#58626) Signed-off-by: Samyabrata Maji <116789799+sammaji@users.noreply.github.com> Co-authored-by: Flora Feng <4florafeng@gmail.com> * [Bugfix] GLM-5.3-Flash: launch the kpool paged MQA logits in the varlen mode its schedule was built with (#55270) Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> * [Bugfix] Accept EOS after grammar finish in outlines backend; reject json_object at validation (#57743) Signed-off-by: SIDDARTHA REDDY <75976672+SIDDARTHAREDDY8@users.noreply.github.com> * [Perf] Batch Mamba2 prefill SSM state saves, removing GPU<->CPU syncs (#49371) Signed-off-by: samuelkim7 <samuelmwkim@gmail.com> Co-authored-by: Claude <noreply@anthropic.com> * [CI] Run DFlash2 NVFP4 acceptance test on B200; skip it on H200 35GB MIG (#58496) Signed-off-by: khluu <khluu000@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Misha Goin <mgoin64@gmail.com> * [Bugfix][MRV2] Treat padded prompt tails as spec-decode rows for hybrid models (#58434) Signed-off-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * [Kimi-K3][Perf] Dispatch GEMM for vision patch embedder (#58527) Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg> * [Minimax-M3][Perf] Use triton_mrope for vision tower + int64 offset fix for triton_mrope (#58526) Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg> Co-authored-by: Kimi <noreply@moonshot.cn> Co-authored-by: Isotr0py <Isotr0py@outlook.com> * [Bugfix][Quantization] Refresh online NVFP4 scales before reload post-processing (#57954) Signed-off-by: S1ro1 <matej.sirovatka@gmail.com> Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com> Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: aoshen02 <aoshen@inferact.ai> * [Perf][DSv4.1] Restore the fused query RMSNorm + MXFP8 quantization path (#57679) Signed-off-by: Juntian Liu <Juntianl777@gmail.com> * [Bugfix][LogitsProcessor] Validate ':' separator in custom logits processor FQCN (#56020) Signed-off-by: 100milliongold <gadian88@gmail.com> * [ROCm][CI][AITER Coverage] Harden MoE sorting-backend/dispatch env-var test matrix (#58393) Signed-off-by: Divakar Verma <divakar.verma@amd.com> * [gRPC] Fix ping tolerance so long non-streaming RPCs are not dropped (#55102) Signed-off-by: Wei Gong <wei@together.ai> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> * [Perf][Rust Frontend] Make histogram observations lock-free (#58574) Co-authored-by: jthomson04 <jwillthomson19@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com> * [ROCm][Perf] MXFP8 GEMM on native 32x32 block scales for gfx950 (#58510) Signed-off-by: fai <fangzhouai@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * [Bugfix][Qwen4Exp] Keep pinned PLE prefetch ids out of the CUDA graph pool (#58489) Signed-off-by: Juntian Liu <Juntianl777@gmail.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [Perf][Engram] Serialize offloaded lookups and pack host tables into huge pages (#56926) Signed-off-by: Juntian Liu <Juntianl777@gmail.com> Co-authored-by: Kimi Code <noreply@moonshot.cn> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [Bugfix][Quantization] Fix MXFP8 startup crash on layers below mm_mxfp8 shape limits (#54223) Signed-off-by: samuelkim7 <samuelmwkim@gmail.com> Signed-off-by: mgoin <mgoin64@gmail.com> Co-authored-by: mgoin <mgoin64@gmail.com> * [ROCm] Cut 69 wasted contiguous copies per decode step from the skinny GEMM path (#58566) Signed-off-by: lifulu <fululi12@amd.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Nick Hill <nickhill123@gmail.com> * [ROCm][Build] Filter crate tags from vLLM version detection (#57744) Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Signed-off-by: reidliu41 <reid201711@gmail.com> * [Perf][Distributed] Add low-SM multimem reduce-scatter for SM100/SM103 (#55072) Signed-off-by: Yongye Zhu <zyy1102000@gmail.com> Signed-off-by: Summer Yang <girasoleyang@gmail.com> Co-authored-by: Summer Yang <girasoleyang@gmail.com> * [PP][XPU]Add the flag to control microbatch feature on MRV2+PP (#55145) Signed-off-by: yisheng <yi.sheng@intel.com> Co-authored-by: Kunshang Ji <kunshang.ji@intel.com> * [Feature] Triton kernel dispatcher (#43048) Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com> * [ROCm] Fix CI runtime and tests for MI355 DPX (#58244) Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Co-authored-by: Mahesh Kunreddi <mahesh.kunreddi@amd.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [CI][ROCm] Prevent Model Executor apt stalls (#58607) Signed-off-by: Andreas Karatzas <akaratza@amd.com> Co-authored-by: Codex <noreply@openai.com> * [Qwen4Exp][ROCm] PLE n-gram table CPU offload (#57497) Signed-off-by: Mathew Odden <modden@redhat.com> Signed-off-by: Andreas Karatzas <akaratza@amd.com> Co-authored-by: opencode+deepseek-v4-flash+vllm <opencode+deepseek-v4-flash+vllm@example.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> Co-authored-by: OpenAI Codex <noreply@openai.com> * [MM] Add Triton kernel for mm_input_normal. (#56798) Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io> Signed-off-by: wang.yuqi <noooop@126.com> Signed-off-by: Isotr0py <Isotr0py@outlook.com> Co-authored-by: Isotr0py <2037008807@qq.com> Co-authored-by: Isotr0py <Isotr0py@outlook.com> * [Structured Outputs] Parse Lark grammars natively in the xgrammar backend (#58321) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com> * [ROCm] Credit ROCm/aiter for the block32 GEMM's packed kernel and in-launch split-K (#58659) Signed-off-by: Lingpeng Jin <103567126+valarLip@users.noreply.github.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [Frontend] Handle Disable Thinking in /v1/messages (#58613) Signed-off-by: jryberg <johan.ryberg@security.ntt> Signed-off-by: Robert Shaw <robshaw@redhat.com> Co-authored-by: jryberg <johan.ryberg@security.ntt> Co-authored-by: Robert Shaw <robshaw@redhat.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * [Bugfix] Stop leaking the internal field name in the max_tokens validation error (#58336) Signed-off-by: shallow10 <495593563@qq.com> * [Bugfix][KV Cache][MLA] Align packed block strides for V3.2 sparse MLA (#55528) Signed-off-by: lz <145014769+200lz@users.noreply.github.com> Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com> Co-authored-by: OpenAI Codex <codex@openai.com> Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com> Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [Perf][DSv4] Fuse inverse RoPE + FP8 quant into FlashInfer sparse MLA (#58621) Signed-off-by: Yongye Zhu <zyy1102000@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * [UX][Frontend] Introduce `vllm preload` cli for fast restart (#56680) Signed-off-by: Isotr0py <Isotr0py@outlook.com> * [MoE] Defer the TRTLLM-Gen top-k finalize on the modular path (#58635) Signed-off-by: Yongye Zhu <zyy1102000@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * [Docs] Add return annotation to `fused_mm_input_norm_triton` (#58687) Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * [CI] Shard (H100) Helion Kernels five ways (#58645) Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * [Core] Model console logging as CLI configuration (#57205) Add `--logging-config` CLI argument which can be supplied as JSON or using dotted arguments. The `--log-level` argument is provided for convenience, and `--log-config-file` is deprecated in favor of `--logging-config.pylogging_config_file`. Signed-off-by: Mark McLoughlin <markmc@redhat.com> Co-authored-by: AI Assistant <noreply@openai.com> * [ROCm][CI] Pass weight_shape in MXFP8 block32 linear tests (#58698) Signed-off-by: Djordje Ramic <djoramic@amd.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [SpecDecode] Add LiLiCorr drafter (#57934) Signed-off-by: Andrii Skliar <askliar@nvidia.com> Signed-off-by: Andrii Skliar <andreyws96@gmail.com> Co-authored-by: Andrii Skliar <askliar@nvidia.com> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com> * [MRV2] Minor model_runner.py code cleanup (#58610) Signed-off-by: Nick Hill <nickhill123@gmail.com> * [Bugfix] Fix generative scoring body cancellation (#57729) Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [Pooling] Preserve reranker tokenization with document limits (#57666) Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com> Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com> * [Kernel][Perf] Register-resident path for per-token-group 8-bit quant (#55330) Signed-off-by: chao.huan <chao.huan@nio.com> Co-authored-by: Misha Goin <mgoin64@gmail.com> * [Perf][Kernel] Vectorized flat abs-max for dynamic per-tensor FP8 quantization (#58194) Signed-off-by: Monishver Chandrasekaran <monishverchandrasekaran@gmail.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: Misha Goin <mgoin64@gmail.com> * [Core][Logging] Fix JSON logging process decoration (#57957) Signed-off-by: Mark McLoughlin <markmc@redhat.com> * [Bugfix] Stop allocator fragmentation from shrinking the KV cache during memory profiling (#58430) Signed-off-by: Robert Shaw <robertgshaw2@gmail.com> Signed-off-by: Robert Shaw <robertgshaw2-redhat@h100-01.nemg-001.lab.rdu2.dc.redhat.com> Signed-off-by: Robert Shaw <robshaw@redhat.com> Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Co-authored-by: Robert Shaw <robertgshaw2-redhat@h100-01.nemg-001.lab.rdu2.dc.redhat.com> Co-authored-by: Robert Shaw <robshaw@redhat.com> Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [Bugfix][Frontend] Keep logprobs of parser-suppressed streaming chunks (#58583) Signed-off-by: errmakov <ide404@gmail.com> Co-authored-by: Yanxiao Zhao <39199723+sdpkjc@users.noreply.github.com> Co-authored-by: Prakhar Agarwal <270064960+agarwalprakhar2511@users.noreply.github.com> Co-authored-by: Flora Feng <4florafeng@gmail.com> * [Bugfix][GLM-5.3-Flash] SM90 sparse MLA: index_kpool mismatch leads to corruption via unread query token (#58704) Signed-off-by: Matt Mastracci <matthew@mastracci.com> Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com> * [Benchmark] Record model_id in bench latency/throughput --output-json (#58112) Signed-off-by: yashasvi <yashasvi@ibm.com> * [Bugfix][GLM-5.3-Flash] kpool corruption with speculative decoding (#58454) Signed-off-by: Matt Mastracci <matthew@mastracci.com> Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com> Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [GLM5.3 Perf] Optimize glm 5.3 metadata op, 1.6~4.8x kernel level performance improvement (#58450) Signed-off-by: yewentao256 <zhyanwentao@126.com> * [Bugfix][KV Connector] Reap expired NIXL leases behind a heartbeated head (#58292) Signed-off-by: GokayAI <60583610+gokay-ai@users.noreply.github.com> Co-authored-by: GokayAI <gokay-ai@users.noreply.github.com> * [ROCm][Kimi-K3] Optimize low-concurrency speculative KDA (#58045) Signed-off-by: jiacao-amd <jiahui.cao@amd.com> Co-authored-by: OpenAI Codex <noreply@openai.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [ROCm][CI] Cover the AITER MQA logits dispatch on gfx950 (#58724) Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com> Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [ROCm][CI] Add quantized MoE serving test for gfx950 (#58748) Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com> Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [ROCm][CI] Test AMD DeepSeek V4 MoE routing against a PyTorch reference (#58740) Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com> Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [Perf] DiffusionGemma: one-pass sampler statistics kernel (#58226) Signed-off-by: Matt Mastracci <matthew@mastracci.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> * [ROCm][CI] Expand single-GPU coverage on MI355 DPX (#57599) Signed-off-by: Sheral Kumar <shekumar@amd.com> Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Signed-off-by: Andreas Karatzas <akaratza@amd.com> Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com> Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> Co-authored-by: OpenAI Codex <codex@openai.com> Co-authored-by: Andreas Karatzas <andreas.karatzas@protonmail.com> * [CI] [MRV2] Restore MRV2 pp dp coverage (#57735) Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com> * [Bugfix][CI] Report subprocess test skips as skips, not passes (#58701) Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [ROCm][CI] Run the MLA attention+quant fusion test on ROCm (#58717) Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com> Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [ROCm] Bump torch 2.13, triton 3.8, torchaudio, torchvision (#50605) Signed-off-by: Rohan Potdar <rohan.potdar@amd.com> Signed-off-by: Rohan138 <rohanpotdar138@gmail.com> Signed-off-by: jpvillam <juan.villamizar@amd.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: jpvillam <juan.villamizar@amd.com> * [Feature][Frontend] Add granite_thinking_parser reasoning parser for Granite 4.2 (#55957) Signed-off-by: Yousaf shah <yousaf.shah@gmail.com> Co-authored-by: sfeng33 <4florafeng@gmail.com> * [Bugfix][Frontend] Respect max_output_tokens in the Harmony tool-call loop (#58551) Signed-off-by: errmakov <ide404@gmail.com> Co-authored-by: Du Bin <8174807+dubin555@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Flora Feng <4florafeng@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [Bugfix][Frontend] Document 404 response for `/generative_scoring` (#58788) Signed-off-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: Kimi Code <noreply@moonshot.cn> * [Perf][Frontend] Defer reasoning usage recounts for non-continuous chat streams (#56067) Signed-off-by: Cheng Rui <286040359@qq.com> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Flora Feng <4florafeng@gmail.com> * [Bugfix] Default missing detail for Responses API input images (#57241) Signed-off-by: Yifan Zong <yzong@redhat.com> Co-authored-by: Ben Browning <56071+bbrowning@users.noreply.github.com> * [watermarking] golden tests for backwards compatibility (#56809) Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai> Signed-off-by: Simon Veitner <sveitner@redhat.com> Co-authored-by: Simon Veitner <sveitner@redhat.com> Co-authored-by: OpenAI Codex <codex@openai.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [Bugfix] Don't drop the rest of the allocator config when toggling expandable segments (#57982) Signed-off-by: Oxana Korzh <okorzh@amd.com> * [AuxOutput] Only require Model Runner V2 on GPU platform (#58205) Signed-off-by: Linkun Chen <github@lkchen.net> * [Pooling] Preserve BERT-family heads for raw logits (#57664) Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com> Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com> * fix: perf: use startswith(x, i) instead of string slicing to avoid O(N^2) (#52580) Signed-off-by: Ricardo-M-L <ricardoporsche001@icloud.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * [Bugfix] V1: fix allowed_token_ids_mask aliasing in InputBatch.swap_states (#48419) Signed-off-by: Rui Zhu <rui.zhu.rz399@yale.edu> Co-authored-by: Claude <noreply@anthropic.com> * [Bugfix][Reasoning] Count Kimi K3 reasoning tokens (#58372) Signed-off-by: Elvir Crncevic <elvircrn@gmail.com> Signed-off-by: Flora Feng <4florafeng@gmail.com> Co-authored-by: Flora Feng <4florafeng@gmail.com> * [CI] Split (H200 MIG/MI300) Basic Correctness into named jobs (#57054) Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com> Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Co-authored-by: Codex <noreply@openai.com> * [Bugfix][Frontend] Fix Inkling tool name leaking into content after reasoning (#58792) Signed-off-by: Baljinder Hothi <baljinder.hothi@cohere.com> Co-authored-by: Flora Feng <4florafeng@gmail.com> * [Core] Bound draft-token RPC waits by the execute-model timeout (#58779) Signed-off-by: VS Chandra Mourya <219748331+vschandramourya@users.noreply.github.com> Co-authored-by: VS Chandra Mourya <219748331+vschandramourya@users.noreply.github.com> * [Perf][DSv4.1] Shard the Engram wkv projection across TP ranks (#58678) Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com> * [RL] Add sharding-aware NCCL M2N weight transfer (#51520) Signed-off-by: Ke Wen <kwen@nvidia.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: aoshen02 <aoshen@inferact.ai> * [Bugfix][ROCm] AMD-Quark mixed-precision DeepSeek-V4.1 support (#57071) Signed-off-by: Xiao Yu <xiao.yu.dc@outlook.com> Co-authored-by: Cursor <cursoragent@cursor.com> * [Bugfix][Frontend][Rust Frontend] Update DeepSeek V4.1 Flash reasoning effort mappings (#58316) Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Bugen Zhao <i@bugenzhao.com> Signed-off-by: zhec <chengyunfei@ruc.edu.cn> * [CI] Only isolate the registry tests that need a fresh process (#58764) Signed-off-by: aarushjain29 <Aarushi.Jain2@amd.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [Spec Decode] Enable Gemma4 DSpark adaptive verification with FlashInfer (#57263) Signed-off-by: zixi-qi <zixi@inferact.ai> Co-authored-by: OpenAI Codex <noreply@openai.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: Nick Hill <nickhill123@gmail.com> * [CI] Split (B200) Miscellaneous Kernels into mHC, FLA Ops and Misc named jobs (#58609) Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Kevin H. Luu <khluu000@gmail.com> * [Security] Gate per-request multimodal processor kwargs (#58830) Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com> Co-authored-by: Cursor <cursoragent@cursor.com> * [Elastic EP] Fix EPLB load statistics during scaling (#58473) Signed-off-by: Itay Alroy <ialroy@nvidia.com> Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com> * [Refactor] Remove dead tests utils (#58803) Signed-off-by: yewentao256 <zhyanwentao@126.com> * [Perf][MoE] Index expert mapping lookups in RoutedExperts.load_weights (#58720) Signed-off-by: Willian <willian@willian.email> Signed-off-by: mgoin <mgoin64@gmail.com> Co-authored-by: mgoin <mgoin64@gmail.com> * [Bugfix] Fix Anthropic Thinking Disabled with P/D (#58786) Signed-off-by: Robert Shaw <robshaw@redhat.com> Co-authored-by: Robert Shaw <robshaw@redhat.com> * [Bugfix][Frontend] Detect Anthropic inline-system merge against the resolved chat template (#58754) Signed-off-by: Robert Shaw <robshaw@redhat.com> Co-authored-by: Robert Shaw <robshaw@redhat.com> * [Mypy] Fix mypy typing for Qwen and Qianfan models (#58046) Signed-off-by: Ashraf Bhuiyan <mbhuiyan@redhat.com> * [CI] Reduce CUDA graph mode test overhead (#58749) * [GLM5.3 Bug] Fix sparse indexer attn topk backend selection (#58594) Signed-off-by: yewentao256 <zhyanwentao@126.com> * [CI] Stabilize batch submission in full CUDA graph tests (#58810) * [Bugfix][DSV4.1] Avoid host sync in ViT CUDA graph replay metadata (#58499) * [Security] Harden message sanitization (#58832) Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk> * [Kernel][DSV4.1] Fuse MoE finalize into the TP all-reduce + mHC boundary (#58586) Signed-off-by: Yongye Zhu <zyy1102000@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * [PCP][DCP] Support DCP target model with non-DCP Dspark (#56723) Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Codex <noreply@openai.com> * [Kernel] Bump FlashKDA to keep the recurrent state in fp32 (#58846) Signed-off-by: Simon Veitner <sveitner@redhat.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com> * [Metrics][KV Offload] Add Prometheus metrics for SimpleCPUOffloadConnector (#57251) Signed-off-by: Vincent <vincexxchan@gmail.com> * [mooncake] support CUSTOM_MEM_POOL in vllm (#49300) Signed-off-by: bruce.xu <bruce.xb@alibaba-inc.com> Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai> Co-authored-by: bruce.xu <bruce.xb@alibaba-inc.com> Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai> Co-authored-by: OpenAI Codex <codex@openai.com> * [Qwen3.8-Flash-Next] Avoid memory fragmentation in QSA indexer logits workspace (#57105) Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg> * [Bugfix] Disable sequence parallelism / async TP under batch invariance and add a TP regression test (#56377) Signed-off-by: LioEinaudi <zhao3024667639@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [ROCm][Kimi-K3] Make VLLM_ROCM_USE_AITER_MOE_SITUV2 select a4w4/a8w4/a16w4 (#58201) Signed-off-by: Hongxia Yang <hongxia.yang@amd.com> * [Bugfix][NIXL] Release a dead peer's NIXL state without waiting for TTL (#50047) Signed-off-by: Yannik Hinteregger <37209495+YannikHinteregger@users.noreply.github.com> Co-authored-by: xijiade.aihemaiti <3146335281@qq.com> * [Bugfix] V1: clear stale allowed_token_ids mask in InputBatch.condense (#43931) Signed-off-by: Varshith <kvarshithgowda@gmail.com> Co-authored-by: Claude <noreply@anthropic.com> * [Perf][DSv4.1] Fuse small-batch WO-A with inverse RoPE and MXFP8 quant on SM100/SM103 (#58634) Signed-off-by: Juntian Liu <Juntianl777@gmail.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * [Attention][CPU] Use zentorch SDPA for CPU MLA prefill (#54967) Signed-off-by: Rakul Chauhan <rakul.chauhan@amd.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Cursor <cursoragent@cursor.com> * [Perf][Attention] Remove D2H sync from FlashInfer SM90 sparse MLA plan under async scheduling (#58684) Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai> * [CI] Split (H200 MIG 35GB) Spec Decode Speculators + MTP into 4 named jobs (#57237) Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com> Co-authored-by: Thang Nguyen <thang.nguyen@inferact.ai> Co-authored-by: Kimi Code <noreply@moonshot.ai> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * [ROCm][Perf] Enable layer-aware CSA2 multi-stream overlap for DeepSeek-V4.1-Flash (#57407) Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com> * [Perf][Pooling] Avoid blocking seq_lens GPU-to-CPU copy for pooling in FlashInfer metadata builder (#57214) Signed-off-by: frankwang28 <frank.wbb@hotmail.com> Co-authored-by: Misha Goin <mgoin64@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com> * [Bugfix] Support repsonse_format + tool_choice=auto (#56086) Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com> Co-authored-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com> Co-authored-by: pablopupo <145598901+pablopupo@users.noreply.github.com> Co-authored-by: hubunt <150658615+hubunt@users.noreply.github.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> * [Fast Start] Add `/health` endpoint for the weight cache daemon (#58552) Signed-off-by: Xun Sun <UNIDY2002@outlook.com> Co-authored-by: Isotr0py <Isotr0py@outlook.com> * [Perf][MoE] Use fused MiniMax2 routing with non-unit routed scaling (#58880) Signed-off-by: zjy0516 <riverclouds.zhu@qq.com> * [ROCm][Bugfix] Fall back to default GEMM for CPU tensors on ROCm builds (#58923) Signed-off-by: fai <fangzhouai@gmail.com> * [Bugfix][KV Connector] Retry Mooncake bootstrap registration on timeout (reopens #55763) (#58919) * [Frontend] Switch Python Harmony dependency to oss-harmony (#55128) Signed-off-by: Anton Peganov <apeganov@nvidia.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: OpenAI Codex <codex@openai.com> * [ROCm] Bump AITER to v0.1.23 (#58867) Signed-off-by: fai <fangzhouai@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [Bugfix][Frontend] Count Responses reasoning tokens per tool round (#58927) Signed-off-by: sfeng33 <4florafeng@gmail.com> * [KimiViT][Perf] Fuse per-layer QK RoPE into one in-place kernel (#58651) * [Mypy] Fix mypy typing for Ultravox and Unlimited-OCR models (#58239) Signed-off-by: Ashraf Bhuiyan <mbhuiyan@redhat.com> * [Bugfix][Frontend] Sample batched chat completions from the adjusted requests (#58929) Signed-off-by: sfeng33 <4florafeng@gmail.com> * [Bugfix][Frontend] Use a fresh parser per choice in non-streaming chat completions (#58939) Signed-off-by: sfeng33 <4florafeng@gmail.com> * [Bugfix][EPD] Skip sampling for encoder-only async steps (#58490) Signed-off-by: Tianyu Guo <guoty@inferact.ai> * [CPU] Build CPU wheels on Ubuntu 22.04 with AMX-FP8 support (#58515) Signed-off-by: zhejiangxiaomai <zhenhui.zhao@intel.com> Signed-off-by: jiang1.li <jiang1.li@intel.com> Co-authored-by: jiang1.li <jiang1.li@intel.com> * [Test][ROCm] Stabilize the mixed OLMoE LoRA test (#58945) Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com> * [Model] Enable LoRA support for RobertaForSequenceClassification (#58884) Signed-off-by: jz-yolo <jz-yolo@users.noreply.github.com> Signed-off-by: Jane Zhu <jane.zhu@slack-corp.com> Co-authored-by: jz-yolo <jz-yolo@users.noreply.github.com> * [Bugfix][Frontend] Apply Harmony adjust_request in batched chat completions (#58958) Signed-off-by: sfeng33 <4florafeng@gmail.com> * [Minimax-M3] Add Encoder CUDA graph support (#58673) Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg> Co-authored-by: Codex <codex@openai.com> * [Perf][MRV2] Reuse Mamba/GDN metadata across KV cache groups (#58762) Signed-off-by: Luca Motz <luca.motz@icloud.com> Co-authored-by: Xin Yang <xyangx@amazon.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com> * [Bugfix] Fix the two multimodal root tests that fail on main (OpenPangu-VL embed merge, MiMo sink test fixture) (#58900) Signed-off-by: khluu <khluu000@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * [Benchmark] Add Responses API backend to vllm bench serve (#54628) Signed-off-by: QHarshil <harshil_c@hotmail.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Chauncey <chaunceyjiang@gmail.com> * [CI] Allowlist-shrink batch 1: wire 17 root-level tests + drop 5 stale watermarking entries into misc.yaml (#58055) Signed-off-by: Turner Jabbour <doubleujabbour@gmail.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> * [CPU] Use accelerator memory API in DiffusionGemma (#58964) Signed-off-by: zhejiangxiaomai <zhenhui.zhao@intel.com> Co-authored-by: Li, Jiang <jiang1.li@intel.com> * [CPU] Add video inferencing via torchcodec on s390x (#58693) Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com> Co-authored-by: Li, Jiang <jiang1.li@intel.com> * [Skills] Update kernel-microbenchmark to include ROCm (#58646) Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg> Co-authored-by: Kimi <noreply@moonshot.cn> * [Misc] Name each backend and its kernel block sizes in block-size errors (#58557) Signed-off-by: Eugenio "Jay" Zuccarelli <11176606+jayzuccarelli@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [CI] Shard (H200 MIG 35GB / MI355 DPX) Entrypoints Integration (Pooling) into named jobs (#58652) Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com> Signed-off-by: khluu <khluu000@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: khluu <khluu000@gmail.com> * [CI] Split Dynamic Shapes out of (H200 MIG 35GB) PyTorch Compilation + (MI300) mirror (#58451) Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * [Bugfix][Qwen4Exp] Release the profiling KV cache held by QSA key views (#58961) Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [CPU] Include vLLM Recipes tooling in release image to deploy models using vLLM Recipes (#58796) Signed-off-by: louie-tsai <louie.tsai@intel.com> * Revert "[CI] Shard (H200 MIG 35GB / MI355 DPX) Entrypoints Integration (Pooling) into named jobs (#58652)" (#59011) Signed-off-by: khluu <khluu000@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * [Perf][Qwen4Exp] Fuse HC down projection and SiLU on NVIDIA (#58957) Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg> * [Bugfix][Model] Gemma4: register aliased embedding scalars as buffers (#54213) Signed-off-by: Yannick Schnider <Yannick.Schnider1@ibm.com> * [Bugfix][Spec Decode] Implement get_top_tokens() on the ROCm DeepSeek V4 MTP drafter (#57568) Signed-off-by: BaoYunkai <ybao@amd.com> * [Perf][Qwen3.8] Reduce PLE metadata construction overhead (#58114) Signed-off-by: zjy0516 <riverclouds.zhu@qq.com> Co-authored-by: OpenAI Codex <codex@openai.com> * [ROCm][Refactor] Move DeepSeek-V4/V4.1 multi-stream overlap gate to ROCm platform (#58983) Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com> * [Multimodal] Avoid extra d2d for encoder cudagraph with fused input norm (#56711) Signed-off-by: Isotr0py <Isotr0py@outlook.com> * [Bugfix][Logging] Preserve application log record factories (#58747) Signed-off-by: mgoin <mgoin64@gmail.com> Co-authored-by: OpenAI Codex <noreply@openai.com> * [Bugfix] Use the correct repository revision for secondary artifact loaders (#57461) Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com> Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [ROCm][Perf] Kimi-K3 Enable sharded latent MoE up-projection under EP (#54956) Signed-off-by: Xavier Aguilar <xavier.aguilarfruto@amd.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [Feature] Add fixed-token prefill scoring (#54335) Signed-off-by: aoshen02 <aoshen@inferact.ai> Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com> Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> * [LoRA] Support variable num_labels for sequence classification (#57766) Signed-off-by: linitra24 <renshuang.zhou@daocloud.io> Co-authored-by: Jee Jee Li <pandaleefree@gmail.com> * [ROCm][Perf] Replace torch.topk in DSA candidate block selection (#58208) Signed-off-by: fai <fangzhouai@gmail.com> Co-authored-by: TJian <tunjian.tan@embeddedllm.com> Co-authored-by: Cursor Agent <cursoragent@cursor.com> * [ROCm]Keep LMCache OpenTelemetry on the image's 1.40 stack (#59056) Signed-off-by: Micah Williamson <micah.williamson@amd.com> * [Bugfix][Kimi-K3] Refresh DSpark context KV cache pointers after the KV cache is re-bound (#58814) Signed-off-by: Oxana Korzh <okorzh@amd.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * [KVConnector][NIXL] Support packed MLA KV layouts in pipeline-parallel push prefill (#50499) Signed-off-by: zixi-qi <zixi@inferact.ai> * [Quant] Use canonical N-first weight format for CT WNA16 MoE (#52798) Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: HDCharles <{"message":"Not Found","documentation_url":"https://docs.github.com/rest/users/emails#list-email-addresses-for-the-authenticated-user","status":"404"}> Co-authored-by: OpenAI Codex <codex@openai.com> Co-authored-by: OpenAI Codex <noreply@openai.com> * [CI][Bugfix] Relax packed_qk_rope_ correctness test to one ULP (#59008) Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [ROCm][CI] Pin OpenTelemetry to LMCache's cap in the ROCm images (#59051) Signed-off-by: Rohan Potdar <rohan.potdar@amd.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * [ROCm][CI] Increase timeout for Entrypoints Unit (#59076) Signed-off-by: Djordje Ramic <djoramic@amd.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [ROCm][MLA] Add an AITER ASM round-robin decode route for DCP multi-token verify (#56861) Signed-off-by: Xiaohu Guo <Xiaohu.Guo@amd.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: seungrokj <144636725+seungrokj@users.noreply.github.com> Co-authored-by: Cursor <cursoragent@cursor.com> * [Bugfix] Don't sync-police or retry FlashInfer all-reduce workspace creation in eager mode (#58498) Signed-off-by: khluu <khluu000@gmail.com> Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com> Co-authored-by: Nick Hill <nickhill123@gmail.com> * [Bugfix] Fix resumable request + async scheduling handoff race (#58259) Signed-off-by: Yifan Zong <yzong@redhat.com> * [Model Runner V2][Spec Decode] Support spec decode with draft model (#43091) Signed-off-by: Icey <1790571317@qq.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [Perf][MRV2] Allow FULL decode graphs for one-token prompt tails (#58400) Signed-off-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Yongye Zhu <zyy1102000@gmail.com> * [Bugfix][Scheduler] Preserve logprobs across streaming continuations (#57447) Signed-off-by: 0xsensei <prblmslvr.aditya@gmail.com> * [Mypy] Fix mypy typing for Voxtral and vision models (#58251) Signed-off-by: Ashraf Bhuiyan <mbhuiyan@redhat.com> Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com> * [Core] Rework scheduler `skipped_waiting` queue (#58947) Signed-off-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com> * [Bugfix][MRV2][Spec Decode] Reject draft slots that were never proposed (#58784) Signed-off-by: zixi-qi <zixi@inferact.ai> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Nick Hill <nickhill123@gmail.com> * [ROCm][CI] Add missing test coverage for upstream parity (#50519) Signed-off-by: Andreas Karatzas <akaratza@amd.com> Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com> Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Co-authored-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com> Co-authored-by: OpenAI Codex <codex@openai.com> Co-authored-by: OpenAI Codex <noreply@openai.com> * [Bugfix][Scheduler] Refresh max tokens for streaming continuations (#57676) Signed-off-by: 0xsensei <prblmslvr.aditya@gmail.com> Co-authored-by: OpenAI Codex <codex@openai.com> * [Perf][Spec Decode] Avoid triton recompiles in the acceptance estimator (#57107) Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai> Co-authored-by: Woosuk Kwon <woosuk@inferact.ai> * [CI/Build] Skip the snapshot runtime on CUDA 12.x images (#59118) Signed-off-by: khluu <khluu000@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit aedaba8664f67d8ae1538e5e5cec2b1ce3f258dd) * [XPU][CI]Skip test_abort_timeout_on_prefiller in nightly (#58307) Signed-off-by: zengxian <xiangdong.zeng@intel.com> Co-authored-by: Kunshang Ji <kunshang.ji@intel.com> (cherry picked from commit 09c47db1ca080793dc2351144cb39513fd7984ca) * [Bugfix][Engram] Keep THP tables private when resolving shared memory (#59068) Signed-off-by: Ren Yuzhou <54501155+yuzhouo7@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com> (cherry picked from commit ec5e0c352f079c2cb8f46752fd0317a2745050bf) * [ROCm][CI] Expand MI355 mirrors and route MIG-sized jobs to DPX (#59137) Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com> (cherry picked from commit 6ebb5bd64f5ef75dbe1471ee88c009003b3d03ec) * [Bugfix][Mamba] Keep the prompt-end prefill checkpoint under sparse retention (#59146) Signed-off-by: Jared Wen <jaredwen@inferact.ai> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com> (cherry picked from commit d882bddbeab6b4a0d5861dfcb171bf61ce2109d6) * [Core][BugFix] Tag prefix-cache extra keys by source (#51899) Signed-off-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com> Co-authored-by: Tai An <antai12232931@outlook.com> Co-authored-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 765872e7ed5b6f372f0898a67e48d3365fa28139) * [Bugfix][Frontend] Reject LoRA adapters named after a served model (#59286) Signed-off-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 105a4e097b905127b1d5f09a7e93bd1095e87a6c) * [Dependency] Upgrade FlashInfer to 0.7.0.post1 (#59323) Signed-off-by: wzhao18 <wzhao18.sz@gmail.com> (cherry picked from commit 678baf53724e06cef7f07628ae8fd4cc6c96f11a) * [Bugfix][HiSparse] Resolve MTP verification rows with a union residency kernel (#59235) Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 2798f668608155bc8a8c74cb97e3ddd0d3053085) * [Bugfix][HiSparse] Never allocate GPU pages without host backing (#59036) Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com> Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Co-authored-by: Matthew Bonanni <mbonanni@redhat.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 90e13fc757cffc11b55689d9cc33d844d9606516) * [CPU][Whisper] Support W4A16 quantized Whisper on the CPU WNA16 kernel (#58268) Signed-off-by: Harshal Adhav <harshal.adhav@amd.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Li, Jiang <jiang1.li@intel.com> (cherry picked from commit 5faf81a4297921c699f9aa58f9c6395e95ede718) * [Core] Bound UniProc EngineCore startup threads to available CPUs (#58946) Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com> (cherry picked from commit 866fa130fac1e1252330a680fbed45ad05eba64f) * [KV-Offloading][TP] : Expand replicated_layout detection to multi-group MLA (#57652) (cherry picked from commit 5463fe4962785cdc3383477bf3af6533e7647dfd) * [Bugfix][HiSparse] Preserve host prefix publication after request completion (#59007) Signed-off-by: Yueh-Ting Chen <yueh.ting.chen@gmail.com> Co-authored-by: Matthew Bonanni <mbonanni@redhat.com> (cherry picked from commit ff1b87cca25690fef6bd12667fd0d26690d949af) * [Bugfix][HiSparse] Adopt GPU prefix copies after the hit's allocation (#59282) Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> (cherry picked from commit 3eb6cec22ad9bb098393021b956feaa97081fc85) * [Bugfix][HiSparse] Stop the host pool feeding device KV cache residency metrics (#58725) Signed-off-by: Yueh-Ting Chen <yueh.ting.chen@gmail.com> Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Co-authored-by: Matthew Bonanni <mbonanni@redhat.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 73c7cae4d746f64c22677348d1cc120eea6f7439) * [Bugfix][Core] Fix mamba prefill checkpoint block reservation and prompt-end eviction in align mode (#59175) Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com> Co-authored-by: Nick Hill <nickhill123@gmail.com> (cherry picked from commit 7e583e615c20ee4ff0cd82aac592c7d6310aa7c3) * [Core] Include the LoRA path in prefix-cache block hashes (#59335) Signed-off-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit c055c1e075ed461cff922aa710d387281f651940) * [Perf][PP] Skip sampled-token broadcasts whose requests leave the engine (#58542) Signed-off-by: LostFox11 <wangziyue17@huawei.com> Signed-off-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: LostFox11 <wangziyue17@huawei.com> Co-authored-by: Nick Hill <nickhill123@gmail.com> (cherry picked from commit 7314360c9eebb3b4b460ec49f6606ef0a08fcae2) * [CPU][Zen] Add DA8W4 (W4A8) int4 support for dense and MoE layers (#54024) Signed-off-by: R <Ganesh.R@amd.com> Signed-off-by: Ganesh R <Ganesh.R@amd.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> (cherry picked from commit e12291d7332db897885fdc0e6ed61b28c969d743) * [Model Runner V2] Support randomized dummy inputs (#58411) Signed-off-by: Robert Shaw <robshaw@redhat.com> Co-authored-by: Robert Shaw <robshaw@redhat.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Nick Hill <nickhill123@gmail.com> (cherry picked from commit e5e38ba9b7d18f9746d989e389a51a94b0f96f6f) * [Bugfix][HiSparse] Fix a chunked-prefill preemption livelock (#59494) Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 3a6963664537ed21172e2ec12e96e3a2dcd3718c) * [Bugfix][HiSparse] Size the KV cache from the groups HiSparse allocates (#59450) Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com> (cherry picked from commit f7999d2e4489126f1c21868699f3890021f01b73) * [Bugfix][HiSparse] Fix MTP acceptance collapse under FULL graphs with a saturated GPU pool (#59309) Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com> Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: Matthew Bonanni <mbonanni@redhat.com> (cherry picked from commit 4056c8ac1f8a7e8fb50cf1c56ff96649fcadccc6) * [CI] Drop a test that depends on #57930 from the #59309 backport Resolving the #59309 cherry-pick conflict in tests/v1/kv_connector/unit/test_hisparse_connector.py pulled in test_scheduled_prefix_hit_publishes_adopted_copies from #57930, which is not on this branch; it imports _allocate_scheduled from tests.v1.core.test_prefix_caching and fails on every platform. After this change the file matches the upstream #59309 diff. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: khluu <khluu000@gmail.com> * [Bugfix] Fix minimax-m3 multimodal processor compatability with Transformers v5.18 (#59613) Signed-off-by: Isotr0py <Isotr0py@outlook.com> (cherry picked from commit b558f160a2c0abcb5902acc3c91a14c38a4af173) * [Misc] Add Transformers version upper bound in requirements (#59614) Signed-off-by: Isotr0py <Isotr0py@outlook.com> Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> (cherry picked from commit 58b3298457dde7b4554b3b4e20b238c0ac2c3a65) * Forward-port the tkv/turbo-attn vLLM seam onto upstream v0.31.0 Squashes arbicity/vllm-turbo main @969c125e9 (upstream v0.28.0 + the seam: b0a14f2eb #28, 76a3e6d72, 6d4c75611 #29) into one commit and replays it onto upstream v0.31.0 (db9527a46, the commit vllm/vllm-openai:v0.31.0 is built from) as a 3-way merge against the real base. Upstream's v0.29-v0.31 KV-cache layout refactor (vllm#51718 and follow-ups) replaced the allocator the seam patched: one backing allocation, per-layer [B, H, N, C] views placed by the engine, page geometry read off the spec, and AttentionBackend.customize_spec applied to every layer's spec by both model runners. The seam is re-expressed on that and shrinks from 49 files to 18. Kept (re-applied on upstream's structure): - plugin KV-cache dtype registry; --kv-cache-dtype choices/type - TURBO_ATTN backend slot (now a distinct enum value: two None members made CUSTOM an alias of TURBO_ATTN), turbo-attn spelling, auto-default for plugin dtypes (#29), CUDA candidate once registered - lifecycle hooks on_model_loaded / on_draft_model_loaded / on_kv_cache_initialized / adjust_kv_budget, fail-loud dispatch to every backend in use - MLA wrapping in the selector; MLA chunked-context _get_gather_op - _tq_layer_idx injection for tkv layers - aggregated_layer_count: fused (composite) pages, now one shared page per fused set in upstream's single-allocation planner Moved to upstream's extension points (turbo-attn plugin side): - get_kv_cache_spec_class (Attention, MLAAttention, hybrid alignment) -> AttentionBackend.customize_spec - KVCacheSpec.get_manager_class -> KVCacheSpecRegistry MRO lookup - get_supported_kv_cache_dtypes -> supported_kv_cache_dtypes ClassVar - get_kv_cache_shape(kv_cache_spec=...) passthrough and the backend-managed-dtype shape coercion -> spec-driven views Dropped: - spec_decode_warmup.py: upstream registers the same kernels with its JIT warmup registry (aed894c19, vllm#56323) - turbo_attn_warmup.py, utils/cutedsl_cache.py: superseded by turbo-attn's own prefill prewarm (on_kv_cache_initialized) and CuTeDSL cache - per-group BlockPools, sampler reserve, padded-page block fill, drain hook / on_kv_manager_created, KVBlockZeroer clamp: conflict with upstream's single-pool allocator; no turbo-attn consumer for the hooks - rotary fast-path registry: upstream guards the import (1f60771c7, vllm#42679) - --kv-cache-dtype-skip-layers-dtype, VLLM_KV_CACHE_SKIP_LAYERS_DTYPE, Triton fused fp8 GEMM hook, gsm8k startup waits, notify-turbo-attn workflow, draft-backend inheritance: unused or superseded - FA2 varlen paged split-K patch: never reached the overlay image (the image installs no compiled _vllm_fa2_C) and the FA pin moved PROTOCOL.md rewritten for the v0.31.0 seam. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Signed-off-by: Lai, Yejing <yejing.lai@intel.com> Signed-off-by: priyansh jain <priyansh.jain2@amd.com> Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com> Signed-off-by: reidliu41 <reid201711@gmail.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com> Signed-off-by: RyanMa29 <ziyang.ma@intel.com> Signed-off-by: R <Ganesh.R@amd.com> Signed-off-by: Shrey Gajjar <shreygajjar007@gmail.com> Signed-off-by: Zijing Liu <liuzijing2014@gmail.com> Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com> Signed-off-by: fai <fangzhouai@gmail.com> Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Signed-off-by: Linze-Shi <linzeshi0@gmail.com> Signed-off-by: yewentao256 <zhyanwentao@126.com> Signed-off-by: Yueh-Ting Chen <yueh.ting.chen@gmail.com> Signed-off-by: Chenglun Hu <chenglunhu@gmail.com> Signed-off-by: hclsys <chenglunhu@gmail.com> Signed-off-by: Wauplin <lucainp@gmail.com> Signed-off-by: Thomas Ortner <boh@zurich.ibm.com> Signed-off-by: nightcityblade <nightcityblade@gmail.com> Signed-off-by: Nick Hill <nickhill123@gmail.com> Signed-off-by: mgoin <mgoin64@gmail.com> Signed-off-by: Felix Marty <Felix.Marty@amd.com> Signed-off-by: Vincent Cave <vincent.cave@amd.com> Signed-off-by: Shiksha Patel <shikpate@amd.com> Signed-off-by: Matt Mastracci <matthew@mastracci.com> Signed-off-by: Aaron Kang <aaron.h.kang@icloud.com> Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com> Signed-off-by: Juntian Liu <Juntianl777@gmail.com> Signed-off-by: Jellow <49915976+CZT0@users.noreply.github.com> Signed-off-by: Jellow <dvdx@foxmail.com> Signed-off-by: Lin, Fanli <fanli.lin@intel.com> Signed-off-by: Fanli Lin <fanli.lin@intel.com> Signed-off-by: Sunita Nadampalli <nadampal@amazon.com> Signed-off-by: Milosz Grunwald <milosz.grunwald@intel.com> Signed-off-by: haosenwang1018 <haosenwang1018@users.noreply.github.com> Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com> Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai> Signed-off-by: khushali9 <khushali.desai9@gmail.com> Signed-off-by: Samyabrata Maji <116789799+sammaji@users.noreply.github.com> Signed-off-by: SIDDARTHA REDDY <75976672+SIDDARTHAREDDY8@users.noreply.github.com> Signed-off-by: samuelkim7 <samuelmwkim@gmail.com> Signed-off-by: khluu <khluu000@gmail.com> Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg> Signed-off-by: S1ro1 <matej.sirovatka@gmail.com> Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com> Signed-off-by: 100milliongold <gadian88@gmail.com> Signed-off-by: Divakar Verma <divakar.verma@amd.com> Signed-off-by: Wei Gong <wei@together.ai> Signed-off-by: lifulu <fululi12@amd.com> Signed-off-by: Yongye Zhu <zyy1102000@gmail.com> Signed-off-by: Summer Yang <girasoleyang@gmail.com> Signed-off-by: yisheng <yi.sheng@intel.com> Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com> Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Signed-off-by: Andreas Karatzas <akaratza@amd.com> Signed-off-by: Mathew Odden <modden@redhat.com> Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io> Signed-off-by: wang.yuqi <noooop@126.com> Signed-off-by: Isotr0py <Isotr0py@outlook.com> Signed-off-by: Lingpeng Jin <103567126+valarLip@users.noreply.github.com> Signed-off-by: jryberg <johan.ryberg@security.ntt> Signed-off-by: Robert Shaw <robshaw@redhat.com> Signed-off-by: shallow10 <495593563@qq.com> Signed-off-by: lz <145014769+200lz@users.noreply.github.com> Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com> Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com> Signed-off-by: Mark McLoughlin <markmc@redhat.com> Signed-off-by: Djordje Ramic <djoramic@amd.com> Signed-off-by: Andrii Skliar <askliar@nvidia.com> Signed-off-by: Andrii Skliar <andreyws96@gmail.com> Signed-off-by: chao.huan <chao.huan@nio.com> Signed-off-by: Monishver Chandrasekaran <monishverchandrasekaran@gmail.com> Signed-off-by: Robert Shaw <robertgshaw2@gmail.com> Signed-off-by: Robert Shaw <robertgshaw2-redhat@h100-01.nemg-001.lab.rdu2.dc.redhat.com> Signed-off-by: errmakov <ide404@gmail.com> Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com> Signed-off-by: yashasvi <yashasvi@ibm.com> Signed-off-by: GokayAI <60583610+gokay-ai@users.noreply.github.com> Signed-off-by: jiacao-amd <jiahui.cao@amd.com> Signed-off-by: Sheral Kumar <shekumar@amd.com> Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com> Signed-off-by: Rohan Potdar <rohan.potdar@amd.com> Signed-off-by: Rohan138 <rohanpotdar138@gmail.com> Signed-off-by: jpvillam <juan.villamizar@amd.com> Signed-off-by: Yousaf shah <yousaf.shah@gmail.com> Signed-off-by: Cheng Rui <286040359@qq.com> Signed-off-by: Yifan Zong <yzong@redhat.com> Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai> Signed-off-by: Simon Veitner <sveitner@redhat.com> Signed-off-by: Oxana Korzh <okorzh@amd.com> Signed-off-by: Linkun Chen <github@lkchen.net> Signed-off-by: Ricardo-M-L <ricardoporsche001@icloud.com> Signed-off-by: Rui Zhu <rui.zhu.rz399@yale.edu> Signed-off-by: Elvir Crncevic <elvircrn@gmail.com> Signed-off-by: Flora Feng <4florafeng@gmail.com> Signed-off-by: Baljinder Hothi <baljinder.hothi@cohere.com> Signed-off-by: VS Chandra Mourya <219748331+vschandramourya@users.noreply.github.com> Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com> Signed-off-by: Ke Wen <kwen@nvidia.com> Signed-off-by: Xiao Yu <xiao.yu.dc@outlook.com> Signed-off-by: zhec <chengyunfei@ruc.edu.cn> Signed-off-by: aarushjain29 <Aarushi.Jain2@amd.com> Signed-off-by: zixi-qi <zixi@inferact.ai> Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com> Signed-off-by: Itay Alroy <ialroy@nvidia.com> Signed-off-by: Willian <willian@willian.email> Signed-off-by: Ashraf Bhuiyan <mbhuiyan@redhat.com> Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk> Signed-off-by: Vincent <vincexxchan@gmail.com> Signed-off-by: bruce.xu <bruce.xb@alibaba-inc.com> Signed-off-by: LioEinaudi <zhao3024667639@gmail.com> Signed-off-by: Hongxia Yang <hongxia.yang@amd.com> Signed-off-by: Yannik Hinteregger <37209495+YannikHinteregger@users.noreply.github.com> Signed-off-by: Varshith <kvarshithgowda@gmail.com> Signed-off-by: Rakul Chauhan <rakul.chauhan@amd.com> Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai> Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com> Signed-off-by: frankwang28 <frank.wbb@hotmail.com> Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com> Signed-off-by: Xun Sun <UNIDY2002@outlook.com> Signed-off-by: zjy0516 <riverclouds.zhu@qq.com> Signed-off-by: Anton Peganov <apeganov@nvidia.com> Signed-off-by: sfeng33 <4florafeng@gmail.com> Signed-off-by: Tianyu Guo <guoty@inferact.ai> Signed-off-by: zhejiangxiaomai <zhenhui.zhao@intel.com> Signed-off-by: jiang1.li <jiang1.li@intel.com> Signed-off-by: jz-yolo <jz-yolo@users.noreply.github.com> Signed-off-by: Jane Zhu <jane.zhu@slack-corp.com> Signed-off-by: Luca Motz <luca.motz@icloud.com> Signed-off-by: QHarshil <harshil_c@hotmail.com> Signed-off-by: Turner Jabbour <doubleujabbour@gmail.com> Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com> Signed-off-by: Eugenio "Jay" Zuccarelli <11176606+jayzuccarelli@users.noreply.github.com> Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com> Signed-off-by: louie-tsai <louie.tsai@intel.com> Signed-off-by: Yannick Schnider <Yannick.Schnider1@ibm.com> Signed-off-by: BaoYunkai <ybao@amd.com> Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com> Signed-off-by: Xavier Aguilar <xavier.aguilarfruto@amd.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> Signed-off-by: linitra24 <renshuang.zhou@daocloud.io> Signed-off-by: Micah Williamson <micah.williamson@amd.com> Signed-off-by: Stefan Kon…

* [Bugfix][XPU] store the pointer raw bit pattern instead of its numeric value (#54514) Signed-off-by: Lai, Yejing <yejing.lai@intel.com> Co-authored-by: Kunshang Ji <kunshang.ji@intel.com> * [Attention][CPU] Run Zen CPU encoder attention on zentorch SDPA (#54508) Signed-off-by: priyansh jain <priyansh.jain2@amd.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [MRV2] Validate MRV2 entrypoint logits processors (#57728) Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com> * [Bugfix][Rust Frontend] Prevent MM timing from enabling debug tracing (#58378) Co-authored-by: Bugen Zhao <i@bugenzhao.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: reidliu41 <reid201711@gmail.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com> * [XPU][UT] Align HF and vLLM inputs for Qwen2 embedding test by preventing Sentence Transformers from applying chat template (#58117) Signed-off-by: RyanMa29 <ziyang.ma@intel.com> * [CPU] Gate the AVX10.2 paths on compiler support (#58133) Signed-off-by: R <Ganesh.R@amd.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Li, Jiang <jiang1.li@intel.com> * [Perf][Frontend] Offload streaming derender detokenization (#57528) Signed-off-by: Shrey Gajjar <shreygajjar007@gmail.com> * [Multimodal] Reuse the supplied tokenizer in the MiniMax-M3 VL processor (#58460) Signed-off-by: Zijing Liu <liuzijing2014@gmail.com> * [XPU] enable XPU GRAPH by default (#51600) Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com> * [ROCm][DSv4.1][Perf] Emit MXFP8 from the sparse decode reduce and run wo_a as a grouped FP8 GEMM (#58456) Signed-off-by: fai <fangzhouai@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * Remove `.gemini/` and `CLAUDE.md` (#58541) Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * [Bugfix][Pooling] Fix JinaVL label configuration and restore multimodal tests (#57347) Signed-off-by: Linze-Shi <linzeshi0@gmail.com> * [Chore] Use Transformers v5 names and drop redundant processor `use_fast` (#58550) Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * [Refactor] Remove dead or duplicate tests (#58446) Signed-off-by: yewentao256 <zhyanwentao@126.com> * [Perf][Attention] Bound FlashInfer prefill dequantization scratch (#57918) Signed-off-by: Yueh-Ting Chen <yueh.ting.chen@gmail.com> * fix(config): apply presence_penalty/frequency_penalty from override-generation-config (#50769) Signed-off-by: Chenglun Hu <chenglunhu@gmail.com> Signed-off-by: hclsys <chenglunhu@gmail.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> * [Bugfix] Resolve the Hub revision once per repo (#56092) Signed-off-by: Wauplin <lucainp@gmail.com> Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [Frontend] Remove the slow tokenizer mode (#58545) Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Revert "[DSpark] Support pipeline-parallel targets in aggregated serving (#56956)" (#58484) * [transformer] RMSNorm matching for alternative rsqrt (#54461) Signed-off-by: Thomas Ortner <boh@zurich.ibm.com> Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> * [Bugfix] Count unsplit Idefics3 image patches (#48760) Signed-off-by: nightcityblade <nightcityblade@gmail.com> Co-authored-by: nightcityblade <nightcityblade@gmail.com> Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk> * [Bugfix] Keep JIT warmup under enforce-eager when fault tolerance is on (#58593) Signed-off-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: Kimi <noreply@moonshot.cn> * [Core] Skip JIT monitor when JIT warmup is disabled (#58590) Signed-off-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: Kimi Code <noreply@moonshot.cn> * [Fast Start] Wait for weight cache daemon readiness (#58370) * [Bugfix][Quantization] Add Humming to the W4A8 (INT4xFP8) MoE oracle (#58427) Signed-off-by: mgoin <mgoin64@gmail.com> * [Refactor] Move auxiliary files out of the repository root (#58572) Signed-off-by: mgoin <mgoin64@gmail.com> * [Cleanup] Remove online quantization support in `fp8.py` in favor of online shorthands (#53585) Signed-off-by: Felix Marty <Felix.Marty@amd.com> Signed-off-by: mgoin <mgoin64@gmail.com> Co-authored-by: mgoin <mgoin64@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [ROCm] Fix misrouting race-condition in multi-decode P/D disagg with mori-io (#51681) Signed-off-by: Vincent Cave <vincent.cave@amd.com> Signed-off-by: Shiksha Patel <shikpate@amd.com> Co-authored-by: Shiksha Patel <shikpate@amd.com> Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [Perf] DiffusionGemma: constrained reads over the request's logprob_token_ids (#58216) Signed-off-by: Matt Mastracci <matthew@mastracci.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Misha Goin <mgoin64@gmail.com> * [Bugfix] Pass quant_config to DiffusionGemma's ParallelLMHead (#48521) Signed-off-by: Aaron Kang <aaron.h.kang@icloud.com> Co-authored-by: Misha Goin <mgoin64@gmail.com> * [ROCm][CI] skip the ROCm MRV1 default where MRV1 cannot serve the config (#58535) Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [DFlash] Capture the context K/V precompute in the draft CUDA graph (#57632) Signed-off-by: Juntian Liu <Juntianl777@gmail.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * [Bugfix][Outlines] Fix EOS termination and unconstrained masks after rejected drafts (#58612) * [Bugfix][KV Cache] Fix incremental multimodal block hashing (#51694) Signed-off-by: Jellow <49915976+CZT0@users.noreply.github.com> Signed-off-by: Jellow <dvdx@foxmail.com> * [XPU][CI] enable prompt embeds tests on XPU (#58283) Signed-off-by: Lin, Fanli <fanli.lin@intel.com> Signed-off-by: Fanli Lin <fanli.lin@intel.com> Co-authored-by: Kunshang Ji <kunshang.ji@intel.com> * [CI] Report to CRCR after all jobs finish, gated on the build's long pole (#58628) * [PD][PushConnector] Record last activity of remotes on the D side (#52245) Signed-off-by: Sunita Nadampalli <nadampal@amazon.com> Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai> * [BUGFIX] fix ovis2_5 multimodal tokens (#52623) Signed-off-by: Milosz Grunwald <milosz.grunwald@intel.com> * [Bugfix][Core] Keep every multimodal feature in the partial-block KV event (#58288) Signed-off-by: haosenwang1018 <haosenwang1018@users.noreply.github.com> Co-authored-by: haosenwang1018 <haosenwang1018@users.noreply.github.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> * [ROCm][CI] Mirror the DSv4-Flash disaggregated DP EP group on MI355 (#58558) Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [Bugfix][Quantization] Give LM heads standard linear metadata (#58444) Signed-off-by: mgoin <mgoin64@gmail.com> * [Bugfix][Mamba] Restore prompt-tail prefix-cache hits with MTP (#58368) Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: Benjamin Chislett <bchislett@nvidia.com> * [Perf] Parallelize registered CUDA Triton kernel warmup at startup (#58582) Signed-off-by: mgoin <mgoin64@gmail.com> Co-authored-by: Codex <noreply@openai.com> * [KV Connector] Fix DecodeBench fp8 fill values and add a startup fill mode (#58472) Signed-off-by: Zijing Liu <liuzijing2014@gmail.com> Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai> Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Release prompt_embeds tensor when its InputBatch slot is freed (#57988) Signed-off-by: khushali9 <khushali.desai9@gmail.com> * [Bugfix][KVConnector] Finalize saves on steps without a forward (#57775) Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai> Co-authored-by: OpenAI Codex <noreply@openai.com> Co-authored-by: Kimi Code <noreply@moonshot.ai> * [Bugfix][Frontend] Count reasoning tokens for Harmony, DeepSeek-V3 and Step3 parsers (#58626) Signed-off-by: Samyabrata Maji <116789799+sammaji@users.noreply.github.com> Co-authored-by: Flora Feng <4florafeng@gmail.com> * [Bugfix] GLM-5.3-Flash: launch the kpool paged MQA logits in the varlen mode its schedule was built with (#55270) Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> * [Bugfix] Accept EOS after grammar finish in outlines backend; reject json_object at validation (#57743) Signed-off-by: SIDDARTHA REDDY <75976672+SIDDARTHAREDDY8@users.noreply.github.com> * [Perf] Batch Mamba2 prefill SSM state saves, removing GPU<->CPU syncs (#49371) Signed-off-by: samuelkim7 <samuelmwkim@gmail.com> Co-authored-by: Claude <noreply@anthropic.com> * [CI] Run DFlash2 NVFP4 acceptance test on B200; skip it on H200 35GB MIG (#58496) Signed-off-by: khluu <khluu000@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Misha Goin <mgoin64@gmail.com> * [Bugfix][MRV2] Treat padded prompt tails as spec-decode rows for hybrid models (#58434) Signed-off-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * [Kimi-K3][Perf] Dispatch GEMM for vision patch embedder (#58527) Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg> * [Minimax-M3][Perf] Use triton_mrope for vision tower + int64 offset fix for triton_mrope (#58526) Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg> Co-authored-by: Kimi <noreply@moonshot.cn> Co-authored-by: Isotr0py <Isotr0py@outlook.com> * [Bugfix][Quantization] Refresh online NVFP4 scales before reload post-processing (#57954) Signed-off-by: S1ro1 <matej.sirovatka@gmail.com> Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com> Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: aoshen02 <aoshen@inferact.ai> * [Perf][DSv4.1] Restore the fused query RMSNorm + MXFP8 quantization path (#57679) Signed-off-by: Juntian Liu <Juntianl777@gmail.com> * [Bugfix][LogitsProcessor] Validate ':' separator in custom logits processor FQCN (#56020) Signed-off-by: 100milliongold <gadian88@gmail.com> * [ROCm][CI][AITER Coverage] Harden MoE sorting-backend/dispatch env-var test matrix (#58393) Signed-off-by: Divakar Verma <divakar.verma@amd.com> * [gRPC] Fix ping tolerance so long non-streaming RPCs are not dropped (#55102) Signed-off-by: Wei Gong <wei@together.ai> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> * [Perf][Rust Frontend] Make histogram observations lock-free (#58574) Co-authored-by: jthomson04 <jwillthomson19@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com> * [ROCm][Perf] MXFP8 GEMM on native 32x32 block scales for gfx950 (#58510) Signed-off-by: fai <fangzhouai@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * [Bugfix][Qwen4Exp] Keep pinned PLE prefetch ids out of the CUDA graph pool (#58489) Signed-off-by: Juntian Liu <Juntianl777@gmail.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [Perf][Engram] Serialize offloaded lookups and pack host tables into huge pages (#56926) Signed-off-by: Juntian Liu <Juntianl777@gmail.com> Co-authored-by: Kimi Code <noreply@moonshot.cn> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [Bugfix][Quantization] Fix MXFP8 startup crash on layers below mm_mxfp8 shape limits (#54223) Signed-off-by: samuelkim7 <samuelmwkim@gmail.com> Signed-off-by: mgoin <mgoin64@gmail.com> Co-authored-by: mgoin <mgoin64@gmail.com> * [ROCm] Cut 69 wasted contiguous copies per decode step from the skinny GEMM path (#58566) Signed-off-by: lifulu <fululi12@amd.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Nick Hill <nickhill123@gmail.com> * [ROCm][Build] Filter crate tags from vLLM version detection (#57744) Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Signed-off-by: reidliu41 <reid201711@gmail.com> * [Perf][Distributed] Add low-SM multimem reduce-scatter for SM100/SM103 (#55072) Signed-off-by: Yongye Zhu <zyy1102000@gmail.com> Signed-off-by: Summer Yang <girasoleyang@gmail.com> Co-authored-by: Summer Yang <girasoleyang@gmail.com> * [PP][XPU]Add the flag to control microbatch feature on MRV2+PP (#55145) Signed-off-by: yisheng <yi.sheng@intel.com> Co-authored-by: Kunshang Ji <kunshang.ji@intel.com> * [Feature] Triton kernel dispatcher (#43048) Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com> * [ROCm] Fix CI runtime and tests for MI355 DPX (#58244) Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Co-authored-by: Mahesh Kunreddi <mahesh.kunreddi@amd.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [CI][ROCm] Prevent Model Executor apt stalls (#58607) Signed-off-by: Andreas Karatzas <akaratza@amd.com> Co-authored-by: Codex <noreply@openai.com> * [Qwen4Exp][ROCm] PLE n-gram table CPU offload (#57497) Signed-off-by: Mathew Odden <modden@redhat.com> Signed-off-by: Andreas Karatzas <akaratza@amd.com> Co-authored-by: opencode+deepseek-v4-flash+vllm <opencode+deepseek-v4-flash+vllm@example.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> Co-authored-by: OpenAI Codex <noreply@openai.com> * [MM] Add Triton kernel for mm_input_normal. (#56798) Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io> Signed-off-by: wang.yuqi <noooop@126.com> Signed-off-by: Isotr0py <Isotr0py@outlook.com> Co-authored-by: Isotr0py <2037008807@qq.com> Co-authored-by: Isotr0py <Isotr0py@outlook.com> * [Structured Outputs] Parse Lark grammars natively in the xgrammar backend (#58321) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com> * [ROCm] Credit ROCm/aiter for the block32 GEMM's packed kernel and in-launch split-K (#58659) Signed-off-by: Lingpeng Jin <103567126+valarLip@users.noreply.github.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [Frontend] Handle Disable Thinking in /v1/messages (#58613) Signed-off-by: jryberg <johan.ryberg@security.ntt> Signed-off-by: Robert Shaw <robshaw@redhat.com> Co-authored-by: jryberg <johan.ryberg@security.ntt> Co-authored-by: Robert Shaw <robshaw@redhat.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * [Bugfix] Stop leaking the internal field name in the max_tokens validation error (#58336) Signed-off-by: shallow10 <495593563@qq.com> * [Bugfix][KV Cache][MLA] Align packed block strides for V3.2 sparse MLA (#55528) Signed-off-by: lz <145014769+200lz@users.noreply.github.com> Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com> Co-authored-by: OpenAI Codex <codex@openai.com> Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com> Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [Perf][DSv4] Fuse inverse RoPE + FP8 quant into FlashInfer sparse MLA (#58621) Signed-off-by: Yongye Zhu <zyy1102000@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * [UX][Frontend] Introduce `vllm preload` cli for fast restart (#56680) Signed-off-by: Isotr0py <Isotr0py@outlook.com> * [MoE] Defer the TRTLLM-Gen top-k finalize on the modular path (#58635) Signed-off-by: Yongye Zhu <zyy1102000@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * [Docs] Add return annotation to `fused_mm_input_norm_triton` (#58687) Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * [CI] Shard (H100) Helion Kernels five ways (#58645) Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * [Core] Model console logging as CLI configuration (#57205) Add `--logging-config` CLI argument which can be supplied as JSON or using dotted arguments. The `--log-level` argument is provided for convenience, and `--log-config-file` is deprecated in favor of `--logging-config.pylogging_config_file`. Signed-off-by: Mark McLoughlin <markmc@redhat.com> Co-authored-by: AI Assistant <noreply@openai.com> * [ROCm][CI] Pass weight_shape in MXFP8 block32 linear tests (#58698) Signed-off-by: Djordje Ramic <djoramic@amd.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [SpecDecode] Add LiLiCorr drafter (#57934) Signed-off-by: Andrii Skliar <askliar@nvidia.com> Signed-off-by: Andrii Skliar <andreyws96@gmail.com> Co-authored-by: Andrii Skliar <askliar@nvidia.com> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com> * [MRV2] Minor model_runner.py code cleanup (#58610) Signed-off-by: Nick Hill <nickhill123@gmail.com> * [Bugfix] Fix generative scoring body cancellation (#57729) Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [Pooling] Preserve reranker tokenization with document limits (#57666) Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com> Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com> * [Kernel][Perf] Register-resident path for per-token-group 8-bit quant (#55330) Signed-off-by: chao.huan <chao.huan@nio.com> Co-authored-by: Misha Goin <mgoin64@gmail.com> * [Perf][Kernel] Vectorized flat abs-max for dynamic per-tensor FP8 quantization (#58194) Signed-off-by: Monishver Chandrasekaran <monishverchandrasekaran@gmail.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: Misha Goin <mgoin64@gmail.com> * [Core][Logging] Fix JSON logging process decoration (#57957) Signed-off-by: Mark McLoughlin <markmc@redhat.com> * [Bugfix] Stop allocator fragmentation from shrinking the KV cache during memory profiling (#58430) Signed-off-by: Robert Shaw <robertgshaw2@gmail.com> Signed-off-by: Robert Shaw <robertgshaw2-redhat@h100-01.nemg-001.lab.rdu2.dc.redhat.com> Signed-off-by: Robert Shaw <robshaw@redhat.com> Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Co-authored-by: Robert Shaw <robertgshaw2-redhat@h100-01.nemg-001.lab.rdu2.dc.redhat.com> Co-authored-by: Robert Shaw <robshaw@redhat.com> Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [Bugfix][Frontend] Keep logprobs of parser-suppressed streaming chunks (#58583) Signed-off-by: errmakov <ide404@gmail.com> Co-authored-by: Yanxiao Zhao <39199723+sdpkjc@users.noreply.github.com> Co-authored-by: Prakhar Agarwal <270064960+agarwalprakhar2511@users.noreply.github.com> Co-authored-by: Flora Feng <4florafeng@gmail.com> * [Bugfix][GLM-5.3-Flash] SM90 sparse MLA: index_kpool mismatch leads to corruption via unread query token (#58704) Signed-off-by: Matt Mastracci <matthew@mastracci.com> Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com> * [Benchmark] Record model_id in bench latency/throughput --output-json (#58112) Signed-off-by: yashasvi <yashasvi@ibm.com> * [Bugfix][GLM-5.3-Flash] kpool corruption with speculative decoding (#58454) Signed-off-by: Matt Mastracci <matthew@mastracci.com> Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com> Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [GLM5.3 Perf] Optimize glm 5.3 metadata op, 1.6~4.8x kernel level performance improvement (#58450) Signed-off-by: yewentao256 <zhyanwentao@126.com> * [Bugfix][KV Connector] Reap expired NIXL leases behind a heartbeated head (#58292) Signed-off-by: GokayAI <60583610+gokay-ai@users.noreply.github.com> Co-authored-by: GokayAI <gokay-ai@users.noreply.github.com> * [ROCm][Kimi-K3] Optimize low-concurrency speculative KDA (#58045) Signed-off-by: jiacao-amd <jiahui.cao@amd.com> Co-authored-by: OpenAI Codex <noreply@openai.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [ROCm][CI] Cover the AITER MQA logits dispatch on gfx950 (#58724) Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com> Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [ROCm][CI] Add quantized MoE serving test for gfx950 (#58748) Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com> Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [ROCm][CI] Test AMD DeepSeek V4 MoE routing against a PyTorch reference (#58740) Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com> Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [Perf] DiffusionGemma: one-pass sampler statistics kernel (#58226) Signed-off-by: Matt Mastracci <matthew@mastracci.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> * [ROCm][CI] Expand single-GPU coverage on MI355 DPX (#57599) Signed-off-by: Sheral Kumar <shekumar@amd.com> Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Signed-off-by: Andreas Karatzas <akaratza@amd.com> Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com> Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> Co-authored-by: OpenAI Codex <codex@openai.com> Co-authored-by: Andreas Karatzas <andreas.karatzas@protonmail.com> * [CI] [MRV2] Restore MRV2 pp dp coverage (#57735) Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com> * [Bugfix][CI] Report subprocess test skips as skips, not passes (#58701) Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [ROCm][CI] Run the MLA attention+quant fusion test on ROCm (#58717) Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com> Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [ROCm] Bump torch 2.13, triton 3.8, torchaudio, torchvision (#50605) Signed-off-by: Rohan Potdar <rohan.potdar@amd.com> Signed-off-by: Rohan138 <rohanpotdar138@gmail.com> Signed-off-by: jpvillam <juan.villamizar@amd.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: jpvillam <juan.villamizar@amd.com> * [Feature][Frontend] Add granite_thinking_parser reasoning parser for Granite 4.2 (#55957) Signed-off-by: Yousaf shah <yousaf.shah@gmail.com> Co-authored-by: sfeng33 <4florafeng@gmail.com> * [Bugfix][Frontend] Respect max_output_tokens in the Harmony tool-call loop (#58551) Signed-off-by: errmakov <ide404@gmail.com> Co-authored-by: Du Bin <8174807+dubin555@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Flora Feng <4florafeng@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [Bugfix][Frontend] Document 404 response for `/generative_scoring` (#58788) Signed-off-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: Kimi Code <noreply@moonshot.cn> * [Perf][Frontend] Defer reasoning usage recounts for non-continuous chat streams (#56067) Signed-off-by: Cheng Rui <286040359@qq.com> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Flora Feng <4florafeng@gmail.com> * [Bugfix] Default missing detail for Responses API input images (#57241) Signed-off-by: Yifan Zong <yzong@redhat.com> Co-authored-by: Ben Browning <56071+bbrowning@users.noreply.github.com> * [watermarking] golden tests for backwards compatibility (#56809) Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai> Signed-off-by: Simon Veitner <sveitner@redhat.com> Co-authored-by: Simon Veitner <sveitner@redhat.com> Co-authored-by: OpenAI Codex <codex@openai.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [Bugfix] Don't drop the rest of the allocator config when toggling expandable segments (#57982) Signed-off-by: Oxana Korzh <okorzh@amd.com> * [AuxOutput] Only require Model Runner V2 on GPU platform (#58205) Signed-off-by: Linkun Chen <github@lkchen.net> * [Pooling] Preserve BERT-family heads for raw logits (#57664) Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com> Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com> * fix: perf: use startswith(x, i) instead of string slicing to avoid O(N^2) (#52580) Signed-off-by: Ricardo-M-L <ricardoporsche001@icloud.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * [Bugfix] V1: fix allowed_token_ids_mask aliasing in InputBatch.swap_states (#48419) Signed-off-by: Rui Zhu <rui.zhu.rz399@yale.edu> Co-authored-by: Claude <noreply@anthropic.com> * [Bugfix][Reasoning] Count Kimi K3 reasoning tokens (#58372) Signed-off-by: Elvir Crncevic <elvircrn@gmail.com> Signed-off-by: Flora Feng <4florafeng@gmail.com> Co-authored-by: Flora Feng <4florafeng@gmail.com> * [CI] Split (H200 MIG/MI300) Basic Correctness into named jobs (#57054) Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com> Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Co-authored-by: Codex <noreply@openai.com> * [Bugfix][Frontend] Fix Inkling tool name leaking into content after reasoning (#58792) Signed-off-by: Baljinder Hothi <baljinder.hothi@cohere.com> Co-authored-by: Flora Feng <4florafeng@gmail.com> * [Core] Bound draft-token RPC waits by the execute-model timeout (#58779) Signed-off-by: VS Chandra Mourya <219748331+vschandramourya@users.noreply.github.com> Co-authored-by: VS Chandra Mourya <219748331+vschandramourya@users.noreply.github.com> * [Perf][DSv4.1] Shard the Engram wkv projection across TP ranks (#58678) Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com> * [RL] Add sharding-aware NCCL M2N weight transfer (#51520) Signed-off-by: Ke Wen <kwen@nvidia.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: aoshen02 <aoshen@inferact.ai> * [Bugfix][ROCm] AMD-Quark mixed-precision DeepSeek-V4.1 support (#57071) Signed-off-by: Xiao Yu <xiao.yu.dc@outlook.com> Co-authored-by: Cursor <cursoragent@cursor.com> * [Bugfix][Frontend][Rust Frontend] Update DeepSeek V4.1 Flash reasoning effort mappings (#58316) Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Bugen Zhao <i@bugenzhao.com> Signed-off-by: zhec <chengyunfei@ruc.edu.cn> * [CI] Only isolate the registry tests that need a fresh process (#58764) Signed-off-by: aarushjain29 <Aarushi.Jain2@amd.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [Spec Decode] Enable Gemma4 DSpark adaptive verification with FlashInfer (#57263) Signed-off-by: zixi-qi <zixi@inferact.ai> Co-authored-by: OpenAI Codex <noreply@openai.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: Nick Hill <nickhill123@gmail.com> * [CI] Split (B200) Miscellaneous Kernels into mHC, FLA Ops and Misc named jobs (#58609) Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Kevin H. Luu <khluu000@gmail.com> * [Security] Gate per-request multimodal processor kwargs (#58830) Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com> Co-authored-by: Cursor <cursoragent@cursor.com> * [Elastic EP] Fix EPLB load statistics during scaling (#58473) Signed-off-by: Itay Alroy <ialroy@nvidia.com> Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com> * [Refactor] Remove dead tests utils (#58803) Signed-off-by: yewentao256 <zhyanwentao@126.com> * [Perf][MoE] Index expert mapping lookups in RoutedExperts.load_weights (#58720) Signed-off-by: Willian <willian@willian.email> Signed-off-by: mgoin <mgoin64@gmail.com> Co-authored-by: mgoin <mgoin64@gmail.com> * [Bugfix] Fix Anthropic Thinking Disabled with P/D (#58786) Signed-off-by: Robert Shaw <robshaw@redhat.com> Co-authored-by: Robert Shaw <robshaw@redhat.com> * [Bugfix][Frontend] Detect Anthropic inline-system merge against the resolved chat template (#58754) Signed-off-by: Robert Shaw <robshaw@redhat.com> Co-authored-by: Robert Shaw <robshaw@redhat.com> * [Mypy] Fix mypy typing for Qwen and Qianfan models (#58046) Signed-off-by: Ashraf Bhuiyan <mbhuiyan@redhat.com> * [CI] Reduce CUDA graph mode test overhead (#58749) * [GLM5.3 Bug] Fix sparse indexer attn topk backend selection (#58594) Signed-off-by: yewentao256 <zhyanwentao@126.com> * [CI] Stabilize batch submission in full CUDA graph tests (#58810) * [Bugfix][DSV4.1] Avoid host sync in ViT CUDA graph replay metadata (#58499) * [Security] Harden message sanitization (#58832) Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk> * [Kernel][DSV4.1] Fuse MoE finalize into the TP all-reduce + mHC boundary (#58586) Signed-off-by: Yongye Zhu <zyy1102000@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * [PCP][DCP] Support DCP target model with non-DCP Dspark (#56723) Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Codex <noreply@openai.com> * [Kernel] Bump FlashKDA to keep the recurrent state in fp32 (#58846) Signed-off-by: Simon Veitner <sveitner@redhat.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com> * [Metrics][KV Offload] Add Prometheus metrics for SimpleCPUOffloadConnector (#57251) Signed-off-by: Vincent <vincexxchan@gmail.com> * [mooncake] support CUSTOM_MEM_POOL in vllm (#49300) Signed-off-by: bruce.xu <bruce.xb@alibaba-inc.com> Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai> Co-authored-by: bruce.xu <bruce.xb@alibaba-inc.com> Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai> Co-authored-by: OpenAI Codex <codex@openai.com> * [Qwen3.8-Flash-Next] Avoid memory fragmentation in QSA indexer logits workspace (#57105) Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg> * [Bugfix] Disable sequence parallelism / async TP under batch invariance and add a TP regression test (#56377) Signed-off-by: LioEinaudi <zhao3024667639@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [ROCm][Kimi-K3] Make VLLM_ROCM_USE_AITER_MOE_SITUV2 select a4w4/a8w4/a16w4 (#58201) Signed-off-by: Hongxia Yang <hongxia.yang@amd.com> * [Bugfix][NIXL] Release a dead peer's NIXL state without waiting for TTL (#50047) Signed-off-by: Yannik Hinteregger <37209495+YannikHinteregger@users.noreply.github.com> Co-authored-by: xijiade.aihemaiti <3146335281@qq.com> * [Bugfix] V1: clear stale allowed_token_ids mask in InputBatch.condense (#43931) Signed-off-by: Varshith <kvarshithgowda@gmail.com> Co-authored-by: Claude <noreply@anthropic.com> * [Perf][DSv4.1] Fuse small-batch WO-A with inverse RoPE and MXFP8 quant on SM100/SM103 (#58634) Signed-off-by: Juntian Liu <Juntianl777@gmail.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * [Attention][CPU] Use zentorch SDPA for CPU MLA prefill (#54967) Signed-off-by: Rakul Chauhan <rakul.chauhan@amd.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Cursor <cursoragent@cursor.com> * [Perf][Attention] Remove D2H sync from FlashInfer SM90 sparse MLA plan under async scheduling (#58684) Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai> * [CI] Split (H200 MIG 35GB) Spec Decode Speculators + MTP into 4 named jobs (#57237) Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com> Co-authored-by: Thang Nguyen <thang.nguyen@inferact.ai> Co-authored-by: Kimi Code <noreply@moonshot.ai> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * [ROCm][Perf] Enable layer-aware CSA2 multi-stream overlap for DeepSeek-V4.1-Flash (#57407) Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com> * [Perf][Pooling] Avoid blocking seq_lens GPU-to-CPU copy for pooling in FlashInfer metadata builder (#57214) Signed-off-by: frankwang28 <frank.wbb@hotmail.com> Co-authored-by: Misha Goin <mgoin64@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com> * [Bugfix] Support repsonse_format + tool_choice=auto (#56086) Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com> Co-authored-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com> Co-authored-by: pablopupo <145598901+pablopupo@users.noreply.github.com> Co-authored-by: hubunt <150658615+hubunt@users.noreply.github.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> * [Fast Start] Add `/health` endpoint for the weight cache daemon (#58552) Signed-off-by: Xun Sun <UNIDY2002@outlook.com> Co-authored-by: Isotr0py <Isotr0py@outlook.com> * [Perf][MoE] Use fused MiniMax2 routing with non-unit routed scaling (#58880) Signed-off-by: zjy0516 <riverclouds.zhu@qq.com> * [ROCm][Bugfix] Fall back to default GEMM for CPU tensors on ROCm builds (#58923) Signed-off-by: fai <fangzhouai@gmail.com> * [Bugfix][KV Connector] Retry Mooncake bootstrap registration on timeout (reopens #55763) (#58919) * [Frontend] Switch Python Harmony dependency to oss-harmony (#55128) Signed-off-by: Anton Peganov <apeganov@nvidia.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: OpenAI Codex <codex@openai.com> * [ROCm] Bump AITER to v0.1.23 (#58867) Signed-off-by: fai <fangzhouai@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [Bugfix][Frontend] Count Responses reasoning tokens per tool round (#58927) Signed-off-by: sfeng33 <4florafeng@gmail.com> * [KimiViT][Perf] Fuse per-layer QK RoPE into one in-place kernel (#58651) * [Mypy] Fix mypy typing for Ultravox and Unlimited-OCR models (#58239) Signed-off-by: Ashraf Bhuiyan <mbhuiyan@redhat.com> * [Bugfix][Frontend] Sample batched chat completions from the adjusted requests (#58929) Signed-off-by: sfeng33 <4florafeng@gmail.com> * [Bugfix][Frontend] Use a fresh parser per choice in non-streaming chat completions (#58939) Signed-off-by: sfeng33 <4florafeng@gmail.com> * [Bugfix][EPD] Skip sampling for encoder-only async steps (#58490) Signed-off-by: Tianyu Guo <guoty@inferact.ai> * [CPU] Build CPU wheels on Ubuntu 22.04 with AMX-FP8 support (#58515) Signed-off-by: zhejiangxiaomai <zhenhui.zhao@intel.com> Signed-off-by: jiang1.li <jiang1.li@intel.com> Co-authored-by: jiang1.li <jiang1.li@intel.com> * [Test][ROCm] Stabilize the mixed OLMoE LoRA test (#58945) Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com> * [Model] Enable LoRA support for RobertaForSequenceClassification (#58884) Signed-off-by: jz-yolo <jz-yolo@users.noreply.github.com> Signed-off-by: Jane Zhu <jane.zhu@slack-corp.com> Co-authored-by: jz-yolo <jz-yolo@users.noreply.github.com> * [Bugfix][Frontend] Apply Harmony adjust_request in batched chat completions (#58958) Signed-off-by: sfeng33 <4florafeng@gmail.com> * [Minimax-M3] Add Encoder CUDA graph support (#58673) Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg> Co-authored-by: Codex <codex@openai.com> * [Perf][MRV2] Reuse Mamba/GDN metadata across KV cache groups (#58762) Signed-off-by: Luca Motz <luca.motz@icloud.com> Co-authored-by: Xin Yang <xyangx@amazon.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com> * [Bugfix] Fix the two multimodal root tests that fail on main (OpenPangu-VL embed merge, MiMo sink test fixture) (#58900) Signed-off-by: khluu <khluu000@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * [Benchmark] Add Responses API backend to vllm bench serve (#54628) Signed-off-by: QHarshil <harshil_c@hotmail.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Chauncey <chaunceyjiang@gmail.com> * [CI] Allowlist-shrink batch 1: wire 17 root-level tests + drop 5 stale watermarking entries into misc.yaml (#58055) Signed-off-by: Turner Jabbour <doubleujabbour@gmail.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> * [CPU] Use accelerator memory API in DiffusionGemma (#58964) Signed-off-by: zhejiangxiaomai <zhenhui.zhao@intel.com> Co-authored-by: Li, Jiang <jiang1.li@intel.com> * [CPU] Add video inferencing via torchcodec on s390x (#58693) Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com> Co-authored-by: Li, Jiang <jiang1.li@intel.com> * [Skills] Update kernel-microbenchmark to include ROCm (#58646) Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg> Co-authored-by: Kimi <noreply@moonshot.cn> * [Misc] Name each backend and its kernel block sizes in block-size errors (#58557) Signed-off-by: Eugenio "Jay" Zuccarelli <11176606+jayzuccarelli@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [CI] Shard (H200 MIG 35GB / MI355 DPX) Entrypoints Integration (Pooling) into named jobs (#58652) Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com> Signed-off-by: khluu <khluu000@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: khluu <khluu000@gmail.com> * [CI] Split Dynamic Shapes out of (H200 MIG 35GB) PyTorch Compilation + (MI300) mirror (#58451) Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * [Bugfix][Qwen4Exp] Release the profiling KV cache held by QSA key views (#58961) Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [CPU] Include vLLM Recipes tooling in release image to deploy models using vLLM Recipes (#58796) Signed-off-by: louie-tsai <louie.tsai@intel.com> * Revert "[CI] Shard (H200 MIG 35GB / MI355 DPX) Entrypoints Integration (Pooling) into named jobs (#58652)" (#59011) Signed-off-by: khluu <khluu000@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * [Perf][Qwen4Exp] Fuse HC down projection and SiLU on NVIDIA (#58957) Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg> * [Bugfix][Model] Gemma4: register aliased embedding scalars as buffers (#54213) Signed-off-by: Yannick Schnider <Yannick.Schnider1@ibm.com> * [Bugfix][Spec Decode] Implement get_top_tokens() on the ROCm DeepSeek V4 MTP drafter (#57568) Signed-off-by: BaoYunkai <ybao@amd.com> * [Perf][Qwen3.8] Reduce PLE metadata construction overhead (#58114) Signed-off-by: zjy0516 <riverclouds.zhu@qq.com> Co-authored-by: OpenAI Codex <codex@openai.com> * [ROCm][Refactor] Move DeepSeek-V4/V4.1 multi-stream overlap gate to ROCm platform (#58983) Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com> * [Multimodal] Avoid extra d2d for encoder cudagraph with fused input norm (#56711) Signed-off-by: Isotr0py <Isotr0py@outlook.com> * [Bugfix][Logging] Preserve application log record factories (#58747) Signed-off-by: mgoin <mgoin64@gmail.com> Co-authored-by: OpenAI Codex <noreply@openai.com> * [Bugfix] Use the correct repository revision for secondary artifact loaders (#57461) Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com> Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [ROCm][Perf] Kimi-K3 Enable sharded latent MoE up-projection under EP (#54956) Signed-off-by: Xavier Aguilar <xavier.aguilarfruto@amd.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [Feature] Add fixed-token prefill scoring (#54335) Signed-off-by: aoshen02 <aoshen@inferact.ai> Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com> Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> * [LoRA] Support variable num_labels for sequence classification (#57766) Signed-off-by: linitra24 <renshuang.zhou@daocloud.io> Co-authored-by: Jee Jee Li <pandaleefree@gmail.com> * [ROCm][Perf] Replace torch.topk in DSA candidate block selection (#58208) Signed-off-by: fai <fangzhouai@gmail.com> Co-authored-by: TJian <tunjian.tan@embeddedllm.com> Co-authored-by: Cursor Agent <cursoragent@cursor.com> * [ROCm]Keep LMCache OpenTelemetry on the image's 1.40 stack (#59056) Signed-off-by: Micah Williamson <micah.williamson@amd.com> * [Bugfix][Kimi-K3] Refresh DSpark context KV cache pointers after the KV cache is re-bound (#58814) Signed-off-by: Oxana Korzh <okorzh@amd.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * [KVConnector][NIXL] Support packed MLA KV layouts in pipeline-parallel push prefill (#50499) Signed-off-by: zixi-qi <zixi@inferact.ai> * [Quant] Use canonical N-first weight format for CT WNA16 MoE (#52798) Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: HDCharles <{"message":"Not Found","documentation_url":"https://docs.github.com/rest/users/emails#list-email-addresses-for-the-authenticated-user","status":"404"}> Co-authored-by: OpenAI Codex <codex@openai.com> Co-authored-by: OpenAI Codex <noreply@openai.com> * [CI][Bugfix] Relax packed_qk_rope_ correctness test to one ULP (#59008) Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [ROCm][CI] Pin OpenTelemetry to LMCache's cap in the ROCm images (#59051) Signed-off-by: Rohan Potdar <rohan.potdar@amd.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * [ROCm][CI] Increase timeout for Entrypoints Unit (#59076) Signed-off-by: Djordje Ramic <djoramic@amd.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [ROCm][MLA] Add an AITER ASM round-robin decode route for DCP multi-token verify (#56861) Signed-off-by: Xiaohu Guo <Xiaohu.Guo@amd.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: seungrokj <144636725+seungrokj@users.noreply.github.com> Co-authored-by: Cursor <cursoragent@cursor.com> * [Bugfix] Don't sync-police or retry FlashInfer all-reduce workspace creation in eager mode (#58498) Signed-off-by: khluu <khluu000@gmail.com> Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com> Co-authored-by: Nick Hill <nickhill123@gmail.com> * [Bugfix] Fix resumable request + async scheduling handoff race (#58259) Signed-off-by: Yifan Zong <yzong@redhat.com> * [Model Runner V2][Spec Decode] Support spec decode with draft model (#43091) Signed-off-by: Icey <1790571317@qq.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [Perf][MRV2] Allow FULL decode graphs for one-token prompt tails (#58400) Signed-off-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Yongye Zhu <zyy1102000@gmail.com> * [Bugfix][Scheduler] Preserve logprobs across streaming continuations (#57447) Signed-off-by: 0xsensei <prblmslvr.aditya@gmail.com> * [Mypy] Fix mypy typing for Voxtral and vision models (#58251) Signed-off-by: Ashraf Bhuiyan <mbhuiyan@redhat.com> Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com> * [Core] Rework scheduler `skipped_waiting` queue (#58947) Signed-off-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com> * [Bugfix][MRV2][Spec Decode] Reject draft slots that were never proposed (#58784) Signed-off-by: zixi-qi <zixi@inferact.ai> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Nick Hill <nickhill123@gmail.com> * [ROCm][CI] Add missing test coverage for upstream parity (#50519) Signed-off-by: Andreas Karatzas <akaratza@amd.com> Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com> Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Co-authored-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com> Co-authored-by: OpenAI Codex <codex@openai.com> Co-authored-by: OpenAI Codex <noreply@openai.com> * [Bugfix][Scheduler] Refresh max tokens for streaming continuations (#57676) Signed-off-by: 0xsensei <prblmslvr.aditya@gmail.com> Co-authored-by: OpenAI Codex <codex@openai.com> * [Perf][Spec Decode] Avoid triton recompiles in the acceptance estimator (#57107) Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai> Co-authored-by: Woosuk Kwon <woosuk@inferact.ai> * [CI/Build] Skip the snapshot runtime on CUDA 12.x images (#59118) Signed-off-by: khluu <khluu000@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit aedaba8664f67d8ae1538e5e5cec2b1ce3f258dd) * [XPU][CI]Skip test_abort_timeout_on_prefiller in nightly (#58307) Signed-off-by: zengxian <xiangdong.zeng@intel.com> Co-authored-by: Kunshang Ji <kunshang.ji@intel.com> (cherry picked from commit 09c47db1ca080793dc2351144cb39513fd7984ca) * [Bugfix][Engram] Keep THP tables private when resolving shared memory (#59068) Signed-off-by: Ren Yuzhou <54501155+yuzhouo7@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com> (cherry picked from commit ec5e0c352f079c2cb8f46752fd0317a2745050bf) * [ROCm][CI] Expand MI355 mirrors and route MIG-sized jobs to DPX (#59137) Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com> (cherry picked from commit 6ebb5bd64f5ef75dbe1471ee88c009003b3d03ec) * [Bugfix][Mamba] Keep the prompt-end prefill checkpoint under sparse retention (#59146) Signed-off-by: Jared Wen <jaredwen@inferact.ai> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com> (cherry picked from commit d882bddbeab6b4a0d5861dfcb171bf61ce2109d6) * [Core][BugFix] Tag prefix-cache extra keys by source (#51899) Signed-off-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com> Co-authored-by: Tai An <antai12232931@outlook.com> Co-authored-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 765872e7ed5b6f372f0898a67e48d3365fa28139) * [Bugfix][Frontend] Reject LoRA adapters named after a served model (#59286) Signed-off-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 105a4e097b905127b1d5f09a7e93bd1095e87a6c) * [Dependency] Upgrade FlashInfer to 0.7.0.post1 (#59323) Signed-off-by: wzhao18 <wzhao18.sz@gmail.com> (cherry picked from commit 678baf53724e06cef7f07628ae8fd4cc6c96f11a) * [Bugfix][HiSparse] Resolve MTP verification rows with a union residency kernel (#59235) Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 2798f668608155bc8a8c74cb97e3ddd0d3053085) * [Bugfix][HiSparse] Never allocate GPU pages without host backing (#59036) Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com> Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Co-authored-by: Matthew Bonanni <mbonanni@redhat.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 90e13fc757cffc11b55689d9cc33d844d9606516) * [CPU][Whisper] Support W4A16 quantized Whisper on the CPU WNA16 kernel (#58268) Signed-off-by: Harshal Adhav <harshal.adhav@amd.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Li, Jiang <jiang1.li@intel.com> (cherry picked from commit 5faf81a4297921c699f9aa58f9c6395e95ede718) * [Core] Bound UniProc EngineCore startup threads to available CPUs (#58946) Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com> (cherry picked from commit 866fa130fac1e1252330a680fbed45ad05eba64f) * [KV-Offloading][TP] : Expand replicated_layout detection to multi-group MLA (#57652) (cherry picked from commit 5463fe4962785cdc3383477bf3af6533e7647dfd) * [Bugfix][HiSparse] Preserve host prefix publication after request completion (#59007) Signed-off-by: Yueh-Ting Chen <yueh.ting.chen@gmail.com> Co-authored-by: Matthew Bonanni <mbonanni@redhat.com> (cherry picked from commit ff1b87cca25690fef6bd12667fd0d26690d949af) * [Bugfix][HiSparse] Adopt GPU prefix copies after the hit's allocation (#59282) Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> (cherry picked from commit 3eb6cec22ad9bb098393021b956feaa97081fc85) * [Bugfix][HiSparse] Stop the host pool feeding device KV cache residency metrics (#58725) Signed-off-by: Yueh-Ting Chen <yueh.ting.chen@gmail.com> Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Co-authored-by: Matthew Bonanni <mbonanni@redhat.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 73c7cae4d746f64c22677348d1cc120eea6f7439) * [Bugfix][Core] Fix mamba prefill checkpoint block reservation and prompt-end eviction in align mode (#59175) Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com> Co-authored-by: Nick Hill <nickhill123@gmail.com> (cherry picked from commit 7e583e615c20ee4ff0cd82aac592c7d6310aa7c3) * [Core] Include the LoRA path in prefix-cache block hashes (#59335) Signed-off-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit c055c1e075ed461cff922aa710d387281f651940) * [Perf][PP] Skip sampled-token broadcasts whose requests leave the engine (#58542) Signed-off-by: LostFox11 <wangziyue17@huawei.com> Signed-off-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: LostFox11 <wangziyue17@huawei.com> Co-authored-by: Nick Hill <nickhill123@gmail.com> (cherry picked from commit 7314360c9eebb3b4b460ec49f6606ef0a08fcae2) * [CPU][Zen] Add DA8W4 (W4A8) int4 support for dense and MoE layers (#54024) Signed-off-by: R <Ganesh.R@amd.com> Signed-off-by: Ganesh R <Ganesh.R@amd.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> (cherry picked from commit e12291d7332db897885fdc0e6ed61b28c969d743) * [Model Runner V2] Support randomized dummy inputs (#58411) Signed-off-by: Robert Shaw <robshaw@redhat.com> Co-authored-by: Robert Shaw <robshaw@redhat.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Nick Hill <nickhill123@gmail.com> (cherry picked from commit e5e38ba9b7d18f9746d989e389a51a94b0f96f6f) * [Bugfix][HiSparse] Fix a chunked-prefill preemption livelock (#59494) Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 3a6963664537ed21172e2ec12e96e3a2dcd3718c) * [Bugfix][HiSparse] Size the KV cache from the groups HiSparse allocates (#59450) Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com> (cherry picked from commit f7999d2e4489126f1c21868699f3890021f01b73) * [Bugfix][HiSparse] Fix MTP acceptance collapse under FULL graphs with a saturated GPU pool (#59309) Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com> Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: Matthew Bonanni <mbonanni@redhat.com> (cherry picked from commit 4056c8ac1f8a7e8fb50cf1c56ff96649fcadccc6) * [CI] Drop a test that depends on #57930 from the #59309 backport Resolving the #59309 cherry-pick conflict in tests/v1/kv_connector/unit/test_hisparse_connector.py pulled in test_scheduled_prefix_hit_publishes_adopted_copies from #57930, which is not on this branch; it imports _allocate_scheduled from tests.v1.core.test_prefix_caching and fails on every platform. After this change the file matches the upstream #59309 diff. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: khluu <khluu000@gmail.com> * [Bugfix] Fix minimax-m3 multimodal processor compatability with Transformers v5.18 (#59613) Signed-off-by: Isotr0py <Isotr0py@outlook.com> (cherry picked from commit b558f160a2c0abcb5902acc3c91a14c38a4af173) * [Misc] Add Transformers version upper bound in requirements (#59614) Signed-off-by: Isotr0py <Isotr0py@outlook.com> Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> (cherry picked from commit 58b3298457dde7b4554b3b4e20b238c0ac2c3a65) * Forward-port the tkv/turbo-attn vLLM seam onto upstream v0.31.0 Squashes arbicity/vllm-turbo main @969c125e9 (upstream v0.28.0 + the seam: b0a14f2eb #28, 76a3e6d72, 6d4c75611 #29) into one commit and replays it onto upstream v0.31.0 (db9527a46, the commit vllm/vllm-openai:v0.31.0 is built from) as a 3-way merge against the real base. Upstream's v0.29-v0.31 KV-cache layout refactor (vllm#51718 and follow-ups) replaced the allocator the seam patched: one backing allocation, per-layer [B, H, N, C] views placed by the engine, page geometry read off the spec, and AttentionBackend.customize_spec applied to every layer's spec by both model runners. The seam is re-expressed on that and shrinks from 49 files to 27. Kept (re-applied on upstream's structure): - plugin KV-cache dtype registry; --kv-cache-dtype choices/type - TURBO_ATTN backend slot (now a distinct enum value: two None members made CUSTOM an alias of TURBO_ATTN), turbo-attn spelling, auto-default for plugin dtypes (#29), CUDA candidate once registered - lifecycle hooks on_model_loaded / on_draft_model_loaded / on_kv_cache_initialized / adjust_kv_budget, fail-loud dispatch to every backend in use - MLA wrapping in the selector; MLA chunked-context _get_gather_op - _tq_layer_idx injection for tkv layers - aggregated_layer_count: fused (composite) pages, now one shared page per fused set in upstream's single-allocation planner - per-group KV pool for hybrid models, re-expressed on v0.31's single backing allocation: the O(1) Mamba/GDN state groups get their own BlockPool sized for max_num_seqs, a region of the allocation after the attention pool; their pages are not unified with attention pages; per-pool admission, events and usage; CoW copies and warmup/profiling block ids per pool; VLLM_SAMPLER_RESERVE_MIB headroom Moved to upstream's extension points (turbo-attn plugin side): - get_kv_cache_spec_class (Attention, MLAAttention, hybrid alignment) -> AttentionBackend.customize_spec - KVCacheSpec.get_manager_class -> KVCacheSpecRegistry MRO lookup - get_supported_kv_cache_dtypes -> supported_kv_cache_dtypes ClassVar - get_kv_cache_shape(kv_cache_spec=...) passthrough and the backend-managed-dtype shape coercion -> spec-driven views Dropped: - spec_decode_warmup.py: upstream registers the same kernels with its JIT warmup registry (aed894c19, vllm#56323) - turbo_attn_warmup.py, utils/cutedsl_cache.py: superseded by turbo-attn's own prefill prewarm (on_kv_cache_initialized) and CuTeDSL cache - padded-page block fill (the split pool removes the page unification it worked around), drain hook / on_kv_manager_created, KVBlockZeroer clamp: no turbo-attn consumer - rotary fast-path registry: upstream guards the import (1f60771c7, vllm#42679) - --kv-cache-dtype-skip-layers-dtype, VLLM_KV_CACHE_SKIP_LAYERS_DTYPE, Triton fused fp8 GEMM hook, gsm8k startup waits, notify-turbo-attn workflow, draft-backend inheritance: unused or superseded - FA2 varlen paged split-K patch: never reached the overlay image (the image installs no compiled _vllm_fa2_C) and the FA pin moved PROTOCOL.md rewritten for the v0.31.0 seam. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Signed-off-by: Lai, Yejing <yejing.lai@intel.com> Signed-off-by: priyansh jain <priyansh.jain2@amd.com> Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com> Signed-off-by: reidliu41 <reid201711@gmail.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com> Signed-off-by: RyanMa29 <ziyang.ma@intel.com> Signed-off-by: R <Ganesh.R@amd.com> Signed-off-by: Shrey Gajjar <shreygajjar007@gmail.com> Signed-off-by: Zijing Liu <liuzijing2014@gmail.com> Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com> Signed-off-by: fai <fangzhouai@gmail.com> Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Signed-off-by: Linze-Shi <linzeshi0@gmail.com> Signed-off-by: yewentao256 <zhyanwentao@126.com> Signed-off-by: Yueh-Ting Chen <yueh.ting.chen@gmail.com> Signed-off-by: Chenglun Hu <chenglunhu@gmail.com> Signed-off-by: hclsys <chenglunhu@gmail.com> Signed-off-by: Wauplin <lucainp@gmail.com> Signed-off-by: Thomas Ortner <boh@zurich.ibm.com> Signed-off-by: nightcityblade <nightcityblade@gmail.com> Signed-off-by: Nick Hill <nickhill123@gmail.com> Signed-off-by: mgoin <mgoin64@gmail.com> Signed-off-by: Felix Marty <Felix.Marty@amd.com> Signed-off-by: Vincent Cave <vincent.cave@amd.com> Signed-off-by: Shiksha Patel <shikpate@amd.com> Signed-off-by: Matt Mastracci <matthew@mastracci.com> Signed-off-by: Aaron Kang <aaron.h.kang@icloud.com> Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com> Signed-off-by: Juntian Liu <Juntianl777@gmail.com> Signed-off-by: Jellow <49915976+CZT0@users.noreply.github.com> Signed-off-by: Jellow <dvdx@foxmail.com> Signed-off-by: Lin, Fanli <fanli.lin@intel.com> Signed-off-by: Fanli Lin <fanli.lin@intel.com> Signed-off-by: Sunita Nadampalli <nadampal@amazon.com> Signed-off-by: Milosz Grunwald <milosz.grunwald@intel.com> Signed-off-by: haosenwang1018 <haosenwang1018@users.noreply.github.com> Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com> Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai> Signed-off-by: khushali9 <khushali.desai9@gmail.com> Signed-off-by: Samyabrata Maji <116789799+sammaji@users.noreply.github.com> Signed-off-by: SIDDARTHA REDDY <75976672+SIDDARTHAREDDY8@users.noreply.github.com> Signed-off-by: samuelkim7 <samuelmwkim@gmail.com> Signed-off-by: khluu <khluu000@gmail.com> Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg> Signed-off-by: S1ro1 <matej.sirovatka@gmail.com> Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com> Signed-off-by: 100milliongold <gadian88@gmail.com> Signed-off-by: Divakar Verma <divakar.verma@amd.com> Signed-off-by: Wei Gong <wei@together.ai> Signed-off-by: lifulu <fululi12@amd.com> Signed-off-by: Yongye Zhu <zyy1102000@gmail.com> Signed-off-by: Summer Yang <girasoleyang@gmail.com> Signed-off-by: yisheng <yi.sheng@intel.com> Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com> Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Signed-off-by: Andreas Karatzas <akaratza@amd.com> Signed-off-by: Mathew Odden <modden@redhat.com> Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io> Signed-off-by: wang.yuqi <noooop@126.com> Signed-off-by: Isotr0py <Isotr0py@outlook.com> Signed-off-by: Lingpeng Jin <103567126+valarLip@users.noreply.github.com> Signed-off-by: jryberg <johan.ryberg@security.ntt> Signed-off-by: Robert Shaw <robshaw@redhat.com> Signed-off-by: shallow10 <495593563@qq.com> Signed-off-by: lz <145014769+200lz@users.noreply.github.com> Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com> Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com> Signed-off-by: Mark McLoughlin <markmc@redhat.com> Signed-off-by: Djordje Ramic <djoramic@amd.com> Signed-off-by: Andrii Skliar <askliar@nvidia.com> Signed-off-by: Andrii Skliar <andreyws96@gmail.com> Signed-off-by: chao.huan <chao.huan@nio.com> Signed-off-by: Monishver Chandrasekaran <monishverchandrasekaran@gmail.com> Signed-off-by: Robert Shaw <robertgshaw2@gmail.com> Signed-off-by: Robert Shaw <robertgshaw2-redhat@h100-01.nemg-001.lab.rdu2.dc.redhat.com> Signed-off-by: errmakov <ide404@gmail.com> Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com> Signed-off-by: yashasvi <yashasvi@ibm.com> Signed-off-by: GokayAI <60583610+gokay-ai@users.noreply.github.com> Signed-off-by: jiacao-amd <jiahui.cao@amd.com> Signed-off-by: Sheral Kumar <shekumar@amd.com> Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com> Signed-off-by: Rohan Potdar <rohan.potdar@amd.com> Signed-off-by: Rohan138 <rohanpotdar138@gmail.com> Signed-off-by: jpvillam <juan.villamizar@amd.com> Signed-off-by: Yousaf shah <yousaf.shah@gmail.com> Signed-off-by: Cheng Rui <286040359@qq.com> Signed-off-by: Yifan Zong <yzong@redhat.com> Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai> Signed-off-by: Simon Veitner <sveitner@redhat.com> Signed-off-by: Oxana Korzh <okorzh@amd.com> Signed-off-by: Linkun Chen <github@lkchen.net> Signed-off-by: Ricardo-M-L <ricardoporsche001@icloud.com> Signed-off-by: Rui Zhu <rui.zhu.rz399@yale.edu> Signed-off-by: Elvir Crncevic <elvircrn@gmail.com> Signed-off-by: Flora Feng <4florafeng@gmail.com> Signed-off-by: Baljinder Hothi <baljinder.hothi@cohere.com> Signed-off-by: VS Chandra Mourya <219748331+vschandramourya@users.noreply.github.com> Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com> Signed-off-by: Ke Wen <kwen@nvidia.com> Signed-off-by: Xiao Yu <xiao.yu.dc@outlook.com> Signed-off-by: zhec <chengyunfei@ruc.edu.cn> Signed-off-by: aarushjain29 <Aarushi.Jain2@amd.com> Signed-off-by: zixi-qi <zixi@inferact.ai> Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com> Signed-off-by: Itay Alroy <ialroy@nvidia.com> Signed-off-by: Willian <willian@willian.email> Signed-off-by: Ashraf Bhuiyan <mbhuiyan@redhat.com> Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk> Signed-off-by: Vincent <vincexxchan@gmail.com> Signed-off-by: bruce.xu <bruce.xb@alibaba-inc.com> Signed-off-by: LioEinaudi <zhao3024667639@gmail.com> Signed-off-by: Hongxia Yang <hongxia.yang@amd.com> Signed-off-by: Yannik Hinteregger <37209495+YannikHinteregger@users.noreply.github.com> Signed-off-by: Varshith <kvarshithgowda@gmail.com> Signed-off-by: Rakul Chauhan <rakul.chauhan@amd.com> Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai> Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com> Signed-off-by: frankwang28 <frank.wbb@hotmail.com> Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com> Signed-off-by: Xun Sun <UNIDY2002@outlook.com> Signed-off-by: zjy0516 <riverclouds.zhu@qq.com> Signed-off-by: Anton Peganov <apeganov@nvidia.com> Signed-off-by: sfeng33 <4florafeng@gmail.com> Signed-off-by: Tianyu Guo <guoty@inferact.ai> Signed-off-by: zhejiangxiaomai <zhenhui.zhao@intel.com> Signed-off-by: jiang1.li <jiang1.li@intel.com> Signed-off-by: jz-yolo <jz-yolo@users.noreply.github.com> Signed-off-by: Jane Zhu <jane.zhu@slack-corp.com> Signed-off-by: Luca Motz <luca.motz@icloud.com> Signed-off-by: QHarshil <harshil_c@hotmail.com> Signed-off-by: Turner Jabbour <doubleujabbour@gmail.com> Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com> Signed-off-by: Eugenio "Jay" Zuccarelli <11176606+jayzuccarelli@users.noreply.github.com> Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com> Signed-off-by: louie-tsai <louie.tsai@intel.com> Signed-off-by: Yannick Schnider <Yannick.Schnider1@ib…

### What this PR does / why we need it? Bump the verified vLLM main pin from `8cc9aa5` to `c4df37df` and adapt vllm-ascend to the upstream changes in that range. The v0.30.0 release lane is kept intact: each adaptation is gated on `vllm_version_is("0.30.0")` (or a version-aware helper) so both the `main` and the `v0.30.0` CPU UT lanes stay green. The range contains three breaking upstream changes plus one release-lane fix: - vllm#58947 reworked the scheduler waiting queues. - vllm#58400 extended the MRV2 `InputBatch`/`BatchReqState` contract. - vllm#53934 turned `CudaGraphManager.dp_size` into a property and changed `EPLBController.maybe_register_model`. - The Ascend Qwen3.5-VL NPU-preprocessing feature imported a main-only fused input norm, breaking the release lane. | Files | Upstream vLLM change (why it breaks vllm-ascend) | vllm-ascend adaptation | |-------|--------------------------------------------------|------------------------| | `.github/vllm-main-verified.commit` | — (main-lane pin only) | Verified main commit bumped `8cc9aa5` -> `c4df37df` | | `tests/ut/core/test_circular_buffer.py` | v0.29.0 lane removed | Drop the now-obsolete `vllm_version_is("0.29.0")` guard and always pass `replay_boundaries` | | `tests/ut/core/test_dyntra_lb_recompute_scheduler.py` | [vllm#58947](vllm-project/vllm#58947) replaced the single `skipped_waiting` queue with `kv_holding_waiting` + `deferred_waiting` | Import and use the `_is_skipped()` helper instead of `request in scheduler.skipped_waiting` | | `tests/ut/core/test_dyntra_lb_scheduler.py` | Same [vllm#58947](vllm-project/vllm#58947) waiting-queue rework | Add `_mark_skipped` / `_is_skipped` / `_summary_waiting_kwargs` helpers and a version-aware promote hook (`_try_promote_blocked_waiting_request` vs `_handle_blocked_waiting_request`) | | `tests/ut/core/test_profiling_chunk.py` | Same [vllm#58947](vllm-project/vllm#58947); renamed blocked helper and admission cap | Version-aware `_handle_blocked_waiting_request`/`max_num_active_reqs` (main) vs `_try_promote_blocked_waiting_request`/`max_num_running_reqs` (v0.30.0) | | `tests/ut/core/test_recompute_scheduler.py` | Same [vllm#58947](vllm-project/vllm#58947) | Use `_is_skipped` instead of `request in scheduler.skipped_waiting` | | `tests/ut/distributed/kv_transfer/kv_p2p/test_remote_prefill_lifecycle.py` | Same [vllm#58947](vllm-project/vllm#58947) replaced `skipped_waiting` | `_skipped_waiting_queue()` resolver: `skipped_waiting` (v0.30.0) / `kv_holding_waiting` (main) | | `tests/ut/models/test_qwen3_5_vl_preprocess.py` | Qwen3.5-VL fused input norm module split (v0.30.0 `FusedInputNorm` in `models.vision`; main `build_mm_input_norm` in `layers.fusion`) | Import the model unconditionally and drop the `skipif`, now that the builder is dual-version | | `tests/ut/patch/platform/test_patch_balance_schedule.py` | Same [vllm#58947](vllm-project/vllm#58947) | Version-aware scheduler method-seam guard (`_handle_blocked_waiting_request`/`_holds_kv_blocks` on main, `_select_waiting_queue_for_scheduling`/`_try_promote_blocked_waiting_request` on v0.30.0) | | `tests/ut/worker/test_model_runner_v2.py` | [vllm#58400](vllm-project/vllm#58400) added `num_draft_tokens_np`/`prefill_runs_as_decode_np`/`decode_graph_eligible` to `BatchReqState` | Supply the new fields (version-gated) in the `_make_batch_state` / `_prepare_inputs_runner` helpers | | `tests/ut/worker/test_v2_dummy_padding.py` | [vllm#58400](vllm-project/vllm#58400) renamed the 4th arg of `get_uniform_decode_token_count` | Pass `batch.has_prefill` (v0.30.0) / `batch.decode_graph_eligible` (main) | | `tests/ut/worker/v2/test_aclgraph.py` | [vllm#53934](vllm-project/vllm#53934) made `CudaGraphManager.dp_size` a read-only property backed by `vllm_config`, so `manager.dp_size = 2` raised `AttributeError` | Set `manager.vllm_config.parallel_config.data_parallel_size` on main; keep the instance attribute on v0.30.0 | | `tests/ut/worker/v2/test_eplb_controller.py` | [vllm#53934](vllm-project/vllm#53934) removed `load_dummy_weights` from `EPLBController.maybe_register_model`, so the 3-arg call raised `TypeError` | Call `maybe_register_model(model, model_config)` on main; keep the 3-arg form on v0.30.0 | | `vllm_ascend/_310p/worker/v2/model_runner.py` | [vllm#58400](vllm-project/vllm#58400) added a required `decode_graph_eligible` to `InputBatch`; the 310P path crashed with `TypeError: Ascend310PInputBatch.__init__() missing 1 required positional argument: 'decode_graph_eligible'` | Pass `decode_graph_eligible = not batch_has_prefill` into the 310P input batch on main | | `vllm_ascend/core/dyntra_lb_scheduler.py` | [vllm#58947](vllm-project/vllm#58947) replaced the single `skipped_waiting` queue with `kv_holding_waiting` (queue) + `deferred_waiting` (set), removed `_select_waiting_queue_for_scheduling`, renamed `_try_promote_blocked_waiting_request` -> `_handle_blocked_waiting_request`, and renamed the admission cap `max_num_running_reqs` -> `max_num_active_reqs`; the v0.30.0 copy crashed with `AttributeError: ... has no attribute 'skipped_waiting' / '_select_waiting_queue_for_scheduling'` | Version-split the waiting bookkeeping: `_skipped_waiting_requests()` / `_prepare_blocked_waiting_request()` helpers, a single versioned `_skip_request()` skip site, async-KV-load skip-before-pop, version-aware re-queue and `prefill_capacity_bound`, and `print_scheduler_summary` chains the right queues per lane | | `vllm_ascend/core/recompute_scheduler.py` | Same [vllm#58947](vllm-project/vllm#58947) waiting-queue rework | Same version-split (`_skip_request`, `kv_holding_waiting`+`deferred_waiting` on main, `skipped_waiting` on v0.30.0; version-aware `max_num_active_reqs` / `prefill_capacity_bound`) | | `vllm_ascend/core/scheduler_profiling_chunk.py` | Same [vllm#58947](vllm-project/vllm#58947) rework; also drops the now-unsupported v0.29.0 lane | Rebase the copied `schedule()` onto v0.30.0/main, version-split the waiting section and re-queue, remove 8 obsolete `vllm_version_is("0.29.0")` branches | | `vllm_ascend/models/qwen3_5_vl.py` | v0.30.0 keeps the fused input norm as `FusedInputNorm` in `vllm.model_executor.models.vision`; main exposes `build_mm_input_norm` in `vllm.model_executor.layers.fusion.mm_input_norm`. The unconditional main-only import broke release-lane loading | Route the builder through `FusedInputNorm.from_model_config` on v0.30.0, keep `build_mm_input_norm` on main | | `vllm_ascend/patch/platform/patch_balance_schedule.py` | Same [vllm#58947](vllm-project/vllm#58947) rework | Same version-split, keeping the three balance deltas intact | | `vllm_ascend/worker/v2/model_runner.py` | [vllm#58400](vllm-project/vllm#58400) added `decode_graph_eligible`/`prefill_runs_as_decode_np` to `BatchReqState`/`InputBatch` and renamed the 4th arg of `get_uniform_decode_token_count` from `has_prefill` to `decode_graph_eligible` | Forward the new fields into `AscendInputBatch` on main; version-gated `_pd_uniform_decode_token_count()` passing `has_prefill` (v0.30.0) vs `not has_prefill` (main) | ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? - vLLM main: vllm-project/vllm@8cc9aa5 --------- Signed-off-by: hfadzxy <starmoon_zhang@163.com> Signed-off-by: zhangxinyuehfad <starmoon_zhang@163.com>

…tibility (#18229) ### What this PR does / why we need it? Upgrade the release dependency from vLLM v0.30.0 to [v0.31.0](https://github.com/vllm-project/vllm/commit/db9527a46873454610df6dbedf79a36d6bf1a7f6) while keeping the independently supported verified-main target at [c4df37d](https://github.com/vllm-project/vllm/commit/c4df37dfcc7f72ca41cd43801a468526ecf99a2b). Rebase on latest Ascend main `c2382c6`, preserving the CPP profiling changes from #18622. The file mapping below describes the diff for `23c8b15` against `c2382c6`. The two targets have different APIs despite both being supported: retain version-dependent forwarding for profiling randomization, PP batch lengths, Mamba allocation, EPLB setup and encoder-only prefill admission. Remove obsolete 0.30 branches only where both targets now share the contract. | Contract | Verified main `c4df37d` | Release v0.31.0 `db9527a` | | --- | --- | --- | | Profiling | Parent calls omit `randomize_inputs` | Forward the flag through V1/V2/310P and MC2 | | PP batch state | No `max_seq_len_np` | Preserve the field in `AscendInputBatch` | | Mamba allocation | Parent has no `prefill_end` | Forward checkpoint boundary | | EPLB setup | Update existing state/communicator | Construct state from model and mapping | | Non-chunked prefill | Keep encoder-only budget exception | Keep release budget rejection | #### Changes organized by downstream file Upstream links identify the reason or contract being adapted. For existing main behavior that becomes common to both targets, the change is removal of the old release branch, not a new port of the complete upstream implementation. File links are pinned to this PR head; [the complete fixed diff](https://github.com/vllm-project/vllm-ascend/compare/c2382c67aa6be12f1c822733cd47d021f61b0b23...23c8b15c2709978463490b0e8531bf5f9f27b9c1) shows the exact edits. <details> <summary>Dependency markers, images and documentation (13 files)</summary> | Downstream file | Upstream change / contract | Ascend adaptation | | --- | --- | --- | | [`.github/vllm-release-tag.commit`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/.github/vllm-release-tag.commit) | Release target advances to [v0.31.0](https://github.com/vllm-project/vllm/releases/tag/v0.31.0); verified main remains pinned independently. | Advance the release marker to `v0.31.0`; leave the verified-main marker at `c4df37d`. | | [`Dockerfile`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/Dockerfile) | Release target advances to [v0.31.0](https://github.com/vllm-project/vllm/releases/tag/v0.31.0); verified main remains pinned independently. | Set the release `VLLM_TAG` default to `v0.31.0`; existing built images are not changed by this default. | | [`Dockerfile.310p`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/Dockerfile.310p) | Release target advances to [v0.31.0](https://github.com/vllm-project/vllm/releases/tag/v0.31.0); verified main remains pinned independently. | Set the release `VLLM_TAG` default to `v0.31.0`; existing built images are not changed by this default. | | [`Dockerfile.310p.openEuler`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/Dockerfile.310p.openEuler) | Release target advances to [v0.31.0](https://github.com/vllm-project/vllm/releases/tag/v0.31.0); verified main remains pinned independently. | Set the release `VLLM_TAG` default to `v0.31.0`; existing built images are not changed by this default. | | [`Dockerfile.a3`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/Dockerfile.a3) | Release target advances to [v0.31.0](https://github.com/vllm-project/vllm/releases/tag/v0.31.0); verified main remains pinned independently. | Set the release `VLLM_TAG` default to `v0.31.0`; existing built images are not changed by this default. | | [`Dockerfile.a3.openEuler`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/Dockerfile.a3.openEuler) | Release target advances to [v0.31.0](https://github.com/vllm-project/vllm/releases/tag/v0.31.0); verified main remains pinned independently. | Set the release `VLLM_TAG` default to `v0.31.0`; existing built images are not changed by this default. | | [`Dockerfile.a5`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/Dockerfile.a5) | Release target advances to [v0.31.0](https://github.com/vllm-project/vllm/releases/tag/v0.31.0); verified main remains pinned independently. | Set the release `VLLM_TAG` default to `v0.31.0`; existing built images are not changed by this default. | | [`Dockerfile.a5.openEuler`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/Dockerfile.a5.openEuler) | Release target advances to [v0.31.0](https://github.com/vllm-project/vllm/releases/tag/v0.31.0); verified main remains pinned independently. | Set the release `VLLM_TAG` default to `v0.31.0`; existing built images are not changed by this default. | | [`Dockerfile.openEuler`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/Dockerfile.openEuler) | Release target advances to [v0.31.0](https://github.com/vllm-project/vllm/releases/tag/v0.31.0); verified main remains pinned independently. | Set the release `VLLM_TAG` default to `v0.31.0`; existing built images are not changed by this default. | | [`README.md`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/README.md) | Release target advances to [v0.31.0](https://github.com/vllm-project/vllm/releases/tag/v0.31.0); verified main remains pinned independently. | Update the release CI support version to 0.31.0. | | [`README.zh.md`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/README.zh.md) | Release target advances to [v0.31.0](https://github.com/vllm-project/vllm/releases/tag/v0.31.0); verified main remains pinned independently. | Update the Chinese release CI support version to 0.31.0. | | [`docs/source/tutorials/models/DeepSeek-V4.1-Flash.md`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/docs/source/tutorials/models/DeepSeek-V4.1-Flash.md) | Release target advances to [v0.31.0](https://github.com/vllm-project/vllm/releases/tag/v0.31.0); verified main remains pinned independently. | Use separate release and verified-main marker commands for the two build targets. | | [`vllm_ascend/ops/triton/docs/v2/spec_decode/prepare_dflash_inputs.md`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/ops/triton/docs/v2/spec_decode/prepare_dflash_inputs.md) | [vLLM #57632](https://github.com/vllm-project/vllm/pull/57632) passes `precompute_context_kv` into DFlash FULL-graph capture; wrapped draft models must expose the underlying callback. | Update the documented DFlash ABI version to 0.31.0; no new kernel behavior is introduced by this document change. | </details> <details> <summary>Model runners, graphs and platform integration (14 files)</summary> | Downstream file | Upstream change / contract | Ascend adaptation | | --- | --- | --- | | [`vllm_ascend/_310p/model_runner_310p.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/_310p/model_runner_310p.py) | [vLLM #58411](https://github.com/vllm-project/vllm/pull/58411) adds `randomize_inputs` for memory profiling to avoid all-zero MoE input concentration. Present in release; absent from the pinned-main parent calls. | Forward the profiling randomization flag through the 310P V1 override to the NPU V1 runner. | | [`vllm_ascend/_310p/worker/v2/model_runner.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/_310p/worker/v2/model_runner.py) | [vLLM #58411](https://github.com/vllm-project/vllm/pull/58411) adds `randomize_inputs` for memory profiling to avoid all-zero MoE input concentration. Present in release; absent from the pinned-main parent calls. [vLLM #58400](https://github.com/vllm-project/vllm/pull/58400) adds `prefill_runs_as_decode_np` and `decode_graph_eligible` for single-token prompt tails with existing context; `has_prefill` alone no longer determines FULL-decode eligibility. [vLLM #56456](https://github.com/vllm-project/vllm/pull/56456) adds trailing `num_active_loras` to `prepare_inputs`, keeping dispatch and fast-prefill graph/padding selection consistent. | Forward release-only randomization; accept `num_active_loras` in the common prepare signature, use the shared PCP/input-batch contract and compute complete prompt-tail/decode-graph metadata in CPU input preparation. | | [`vllm_ascend/_310p/worker/v2/sampler.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/_310p/worker/v2/sampler.py) | [vLLM #56497](https://github.com/vllm-project/vllm/pull/56497) introduces V2 `LogitsContext` and custom logits-processor lifecycle; `Sampler.add_request` no longer takes `prompt_len`. | Adopt `add_request(req_idx, sampling_params)` and remove the legacy prompt-length argument; retain the 310P CPU generator/sampling implementation. | | [`vllm_ascend/attention/attention_v1.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/attention/attention_v1.py) | [vLLM #58400](https://github.com/vllm-project/vllm/pull/58400) adds `prefill_runs_as_decode_np` and `decode_graph_eligible` for single-token prompt tails with existing context; `has_prefill` alone no longer determines FULL-decode eligibility. | Remove the 0.30 condition around V2 speculative-padding handling while retaining actual prefill/decode metadata semantics. | | [`vllm_ascend/patch/worker/patch_v2/patch_model_runner.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/patch/worker/patch_v2/patch_model_runner.py) | [vLLM #56723](https://github.com/vllm-project/vllm/pull/56723) adds `dcp_sharded` to distinguish sharded attention KV from replicated Mamba/GDN state when target DCP and DCP=1 draft coexist. [vLLM #45635](https://github.com/vllm-project/vllm/pull/45635) moves routed-expert output to `AuxOutputConfig`/the AuxOutput connector; [vLLM #57980](https://github.com/vllm-project/vllm/pull/57980) removes the legacy runner initialization keyword. | Read `dcp_sharded` from cache specs and forward it to block tables; initialize the AuxOutput connector from `aux_output_config.enabled`; drop the legacy `attn_cg_support` keyword. | | [`vllm_ascend/platform.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/platform.py) | [vLLM #45635](https://github.com/vllm-project/vllm/pull/45635) moves routed-expert output to `AuxOutputConfig`/the AuxOutput connector; [vLLM #57980](https://github.com/vllm-project/vllm/pull/57980) removes the legacy runner initialization keyword. | Apply the existing V1 AuxOutput configuration guard to both targets, removing the old 0.30 early return. | | [`vllm_ascend/worker/model_runner_v1.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/worker/model_runner_v1.py) | [vLLM #58411](https://github.com/vllm-project/vllm/pull/58411) adds `randomize_inputs` for memory profiling to avoid all-zero MoE input concentration. Present in release; absent from the pinned-main parent calls. | Accept `randomize_inputs` in profiling/dummy execution and enter `maybe_randomize_inputs` inside forward context when requested; preserve the Ascend execution path. | | [`vllm_ascend/worker/v2/aclgraph_utils.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/worker/v2/aclgraph_utils.py) | [vLLM #57980](https://github.com/vllm-project/vllm/pull/57980) replaces the legacy PCP helper with descriptor-based `partition_batch`, restores auxiliary hidden states with sampling outputs, and renames the DCP preparation helper. [vLLM #57632](https://github.com/vllm-project/vllm/pull/57632) passes `precompute_context_kv` into DFlash FULL-graph capture; wrapped draft models must expose the underlying callback. | Use `prepare_dcp_local_seq_lens`; delegate model-wrapper attribute lookup to the original model so DFlash context-KV callbacks remain accessible. | | [`vllm_ascend/worker/v2/input_batch.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/worker/v2/input_batch.py) | [vLLM #58275](https://github.com/vllm-project/vllm/pull/58275)/[vLLM #57980](https://github.com/vllm-project/vllm/pull/57980) expose `max_query_len`/`is_padding` on dummy batches and an explicit query-length bound for autoregressive graph capture. | Always retain prompt-tail and graph-eligibility fields and forward dummy-batch `max_query_len`/`is_padding`; remove obsolete 0.30 branches. | | [`vllm_ascend/worker/v2/model_runner.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/worker/v2/model_runner.py) | [vLLM #58411](https://github.com/vllm-project/vllm/pull/58411) adds `randomize_inputs` for memory profiling to avoid all-zero MoE input concentration. Present in release; absent from the pinned-main parent calls. [vLLM #58542](https://github.com/vllm-project/vllm/pull/58542) adds `max_seq_len_np` for PP sampled-token broadcast handling. Present in release batch state; absent from pinned main. [vLLM #58400](https://github.com/vllm-project/vllm/pull/58400) adds `prefill_runs_as_decode_np` and `decode_graph_eligible` for single-token prompt tails with existing context; `has_prefill` alone no longer determines FULL-decode eligibility. [vLLM #57980](https://github.com/vllm-project/vllm/pull/57980) replaces the legacy PCP helper with descriptor-based `partition_batch`, restores auxiliary hidden states with sampling outputs, and renames the DCP preparation helper. [vLLM #56456](https://github.com/vllm-project/vllm/pull/56456) adds trailing `num_active_loras` to `prepare_inputs`, keeping dispatch and fast-prefill graph/padding selection consistent. [vLLM #54335](https://github.com/vllm-project/vllm/pull/54335) routes fixed-token scoring through the same unpadded logits path as prompt logprobs, which is incompatible with Ascend lmhead-TP collective row layouts. | Use release-only kwargs for profiling/execute and MC2 dummy execution; forward release-only `max_seq_len_np`. After PD request reclassification, recompute `has_prefill`, decode-graph eligibility and uniform token count. Accept `num_active_loras` in the common prepare signature without adding FastPrefillHelper behavior. Use descriptor-based PCP partitioning and restore auxiliary states before replicated drafting. Extend the lmhead-TP guard to fixed-token scoring, tolerating workers without `token_id_scores`. | | [`vllm_ascend/worker/v2/model_states/mamba_hybrid.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/worker/v2/model_states/mamba_hybrid.py) | [vLLM #58400](https://github.com/vllm-project/vllm/pull/58400) requires decode-like prompt tails to use decode-row Mamba metadata for speculative rollback. | Build a local mask `is_prefilling_np & ~prefill_runs_as_decode_np` so prompt tails are decode rows; do not mutate the batch's original prefill mask. | | [`vllm_ascend/worker/v2/pcp_manager.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/worker/v2/pcp_manager.py) | [vLLM #57980](https://github.com/vllm-project/vllm/pull/57980) replaces the legacy PCP helper with descriptor-based `partition_batch`, restores auxiliary hidden states with sampling outputs, and renames the DCP preparation helper. | Unify descriptor-based partitioning, including CPP profile capture after #18622, and the `(batch, hidden, aux)` sampling restore contract; restore and trim auxiliary states together with hidden states to the unpadded global token extent. | | [`vllm_ascend/worker/v2/spec_decode/autoregressive/aclgraph.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/worker/v2/spec_decode/autoregressive/aclgraph.py) | [vLLM #58275](https://github.com/vllm-project/vllm/pull/58275)/[vLLM #57980](https://github.com/vllm-project/vllm/pull/57980) expose `max_query_len`/`is_padding` on dummy batches and an explicit query-length bound for autoregressive graph capture. | Always pass `desc.max_query_len or desc.uniform_token_count` to autoregressive capture. | | [`vllm_ascend/worker/v2/spec_decode/dflash/aclgraph.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/worker/v2/spec_decode/dflash/aclgraph.py) | [vLLM #57632](https://github.com/vllm-project/vllm/pull/57632) passes `precompute_context_kv` into DFlash FULL-graph capture; wrapped draft models must expose the underlying callback. | Always receive and forward `precompute_context_kv` under the Ascend communicator/model-capture wrappers. | </details> <details> <summary>KV allocation and block tables (6 files)</summary> | Downstream file | Upstream change / contract | Ascend adaptation | | --- | --- | --- | | [`vllm_ascend/_310p/worker/v2/block_table.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/_310p/worker/v2/block_table.py) | [vLLM #56723](https://github.com/vllm-project/vllm/pull/56723) adds `dcp_sharded` to distinguish sharded attention KV from replicated Mamba/GDN state when target DCP and DCP=1 draft coexist. [vLLM #56734](https://github.com/vllm-project/vllm/pull/56734) uses `idx_mapping=-1` for dummy draft/padding requests and masks their persistent block-table reads to avoid stale-row KV writes. | Unify the constructor with `dcp_sharded`; skip negative request indices and retain PAD slots in the CPU mapping path. Keep the existing CP=1 restriction and persistent CPU/device buffers. | | [`vllm_ascend/models/glm5next/kv_cache.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/models/glm5next/kv_cache.py) | [vLLM #59175](https://github.com/vllm-project/vllm/pull/59175) adds `prefill_end` to Mamba checkpoint allocation, including prompt-end retention. Release has the new signature; pinned main has the old one. | Accept `prefill_end` for call compatibility; keep one private Kpool tail block per request. | | [`vllm_ascend/ops/triton/v2/block_table/compute_slot_mappings.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/ops/triton/v2/block_table/compute_slot_mappings.py) | [vLLM #56734](https://github.com/vllm-project/vllm/pull/56734) uses `idx_mapping=-1` for dummy draft/padding requests and masks their persistent block-table reads to avoid stale-row KV writes. | Unify the slot-mapping kernel, mask block-table loads with `is_real_req`, and emit PAD for dummy rows; retain Ascend block/window and address-calculation behavior. | | [`vllm_ascend/patch/platform/patch_kv_cache_utils.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/patch/platform/patch_kv_cache_utils.py) | The pinned `_annotate_eagle_groups` contract uses `use_trailing_layer_fallback`, replacing the DeepSeek-V4-specific fallback keyword; the introducing PR was not independently identified. Fixed contracts: [release: kv_cache_utils.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/v1/core/kv_cache_utils.py#L2170). | Always pass `use_trailing_layer_fallback=True`; retain existing Ascend cache grouping/annotation behavior. | | [`vllm_ascend/patch/platform/patch_mamba_manager.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/patch/platform/patch_mamba_manager.py) | [vLLM #59175](https://github.com/vllm-project/vllm/pull/59175) adds `prefill_end` to Mamba checkpoint allocation, including prompt-end retention. Release has the new signature; pinned main has the old one. | Accept `prefill_end`, forwarding it only to the release parent while retaining the existing Ascend allocation logic. | | [`vllm_ascend/worker/v2/block_table.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/worker/v2/block_table.py) | [vLLM #56723](https://github.com/vllm-project/vllm/pull/56723) adds `dcp_sharded` to distinguish sharded attention KV from replicated Mamba/GDN state when target DCP and DCP=1 draft coexist. | Use one constructor with optional per-group `dcp_sharded` validation/forwarding; retain Ascend int32 slot mappings. | </details> <details> <summary>Schedulers and profiling (5 files)</summary> | Downstream file | Upstream change / contract | Ascend adaptation | | --- | --- | --- | | [`vllm_ascend/core/dyntra_lb_scheduler.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/core/dyntra_lb_scheduler.py) | [vLLM #59029](https://github.com/vllm-project/vllm/pull/59029) gives main an encoder-only exception to the non-chunked prefill budget check; release keeps the budget rejection condition. [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) replaces `skipped_waiting` with `kv_holding_waiting` and `deferred_waiting`, prioritizes KV holders, and uses `_handle_blocked_waiting_request`. [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) and [vLLM #58259](https://github.com/vllm-project/vllm/pull/58259) define the upstream resumable-session queue/handoff lifecycle: stopping a turn can re-enqueue buffered continuation input. Both targets admit against `max_num_active_reqs` and include parked streaming sessions in occupied slots; related queue/Elastic-EP changes are [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) and [vLLM #53934](https://github.com/vllm-project/vllm/pull/53934). Fixed contracts: [release: scheduler.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/v1/core/sched/scheduler.py#L127), [release: scheduler.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/v1/core/sched/scheduler.py#L892). | Use KV-holder-first queue selection/re-enqueueing, the common blocked-request helper, deferred bookkeeping and active-slot admission including parked streams. Preserve the main-only encoder exception. Remove secondary stop cleanup so upstream can retain/re-enqueue resumable continuations. | | [`vllm_ascend/core/recompute_scheduler.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/core/recompute_scheduler.py) | [vLLM #59029](https://github.com/vllm-project/vllm/pull/59029) gives main an encoder-only exception to the non-chunked prefill budget check; release keeps the budget rejection condition. [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) replaces `skipped_waiting` with `kv_holding_waiting` and `deferred_waiting`, prioritizes KV holders, and uses `_handle_blocked_waiting_request`. Both targets admit against `max_num_active_reqs` and include parked streaming sessions in occupied slots; related queue/Elastic-EP changes are [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) and [vLLM #53934](https://github.com/vllm-project/vllm/pull/53934). Fixed contracts: [release: scheduler.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/v1/core/sched/scheduler.py#L127), [release: scheduler.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/v1/core/sched/scheduler.py#L892). | Fold the legacy queues into the common KV-holder/deferred path, count parked streams for active admission, and preserve the version-dependent encoder condition. Remove the redundant remote-KV waiting debug log as a local cleanup. | | [`vllm_ascend/core/scheduler_profiling_chunk.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/core/scheduler_profiling_chunk.py) | [vLLM #59029](https://github.com/vllm-project/vllm/pull/59029) gives main an encoder-only exception to the non-chunked prefill budget check; release keeps the budget rejection condition. [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) replaces `skipped_waiting` with `kv_holding_waiting` and `deferred_waiting`, prioritizes KV holders, and uses `_handle_blocked_waiting_request`. Both targets admit against `max_num_active_reqs` and include parked streaming sessions in occupied slots; related queue/Elastic-EP changes are [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) and [vLLM #53934](https://github.com/vllm-project/vllm/pull/53934). Fixed contracts: [release: scheduler.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/v1/core/sched/scheduler.py#L127), [release: scheduler.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/v1/core/sched/scheduler.py#L892). | Use the common blocked-request/queue lifecycle, active admission including parked streams, and the version-dependent encoder budget condition. | | [`vllm_ascend/patch/platform/patch_balance_schedule.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/patch/platform/patch_balance_schedule.py) | [vLLM #59029](https://github.com/vllm-project/vllm/pull/59029) gives main an encoder-only exception to the non-chunked prefill budget check; release keeps the budget rejection condition. [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) replaces `skipped_waiting` with `kv_holding_waiting` and `deferred_waiting`, prioritizes KV holders, and uses `_handle_blocked_waiting_request`. | Fold six obsolete 0.30 queue branches and add the release/main encoder budget condition. Document the shared `if not request_queue` drained-queue guard. Retain baseline balancing, running-only DP snapshots/admission accounting and connector-snapshot cleanup; complete upstream schedule synchronization is outside this patch. | | [`vllm_ascend/profiling_config.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/profiling_config.py) | [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) replaces `skipped_waiting` with `kv_holding_waiting` and `deferred_waiting`, prioritizes KV holders, and uses `_handle_blocked_waiting_request`. | Retarget the profiling wrapper from the old blocked-status promotion helper to `_handle_blocked_waiting_request`. | </details> <details> <summary>EPLB state and routing (2 files)</summary> | Downstream file | Upstream change / contract | Ascend adaptation | | --- | --- | --- | | [`vllm_ascend/distributed/eplb/eplb_state.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/distributed/eplb/eplb_state.py) | [vLLM #53934](https://github.com/vllm-project/vllm/pull/53934) changes main's EPLB setup to update existing state and reuse its communicator. Release still creates state from `(model, model_config, mapping)`. | After the parent's `update_mapping`, refresh Ascend replica routing tables; consistently resolve an unspecified NPU device index to the current device. | | [`vllm_ascend/worker/v2/eplb.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/worker/v2/eplb.py) | [vLLM #53934](https://github.com/vllm-project/vllm/pull/53934) changes main's EPLB setup to update existing state and reuse its communicator. Release still creates state from `(model, model_config, mapping)`. | Create `AscendEplbState` on release with the current policy/optional expert arguments; call the main parent to update the existing state and communicator. | </details> <details> <summary>Models and MoE operators (6 files)</summary> | Downstream file | Upstream change / contract | Ascend adaptation | | --- | --- | --- | | [`vllm_ascend/models/kimi_k3.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/models/kimi_k3.py) | [vLLM #50592](https://github.com/vllm-project/vllm/pull/50592)/[vLLM #50593](https://github.com/vllm-project/vllm/pull/50593) use `prefix_delta` and the three-value AttnRes layer contract instead of the old prepared-input/deferred-MLP contract. | Remove old layer/model contracts, propagate `prefix_delta` and return the three-value AttnRes state; preserve Ascend fused operations, DSpark aux selection and SP/PP transport/output boundaries. | | [`vllm_ascend/ops/fused_moe/gate_linear.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/ops/fused_moe/gate_linear.py) | [vLLM #58234](https://github.com/vllm-project/vllm/pull/58234) unifies MoE gate layers under `GateLinear` with the current linear constructor and replaces Step3.5's dedicated FP32 gate class. | Use the current constructor parameters (`skip_bias_add`, `quant_config`, `return_bias`) and retain BF16-to-FP32 gate/fallback semantics. CRLF-to-LF normalization contributes to the large line diff. | | [`vllm_ascend/ops/fused_moe/router/grouped_topk_router.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/ops/fused_moe/router/grouped_topk_router.py) | [vLLM #58880](https://github.com/vllm-project/vllm/pull/58880) removes `routed_scaling_factor` from routing-method classification, allowing MiniMax classification with non-unit scaling. | Stop passing `routed_scaling_factor` to classification; downstream routing-weight scaling is retained. | | [`vllm_ascend/patch/__init__.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/patch/__init__.py) | The release target is 0.31.0; [vLLM #56994](https://github.com/vllm-project/vllm/pull/56994) is context for the retained GLM reasoning patch comment, not a new model capability in this PR. | Update only the supported-version comment for the retained GLM reasoning patch. | | [`vllm_ascend/patch/worker/patch_step3p5.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/patch/worker/patch_step3p5.py) | [vLLM #58234](https://github.com/vllm-project/vllm/pull/58234) unifies MoE gate layers under `GateLinear` with the current linear constructor and replaces Step3.5's dedicated FP32 gate class. | Always use `GateLinear` with FP32 parameter/output types instead of the legacy FP32ReplicatedLinear branch. | </details> <details> <summary>Token scoring and Triton helpers (2 files)</summary> | Downstream file | Upstream change / contract | Ascend adaptation | | --- | --- | --- | | [`vllm_ascend/patch/worker/patch_triton.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/patch/worker/patch_triton.py) | [vLLM #58651](https://github.com/vllm-project/vllm/pull/58651) uses `triton.next_power_of_2` in packed-QK RoPE constexpr evaluation; the Ascend replacement must also satisfy the installed Triton JIT/constexpr-function contract. | Use a constexpr-aware `next_power_of_2` replacement; register with `triton.constexpr_function` when available, otherwise unwrap `tl.constexpr.value` in the fallback. | | [`vllm_ascend/worker/v2/sample/logprob.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/worker/v2/sample/logprob.py) | [vLLM #54335](https://github.com/vllm-project/vllm/pull/54335) adds fixed-token prompt scoring and broadcasts candidate IDs with `expand`; the token-ID row stride can be zero or non-contiguous. | Pass `token_ids.stride(0)` into the Triton kernel and use it for row addressing; support broadcast stride zero and strided candidate IDs. | </details> BalanceScheduler scope: this PR folds the old queue branches and adapts encoder-only admission. Its baseline DP balancing, running-only snapshot/accounting and connector-snapshot cleanup remain; complete scheduler-body synchronization and a strict AST equivalence requirement are deferred. The baseline drift guard may skip and does not prove complete upstream equivalence. ### Does this PR introduce _any_ user-facing change? The default release dependency becomes vLLM v0.31.0; the verified-main pin remains unchanged. Fixed-token prompt scoring handles broadcast/strided IDs correctly and reports an explicit error when used with unsupported lmhead TP. Resumable Dyntra continuations retain upstream stop/queue behavior, and PD prompt-tail graph/Mamba metadata follow the supported upstream contracts. ### How was this patch tested? #### Test changes organized by file The table describes test intent and fixture changes; it does not claim that every changed test was selected or executed on NPU hardware. <details> <summary>Unit, PR E2E and nightly test mapping (37 files)</summary> | Downstream file | Upstream change / contract | Ascend adaptation | | --- | --- | --- | | [`tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_compute_token_logprobs.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_compute_token_logprobs.py) | [vLLM #54335](https://github.com/vllm-project/vllm/pull/54335) | Add NPU numerical comparisons for broadcast stride=0 and sliced stride=8 token IDs; outside the PR CI selector. | | [`tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_num_nans.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_num_nans.py) | Release-version cleanup; no new upstream behavior. | Update the default version label to 0.31.0; no algorithm change. | | [`tests/e2e/pull_request/four_card/test_kimi_k3.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/e2e/pull_request/four_card/test_kimi_k3.py) | [vLLM #54222](https://github.com/vllm-project/vllm/pull/54222) | Check equality between P-worker `remote_prefill_cached_tokens` and the response cache count; zero hits are valid for a new prompt. | | [`tests/e2e/pull_request/one_card/test_model_runner_v1_with_device.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/e2e/pull_request/one_card/test_model_runner_v1_with_device.py) | [vLLM #45635](https://github.com/vllm-project/vllm/pull/45635), [vLLM #57980](https://github.com/vllm-project/vllm/pull/57980) | Use the common AuxOutput configuration object in V1 guard fixtures. | | [`tests/e2e/pull_request/one_card/test_thinking_budget.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/e2e/pull_request/one_card/test_thinking_budget.py) | [vLLM #56497](https://github.com/vllm-project/vllm/pull/56497) | Construct `LogitsContext` with expanded mapping, input IDs, positions and sequence bounds; call `apply(logits, ctx)`. | | [`tests/ut/_310p/test_model_runner_v2_310p.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/_310p/test_model_runner_v2_310p.py) | [vLLM #56456](https://github.com/vllm-project/vllm/pull/56456), [vLLM #56497](https://github.com/vllm-project/vllm/pull/56497) | Use the current sampler request signature and pass the added prepare-input LoRA argument in 310P fixtures. | | [`tests/ut/attention/test_mla_cp.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/attention/test_mla_cp.py) | [vLLM #58149](https://github.com/vllm-project/vllm/pull/58149) makes `ModelState.max_model_len` a model-config-backed property; [vLLM #57913](https://github.com/vllm-project/vllm/pull/57913) moves multimodal capability to `ModelConfig`; [vLLM #56723](https://github.com/vllm-project/vllm/pull/56723) updates DCP state/metadata contracts. | Mock `get_dcp_world_size_and_rank` and remove legacy DCP group fixture branching. | | [`tests/ut/core/test_circular_buffer.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/core/test_circular_buffer.py) | Pinned cache allocation requires replay boundaries ([vLLM #58368](https://github.com/vllm-project/vllm/pull/58368)); DeepSeek-V4.1 architecture metadata and Engram configuration ([vLLM #56214](https://github.com/vllm-project/vllm/pull/56214)) use their current configuration contracts. Individual configuration migrations are linked to fixed source where not precisely attributed. Fixed contracts: [release: single_type_kv_cache_manager.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/v1/core/single_type_kv_cache_manager.py#L482), [release: model_arch.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/config/model_arch.py#L59), [release: engram.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/config/engram.py#L89). | Pass `replay_boundaries=[tokens-1]` to the current cache-block API. | | [`tests/ut/core/test_dyntra_lb_recompute_scheduler.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/core/test_dyntra_lb_recompute_scheduler.py) | [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) | Use the shared KV-holder/deferred queue interfaces in Dyntra/recompute fixtures. | | [`tests/ut/core/test_dyntra_lb_scheduler.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/core/test_dyntra_lb_scheduler.py) | [vLLM #58259](https://github.com/vllm-project/vllm/pull/58259), [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) | Regress synchronous/asynchronous resumable continuation handling for running, paused and preempted requests, buffered/unbuffered input, and final cleanup; update queue seams. | | [`tests/ut/core/test_profiling_chunk.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/core/test_profiling_chunk.py) | [vLLM #53934](https://github.com/vllm-project/vllm/pull/53934), [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) | Update queue fixtures/blocked-request helper and active-request capacity field. | | [`tests/ut/core/test_recompute_scheduler.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/core/test_recompute_scheduler.py) | [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) | Remove legacy queue branching and use the shared blocked-request seam. | | [`tests/ut/core/test_scheduler_connector_block_state.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/core/test_scheduler_connector_block_state.py) | [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947), [vLLM #59029](https://github.com/vllm-project/vllm/pull/59029) | Supply the baseline `balance_queue` fixture; connector-snapshot cleanup itself predates this change. | | [`tests/ut/distributed/eplb/test_state.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/distributed/eplb/test_state.py) | [vLLM #53934](https://github.com/vllm-project/vllm/pull/53934) | Verify mapping updates refresh Ascend routing tables. | | [`tests/ut/distributed/kv_transfer/kv_p2p/test_remote_prefill_lifecycle.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/distributed/kv_transfer/kv_p2p/test_remote_prefill_lifecycle.py) | [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) | Use the shared queue lifecycle rather than version-specific skipped-waiting fixtures. | | [`tests/ut/kv_offload/test_preempt_offload_connector.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/kv_offload/test_preempt_offload_connector.py) | Pinned cache allocation requires replay boundaries ([vLLM #58368](https://github.com/vllm-project/vllm/pull/58368)); DeepSeek-V4.1 architecture metadata and Engram configuration ([vLLM #56214](https://github.com/vllm-project/vllm/pull/56214)) use their current configuration contracts. Individual configuration migrations are linked to fixed source where not precisely attributed. Fixed contracts: [release: single_type_kv_cache_manager.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/v1/core/single_type_kv_cache_manager.py#L482), [release: model_arch.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/config/model_arch.py#L59), [release: engram.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/config/engram.py#L89). | Always provide prefix replay and request token-count fields in the offload fixture. | | [`tests/ut/models/test_deepseek_v41_release_config.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/models/test_deepseek_v41_release_config.py) | Pinned cache allocation requires replay boundaries ([vLLM #58368](https://github.com/vllm-project/vllm/pull/58368)); DeepSeek-V4.1 architecture metadata and Engram configuration ([vLLM #56214](https://github.com/vllm-project/vllm/pull/56214)) use their current configuration contracts. Individual configuration migrations are linked to fixed source where not precisely attributed. Fixed contracts: [release: single_type_kv_cache_manager.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/v1/core/single_type_kv_cache_manager.py#L482), [release: model_arch.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/config/model_arch.py#L59), [release: engram.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/config/engram.py#L89). | Assert obsolete flattened multimodal-prefix flags are absent after architecture conversion. | | [`tests/ut/models/test_glm5next_kv_cache.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/models/test_glm5next_kv_cache.py) | [vLLM #59175](https://github.com/vllm-project/vllm/pull/59175) | Pass the new allocation argument while checking the private-tail block policy. | | [`tests/ut/models/test_kimi_k3_adapter.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/models/test_kimi_k3_adapter.py) | [vLLM #50592](https://github.com/vllm-project/vllm/pull/50592), [vLLM #50593](https://github.com/vllm-project/vllm/pull/50593) | Update layer signatures/three-value returns, auxiliary output expectations and AST-extracted fixtures. | | [`tests/ut/models/test_kimi_k3_pp_sp.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/models/test_kimi_k3_pp_sp.py) | [vLLM #50592](https://github.com/vllm-project/vllm/pull/50592), [vLLM #50593](https://github.com/vllm-project/vllm/pull/50593) | Use the common Kimi AttnRes contract in PP/SP checks. | | [`tests/ut/patch/platform/test_patch_balance_schedule.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/patch/platform/test_patch_balance_schedule.py) | [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947), [vLLM #59029](https://github.com/vllm-project/vllm/pull/59029) | Use common queue fixtures; add eight encoder/chunked/prompt-budget cases and empty/fully deferred queue regressions with/without KV blocks. Correct the structural guard assertion to `if not request_queue`. Keep the baseline AST drift check, which can skip. | | [`tests/ut/patch/platform/test_patch_engram_config.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/patch/platform/test_patch_engram_config.py) | Pinned cache allocation requires replay boundaries ([vLLM #58368](https://github.com/vllm-project/vllm/pull/58368)); DeepSeek-V4.1 architecture metadata and Engram configuration ([vLLM #56214](https://github.com/vllm-project/vllm/pull/56214)) use their current configuration contracts. Individual configuration migrations are linked to fixed source where not precisely attributed. Fixed contracts: [release: single_type_kv_cache_manager.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/v1/core/single_type_kv_cache_manager.py#L482), [release: model_arch.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/config/model_arch.py#L59), [release: engram.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/config/engram.py#L89). | Expect shared-memory default resolution for this supported fixture configuration; no universal shared-memory default is claimed. | | [`tests/ut/patch/platform/test_prefix_cache_cp_patches.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/patch/platform/test_prefix_cache_cp_patches.py) | [vLLM #59175](https://github.com/vllm-project/vllm/pull/59175) | Supply an explicit `prefill_end` allocation boundary. | | [`tests/ut/spec_decode/test_eagle_proposer.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/spec_decode/test_eagle_proposer.py) | [vLLM #58149](https://github.com/vllm-project/vllm/pull/58149) makes `ModelState.max_model_len` a model-config-backed property; [vLLM #57913](https://github.com/vllm-project/vllm/pull/57913) moves multimodal capability to `ModelConfig`; [vLLM #56723](https://github.com/vllm-project/vllm/pull/56723) updates DCP state/metadata contracts. | Use the current model-config multimodal capability interface and remove obsolete skips/branches. | | [`tests/ut/spec_decode/test_mla_dspark.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/spec_decode/test_mla_dspark.py) | [vLLM #57632](https://github.com/vllm-project/vllm/pull/57632) | Assert DFlash context-KV callback forwarding and the shared capture signature. | | [`tests/ut/test_minimax_m3_kv_block_size.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/test_minimax_m3_kv_block_size.py) | Release-version cleanup; no new upstream behavior. | Remove only an unused version-helper import. | | [`tests/ut/test_platform.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/test_platform.py) | [vLLM #45635](https://github.com/vllm-project/vllm/pull/45635), [vLLM #57980](https://github.com/vllm-project/vllm/pull/57980) | Use the current AuxOutput configuration guard for both target versions. | | [`tests/ut/worker/test_attn_utils_v2.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/worker/test_attn_utils_v2.py) | [vLLM #58149](https://github.com/vllm-project/vllm/pull/58149) makes `ModelState.max_model_len` a model-config-backed property; [vLLM #57913](https://github.com/vllm-project/vllm/pull/57913) moves multimodal capability to `ModelConfig`; [vLLM #56723](https://github.com/vllm-project/vllm/pull/56723) updates DCP state/metadata contracts. | Initialize `model_state.model_config` instead of assigning the read-only length property. | | [`tests/ut/worker/test_dcp_speculator_metadata_v2.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/worker/test_dcp_speculator_metadata_v2.py) | [vLLM #58149](https://github.com/vllm-project/vllm/pull/58149) makes `ModelState.max_model_len` a model-config-backed property; [vLLM #57913](https://github.com/vllm-project/vllm/pull/57913) moves multimodal capability to `ModelConfig`; [vLLM #56723](https://github.com/vllm-project/vllm/pull/56723) updates DCP state/metadata contracts. | Assert prefill metadata over the padded request extent. | | [`tests/ut/worker/test_model_runner_v2.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/worker/test_model_runner_v2.py) | [vLLM #58411](https://github.com/vllm-project/vllm/pull/58411) adds `randomize_inputs` for memory profiling to avoid all-zero MoE input concentration. Present in release; absent from the pinned-main parent calls. [vLLM #58542](https://github.com/vllm-project/vllm/pull/58542) adds `max_seq_len_np` for PP sampled-token broadcast handling. Present in release batch state; absent from pinned main. [vLLM #58400](https://github.com/vllm-project/vllm/pull/58400) adds `prefill_runs_as_decode_np` and `decode_graph_eligible` for single-token prompt tails with existing context; `has_prefill` alone no longer determines FULL-decode eligibility. [vLLM #57980](https://github.com/vllm-project/vllm/pull/57980) replaces the legacy PCP helper with descriptor-based `partition_batch`, restores auxiliary hidden states with sampling outputs, and renames the DCP preparation helper. [vLLM #56456](https://github.com/vllm-project/vllm/pull/56456) adds trailing `num_active_loras` to `prepare_inputs`, keeping dispatch and fast-prefill graph/padding selection consistent. [vLLM #54335](https://github.com/vllm-project/vllm/pull/54335) routes fixed-token scoring through the same unpadded logits path as prompt logprobs, which is incompatible with Ascend lmhead-TP collective row layouts. [vLLM #58149](https://github.com/vllm-project/vllm/pull/58149) makes `ModelState.max_model_len` a model-config-backed property; [vLLM #57913](https://github.com/vllm-project/vllm/pull/57913) moves multimodal capability to `ModelConfig`; [vLLM #56723](https://github.com/vllm-project/vllm/pull/56723) updates DCP state/metadata contracts. | Cover release/main kwargs and PP fields, PD graph reclassification, PCP auxiliary restoration and lmhead scoring guards; update model-state fixtures. | | [`tests/ut/worker/test_model_runner_v2_encoder_decoder.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/worker/test_model_runner_v2_encoder_decoder.py) | [vLLM #58149](https://github.com/vllm-project/vllm/pull/58149) makes `ModelState.max_model_len` a model-config-backed property; [vLLM #57913](https://github.com/vllm-project/vllm/pull/57913) moves multimodal capability to `ModelConfig`; [vLLM #56723](https://github.com/vllm-project/vllm/pull/56723) updates DCP state/metadata contracts. | Update model-state length fixtures through model configuration. | | [`tests/ut/worker/test_model_runner_v2_mamba.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/worker/test_model_runner_v2_mamba.py) | [vLLM #58400](https://github.com/vllm-project/vllm/pull/58400) | Regress prompt-tail masks for FULL/NONE graph modes and zero/three draft tokens; verify the original mask is unchanged. | | [`tests/ut/worker/test_pcp_manager_v2.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/worker/test_pcp_manager_v2.py) | [vLLM #45635](https://github.com/vllm-project/vllm/pull/45635), [vLLM #57980](https://github.com/vllm-project/vllm/pull/57980) | Assert descriptor-based partitioning and hidden/aux restore layout; update AuxOutput fixture configuration. | | [`tests/ut/worker/test_v2_dummy_padding.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/worker/test_v2_dummy_padding.py) | [vLLM #58400](https://github.com/vllm-project/vllm/pull/58400) adds `prefill_runs_as_decode_np` and `decode_graph_eligible` for single-token prompt tails with existing context; `has_prefill` alone no longer determines FULL-decode eligibility. [vLLM #58275](https://github.com/vllm-project/vllm/pull/58275)/[vLLM #57980](https://github.com/vllm-project/vllm/pull/57980) expose `max_query_len`/`is_padding` on dummy batches and an explicit query-length bound for autoregressive graph capture. [vLLM #58149](https://github.com/vllm-project/vllm/pull/58149) makes `ModelState.max_model_len` a model-config-backed property; [vLLM #57913](https://github.com/vllm-project/vllm/pull/57913) moves multimodal capability to `ModelConfig`; [vLLM #56723](https://github.com/vllm-project/vllm/pull/56723) updates DCP state/metadata contracts. | Use the model-config-backed state/dummy descriptor contract and retain graph-eligibility metadata. | | [`tests/ut/worker/v2/test_aclgraph.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/worker/v2/test_aclgraph.py) | [vLLM #57980](https://github.com/vllm-project/vllm/pull/57980), [vLLM #58275](https://github.com/vllm-project/vllm/pull/58275) | Update capture descriptors and dummy-batch keyword expectations. | | [`tests/ut/worker/v2/test_eplb_controller.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/worker/v2/test_eplb_controller.py) | [vLLM #53934](https://github.com/vllm-project/vllm/pull/53934) | Cover release state construction versus main in-place state/communicator reuse. | | [`tests/ut/worker/v2/test_eplb_load_collection_phase.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/worker/v2/test_eplb_load_collection_phase.py) | [vLLM #53934](https://github.com/vllm-project/vllm/pull/53934) | Match each target's EPLB setup signature in load-collection fixtures. | </details> #### Local A3 nightly validation (2026-10-15; frozen head `444e962`) These four model runs used **`444e962b239c3a8fc56b9255d601818e6e028b9a` + vLLM 0.31.0 (`db9527a`)**, which is also the restored PR head; the deferred connector hardening is not included. Prefill 138 + Decode 136, 16 NPUs per node; original image, weights, datasets, launch and benchmark parameters were retained, with only the second-node address mapped from 139 to 136. All four cases exited pytest 0 on both nodes (one passed JUnit case per node). OTT is output token/s; TPOT/TTFT are ms. | Case | Performance requests | OTT / required minimum | TPOT / TTFT | AIME2025 accuracy | Result | | --- | --- | --- | --- | --- | --- | | Kimi-K2.6 PD | 60/60 | 355.1593 / 337.024075 | 20.5 / 3268.2 | 90%, within 93.33±10 | Performance + accuracy passed | | DeepSeek-V4 Flash PD-prefix | Accuracy only | N/A | N/A | 90%, within 90±10 | Accuracy passed | | GLM-5.1 W8A8C8 128K | 256/256 | 811.7645 / 282.105876 | 39.8 / 32665.6 | No accuracy item | Performance passed | | MiniMax-M2.7 PD-prefix | 160/160 | 711.0518 / 698.497 | 34.8 / 17438.2 | 90%, within 93.33±10 | Performance + accuracy passed | All performance requests above, and the valid controls below, were checked for actual 1024-token outputs, not just success flags. **Conditional controls:** historical throughput differences triggered only the affected performance cases against the same PR base `6d1dd70`, without this PR, + vLLM 0.30.0 (`ced6857`). Kimi: 355.1593 vs 359.8514 OTT (**-1.30%**); the earlier 7.27% loss was not reproduced in this comparison. TPOT was 20.5 vs 19.7ms (+0.8ms/+4.06%), and average request latency +3.48%; a single pair does not establish latency equivalence or a stable regression. GLM: 811.7645 vs 813.3506 OTT (**-0.20%**), TPOT 39.8 vs 39.4ms. Kimi's original random request pacing affects aggregate OTT. No alternating reruns or unaffected-model 0.30 controls were added. Archived evidence: `main2main-nightly-20261013/pr18229-update-444e962-20261015/` (`FINAL_REPORT.md`, per-node logs/JUnit, raw AISBench outputs, token-count audit, controls and diagnostic evidence). Both nodes were released after testing. #### Deferred follow-up: multi-group receive-failure reporting (not included) The request-level receive-failure hardening change has been removed from this upgrade PR; the branch is restored to the nightly-tested head `444e962`. The original four nightlies already passed on this head **without** that fix, so it is not required for these normal-path workloads to pass. The fault-path issue remains real: if a multi-group receive actually fails, the existing connector can report flat block IDs and trigger the vLLM 0.31 scheduler guard, terminating Decode EngineCore. This is handled separately in [#18823](https://github.com/vllm-project/vllm-ascend/pull/18823), retaining ordinary non-HMA block-level recovery semantics. The upgrade PR does not claim fault isolation is fixed. Development fault-injection results belong to the deferred connector implementation, not this restored head. No new NPU run was performed when restoring the exact previously tested code. #### Historical validation for head `23c8b15` - New [E2E workflow](https://github.com/vllm-project/vllm-ascend/actions/runs/38562748688) and [Docs link check](https://github.com/vllm-project/vllm-ascend/actions/runs/38562748395) were triggered after the rebase. Results are pending; previous-head CI does not validate this rebased head. - A targeted [comment command](https://github.com/vllm-project/vllm-ascend/pull/18229#issuecomment-6168149194) requests `test_deepseek_v4_mtp_full_decode_only` against both supported upstream targets. The [command workflow](https://github.com/vllm-project/vllm-ascend/actions/runs/38562831465) is queued; its actual test results remain pending. - Ruff check and format check passed for all 72 changed Python files; Python compilation and diff checks passed. Source-extracted execution of both real CPP routing regressions passed against the required partition signatures of the exact main/release refs. This validates dispatch contracts with lightweight dependency stubs and does not replace full pytest or NPU execution. - Existing source-extracted scheduler regressions passed on both version lanes: eight budget scenarios, two queue paths and empty/fully deferred queue regressions, including negative controls. - Local full pytest/Torch/NPU and `bash format.sh ci` remain unavailable in this Windows environment. They require CI validation. - Previous head `9330488` passed pre-commit, Docs and both CPU UT lanes (main 10782 passed / 114 skipped; release 10784 passed / 112 skipped). Its initial NPU matrix passed 41/42 shards; the remaining main A3 shard failed in unchanged upstream teardown with `psutil.NoSuchProcess`, before the MTP golden acceptance assertion. The consequent gate failure and old-head rerun are historical evidence, not a result for the current head. #### Coverage limits and historical evidence The PR CI selector scans `tests/ut` and `tests/e2e/pull_request`; the new broadcast/strided token-logprob numerical tests under `tests/e2e/nightly` are outside that selection. A5, standalone upstream-worker and coverage-upload jobs were skipped in the inspected run. Skips are not coverage. No dedicated packed-QK RoPE or V1 profiling-randomization standalone UT is added by this final diff. Previous head `15f8a3d` passed [E2E](https://github.com/vllm-project/vllm-ascend/actions/runs/38511018361) and [Docs](https://github.com/vllm-project/vllm-ascend/actions/runs/38511018193), including 42 NPU jobs. Its PR-recorded CPU counts were 10574 passed / 113 skipped for verified main and 10576 passed / 111 skipped for release, with no failures. Those counts belong to that historical head and are not asserted as current-head counts. - vLLM main: https://github.com/vllm-project/vllm/commit/c4df37dfcc7f72ca41cd43801a468526ecf99a2b --------- Signed-off-by: zhao-stack <80399320+zhao-stack@users.noreply.github.com> Co-authored-by: shenzhao <shenzhao9@huawei.com>

…tibility (vllm-project#18229) ### What this PR does / why we need it? Upgrade the release dependency from vLLM v0.30.0 to [v0.31.0](https://github.com/vllm-project/vllm/commit/db9527a46873454610df6dbedf79a36d6bf1a7f6) while keeping the independently supported verified-main target at [c4df37d](https://github.com/vllm-project/vllm/commit/c4df37dfcc7f72ca41cd43801a468526ecf99a2b). Rebase on latest Ascend main `c2382c6`, preserving the CPP profiling changes from #18622. The file mapping below describes the diff for `23c8b15` against `c2382c6`. The two targets have different APIs despite both being supported: retain version-dependent forwarding for profiling randomization, PP batch lengths, Mamba allocation, EPLB setup and encoder-only prefill admission. Remove obsolete 0.30 branches only where both targets now share the contract. | Contract | Verified main `c4df37d` | Release v0.31.0 `db9527a` | | --- | --- | --- | | Profiling | Parent calls omit `randomize_inputs` | Forward the flag through V1/V2/310P and MC2 | | PP batch state | No `max_seq_len_np` | Preserve the field in `AscendInputBatch` | | Mamba allocation | Parent has no `prefill_end` | Forward checkpoint boundary | | EPLB setup | Update existing state/communicator | Construct state from model and mapping | | Non-chunked prefill | Keep encoder-only budget exception | Keep release budget rejection | #### Changes organized by downstream file Upstream links identify the reason or contract being adapted. For existing main behavior that becomes common to both targets, the change is removal of the old release branch, not a new port of the complete upstream implementation. File links are pinned to this PR head; [the complete fixed diff](https://github.com/vllm-project/vllm-ascend/compare/c2382c67aa6be12f1c822733cd47d021f61b0b23...23c8b15c2709978463490b0e8531bf5f9f27b9c1) shows the exact edits. <details> <summary>Dependency markers, images and documentation (13 files)</summary> | Downstream file | Upstream change / contract | Ascend adaptation | | --- | --- | --- | | [`.github/vllm-release-tag.commit`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/.github/vllm-release-tag.commit) | Release target advances to [v0.31.0](https://github.com/vllm-project/vllm/releases/tag/v0.31.0); verified main remains pinned independently. | Advance the release marker to `v0.31.0`; leave the verified-main marker at `c4df37d`. | | [`Dockerfile`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/Dockerfile) | Release target advances to [v0.31.0](https://github.com/vllm-project/vllm/releases/tag/v0.31.0); verified main remains pinned independently. | Set the release `VLLM_TAG` default to `v0.31.0`; existing built images are not changed by this default. | | [`Dockerfile.310p`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/Dockerfile.310p) | Release target advances to [v0.31.0](https://github.com/vllm-project/vllm/releases/tag/v0.31.0); verified main remains pinned independently. | Set the release `VLLM_TAG` default to `v0.31.0`; existing built images are not changed by this default. | | [`Dockerfile.310p.openEuler`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/Dockerfile.310p.openEuler) | Release target advances to [v0.31.0](https://github.com/vllm-project/vllm/releases/tag/v0.31.0); verified main remains pinned independently. | Set the release `VLLM_TAG` default to `v0.31.0`; existing built images are not changed by this default. | | [`Dockerfile.a3`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/Dockerfile.a3) | Release target advances to [v0.31.0](https://github.com/vllm-project/vllm/releases/tag/v0.31.0); verified main remains pinned independently. | Set the release `VLLM_TAG` default to `v0.31.0`; existing built images are not changed by this default. | | [`Dockerfile.a3.openEuler`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/Dockerfile.a3.openEuler) | Release target advances to [v0.31.0](https://github.com/vllm-project/vllm/releases/tag/v0.31.0); verified main remains pinned independently. | Set the release `VLLM_TAG` default to `v0.31.0`; existing built images are not changed by this default. | | [`Dockerfile.a5`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/Dockerfile.a5) | Release target advances to [v0.31.0](https://github.com/vllm-project/vllm/releases/tag/v0.31.0); verified main remains pinned independently. | Set the release `VLLM_TAG` default to `v0.31.0`; existing built images are not changed by this default. | | [`Dockerfile.a5.openEuler`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/Dockerfile.a5.openEuler) | Release target advances to [v0.31.0](https://github.com/vllm-project/vllm/releases/tag/v0.31.0); verified main remains pinned independently. | Set the release `VLLM_TAG` default to `v0.31.0`; existing built images are not changed by this default. | | [`Dockerfile.openEuler`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/Dockerfile.openEuler) | Release target advances to [v0.31.0](https://github.com/vllm-project/vllm/releases/tag/v0.31.0); verified main remains pinned independently. | Set the release `VLLM_TAG` default to `v0.31.0`; existing built images are not changed by this default. | | [`README.md`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/README.md) | Release target advances to [v0.31.0](https://github.com/vllm-project/vllm/releases/tag/v0.31.0); verified main remains pinned independently. | Update the release CI support version to 0.31.0. | | [`README.zh.md`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/README.zh.md) | Release target advances to [v0.31.0](https://github.com/vllm-project/vllm/releases/tag/v0.31.0); verified main remains pinned independently. | Update the Chinese release CI support version to 0.31.0. | | [`docs/source/tutorials/models/DeepSeek-V4.1-Flash.md`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/docs/source/tutorials/models/DeepSeek-V4.1-Flash.md) | Release target advances to [v0.31.0](https://github.com/vllm-project/vllm/releases/tag/v0.31.0); verified main remains pinned independently. | Use separate release and verified-main marker commands for the two build targets. | | [`vllm_ascend/ops/triton/docs/v2/spec_decode/prepare_dflash_inputs.md`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/ops/triton/docs/v2/spec_decode/prepare_dflash_inputs.md) | [vLLM #57632](https://github.com/vllm-project/vllm/pull/57632) passes `precompute_context_kv` into DFlash FULL-graph capture; wrapped draft models must expose the underlying callback. | Update the documented DFlash ABI version to 0.31.0; no new kernel behavior is introduced by this document change. | </details> <details> <summary>Model runners, graphs and platform integration (14 files)</summary> | Downstream file | Upstream change / contract | Ascend adaptation | | --- | --- | --- | | [`vllm_ascend/_310p/model_runner_310p.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/_310p/model_runner_310p.py) | [vLLM #58411](https://github.com/vllm-project/vllm/pull/58411) adds `randomize_inputs` for memory profiling to avoid all-zero MoE input concentration. Present in release; absent from the pinned-main parent calls. | Forward the profiling randomization flag through the 310P V1 override to the NPU V1 runner. | | [`vllm_ascend/_310p/worker/v2/model_runner.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/_310p/worker/v2/model_runner.py) | [vLLM #58411](https://github.com/vllm-project/vllm/pull/58411) adds `randomize_inputs` for memory profiling to avoid all-zero MoE input concentration. Present in release; absent from the pinned-main parent calls. [vLLM #58400](https://github.com/vllm-project/vllm/pull/58400) adds `prefill_runs_as_decode_np` and `decode_graph_eligible` for single-token prompt tails with existing context; `has_prefill` alone no longer determines FULL-decode eligibility. [vLLM #56456](https://github.com/vllm-project/vllm/pull/56456) adds trailing `num_active_loras` to `prepare_inputs`, keeping dispatch and fast-prefill graph/padding selection consistent. | Forward release-only randomization; accept `num_active_loras` in the common prepare signature, use the shared PCP/input-batch contract and compute complete prompt-tail/decode-graph metadata in CPU input preparation. | | [`vllm_ascend/_310p/worker/v2/sampler.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/_310p/worker/v2/sampler.py) | [vLLM #56497](https://github.com/vllm-project/vllm/pull/56497) introduces V2 `LogitsContext` and custom logits-processor lifecycle; `Sampler.add_request` no longer takes `prompt_len`. | Adopt `add_request(req_idx, sampling_params)` and remove the legacy prompt-length argument; retain the 310P CPU generator/sampling implementation. | | [`vllm_ascend/attention/attention_v1.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/attention/attention_v1.py) | [vLLM #58400](https://github.com/vllm-project/vllm/pull/58400) adds `prefill_runs_as_decode_np` and `decode_graph_eligible` for single-token prompt tails with existing context; `has_prefill` alone no longer determines FULL-decode eligibility. | Remove the 0.30 condition around V2 speculative-padding handling while retaining actual prefill/decode metadata semantics. | | [`vllm_ascend/patch/worker/patch_v2/patch_model_runner.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/patch/worker/patch_v2/patch_model_runner.py) | [vLLM #56723](https://github.com/vllm-project/vllm/pull/56723) adds `dcp_sharded` to distinguish sharded attention KV from replicated Mamba/GDN state when target DCP and DCP=1 draft coexist. [vLLM #45635](https://github.com/vllm-project/vllm/pull/45635) moves routed-expert output to `AuxOutputConfig`/the AuxOutput connector; [vLLM #57980](https://github.com/vllm-project/vllm/pull/57980) removes the legacy runner initialization keyword. | Read `dcp_sharded` from cache specs and forward it to block tables; initialize the AuxOutput connector from `aux_output_config.enabled`; drop the legacy `attn_cg_support` keyword. | | [`vllm_ascend/platform.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/platform.py) | [vLLM #45635](https://github.com/vllm-project/vllm/pull/45635) moves routed-expert output to `AuxOutputConfig`/the AuxOutput connector; [vLLM #57980](https://github.com/vllm-project/vllm/pull/57980) removes the legacy runner initialization keyword. | Apply the existing V1 AuxOutput configuration guard to both targets, removing the old 0.30 early return. | | [`vllm_ascend/worker/model_runner_v1.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/worker/model_runner_v1.py) | [vLLM #58411](https://github.com/vllm-project/vllm/pull/58411) adds `randomize_inputs` for memory profiling to avoid all-zero MoE input concentration. Present in release; absent from the pinned-main parent calls. | Accept `randomize_inputs` in profiling/dummy execution and enter `maybe_randomize_inputs` inside forward context when requested; preserve the Ascend execution path. | | [`vllm_ascend/worker/v2/aclgraph_utils.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/worker/v2/aclgraph_utils.py) | [vLLM #57980](https://github.com/vllm-project/vllm/pull/57980) replaces the legacy PCP helper with descriptor-based `partition_batch`, restores auxiliary hidden states with sampling outputs, and renames the DCP preparation helper. [vLLM #57632](https://github.com/vllm-project/vllm/pull/57632) passes `precompute_context_kv` into DFlash FULL-graph capture; wrapped draft models must expose the underlying callback. | Use `prepare_dcp_local_seq_lens`; delegate model-wrapper attribute lookup to the original model so DFlash context-KV callbacks remain accessible. | | [`vllm_ascend/worker/v2/input_batch.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/worker/v2/input_batch.py) | [vLLM #58275](https://github.com/vllm-project/vllm/pull/58275)/[vLLM #57980](https://github.com/vllm-project/vllm/pull/57980) expose `max_query_len`/`is_padding` on dummy batches and an explicit query-length bound for autoregressive graph capture. | Always retain prompt-tail and graph-eligibility fields and forward dummy-batch `max_query_len`/`is_padding`; remove obsolete 0.30 branches. | | [`vllm_ascend/worker/v2/model_runner.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/worker/v2/model_runner.py) | [vLLM #58411](https://github.com/vllm-project/vllm/pull/58411) adds `randomize_inputs` for memory profiling to avoid all-zero MoE input concentration. Present in release; absent from the pinned-main parent calls. [vLLM #58542](https://github.com/vllm-project/vllm/pull/58542) adds `max_seq_len_np` for PP sampled-token broadcast handling. Present in release batch state; absent from pinned main. [vLLM #58400](https://github.com/vllm-project/vllm/pull/58400) adds `prefill_runs_as_decode_np` and `decode_graph_eligible` for single-token prompt tails with existing context; `has_prefill` alone no longer determines FULL-decode eligibility. [vLLM #57980](https://github.com/vllm-project/vllm/pull/57980) replaces the legacy PCP helper with descriptor-based `partition_batch`, restores auxiliary hidden states with sampling outputs, and renames the DCP preparation helper. [vLLM #56456](https://github.com/vllm-project/vllm/pull/56456) adds trailing `num_active_loras` to `prepare_inputs`, keeping dispatch and fast-prefill graph/padding selection consistent. [vLLM #54335](https://github.com/vllm-project/vllm/pull/54335) routes fixed-token scoring through the same unpadded logits path as prompt logprobs, which is incompatible with Ascend lmhead-TP collective row layouts. | Use release-only kwargs for profiling/execute and MC2 dummy execution; forward release-only `max_seq_len_np`. After PD request reclassification, recompute `has_prefill`, decode-graph eligibility and uniform token count. Accept `num_active_loras` in the common prepare signature without adding FastPrefillHelper behavior. Use descriptor-based PCP partitioning and restore auxiliary states before replicated drafting. Extend the lmhead-TP guard to fixed-token scoring, tolerating workers without `token_id_scores`. | | [`vllm_ascend/worker/v2/model_states/mamba_hybrid.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/worker/v2/model_states/mamba_hybrid.py) | [vLLM #58400](https://github.com/vllm-project/vllm/pull/58400) requires decode-like prompt tails to use decode-row Mamba metadata for speculative rollback. | Build a local mask `is_prefilling_np & ~prefill_runs_as_decode_np` so prompt tails are decode rows; do not mutate the batch's original prefill mask. | | [`vllm_ascend/worker/v2/pcp_manager.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/worker/v2/pcp_manager.py) | [vLLM #57980](https://github.com/vllm-project/vllm/pull/57980) replaces the legacy PCP helper with descriptor-based `partition_batch`, restores auxiliary hidden states with sampling outputs, and renames the DCP preparation helper. | Unify descriptor-based partitioning, including CPP profile capture after #18622, and the `(batch, hidden, aux)` sampling restore contract; restore and trim auxiliary states together with hidden states to the unpadded global token extent. | | [`vllm_ascend/worker/v2/spec_decode/autoregressive/aclgraph.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/worker/v2/spec_decode/autoregressive/aclgraph.py) | [vLLM #58275](https://github.com/vllm-project/vllm/pull/58275)/[vLLM #57980](https://github.com/vllm-project/vllm/pull/57980) expose `max_query_len`/`is_padding` on dummy batches and an explicit query-length bound for autoregressive graph capture. | Always pass `desc.max_query_len or desc.uniform_token_count` to autoregressive capture. | | [`vllm_ascend/worker/v2/spec_decode/dflash/aclgraph.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/worker/v2/spec_decode/dflash/aclgraph.py) | [vLLM #57632](https://github.com/vllm-project/vllm/pull/57632) passes `precompute_context_kv` into DFlash FULL-graph capture; wrapped draft models must expose the underlying callback. | Always receive and forward `precompute_context_kv` under the Ascend communicator/model-capture wrappers. | </details> <details> <summary>KV allocation and block tables (6 files)</summary> | Downstream file | Upstream change / contract | Ascend adaptation | | --- | --- | --- | | [`vllm_ascend/_310p/worker/v2/block_table.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/_310p/worker/v2/block_table.py) | [vLLM #56723](https://github.com/vllm-project/vllm/pull/56723) adds `dcp_sharded` to distinguish sharded attention KV from replicated Mamba/GDN state when target DCP and DCP=1 draft coexist. [vLLM #56734](https://github.com/vllm-project/vllm/pull/56734) uses `idx_mapping=-1` for dummy draft/padding requests and masks their persistent block-table reads to avoid stale-row KV writes. | Unify the constructor with `dcp_sharded`; skip negative request indices and retain PAD slots in the CPU mapping path. Keep the existing CP=1 restriction and persistent CPU/device buffers. | | [`vllm_ascend/models/glm5next/kv_cache.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/models/glm5next/kv_cache.py) | [vLLM #59175](https://github.com/vllm-project/vllm/pull/59175) adds `prefill_end` to Mamba checkpoint allocation, including prompt-end retention. Release has the new signature; pinned main has the old one. | Accept `prefill_end` for call compatibility; keep one private Kpool tail block per request. | | [`vllm_ascend/ops/triton/v2/block_table/compute_slot_mappings.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/ops/triton/v2/block_table/compute_slot_mappings.py) | [vLLM #56734](https://github.com/vllm-project/vllm/pull/56734) uses `idx_mapping=-1` for dummy draft/padding requests and masks their persistent block-table reads to avoid stale-row KV writes. | Unify the slot-mapping kernel, mask block-table loads with `is_real_req`, and emit PAD for dummy rows; retain Ascend block/window and address-calculation behavior. | | [`vllm_ascend/patch/platform/patch_kv_cache_utils.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/patch/platform/patch_kv_cache_utils.py) | The pinned `_annotate_eagle_groups` contract uses `use_trailing_layer_fallback`, replacing the DeepSeek-V4-specific fallback keyword; the introducing PR was not independently identified. Fixed contracts: [release: kv_cache_utils.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/v1/core/kv_cache_utils.py#L2170). | Always pass `use_trailing_layer_fallback=True`; retain existing Ascend cache grouping/annotation behavior. | | [`vllm_ascend/patch/platform/patch_mamba_manager.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/patch/platform/patch_mamba_manager.py) | [vLLM #59175](https://github.com/vllm-project/vllm/pull/59175) adds `prefill_end` to Mamba checkpoint allocation, including prompt-end retention. Release has the new signature; pinned main has the old one. | Accept `prefill_end`, forwarding it only to the release parent while retaining the existing Ascend allocation logic. | | [`vllm_ascend/worker/v2/block_table.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/worker/v2/block_table.py) | [vLLM #56723](https://github.com/vllm-project/vllm/pull/56723) adds `dcp_sharded` to distinguish sharded attention KV from replicated Mamba/GDN state when target DCP and DCP=1 draft coexist. | Use one constructor with optional per-group `dcp_sharded` validation/forwarding; retain Ascend int32 slot mappings. | </details> <details> <summary>Schedulers and profiling (5 files)</summary> | Downstream file | Upstream change / contract | Ascend adaptation | | --- | --- | --- | | [`vllm_ascend/core/dyntra_lb_scheduler.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/core/dyntra_lb_scheduler.py) | [vLLM #59029](https://github.com/vllm-project/vllm/pull/59029) gives main an encoder-only exception to the non-chunked prefill budget check; release keeps the budget rejection condition. [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) replaces `skipped_waiting` with `kv_holding_waiting` and `deferred_waiting`, prioritizes KV holders, and uses `_handle_blocked_waiting_request`. [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) and [vLLM #58259](https://github.com/vllm-project/vllm/pull/58259) define the upstream resumable-session queue/handoff lifecycle: stopping a turn can re-enqueue buffered continuation input. Both targets admit against `max_num_active_reqs` and include parked streaming sessions in occupied slots; related queue/Elastic-EP changes are [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) and [vLLM #53934](https://github.com/vllm-project/vllm/pull/53934). Fixed contracts: [release: scheduler.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/v1/core/sched/scheduler.py#L127), [release: scheduler.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/v1/core/sched/scheduler.py#L892). | Use KV-holder-first queue selection/re-enqueueing, the common blocked-request helper, deferred bookkeeping and active-slot admission including parked streams. Preserve the main-only encoder exception. Remove secondary stop cleanup so upstream can retain/re-enqueue resumable continuations. | | [`vllm_ascend/core/recompute_scheduler.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/core/recompute_scheduler.py) | [vLLM #59029](https://github.com/vllm-project/vllm/pull/59029) gives main an encoder-only exception to the non-chunked prefill budget check; release keeps the budget rejection condition. [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) replaces `skipped_waiting` with `kv_holding_waiting` and `deferred_waiting`, prioritizes KV holders, and uses `_handle_blocked_waiting_request`. Both targets admit against `max_num_active_reqs` and include parked streaming sessions in occupied slots; related queue/Elastic-EP changes are [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) and [vLLM #53934](https://github.com/vllm-project/vllm/pull/53934). Fixed contracts: [release: scheduler.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/v1/core/sched/scheduler.py#L127), [release: scheduler.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/v1/core/sched/scheduler.py#L892). | Fold the legacy queues into the common KV-holder/deferred path, count parked streams for active admission, and preserve the version-dependent encoder condition. Remove the redundant remote-KV waiting debug log as a local cleanup. | | [`vllm_ascend/core/scheduler_profiling_chunk.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/core/scheduler_profiling_chunk.py) | [vLLM #59029](https://github.com/vllm-project/vllm/pull/59029) gives main an encoder-only exception to the non-chunked prefill budget check; release keeps the budget rejection condition. [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) replaces `skipped_waiting` with `kv_holding_waiting` and `deferred_waiting`, prioritizes KV holders, and uses `_handle_blocked_waiting_request`. Both targets admit against `max_num_active_reqs` and include parked streaming sessions in occupied slots; related queue/Elastic-EP changes are [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) and [vLLM #53934](https://github.com/vllm-project/vllm/pull/53934). Fixed contracts: [release: scheduler.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/v1/core/sched/scheduler.py#L127), [release: scheduler.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/v1/core/sched/scheduler.py#L892). | Use the common blocked-request/queue lifecycle, active admission including parked streams, and the version-dependent encoder budget condition. | | [`vllm_ascend/patch/platform/patch_balance_schedule.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/patch/platform/patch_balance_schedule.py) | [vLLM #59029](https://github.com/vllm-project/vllm/pull/59029) gives main an encoder-only exception to the non-chunked prefill budget check; release keeps the budget rejection condition. [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) replaces `skipped_waiting` with `kv_holding_waiting` and `deferred_waiting`, prioritizes KV holders, and uses `_handle_blocked_waiting_request`. | Fold six obsolete 0.30 queue branches and add the release/main encoder budget condition. Document the shared `if not request_queue` drained-queue guard. Retain baseline balancing, running-only DP snapshots/admission accounting and connector-snapshot cleanup; complete upstream schedule synchronization is outside this patch. | | [`vllm_ascend/profiling_config.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/profiling_config.py) | [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) replaces `skipped_waiting` with `kv_holding_waiting` and `deferred_waiting`, prioritizes KV holders, and uses `_handle_blocked_waiting_request`. | Retarget the profiling wrapper from the old blocked-status promotion helper to `_handle_blocked_waiting_request`. | </details> <details> <summary>EPLB state and routing (2 files)</summary> | Downstream file | Upstream change / contract | Ascend adaptation | | --- | --- | --- | | [`vllm_ascend/distributed/eplb/eplb_state.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/distributed/eplb/eplb_state.py) | [vLLM #53934](https://github.com/vllm-project/vllm/pull/53934) changes main's EPLB setup to update existing state and reuse its communicator. Release still creates state from `(model, model_config, mapping)`. | After the parent's `update_mapping`, refresh Ascend replica routing tables; consistently resolve an unspecified NPU device index to the current device. | | [`vllm_ascend/worker/v2/eplb.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/worker/v2/eplb.py) | [vLLM #53934](https://github.com/vllm-project/vllm/pull/53934) changes main's EPLB setup to update existing state and reuse its communicator. Release still creates state from `(model, model_config, mapping)`. | Create `AscendEplbState` on release with the current policy/optional expert arguments; call the main parent to update the existing state and communicator. | </details> <details> <summary>Models and MoE operators (6 files)</summary> | Downstream file | Upstream change / contract | Ascend adaptation | | --- | --- | --- | | [`vllm_ascend/models/kimi_k3.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/models/kimi_k3.py) | [vLLM #50592](https://github.com/vllm-project/vllm/pull/50592)/[vLLM #50593](https://github.com/vllm-project/vllm/pull/50593) use `prefix_delta` and the three-value AttnRes layer contract instead of the old prepared-input/deferred-MLP contract. | Remove old layer/model contracts, propagate `prefix_delta` and return the three-value AttnRes state; preserve Ascend fused operations, DSpark aux selection and SP/PP transport/output boundaries. | | [`vllm_ascend/ops/fused_moe/gate_linear.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/ops/fused_moe/gate_linear.py) | [vLLM #58234](https://github.com/vllm-project/vllm/pull/58234) unifies MoE gate layers under `GateLinear` with the current linear constructor and replaces Step3.5's dedicated FP32 gate class. | Use the current constructor parameters (`skip_bias_add`, `quant_config`, `return_bias`) and retain BF16-to-FP32 gate/fallback semantics. CRLF-to-LF normalization contributes to the large line diff. | | [`vllm_ascend/ops/fused_moe/router/grouped_topk_router.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/ops/fused_moe/router/grouped_topk_router.py) | [vLLM #58880](https://github.com/vllm-project/vllm/pull/58880) removes `routed_scaling_factor` from routing-method classification, allowing MiniMax classification with non-unit scaling. | Stop passing `routed_scaling_factor` to classification; downstream routing-weight scaling is retained. | | [`vllm_ascend/patch/__init__.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/patch/__init__.py) | The release target is 0.31.0; [vLLM #56994](https://github.com/vllm-project/vllm/pull/56994) is context for the retained GLM reasoning patch comment, not a new model capability in this PR. | Update only the supported-version comment for the retained GLM reasoning patch. | | [`vllm_ascend/patch/worker/patch_step3p5.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/patch/worker/patch_step3p5.py) | [vLLM #58234](https://github.com/vllm-project/vllm/pull/58234) unifies MoE gate layers under `GateLinear` with the current linear constructor and replaces Step3.5's dedicated FP32 gate class. | Always use `GateLinear` with FP32 parameter/output types instead of the legacy FP32ReplicatedLinear branch. | </details> <details> <summary>Token scoring and Triton helpers (2 files)</summary> | Downstream file | Upstream change / contract | Ascend adaptation | | --- | --- | --- | | [`vllm_ascend/patch/worker/patch_triton.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/patch/worker/patch_triton.py) | [vLLM #58651](https://github.com/vllm-project/vllm/pull/58651) uses `triton.next_power_of_2` in packed-QK RoPE constexpr evaluation; the Ascend replacement must also satisfy the installed Triton JIT/constexpr-function contract. | Use a constexpr-aware `next_power_of_2` replacement; register with `triton.constexpr_function` when available, otherwise unwrap `tl.constexpr.value` in the fallback. | | [`vllm_ascend/worker/v2/sample/logprob.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/worker/v2/sample/logprob.py) | [vLLM #54335](https://github.com/vllm-project/vllm/pull/54335) adds fixed-token prompt scoring and broadcasts candidate IDs with `expand`; the token-ID row stride can be zero or non-contiguous. | Pass `token_ids.stride(0)` into the Triton kernel and use it for row addressing; support broadcast stride zero and strided candidate IDs. | </details> BalanceScheduler scope: this PR folds the old queue branches and adapts encoder-only admission. Its baseline DP balancing, running-only snapshot/accounting and connector-snapshot cleanup remain; complete scheduler-body synchronization and a strict AST equivalence requirement are deferred. The baseline drift guard may skip and does not prove complete upstream equivalence. ### Does this PR introduce _any_ user-facing change? The default release dependency becomes vLLM v0.31.0; the verified-main pin remains unchanged. Fixed-token prompt scoring handles broadcast/strided IDs correctly and reports an explicit error when used with unsupported lmhead TP. Resumable Dyntra continuations retain upstream stop/queue behavior, and PD prompt-tail graph/Mamba metadata follow the supported upstream contracts. ### How was this patch tested? #### Test changes organized by file The table describes test intent and fixture changes; it does not claim that every changed test was selected or executed on NPU hardware. <details> <summary>Unit, PR E2E and nightly test mapping (37 files)</summary> | Downstream file | Upstream change / contract | Ascend adaptation | | --- | --- | --- | | [`tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_compute_token_logprobs.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_compute_token_logprobs.py) | [vLLM #54335](https://github.com/vllm-project/vllm/pull/54335) | Add NPU numerical comparisons for broadcast stride=0 and sliced stride=8 token IDs; outside the PR CI selector. | | [`tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_num_nans.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_num_nans.py) | Release-version cleanup; no new upstream behavior. | Update the default version label to 0.31.0; no algorithm change. | | [`tests/e2e/pull_request/four_card/test_kimi_k3.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/e2e/pull_request/four_card/test_kimi_k3.py) | [vLLM #54222](https://github.com/vllm-project/vllm/pull/54222) | Check equality between P-worker `remote_prefill_cached_tokens` and the response cache count; zero hits are valid for a new prompt. | | [`tests/e2e/pull_request/one_card/test_model_runner_v1_with_device.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/e2e/pull_request/one_card/test_model_runner_v1_with_device.py) | [vLLM #45635](https://github.com/vllm-project/vllm/pull/45635), [vLLM #57980](https://github.com/vllm-project/vllm/pull/57980) | Use the common AuxOutput configuration object in V1 guard fixtures. | | [`tests/e2e/pull_request/one_card/test_thinking_budget.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/e2e/pull_request/one_card/test_thinking_budget.py) | [vLLM #56497](https://github.com/vllm-project/vllm/pull/56497) | Construct `LogitsContext` with expanded mapping, input IDs, positions and sequence bounds; call `apply(logits, ctx)`. | | [`tests/ut/_310p/test_model_runner_v2_310p.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/_310p/test_model_runner_v2_310p.py) | [vLLM #56456](https://github.com/vllm-project/vllm/pull/56456), [vLLM #56497](https://github.com/vllm-project/vllm/pull/56497) | Use the current sampler request signature and pass the added prepare-input LoRA argument in 310P fixtures. | | [`tests/ut/attention/test_mla_cp.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/attention/test_mla_cp.py) | [vLLM #58149](https://github.com/vllm-project/vllm/pull/58149) makes `ModelState.max_model_len` a model-config-backed property; [vLLM #57913](https://github.com/vllm-project/vllm/pull/57913) moves multimodal capability to `ModelConfig`; [vLLM #56723](https://github.com/vllm-project/vllm/pull/56723) updates DCP state/metadata contracts. | Mock `get_dcp_world_size_and_rank` and remove legacy DCP group fixture branching. | | [`tests/ut/core/test_circular_buffer.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/core/test_circular_buffer.py) | Pinned cache allocation requires replay boundaries ([vLLM #58368](https://github.com/vllm-project/vllm/pull/58368)); DeepSeek-V4.1 architecture metadata and Engram configuration ([vLLM #56214](https://github.com/vllm-project/vllm/pull/56214)) use their current configuration contracts. Individual configuration migrations are linked to fixed source where not precisely attributed. Fixed contracts: [release: single_type_kv_cache_manager.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/v1/core/single_type_kv_cache_manager.py#L482), [release: model_arch.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/config/model_arch.py#L59), [release: engram.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/config/engram.py#L89). | Pass `replay_boundaries=[tokens-1]` to the current cache-block API. | | [`tests/ut/core/test_dyntra_lb_recompute_scheduler.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/core/test_dyntra_lb_recompute_scheduler.py) | [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) | Use the shared KV-holder/deferred queue interfaces in Dyntra/recompute fixtures. | | [`tests/ut/core/test_dyntra_lb_scheduler.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/core/test_dyntra_lb_scheduler.py) | [vLLM #58259](https://github.com/vllm-project/vllm/pull/58259), [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) | Regress synchronous/asynchronous resumable continuation handling for running, paused and preempted requests, buffered/unbuffered input, and final cleanup; update queue seams. | | [`tests/ut/core/test_profiling_chunk.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/core/test_profiling_chunk.py) | [vLLM #53934](https://github.com/vllm-project/vllm/pull/53934), [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) | Update queue fixtures/blocked-request helper and active-request capacity field. | | [`tests/ut/core/test_recompute_scheduler.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/core/test_recompute_scheduler.py) | [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) | Remove legacy queue branching and use the shared blocked-request seam. | | [`tests/ut/core/test_scheduler_connector_block_state.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/core/test_scheduler_connector_block_state.py) | [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947), [vLLM #59029](https://github.com/vllm-project/vllm/pull/59029) | Supply the baseline `balance_queue` fixture; connector-snapshot cleanup itself predates this change. | | [`tests/ut/distributed/eplb/test_state.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/distributed/eplb/test_state.py) | [vLLM #53934](https://github.com/vllm-project/vllm/pull/53934) | Verify mapping updates refresh Ascend routing tables. | | [`tests/ut/distributed/kv_transfer/kv_p2p/test_remote_prefill_lifecycle.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/distributed/kv_transfer/kv_p2p/test_remote_prefill_lifecycle.py) | [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) | Use the shared queue lifecycle rather than version-specific skipped-waiting fixtures. | | [`tests/ut/kv_offload/test_preempt_offload_connector.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/kv_offload/test_preempt_offload_connector.py) | Pinned cache allocation requires replay boundaries ([vLLM #58368](https://github.com/vllm-project/vllm/pull/58368)); DeepSeek-V4.1 architecture metadata and Engram configuration ([vLLM #56214](https://github.com/vllm-project/vllm/pull/56214)) use their current configuration contracts. Individual configuration migrations are linked to fixed source where not precisely attributed. Fixed contracts: [release: single_type_kv_cache_manager.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/v1/core/single_type_kv_cache_manager.py#L482), [release: model_arch.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/config/model_arch.py#L59), [release: engram.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/config/engram.py#L89). | Always provide prefix replay and request token-count fields in the offload fixture. | | [`tests/ut/models/test_deepseek_v41_release_config.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/models/test_deepseek_v41_release_config.py) | Pinned cache allocation requires replay boundaries ([vLLM #58368](https://github.com/vllm-project/vllm/pull/58368)); DeepSeek-V4.1 architecture metadata and Engram configuration ([vLLM #56214](https://github.com/vllm-project/vllm/pull/56214)) use their current configuration contracts. Individual configuration migrations are linked to fixed source where not precisely attributed. Fixed contracts: [release: single_type_kv_cache_manager.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/v1/core/single_type_kv_cache_manager.py#L482), [release: model_arch.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/config/model_arch.py#L59), [release: engram.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/config/engram.py#L89). | Assert obsolete flattened multimodal-prefix flags are absent after architecture conversion. | | [`tests/ut/models/test_glm5next_kv_cache.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/models/test_glm5next_kv_cache.py) | [vLLM #59175](https://github.com/vllm-project/vllm/pull/59175) | Pass the new allocation argument while checking the private-tail block policy. | | [`tests/ut/models/test_kimi_k3_adapter.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/models/test_kimi_k3_adapter.py) | [vLLM #50592](https://github.com/vllm-project/vllm/pull/50592), [vLLM #50593](https://github.com/vllm-project/vllm/pull/50593) | Update layer signatures/three-value returns, auxiliary output expectations and AST-extracted fixtures. | | [`tests/ut/models/test_kimi_k3_pp_sp.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/models/test_kimi_k3_pp_sp.py) | [vLLM #50592](https://github.com/vllm-project/vllm/pull/50592), [vLLM #50593](https://github.com/vllm-project/vllm/pull/50593) | Use the common Kimi AttnRes contract in PP/SP checks. | | [`tests/ut/patch/platform/test_patch_balance_schedule.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/patch/platform/test_patch_balance_schedule.py) | [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947), [vLLM #59029](https://github.com/vllm-project/vllm/pull/59029) | Use common queue fixtures; add eight encoder/chunked/prompt-budget cases and empty/fully deferred queue regressions with/without KV blocks. Correct the structural guard assertion to `if not request_queue`. Keep the baseline AST drift check, which can skip. | | [`tests/ut/patch/platform/test_patch_engram_config.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/patch/platform/test_patch_engram_config.py) | Pinned cache allocation requires replay boundaries ([vLLM #58368](https://github.com/vllm-project/vllm/pull/58368)); DeepSeek-V4.1 architecture metadata and Engram configuration ([vLLM #56214](https://github.com/vllm-project/vllm/pull/56214)) use their current configuration contracts. Individual configuration migrations are linked to fixed source where not precisely attributed. Fixed contracts: [release: single_type_kv_cache_manager.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/v1/core/single_type_kv_cache_manager.py#L482), [release: model_arch.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/config/model_arch.py#L59), [release: engram.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/config/engram.py#L89). | Expect shared-memory default resolution for this supported fixture configuration; no universal shared-memory default is claimed. | | [`tests/ut/patch/platform/test_prefix_cache_cp_patches.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/patch/platform/test_prefix_cache_cp_patches.py) | [vLLM #59175](https://github.com/vllm-project/vllm/pull/59175) | Supply an explicit `prefill_end` allocation boundary. | | [`tests/ut/spec_decode/test_eagle_proposer.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/spec_decode/test_eagle_proposer.py) | [vLLM #58149](https://github.com/vllm-project/vllm/pull/58149) makes `ModelState.max_model_len` a model-config-backed property; [vLLM #57913](https://github.com/vllm-project/vllm/pull/57913) moves multimodal capability to `ModelConfig`; [vLLM #56723](https://github.com/vllm-project/vllm/pull/56723) updates DCP state/metadata contracts. | Use the current model-config multimodal capability interface and remove obsolete skips/branches. | | [`tests/ut/spec_decode/test_mla_dspark.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/spec_decode/test_mla_dspark.py) | [vLLM #57632](https://github.com/vllm-project/vllm/pull/57632) | Assert DFlash context-KV callback forwarding and the shared capture signature. | | [`tests/ut/test_minimax_m3_kv_block_size.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/test_minimax_m3_kv_block_size.py) | Release-version cleanup; no new upstream behavior. | Remove only an unused version-helper import. | | [`tests/ut/test_platform.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/test_platform.py) | [vLLM #45635](https://github.com/vllm-project/vllm/pull/45635), [vLLM #57980](https://github.com/vllm-project/vllm/pull/57980) | Use the current AuxOutput configuration guard for both target versions. | | [`tests/ut/worker/test_attn_utils_v2.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/worker/test_attn_utils_v2.py) | [vLLM #58149](https://github.com/vllm-project/vllm/pull/58149) makes `ModelState.max_model_len` a model-config-backed property; [vLLM #57913](https://github.com/vllm-project/vllm/pull/57913) moves multimodal capability to `ModelConfig`; [vLLM #56723](https://github.com/vllm-project/vllm/pull/56723) updates DCP state/metadata contracts. | Initialize `model_state.model_config` instead of assigning the read-only length property. | | [`tests/ut/worker/test_dcp_speculator_metadata_v2.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/worker/test_dcp_speculator_metadata_v2.py) | [vLLM #58149](https://github.com/vllm-project/vllm/pull/58149) makes `ModelState.max_model_len` a model-config-backed property; [vLLM #57913](https://github.com/vllm-project/vllm/pull/57913) moves multimodal capability to `ModelConfig`; [vLLM #56723](https://github.com/vllm-project/vllm/pull/56723) updates DCP state/metadata contracts. | Assert prefill metadata over the padded request extent. | | [`tests/ut/worker/test_model_runner_v2.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/worker/test_model_runner_v2.py) | [vLLM #58411](https://github.com/vllm-project/vllm/pull/58411) adds `randomize_inputs` for memory profiling to avoid all-zero MoE input concentration. Present in release; absent from the pinned-main parent calls. [vLLM #58542](https://github.com/vllm-project/vllm/pull/58542) adds `max_seq_len_np` for PP sampled-token broadcast handling. Present in release batch state; absent from pinned main. [vLLM #58400](https://github.com/vllm-project/vllm/pull/58400) adds `prefill_runs_as_decode_np` and `decode_graph_eligible` for single-token prompt tails with existing context; `has_prefill` alone no longer determines FULL-decode eligibility. [vLLM #57980](https://github.com/vllm-project/vllm/pull/57980) replaces the legacy PCP helper with descriptor-based `partition_batch`, restores auxiliary hidden states with sampling outputs, and renames the DCP preparation helper. [vLLM #56456](https://github.com/vllm-project/vllm/pull/56456) adds trailing `num_active_loras` to `prepare_inputs`, keeping dispatch and fast-prefill graph/padding selection consistent. [vLLM #54335](https://github.com/vllm-project/vllm/pull/54335) routes fixed-token scoring through the same unpadded logits path as prompt logprobs, which is incompatible with Ascend lmhead-TP collective row layouts. [vLLM #58149](https://github.com/vllm-project/vllm/pull/58149) makes `ModelState.max_model_len` a model-config-backed property; [vLLM #57913](https://github.com/vllm-project/vllm/pull/57913) moves multimodal capability to `ModelConfig`; [vLLM #56723](https://github.com/vllm-project/vllm/pull/56723) updates DCP state/metadata contracts. | Cover release/main kwargs and PP fields, PD graph reclassification, PCP auxiliary restoration and lmhead scoring guards; update model-state fixtures. | | [`tests/ut/worker/test_model_runner_v2_encoder_decoder.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/worker/test_model_runner_v2_encoder_decoder.py) | [vLLM #58149](https://github.com/vllm-project/vllm/pull/58149) makes `ModelState.max_model_len` a model-config-backed property; [vLLM #57913](https://github.com/vllm-project/vllm/pull/57913) moves multimodal capability to `ModelConfig`; [vLLM #56723](https://github.com/vllm-project/vllm/pull/56723) updates DCP state/metadata contracts. | Update model-state length fixtures through model configuration. | | [`tests/ut/worker/test_model_runner_v2_mamba.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/worker/test_model_runner_v2_mamba.py) | [vLLM #58400](https://github.com/vllm-project/vllm/pull/58400) | Regress prompt-tail masks for FULL/NONE graph modes and zero/three draft tokens; verify the original mask is unchanged. | | [`tests/ut/worker/test_pcp_manager_v2.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/worker/test_pcp_manager_v2.py) | [vLLM #45635](https://github.com/vllm-project/vllm/pull/45635), [vLLM #57980](https://github.com/vllm-project/vllm/pull/57980) | Assert descriptor-based partitioning and hidden/aux restore layout; update AuxOutput fixture configuration. | | [`tests/ut/worker/test_v2_dummy_padding.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/worker/test_v2_dummy_padding.py) | [vLLM #58400](https://github.com/vllm-project/vllm/pull/58400) adds `prefill_runs_as_decode_np` and `decode_graph_eligible` for single-token prompt tails with existing context; `has_prefill` alone no longer determines FULL-decode eligibility. [vLLM #58275](https://github.com/vllm-project/vllm/pull/58275)/[vLLM #57980](https://github.com/vllm-project/vllm/pull/57980) expose `max_query_len`/`is_padding` on dummy batches and an explicit query-length bound for autoregressive graph capture. [vLLM #58149](https://github.com/vllm-project/vllm/pull/58149) makes `ModelState.max_model_len` a model-config-backed property; [vLLM #57913](https://github.com/vllm-project/vllm/pull/57913) moves multimodal capability to `ModelConfig`; [vLLM #56723](https://github.com/vllm-project/vllm/pull/56723) updates DCP state/metadata contracts. | Use the model-config-backed state/dummy descriptor contract and retain graph-eligibility metadata. | | [`tests/ut/worker/v2/test_aclgraph.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/worker/v2/test_aclgraph.py) | [vLLM #57980](https://github.com/vllm-project/vllm/pull/57980), [vLLM #58275](https://github.com/vllm-project/vllm/pull/58275) | Update capture descriptors and dummy-batch keyword expectations. | | [`tests/ut/worker/v2/test_eplb_controller.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/worker/v2/test_eplb_controller.py) | [vLLM #53934](https://github.com/vllm-project/vllm/pull/53934) | Cover release state construction versus main in-place state/communicator reuse. | | [`tests/ut/worker/v2/test_eplb_load_collection_phase.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/worker/v2/test_eplb_load_collection_phase.py) | [vLLM #53934](https://github.com/vllm-project/vllm/pull/53934) | Match each target's EPLB setup signature in load-collection fixtures. | </details> #### Local A3 nightly validation (2026-10-15; frozen head `444e962`) These four model runs used **`444e962b239c3a8fc56b9255d601818e6e028b9a` + vLLM 0.31.0 (`db9527a`)**, which is also the restored PR head; the deferred connector hardening is not included. Prefill 138 + Decode 136, 16 NPUs per node; original image, weights, datasets, launch and benchmark parameters were retained, with only the second-node address mapped from 139 to 136. All four cases exited pytest 0 on both nodes (one passed JUnit case per node). OTT is output token/s; TPOT/TTFT are ms. | Case | Performance requests | OTT / required minimum | TPOT / TTFT | AIME2025 accuracy | Result | | --- | --- | --- | --- | --- | --- | | Kimi-K2.6 PD | 60/60 | 355.1593 / 337.024075 | 20.5 / 3268.2 | 90%, within 93.33±10 | Performance + accuracy passed | | DeepSeek-V4 Flash PD-prefix | Accuracy only | N/A | N/A | 90%, within 90±10 | Accuracy passed | | GLM-5.1 W8A8C8 128K | 256/256 | 811.7645 / 282.105876 | 39.8 / 32665.6 | No accuracy item | Performance passed | | MiniMax-M2.7 PD-prefix | 160/160 | 711.0518 / 698.497 | 34.8 / 17438.2 | 90%, within 93.33±10 | Performance + accuracy passed | All performance requests above, and the valid controls below, were checked for actual 1024-token outputs, not just success flags. **Conditional controls:** historical throughput differences triggered only the affected performance cases against the same PR base `6d1dd70`, without this PR, + vLLM 0.30.0 (`ced6857`). Kimi: 355.1593 vs 359.8514 OTT (**-1.30%**); the earlier 7.27% loss was not reproduced in this comparison. TPOT was 20.5 vs 19.7ms (+0.8ms/+4.06%), and average request latency +3.48%; a single pair does not establish latency equivalence or a stable regression. GLM: 811.7645 vs 813.3506 OTT (**-0.20%**), TPOT 39.8 vs 39.4ms. Kimi's original random request pacing affects aggregate OTT. No alternating reruns or unaffected-model 0.30 controls were added. Archived evidence: `main2main-nightly-20261013/pr18229-update-444e962-20261015/` (`FINAL_REPORT.md`, per-node logs/JUnit, raw AISBench outputs, token-count audit, controls and diagnostic evidence). Both nodes were released after testing. #### Deferred follow-up: multi-group receive-failure reporting (not included) The request-level receive-failure hardening change has been removed from this upgrade PR; the branch is restored to the nightly-tested head `444e962`. The original four nightlies already passed on this head **without** that fix, so it is not required for these normal-path workloads to pass. The fault-path issue remains real: if a multi-group receive actually fails, the existing connector can report flat block IDs and trigger the vLLM 0.31 scheduler guard, terminating Decode EngineCore. This is handled separately in [#18823](https://github.com/vllm-project/vllm-ascend/pull/18823), retaining ordinary non-HMA block-level recovery semantics. The upgrade PR does not claim fault isolation is fixed. Development fault-injection results belong to the deferred connector implementation, not this restored head. No new NPU run was performed when restoring the exact previously tested code. #### Historical validation for head `23c8b15` - New [E2E workflow](https://github.com/vllm-project/vllm-ascend/actions/runs/38562748688) and [Docs link check](https://github.com/vllm-project/vllm-ascend/actions/runs/38562748395) were triggered after the rebase. Results are pending; previous-head CI does not validate this rebased head. - A targeted [comment command](https://github.com/vllm-project/vllm-ascend/pull/18229#issuecomment-6168149194) requests `test_deepseek_v4_mtp_full_decode_only` against both supported upstream targets. The [command workflow](https://github.com/vllm-project/vllm-ascend/actions/runs/38562831465) is queued; its actual test results remain pending. - Ruff check and format check passed for all 72 changed Python files; Python compilation and diff checks passed. Source-extracted execution of both real CPP routing regressions passed against the required partition signatures of the exact main/release refs. This validates dispatch contracts with lightweight dependency stubs and does not replace full pytest or NPU execution. - Existing source-extracted scheduler regressions passed on both version lanes: eight budget scenarios, two queue paths and empty/fully deferred queue regressions, including negative controls. - Local full pytest/Torch/NPU and `bash format.sh ci` remain unavailable in this Windows environment. They require CI validation. - Previous head `9330488` passed pre-commit, Docs and both CPU UT lanes (main 10782 passed / 114 skipped; release 10784 passed / 112 skipped). Its initial NPU matrix passed 41/42 shards; the remaining main A3 shard failed in unchanged upstream teardown with `psutil.NoSuchProcess`, before the MTP golden acceptance assertion. The consequent gate failure and old-head rerun are historical evidence, not a result for the current head. #### Coverage limits and historical evidence The PR CI selector scans `tests/ut` and `tests/e2e/pull_request`; the new broadcast/strided token-logprob numerical tests under `tests/e2e/nightly` are outside that selection. A5, standalone upstream-worker and coverage-upload jobs were skipped in the inspected run. Skips are not coverage. No dedicated packed-QK RoPE or V1 profiling-randomization standalone UT is added by this final diff. Previous head `15f8a3d` passed [E2E](https://github.com/vllm-project/vllm-ascend/actions/runs/38511018361) and [Docs](https://github.com/vllm-project/vllm-ascend/actions/runs/38511018193), including 42 NPU jobs. Its PR-recorded CPU counts were 10574 passed / 113 skipped for verified main and 10576 passed / 111 skipped for release, with no failures. Those counts belong to that historical head and are not asserted as current-head counts. - vLLM main: https://github.com/vllm-project/vllm/commit/c4df37dfcc7f72ca41cd43801a468526ecf99a2b --------- Signed-off-by: zhao-stack <80399320+zhao-stack@users.noreply.github.com> Co-authored-by: shenzhao <shenzhao9@huawei.com>

…m-project#58400) Signed-off-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>

### What this PR does / why we need it? Bump the verified vLLM main pin from `8cc9aa5` to `c4df37df` and adapt vllm-ascend to the upstream changes in that range. The v0.30.0 release lane is kept intact: each adaptation is gated on `vllm_version_is("0.30.0")` (or a version-aware helper) so both the `main` and the `v0.30.0` CPU UT lanes stay green. The range contains three breaking upstream changes plus one release-lane fix: - vllm#58947 reworked the scheduler waiting queues. - vllm#58400 extended the MRV2 `InputBatch`/`BatchReqState` contract. - vllm#53934 turned `CudaGraphManager.dp_size` into a property and changed `EPLBController.maybe_register_model`. - The Ascend Qwen3.5-VL NPU-preprocessing feature imported a main-only fused input norm, breaking the release lane. | Files | Upstream vLLM change (why it breaks vllm-ascend) | vllm-ascend adaptation | |-------|--------------------------------------------------|------------------------| | `.github/vllm-main-verified.commit` | — (main-lane pin only) | Verified main commit bumped `8cc9aa5` -> `c4df37df` | | `tests/ut/core/test_circular_buffer.py` | v0.29.0 lane removed | Drop the now-obsolete `vllm_version_is("0.29.0")` guard and always pass `replay_boundaries` | | `tests/ut/core/test_dyntra_lb_recompute_scheduler.py` | [vllm#58947](vllm-project/vllm#58947) replaced the single `skipped_waiting` queue with `kv_holding_waiting` + `deferred_waiting` | Import and use the `_is_skipped()` helper instead of `request in scheduler.skipped_waiting` | | `tests/ut/core/test_dyntra_lb_scheduler.py` | Same [vllm#58947](vllm-project/vllm#58947) waiting-queue rework | Add `_mark_skipped` / `_is_skipped` / `_summary_waiting_kwargs` helpers and a version-aware promote hook (`_try_promote_blocked_waiting_request` vs `_handle_blocked_waiting_request`) | | `tests/ut/core/test_profiling_chunk.py` | Same [vllm#58947](vllm-project/vllm#58947); renamed blocked helper and admission cap | Version-aware `_handle_blocked_waiting_request`/`max_num_active_reqs` (main) vs `_try_promote_blocked_waiting_request`/`max_num_running_reqs` (v0.30.0) | | `tests/ut/core/test_recompute_scheduler.py` | Same [vllm#58947](vllm-project/vllm#58947) | Use `_is_skipped` instead of `request in scheduler.skipped_waiting` | | `tests/ut/distributed/kv_transfer/kv_p2p/test_remote_prefill_lifecycle.py` | Same [vllm#58947](vllm-project/vllm#58947) replaced `skipped_waiting` | `_skipped_waiting_queue()` resolver: `skipped_waiting` (v0.30.0) / `kv_holding_waiting` (main) | | `tests/ut/models/test_qwen3_5_vl_preprocess.py` | Qwen3.5-VL fused input norm module split (v0.30.0 `FusedInputNorm` in `models.vision`; main `build_mm_input_norm` in `layers.fusion`) | Import the model unconditionally and drop the `skipif`, now that the builder is dual-version | | `tests/ut/patch/platform/test_patch_balance_schedule.py` | Same [vllm#58947](vllm-project/vllm#58947) | Version-aware scheduler method-seam guard (`_handle_blocked_waiting_request`/`_holds_kv_blocks` on main, `_select_waiting_queue_for_scheduling`/`_try_promote_blocked_waiting_request` on v0.30.0) | | `tests/ut/worker/test_model_runner_v2.py` | [vllm#58400](vllm-project/vllm#58400) added `num_draft_tokens_np`/`prefill_runs_as_decode_np`/`decode_graph_eligible` to `BatchReqState` | Supply the new fields (version-gated) in the `_make_batch_state` / `_prepare_inputs_runner` helpers | | `tests/ut/worker/test_v2_dummy_padding.py` | [vllm#58400](vllm-project/vllm#58400) renamed the 4th arg of `get_uniform_decode_token_count` | Pass `batch.has_prefill` (v0.30.0) / `batch.decode_graph_eligible` (main) | | `tests/ut/worker/v2/test_aclgraph.py` | [vllm#53934](vllm-project/vllm#53934) made `CudaGraphManager.dp_size` a read-only property backed by `vllm_config`, so `manager.dp_size = 2` raised `AttributeError` | Set `manager.vllm_config.parallel_config.data_parallel_size` on main; keep the instance attribute on v0.30.0 | | `tests/ut/worker/v2/test_eplb_controller.py` | [vllm#53934](vllm-project/vllm#53934) removed `load_dummy_weights` from `EPLBController.maybe_register_model`, so the 3-arg call raised `TypeError` | Call `maybe_register_model(model, model_config)` on main; keep the 3-arg form on v0.30.0 | | `vllm_ascend/_310p/worker/v2/model_runner.py` | [vllm#58400](vllm-project/vllm#58400) added a required `decode_graph_eligible` to `InputBatch`; the 310P path crashed with `TypeError: Ascend310PInputBatch.__init__() missing 1 required positional argument: 'decode_graph_eligible'` | Pass `decode_graph_eligible = not batch_has_prefill` into the 310P input batch on main | | `vllm_ascend/core/dyntra_lb_scheduler.py` | [vllm#58947](vllm-project/vllm#58947) replaced the single `skipped_waiting` queue with `kv_holding_waiting` (queue) + `deferred_waiting` (set), removed `_select_waiting_queue_for_scheduling`, renamed `_try_promote_blocked_waiting_request` -> `_handle_blocked_waiting_request`, and renamed the admission cap `max_num_running_reqs` -> `max_num_active_reqs`; the v0.30.0 copy crashed with `AttributeError: ... has no attribute 'skipped_waiting' / '_select_waiting_queue_for_scheduling'` | Version-split the waiting bookkeeping: `_skipped_waiting_requests()` / `_prepare_blocked_waiting_request()` helpers, a single versioned `_skip_request()` skip site, async-KV-load skip-before-pop, version-aware re-queue and `prefill_capacity_bound`, and `print_scheduler_summary` chains the right queues per lane | | `vllm_ascend/core/recompute_scheduler.py` | Same [vllm#58947](vllm-project/vllm#58947) waiting-queue rework | Same version-split (`_skip_request`, `kv_holding_waiting`+`deferred_waiting` on main, `skipped_waiting` on v0.30.0; version-aware `max_num_active_reqs` / `prefill_capacity_bound`) | | `vllm_ascend/core/scheduler_profiling_chunk.py` | Same [vllm#58947](vllm-project/vllm#58947) rework; also drops the now-unsupported v0.29.0 lane | Rebase the copied `schedule()` onto v0.30.0/main, version-split the waiting section and re-queue, remove 8 obsolete `vllm_version_is("0.29.0")` branches | | `vllm_ascend/models/qwen3_5_vl.py` | v0.30.0 keeps the fused input norm as `FusedInputNorm` in `vllm.model_executor.models.vision`; main exposes `build_mm_input_norm` in `vllm.model_executor.layers.fusion.mm_input_norm`. The unconditional main-only import broke release-lane loading | Route the builder through `FusedInputNorm.from_model_config` on v0.30.0, keep `build_mm_input_norm` on main | | `vllm_ascend/patch/platform/patch_balance_schedule.py` | Same [vllm#58947](vllm-project/vllm#58947) rework | Same version-split, keeping the three balance deltas intact | | `vllm_ascend/worker/v2/model_runner.py` | [vllm#58400](vllm-project/vllm#58400) added `decode_graph_eligible`/`prefill_runs_as_decode_np` to `BatchReqState`/`InputBatch` and renamed the 4th arg of `get_uniform_decode_token_count` from `has_prefill` to `decode_graph_eligible` | Forward the new fields into `AscendInputBatch` on main; version-gated `_pd_uniform_decode_token_count()` passing `has_prefill` (v0.30.0) vs `not has_prefill` (main) | ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? - vLLM main: vllm-project/vllm@8cc9aa5 --------- Signed-off-by: hfadzxy <starmoon_zhang@163.com> Signed-off-by: zhangxinyuehfad <starmoon_zhang@163.com>

…tibility (vllm-project#18229) ### What this PR does / why we need it? Upgrade the release dependency from vLLM v0.30.0 to [v0.31.0](https://github.com/vllm-project/vllm/commit/db9527a46873454610df6dbedf79a36d6bf1a7f6) while keeping the independently supported verified-main target at [c4df37d](https://github.com/vllm-project/vllm/commit/c4df37dfcc7f72ca41cd43801a468526ecf99a2b). Rebase on latest Ascend main `c2382c6`, preserving the CPP profiling changes from #18622. The file mapping below describes the diff for `23c8b15` against `c2382c6`. The two targets have different APIs despite both being supported: retain version-dependent forwarding for profiling randomization, PP batch lengths, Mamba allocation, EPLB setup and encoder-only prefill admission. Remove obsolete 0.30 branches only where both targets now share the contract. | Contract | Verified main `c4df37d` | Release v0.31.0 `db9527a` | | --- | --- | --- | | Profiling | Parent calls omit `randomize_inputs` | Forward the flag through V1/V2/310P and MC2 | | PP batch state | No `max_seq_len_np` | Preserve the field in `AscendInputBatch` | | Mamba allocation | Parent has no `prefill_end` | Forward checkpoint boundary | | EPLB setup | Update existing state/communicator | Construct state from model and mapping | | Non-chunked prefill | Keep encoder-only budget exception | Keep release budget rejection | #### Changes organized by downstream file Upstream links identify the reason or contract being adapted. For existing main behavior that becomes common to both targets, the change is removal of the old release branch, not a new port of the complete upstream implementation. File links are pinned to this PR head; [the complete fixed diff](https://github.com/vllm-project/vllm-ascend/compare/c2382c67aa6be12f1c822733cd47d021f61b0b23...23c8b15c2709978463490b0e8531bf5f9f27b9c1) shows the exact edits. <details> <summary>Dependency markers, images and documentation (13 files)</summary> | Downstream file | Upstream change / contract | Ascend adaptation | | --- | --- | --- | | [`.github/vllm-release-tag.commit`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/.github/vllm-release-tag.commit) | Release target advances to [v0.31.0](https://github.com/vllm-project/vllm/releases/tag/v0.31.0); verified main remains pinned independently. | Advance the release marker to `v0.31.0`; leave the verified-main marker at `c4df37d`. | | [`Dockerfile`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/Dockerfile) | Release target advances to [v0.31.0](https://github.com/vllm-project/vllm/releases/tag/v0.31.0); verified main remains pinned independently. | Set the release `VLLM_TAG` default to `v0.31.0`; existing built images are not changed by this default. | | [`Dockerfile.310p`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/Dockerfile.310p) | Release target advances to [v0.31.0](https://github.com/vllm-project/vllm/releases/tag/v0.31.0); verified main remains pinned independently. | Set the release `VLLM_TAG` default to `v0.31.0`; existing built images are not changed by this default. | | [`Dockerfile.310p.openEuler`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/Dockerfile.310p.openEuler) | Release target advances to [v0.31.0](https://github.com/vllm-project/vllm/releases/tag/v0.31.0); verified main remains pinned independently. | Set the release `VLLM_TAG` default to `v0.31.0`; existing built images are not changed by this default. | | [`Dockerfile.a3`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/Dockerfile.a3) | Release target advances to [v0.31.0](https://github.com/vllm-project/vllm/releases/tag/v0.31.0); verified main remains pinned independently. | Set the release `VLLM_TAG` default to `v0.31.0`; existing built images are not changed by this default. | | [`Dockerfile.a3.openEuler`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/Dockerfile.a3.openEuler) | Release target advances to [v0.31.0](https://github.com/vllm-project/vllm/releases/tag/v0.31.0); verified main remains pinned independently. | Set the release `VLLM_TAG` default to `v0.31.0`; existing built images are not changed by this default. | | [`Dockerfile.a5`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/Dockerfile.a5) | Release target advances to [v0.31.0](https://github.com/vllm-project/vllm/releases/tag/v0.31.0); verified main remains pinned independently. | Set the release `VLLM_TAG` default to `v0.31.0`; existing built images are not changed by this default. | | [`Dockerfile.a5.openEuler`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/Dockerfile.a5.openEuler) | Release target advances to [v0.31.0](https://github.com/vllm-project/vllm/releases/tag/v0.31.0); verified main remains pinned independently. | Set the release `VLLM_TAG` default to `v0.31.0`; existing built images are not changed by this default. | | [`Dockerfile.openEuler`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/Dockerfile.openEuler) | Release target advances to [v0.31.0](https://github.com/vllm-project/vllm/releases/tag/v0.31.0); verified main remains pinned independently. | Set the release `VLLM_TAG` default to `v0.31.0`; existing built images are not changed by this default. | | [`README.md`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/README.md) | Release target advances to [v0.31.0](https://github.com/vllm-project/vllm/releases/tag/v0.31.0); verified main remains pinned independently. | Update the release CI support version to 0.31.0. | | [`README.zh.md`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/README.zh.md) | Release target advances to [v0.31.0](https://github.com/vllm-project/vllm/releases/tag/v0.31.0); verified main remains pinned independently. | Update the Chinese release CI support version to 0.31.0. | | [`docs/source/tutorials/models/DeepSeek-V4.1-Flash.md`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/docs/source/tutorials/models/DeepSeek-V4.1-Flash.md) | Release target advances to [v0.31.0](https://github.com/vllm-project/vllm/releases/tag/v0.31.0); verified main remains pinned independently. | Use separate release and verified-main marker commands for the two build targets. | | [`vllm_ascend/ops/triton/docs/v2/spec_decode/prepare_dflash_inputs.md`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/ops/triton/docs/v2/spec_decode/prepare_dflash_inputs.md) | [vLLM #57632](https://github.com/vllm-project/vllm/pull/57632) passes `precompute_context_kv` into DFlash FULL-graph capture; wrapped draft models must expose the underlying callback. | Update the documented DFlash ABI version to 0.31.0; no new kernel behavior is introduced by this document change. | </details> <details> <summary>Model runners, graphs and platform integration (14 files)</summary> | Downstream file | Upstream change / contract | Ascend adaptation | | --- | --- | --- | | [`vllm_ascend/_310p/model_runner_310p.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/_310p/model_runner_310p.py) | [vLLM #58411](https://github.com/vllm-project/vllm/pull/58411) adds `randomize_inputs` for memory profiling to avoid all-zero MoE input concentration. Present in release; absent from the pinned-main parent calls. | Forward the profiling randomization flag through the 310P V1 override to the NPU V1 runner. | | [`vllm_ascend/_310p/worker/v2/model_runner.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/_310p/worker/v2/model_runner.py) | [vLLM #58411](https://github.com/vllm-project/vllm/pull/58411) adds `randomize_inputs` for memory profiling to avoid all-zero MoE input concentration. Present in release; absent from the pinned-main parent calls. [vLLM #58400](https://github.com/vllm-project/vllm/pull/58400) adds `prefill_runs_as_decode_np` and `decode_graph_eligible` for single-token prompt tails with existing context; `has_prefill` alone no longer determines FULL-decode eligibility. [vLLM #56456](https://github.com/vllm-project/vllm/pull/56456) adds trailing `num_active_loras` to `prepare_inputs`, keeping dispatch and fast-prefill graph/padding selection consistent. | Forward release-only randomization; accept `num_active_loras` in the common prepare signature, use the shared PCP/input-batch contract and compute complete prompt-tail/decode-graph metadata in CPU input preparation. | | [`vllm_ascend/_310p/worker/v2/sampler.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/_310p/worker/v2/sampler.py) | [vLLM #56497](https://github.com/vllm-project/vllm/pull/56497) introduces V2 `LogitsContext` and custom logits-processor lifecycle; `Sampler.add_request` no longer takes `prompt_len`. | Adopt `add_request(req_idx, sampling_params)` and remove the legacy prompt-length argument; retain the 310P CPU generator/sampling implementation. | | [`vllm_ascend/attention/attention_v1.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/attention/attention_v1.py) | [vLLM #58400](https://github.com/vllm-project/vllm/pull/58400) adds `prefill_runs_as_decode_np` and `decode_graph_eligible` for single-token prompt tails with existing context; `has_prefill` alone no longer determines FULL-decode eligibility. | Remove the 0.30 condition around V2 speculative-padding handling while retaining actual prefill/decode metadata semantics. | | [`vllm_ascend/patch/worker/patch_v2/patch_model_runner.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/patch/worker/patch_v2/patch_model_runner.py) | [vLLM #56723](https://github.com/vllm-project/vllm/pull/56723) adds `dcp_sharded` to distinguish sharded attention KV from replicated Mamba/GDN state when target DCP and DCP=1 draft coexist. [vLLM #45635](https://github.com/vllm-project/vllm/pull/45635) moves routed-expert output to `AuxOutputConfig`/the AuxOutput connector; [vLLM #57980](https://github.com/vllm-project/vllm/pull/57980) removes the legacy runner initialization keyword. | Read `dcp_sharded` from cache specs and forward it to block tables; initialize the AuxOutput connector from `aux_output_config.enabled`; drop the legacy `attn_cg_support` keyword. | | [`vllm_ascend/platform.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/platform.py) | [vLLM #45635](https://github.com/vllm-project/vllm/pull/45635) moves routed-expert output to `AuxOutputConfig`/the AuxOutput connector; [vLLM #57980](https://github.com/vllm-project/vllm/pull/57980) removes the legacy runner initialization keyword. | Apply the existing V1 AuxOutput configuration guard to both targets, removing the old 0.30 early return. | | [`vllm_ascend/worker/model_runner_v1.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/worker/model_runner_v1.py) | [vLLM #58411](https://github.com/vllm-project/vllm/pull/58411) adds `randomize_inputs` for memory profiling to avoid all-zero MoE input concentration. Present in release; absent from the pinned-main parent calls. | Accept `randomize_inputs` in profiling/dummy execution and enter `maybe_randomize_inputs` inside forward context when requested; preserve the Ascend execution path. | | [`vllm_ascend/worker/v2/aclgraph_utils.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/worker/v2/aclgraph_utils.py) | [vLLM #57980](https://github.com/vllm-project/vllm/pull/57980) replaces the legacy PCP helper with descriptor-based `partition_batch`, restores auxiliary hidden states with sampling outputs, and renames the DCP preparation helper. [vLLM #57632](https://github.com/vllm-project/vllm/pull/57632) passes `precompute_context_kv` into DFlash FULL-graph capture; wrapped draft models must expose the underlying callback. | Use `prepare_dcp_local_seq_lens`; delegate model-wrapper attribute lookup to the original model so DFlash context-KV callbacks remain accessible. | | [`vllm_ascend/worker/v2/input_batch.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/worker/v2/input_batch.py) | [vLLM #58275](https://github.com/vllm-project/vllm/pull/58275)/[vLLM #57980](https://github.com/vllm-project/vllm/pull/57980) expose `max_query_len`/`is_padding` on dummy batches and an explicit query-length bound for autoregressive graph capture. | Always retain prompt-tail and graph-eligibility fields and forward dummy-batch `max_query_len`/`is_padding`; remove obsolete 0.30 branches. | | [`vllm_ascend/worker/v2/model_runner.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/worker/v2/model_runner.py) | [vLLM #58411](https://github.com/vllm-project/vllm/pull/58411) adds `randomize_inputs` for memory profiling to avoid all-zero MoE input concentration. Present in release; absent from the pinned-main parent calls. [vLLM #58542](https://github.com/vllm-project/vllm/pull/58542) adds `max_seq_len_np` for PP sampled-token broadcast handling. Present in release batch state; absent from pinned main. [vLLM #58400](https://github.com/vllm-project/vllm/pull/58400) adds `prefill_runs_as_decode_np` and `decode_graph_eligible` for single-token prompt tails with existing context; `has_prefill` alone no longer determines FULL-decode eligibility. [vLLM #57980](https://github.com/vllm-project/vllm/pull/57980) replaces the legacy PCP helper with descriptor-based `partition_batch`, restores auxiliary hidden states with sampling outputs, and renames the DCP preparation helper. [vLLM #56456](https://github.com/vllm-project/vllm/pull/56456) adds trailing `num_active_loras` to `prepare_inputs`, keeping dispatch and fast-prefill graph/padding selection consistent. [vLLM #54335](https://github.com/vllm-project/vllm/pull/54335) routes fixed-token scoring through the same unpadded logits path as prompt logprobs, which is incompatible with Ascend lmhead-TP collective row layouts. | Use release-only kwargs for profiling/execute and MC2 dummy execution; forward release-only `max_seq_len_np`. After PD request reclassification, recompute `has_prefill`, decode-graph eligibility and uniform token count. Accept `num_active_loras` in the common prepare signature without adding FastPrefillHelper behavior. Use descriptor-based PCP partitioning and restore auxiliary states before replicated drafting. Extend the lmhead-TP guard to fixed-token scoring, tolerating workers without `token_id_scores`. | | [`vllm_ascend/worker/v2/model_states/mamba_hybrid.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/worker/v2/model_states/mamba_hybrid.py) | [vLLM #58400](https://github.com/vllm-project/vllm/pull/58400) requires decode-like prompt tails to use decode-row Mamba metadata for speculative rollback. | Build a local mask `is_prefilling_np & ~prefill_runs_as_decode_np` so prompt tails are decode rows; do not mutate the batch's original prefill mask. | | [`vllm_ascend/worker/v2/pcp_manager.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/worker/v2/pcp_manager.py) | [vLLM #57980](https://github.com/vllm-project/vllm/pull/57980) replaces the legacy PCP helper with descriptor-based `partition_batch`, restores auxiliary hidden states with sampling outputs, and renames the DCP preparation helper. | Unify descriptor-based partitioning, including CPP profile capture after #18622, and the `(batch, hidden, aux)` sampling restore contract; restore and trim auxiliary states together with hidden states to the unpadded global token extent. | | [`vllm_ascend/worker/v2/spec_decode/autoregressive/aclgraph.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/worker/v2/spec_decode/autoregressive/aclgraph.py) | [vLLM #58275](https://github.com/vllm-project/vllm/pull/58275)/[vLLM #57980](https://github.com/vllm-project/vllm/pull/57980) expose `max_query_len`/`is_padding` on dummy batches and an explicit query-length bound for autoregressive graph capture. | Always pass `desc.max_query_len or desc.uniform_token_count` to autoregressive capture. | | [`vllm_ascend/worker/v2/spec_decode/dflash/aclgraph.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/worker/v2/spec_decode/dflash/aclgraph.py) | [vLLM #57632](https://github.com/vllm-project/vllm/pull/57632) passes `precompute_context_kv` into DFlash FULL-graph capture; wrapped draft models must expose the underlying callback. | Always receive and forward `precompute_context_kv` under the Ascend communicator/model-capture wrappers. | </details> <details> <summary>KV allocation and block tables (6 files)</summary> | Downstream file | Upstream change / contract | Ascend adaptation | | --- | --- | --- | | [`vllm_ascend/_310p/worker/v2/block_table.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/_310p/worker/v2/block_table.py) | [vLLM #56723](https://github.com/vllm-project/vllm/pull/56723) adds `dcp_sharded` to distinguish sharded attention KV from replicated Mamba/GDN state when target DCP and DCP=1 draft coexist. [vLLM #56734](https://github.com/vllm-project/vllm/pull/56734) uses `idx_mapping=-1` for dummy draft/padding requests and masks their persistent block-table reads to avoid stale-row KV writes. | Unify the constructor with `dcp_sharded`; skip negative request indices and retain PAD slots in the CPU mapping path. Keep the existing CP=1 restriction and persistent CPU/device buffers. | | [`vllm_ascend/models/glm5next/kv_cache.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/models/glm5next/kv_cache.py) | [vLLM #59175](https://github.com/vllm-project/vllm/pull/59175) adds `prefill_end` to Mamba checkpoint allocation, including prompt-end retention. Release has the new signature; pinned main has the old one. | Accept `prefill_end` for call compatibility; keep one private Kpool tail block per request. | | [`vllm_ascend/ops/triton/v2/block_table/compute_slot_mappings.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/ops/triton/v2/block_table/compute_slot_mappings.py) | [vLLM #56734](https://github.com/vllm-project/vllm/pull/56734) uses `idx_mapping=-1` for dummy draft/padding requests and masks their persistent block-table reads to avoid stale-row KV writes. | Unify the slot-mapping kernel, mask block-table loads with `is_real_req`, and emit PAD for dummy rows; retain Ascend block/window and address-calculation behavior. | | [`vllm_ascend/patch/platform/patch_kv_cache_utils.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/patch/platform/patch_kv_cache_utils.py) | The pinned `_annotate_eagle_groups` contract uses `use_trailing_layer_fallback`, replacing the DeepSeek-V4-specific fallback keyword; the introducing PR was not independently identified. Fixed contracts: [release: kv_cache_utils.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/v1/core/kv_cache_utils.py#L2170). | Always pass `use_trailing_layer_fallback=True`; retain existing Ascend cache grouping/annotation behavior. | | [`vllm_ascend/patch/platform/patch_mamba_manager.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/patch/platform/patch_mamba_manager.py) | [vLLM #59175](https://github.com/vllm-project/vllm/pull/59175) adds `prefill_end` to Mamba checkpoint allocation, including prompt-end retention. Release has the new signature; pinned main has the old one. | Accept `prefill_end`, forwarding it only to the release parent while retaining the existing Ascend allocation logic. | | [`vllm_ascend/worker/v2/block_table.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/worker/v2/block_table.py) | [vLLM #56723](https://github.com/vllm-project/vllm/pull/56723) adds `dcp_sharded` to distinguish sharded attention KV from replicated Mamba/GDN state when target DCP and DCP=1 draft coexist. | Use one constructor with optional per-group `dcp_sharded` validation/forwarding; retain Ascend int32 slot mappings. | </details> <details> <summary>Schedulers and profiling (5 files)</summary> | Downstream file | Upstream change / contract | Ascend adaptation | | --- | --- | --- | | [`vllm_ascend/core/dyntra_lb_scheduler.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/core/dyntra_lb_scheduler.py) | [vLLM #59029](https://github.com/vllm-project/vllm/pull/59029) gives main an encoder-only exception to the non-chunked prefill budget check; release keeps the budget rejection condition. [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) replaces `skipped_waiting` with `kv_holding_waiting` and `deferred_waiting`, prioritizes KV holders, and uses `_handle_blocked_waiting_request`. [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) and [vLLM #58259](https://github.com/vllm-project/vllm/pull/58259) define the upstream resumable-session queue/handoff lifecycle: stopping a turn can re-enqueue buffered continuation input. Both targets admit against `max_num_active_reqs` and include parked streaming sessions in occupied slots; related queue/Elastic-EP changes are [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) and [vLLM #53934](https://github.com/vllm-project/vllm/pull/53934). Fixed contracts: [release: scheduler.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/v1/core/sched/scheduler.py#L127), [release: scheduler.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/v1/core/sched/scheduler.py#L892). | Use KV-holder-first queue selection/re-enqueueing, the common blocked-request helper, deferred bookkeeping and active-slot admission including parked streams. Preserve the main-only encoder exception. Remove secondary stop cleanup so upstream can retain/re-enqueue resumable continuations. | | [`vllm_ascend/core/recompute_scheduler.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/core/recompute_scheduler.py) | [vLLM #59029](https://github.com/vllm-project/vllm/pull/59029) gives main an encoder-only exception to the non-chunked prefill budget check; release keeps the budget rejection condition. [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) replaces `skipped_waiting` with `kv_holding_waiting` and `deferred_waiting`, prioritizes KV holders, and uses `_handle_blocked_waiting_request`. Both targets admit against `max_num_active_reqs` and include parked streaming sessions in occupied slots; related queue/Elastic-EP changes are [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) and [vLLM #53934](https://github.com/vllm-project/vllm/pull/53934). Fixed contracts: [release: scheduler.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/v1/core/sched/scheduler.py#L127), [release: scheduler.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/v1/core/sched/scheduler.py#L892). | Fold the legacy queues into the common KV-holder/deferred path, count parked streams for active admission, and preserve the version-dependent encoder condition. Remove the redundant remote-KV waiting debug log as a local cleanup. | | [`vllm_ascend/core/scheduler_profiling_chunk.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/core/scheduler_profiling_chunk.py) | [vLLM #59029](https://github.com/vllm-project/vllm/pull/59029) gives main an encoder-only exception to the non-chunked prefill budget check; release keeps the budget rejection condition. [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) replaces `skipped_waiting` with `kv_holding_waiting` and `deferred_waiting`, prioritizes KV holders, and uses `_handle_blocked_waiting_request`. Both targets admit against `max_num_active_reqs` and include parked streaming sessions in occupied slots; related queue/Elastic-EP changes are [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) and [vLLM #53934](https://github.com/vllm-project/vllm/pull/53934). Fixed contracts: [release: scheduler.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/v1/core/sched/scheduler.py#L127), [release: scheduler.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/v1/core/sched/scheduler.py#L892). | Use the common blocked-request/queue lifecycle, active admission including parked streams, and the version-dependent encoder budget condition. | | [`vllm_ascend/patch/platform/patch_balance_schedule.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/patch/platform/patch_balance_schedule.py) | [vLLM #59029](https://github.com/vllm-project/vllm/pull/59029) gives main an encoder-only exception to the non-chunked prefill budget check; release keeps the budget rejection condition. [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) replaces `skipped_waiting` with `kv_holding_waiting` and `deferred_waiting`, prioritizes KV holders, and uses `_handle_blocked_waiting_request`. | Fold six obsolete 0.30 queue branches and add the release/main encoder budget condition. Document the shared `if not request_queue` drained-queue guard. Retain baseline balancing, running-only DP snapshots/admission accounting and connector-snapshot cleanup; complete upstream schedule synchronization is outside this patch. | | [`vllm_ascend/profiling_config.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/profiling_config.py) | [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) replaces `skipped_waiting` with `kv_holding_waiting` and `deferred_waiting`, prioritizes KV holders, and uses `_handle_blocked_waiting_request`. | Retarget the profiling wrapper from the old blocked-status promotion helper to `_handle_blocked_waiting_request`. | </details> <details> <summary>EPLB state and routing (2 files)</summary> | Downstream file | Upstream change / contract | Ascend adaptation | | --- | --- | --- | | [`vllm_ascend/distributed/eplb/eplb_state.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/distributed/eplb/eplb_state.py) | [vLLM #53934](https://github.com/vllm-project/vllm/pull/53934) changes main's EPLB setup to update existing state and reuse its communicator. Release still creates state from `(model, model_config, mapping)`. | After the parent's `update_mapping`, refresh Ascend replica routing tables; consistently resolve an unspecified NPU device index to the current device. | | [`vllm_ascend/worker/v2/eplb.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/worker/v2/eplb.py) | [vLLM #53934](https://github.com/vllm-project/vllm/pull/53934) changes main's EPLB setup to update existing state and reuse its communicator. Release still creates state from `(model, model_config, mapping)`. | Create `AscendEplbState` on release with the current policy/optional expert arguments; call the main parent to update the existing state and communicator. | </details> <details> <summary>Models and MoE operators (6 files)</summary> | Downstream file | Upstream change / contract | Ascend adaptation | | --- | --- | --- | | [`vllm_ascend/models/kimi_k3.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/models/kimi_k3.py) | [vLLM #50592](https://github.com/vllm-project/vllm/pull/50592)/[vLLM #50593](https://github.com/vllm-project/vllm/pull/50593) use `prefix_delta` and the three-value AttnRes layer contract instead of the old prepared-input/deferred-MLP contract. | Remove old layer/model contracts, propagate `prefix_delta` and return the three-value AttnRes state; preserve Ascend fused operations, DSpark aux selection and SP/PP transport/output boundaries. | | [`vllm_ascend/ops/fused_moe/gate_linear.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/ops/fused_moe/gate_linear.py) | [vLLM #58234](https://github.com/vllm-project/vllm/pull/58234) unifies MoE gate layers under `GateLinear` with the current linear constructor and replaces Step3.5's dedicated FP32 gate class. | Use the current constructor parameters (`skip_bias_add`, `quant_config`, `return_bias`) and retain BF16-to-FP32 gate/fallback semantics. CRLF-to-LF normalization contributes to the large line diff. | | [`vllm_ascend/ops/fused_moe/router/grouped_topk_router.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/ops/fused_moe/router/grouped_topk_router.py) | [vLLM #58880](https://github.com/vllm-project/vllm/pull/58880) removes `routed_scaling_factor` from routing-method classification, allowing MiniMax classification with non-unit scaling. | Stop passing `routed_scaling_factor` to classification; downstream routing-weight scaling is retained. | | [`vllm_ascend/patch/__init__.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/patch/__init__.py) | The release target is 0.31.0; [vLLM #56994](https://github.com/vllm-project/vllm/pull/56994) is context for the retained GLM reasoning patch comment, not a new model capability in this PR. | Update only the supported-version comment for the retained GLM reasoning patch. | | [`vllm_ascend/patch/worker/patch_step3p5.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/patch/worker/patch_step3p5.py) | [vLLM #58234](https://github.com/vllm-project/vllm/pull/58234) unifies MoE gate layers under `GateLinear` with the current linear constructor and replaces Step3.5's dedicated FP32 gate class. | Always use `GateLinear` with FP32 parameter/output types instead of the legacy FP32ReplicatedLinear branch. | </details> <details> <summary>Token scoring and Triton helpers (2 files)</summary> | Downstream file | Upstream change / contract | Ascend adaptation | | --- | --- | --- | | [`vllm_ascend/patch/worker/patch_triton.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/patch/worker/patch_triton.py) | [vLLM #58651](https://github.com/vllm-project/vllm/pull/58651) uses `triton.next_power_of_2` in packed-QK RoPE constexpr evaluation; the Ascend replacement must also satisfy the installed Triton JIT/constexpr-function contract. | Use a constexpr-aware `next_power_of_2` replacement; register with `triton.constexpr_function` when available, otherwise unwrap `tl.constexpr.value` in the fallback. | | [`vllm_ascend/worker/v2/sample/logprob.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/vllm_ascend/worker/v2/sample/logprob.py) | [vLLM #54335](https://github.com/vllm-project/vllm/pull/54335) adds fixed-token prompt scoring and broadcasts candidate IDs with `expand`; the token-ID row stride can be zero or non-contiguous. | Pass `token_ids.stride(0)` into the Triton kernel and use it for row addressing; support broadcast stride zero and strided candidate IDs. | </details> BalanceScheduler scope: this PR folds the old queue branches and adapts encoder-only admission. Its baseline DP balancing, running-only snapshot/accounting and connector-snapshot cleanup remain; complete scheduler-body synchronization and a strict AST equivalence requirement are deferred. The baseline drift guard may skip and does not prove complete upstream equivalence. ### Does this PR introduce _any_ user-facing change? The default release dependency becomes vLLM v0.31.0; the verified-main pin remains unchanged. Fixed-token prompt scoring handles broadcast/strided IDs correctly and reports an explicit error when used with unsupported lmhead TP. Resumable Dyntra continuations retain upstream stop/queue behavior, and PD prompt-tail graph/Mamba metadata follow the supported upstream contracts. ### How was this patch tested? #### Test changes organized by file The table describes test intent and fixture changes; it does not claim that every changed test was selected or executed on NPU hardware. <details> <summary>Unit, PR E2E and nightly test mapping (37 files)</summary> | Downstream file | Upstream change / contract | Ascend adaptation | | --- | --- | --- | | [`tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_compute_token_logprobs.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_compute_token_logprobs.py) | [vLLM #54335](https://github.com/vllm-project/vllm/pull/54335) | Add NPU numerical comparisons for broadcast stride=0 and sliced stride=8 token IDs; outside the PR CI selector. | | [`tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_num_nans.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_num_nans.py) | Release-version cleanup; no new upstream behavior. | Update the default version label to 0.31.0; no algorithm change. | | [`tests/e2e/pull_request/four_card/test_kimi_k3.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/e2e/pull_request/four_card/test_kimi_k3.py) | [vLLM #54222](https://github.com/vllm-project/vllm/pull/54222) | Check equality between P-worker `remote_prefill_cached_tokens` and the response cache count; zero hits are valid for a new prompt. | | [`tests/e2e/pull_request/one_card/test_model_runner_v1_with_device.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/e2e/pull_request/one_card/test_model_runner_v1_with_device.py) | [vLLM #45635](https://github.com/vllm-project/vllm/pull/45635), [vLLM #57980](https://github.com/vllm-project/vllm/pull/57980) | Use the common AuxOutput configuration object in V1 guard fixtures. | | [`tests/e2e/pull_request/one_card/test_thinking_budget.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/e2e/pull_request/one_card/test_thinking_budget.py) | [vLLM #56497](https://github.com/vllm-project/vllm/pull/56497) | Construct `LogitsContext` with expanded mapping, input IDs, positions and sequence bounds; call `apply(logits, ctx)`. | | [`tests/ut/_310p/test_model_runner_v2_310p.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/_310p/test_model_runner_v2_310p.py) | [vLLM #56456](https://github.com/vllm-project/vllm/pull/56456), [vLLM #56497](https://github.com/vllm-project/vllm/pull/56497) | Use the current sampler request signature and pass the added prepare-input LoRA argument in 310P fixtures. | | [`tests/ut/attention/test_mla_cp.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/attention/test_mla_cp.py) | [vLLM #58149](https://github.com/vllm-project/vllm/pull/58149) makes `ModelState.max_model_len` a model-config-backed property; [vLLM #57913](https://github.com/vllm-project/vllm/pull/57913) moves multimodal capability to `ModelConfig`; [vLLM #56723](https://github.com/vllm-project/vllm/pull/56723) updates DCP state/metadata contracts. | Mock `get_dcp_world_size_and_rank` and remove legacy DCP group fixture branching. | | [`tests/ut/core/test_circular_buffer.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/core/test_circular_buffer.py) | Pinned cache allocation requires replay boundaries ([vLLM #58368](https://github.com/vllm-project/vllm/pull/58368)); DeepSeek-V4.1 architecture metadata and Engram configuration ([vLLM #56214](https://github.com/vllm-project/vllm/pull/56214)) use their current configuration contracts. Individual configuration migrations are linked to fixed source where not precisely attributed. Fixed contracts: [release: single_type_kv_cache_manager.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/v1/core/single_type_kv_cache_manager.py#L482), [release: model_arch.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/config/model_arch.py#L59), [release: engram.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/config/engram.py#L89). | Pass `replay_boundaries=[tokens-1]` to the current cache-block API. | | [`tests/ut/core/test_dyntra_lb_recompute_scheduler.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/core/test_dyntra_lb_recompute_scheduler.py) | [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) | Use the shared KV-holder/deferred queue interfaces in Dyntra/recompute fixtures. | | [`tests/ut/core/test_dyntra_lb_scheduler.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/core/test_dyntra_lb_scheduler.py) | [vLLM #58259](https://github.com/vllm-project/vllm/pull/58259), [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) | Regress synchronous/asynchronous resumable continuation handling for running, paused and preempted requests, buffered/unbuffered input, and final cleanup; update queue seams. | | [`tests/ut/core/test_profiling_chunk.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/core/test_profiling_chunk.py) | [vLLM #53934](https://github.com/vllm-project/vllm/pull/53934), [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) | Update queue fixtures/blocked-request helper and active-request capacity field. | | [`tests/ut/core/test_recompute_scheduler.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/core/test_recompute_scheduler.py) | [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) | Remove legacy queue branching and use the shared blocked-request seam. | | [`tests/ut/core/test_scheduler_connector_block_state.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/core/test_scheduler_connector_block_state.py) | [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947), [vLLM #59029](https://github.com/vllm-project/vllm/pull/59029) | Supply the baseline `balance_queue` fixture; connector-snapshot cleanup itself predates this change. | | [`tests/ut/distributed/eplb/test_state.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/distributed/eplb/test_state.py) | [vLLM #53934](https://github.com/vllm-project/vllm/pull/53934) | Verify mapping updates refresh Ascend routing tables. | | [`tests/ut/distributed/kv_transfer/kv_p2p/test_remote_prefill_lifecycle.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/distributed/kv_transfer/kv_p2p/test_remote_prefill_lifecycle.py) | [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947) | Use the shared queue lifecycle rather than version-specific skipped-waiting fixtures. | | [`tests/ut/kv_offload/test_preempt_offload_connector.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/kv_offload/test_preempt_offload_connector.py) | Pinned cache allocation requires replay boundaries ([vLLM #58368](https://github.com/vllm-project/vllm/pull/58368)); DeepSeek-V4.1 architecture metadata and Engram configuration ([vLLM #56214](https://github.com/vllm-project/vllm/pull/56214)) use their current configuration contracts. Individual configuration migrations are linked to fixed source where not precisely attributed. Fixed contracts: [release: single_type_kv_cache_manager.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/v1/core/single_type_kv_cache_manager.py#L482), [release: model_arch.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/config/model_arch.py#L59), [release: engram.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/config/engram.py#L89). | Always provide prefix replay and request token-count fields in the offload fixture. | | [`tests/ut/models/test_deepseek_v41_release_config.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/models/test_deepseek_v41_release_config.py) | Pinned cache allocation requires replay boundaries ([vLLM #58368](https://github.com/vllm-project/vllm/pull/58368)); DeepSeek-V4.1 architecture metadata and Engram configuration ([vLLM #56214](https://github.com/vllm-project/vllm/pull/56214)) use their current configuration contracts. Individual configuration migrations are linked to fixed source where not precisely attributed. Fixed contracts: [release: single_type_kv_cache_manager.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/v1/core/single_type_kv_cache_manager.py#L482), [release: model_arch.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/config/model_arch.py#L59), [release: engram.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/config/engram.py#L89). | Assert obsolete flattened multimodal-prefix flags are absent after architecture conversion. | | [`tests/ut/models/test_glm5next_kv_cache.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/models/test_glm5next_kv_cache.py) | [vLLM #59175](https://github.com/vllm-project/vllm/pull/59175) | Pass the new allocation argument while checking the private-tail block policy. | | [`tests/ut/models/test_kimi_k3_adapter.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/models/test_kimi_k3_adapter.py) | [vLLM #50592](https://github.com/vllm-project/vllm/pull/50592), [vLLM #50593](https://github.com/vllm-project/vllm/pull/50593) | Update layer signatures/three-value returns, auxiliary output expectations and AST-extracted fixtures. | | [`tests/ut/models/test_kimi_k3_pp_sp.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/models/test_kimi_k3_pp_sp.py) | [vLLM #50592](https://github.com/vllm-project/vllm/pull/50592), [vLLM #50593](https://github.com/vllm-project/vllm/pull/50593) | Use the common Kimi AttnRes contract in PP/SP checks. | | [`tests/ut/patch/platform/test_patch_balance_schedule.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/patch/platform/test_patch_balance_schedule.py) | [vLLM #58947](https://github.com/vllm-project/vllm/pull/58947), [vLLM #59029](https://github.com/vllm-project/vllm/pull/59029) | Use common queue fixtures; add eight encoder/chunked/prompt-budget cases and empty/fully deferred queue regressions with/without KV blocks. Correct the structural guard assertion to `if not request_queue`. Keep the baseline AST drift check, which can skip. | | [`tests/ut/patch/platform/test_patch_engram_config.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/patch/platform/test_patch_engram_config.py) | Pinned cache allocation requires replay boundaries ([vLLM #58368](https://github.com/vllm-project/vllm/pull/58368)); DeepSeek-V4.1 architecture metadata and Engram configuration ([vLLM #56214](https://github.com/vllm-project/vllm/pull/56214)) use their current configuration contracts. Individual configuration migrations are linked to fixed source where not precisely attributed. Fixed contracts: [release: single_type_kv_cache_manager.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/v1/core/single_type_kv_cache_manager.py#L482), [release: model_arch.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/config/model_arch.py#L59), [release: engram.py](https://github.com/vllm-project/vllm/blob/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm/config/engram.py#L89). | Expect shared-memory default resolution for this supported fixture configuration; no universal shared-memory default is claimed. | | [`tests/ut/patch/platform/test_prefix_cache_cp_patches.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/patch/platform/test_prefix_cache_cp_patches.py) | [vLLM #59175](https://github.com/vllm-project/vllm/pull/59175) | Supply an explicit `prefill_end` allocation boundary. | | [`tests/ut/spec_decode/test_eagle_proposer.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/spec_decode/test_eagle_proposer.py) | [vLLM #58149](https://github.com/vllm-project/vllm/pull/58149) makes `ModelState.max_model_len` a model-config-backed property; [vLLM #57913](https://github.com/vllm-project/vllm/pull/57913) moves multimodal capability to `ModelConfig`; [vLLM #56723](https://github.com/vllm-project/vllm/pull/56723) updates DCP state/metadata contracts. | Use the current model-config multimodal capability interface and remove obsolete skips/branches. | | [`tests/ut/spec_decode/test_mla_dspark.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/spec_decode/test_mla_dspark.py) | [vLLM #57632](https://github.com/vllm-project/vllm/pull/57632) | Assert DFlash context-KV callback forwarding and the shared capture signature. | | [`tests/ut/test_minimax_m3_kv_block_size.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/test_minimax_m3_kv_block_size.py) | Release-version cleanup; no new upstream behavior. | Remove only an unused version-helper import. | | [`tests/ut/test_platform.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/test_platform.py) | [vLLM #45635](https://github.com/vllm-project/vllm/pull/45635), [vLLM #57980](https://github.com/vllm-project/vllm/pull/57980) | Use the current AuxOutput configuration guard for both target versions. | | [`tests/ut/worker/test_attn_utils_v2.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/worker/test_attn_utils_v2.py) | [vLLM #58149](https://github.com/vllm-project/vllm/pull/58149) makes `ModelState.max_model_len` a model-config-backed property; [vLLM #57913](https://github.com/vllm-project/vllm/pull/57913) moves multimodal capability to `ModelConfig`; [vLLM #56723](https://github.com/vllm-project/vllm/pull/56723) updates DCP state/metadata contracts. | Initialize `model_state.model_config` instead of assigning the read-only length property. | | [`tests/ut/worker/test_dcp_speculator_metadata_v2.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/worker/test_dcp_speculator_metadata_v2.py) | [vLLM #58149](https://github.com/vllm-project/vllm/pull/58149) makes `ModelState.max_model_len` a model-config-backed property; [vLLM #57913](https://github.com/vllm-project/vllm/pull/57913) moves multimodal capability to `ModelConfig`; [vLLM #56723](https://github.com/vllm-project/vllm/pull/56723) updates DCP state/metadata contracts. | Assert prefill metadata over the padded request extent. | | [`tests/ut/worker/test_model_runner_v2.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/worker/test_model_runner_v2.py) | [vLLM #58411](https://github.com/vllm-project/vllm/pull/58411) adds `randomize_inputs` for memory profiling to avoid all-zero MoE input concentration. Present in release; absent from the pinned-main parent calls. [vLLM #58542](https://github.com/vllm-project/vllm/pull/58542) adds `max_seq_len_np` for PP sampled-token broadcast handling. Present in release batch state; absent from pinned main. [vLLM #58400](https://github.com/vllm-project/vllm/pull/58400) adds `prefill_runs_as_decode_np` and `decode_graph_eligible` for single-token prompt tails with existing context; `has_prefill` alone no longer determines FULL-decode eligibility. [vLLM #57980](https://github.com/vllm-project/vllm/pull/57980) replaces the legacy PCP helper with descriptor-based `partition_batch`, restores auxiliary hidden states with sampling outputs, and renames the DCP preparation helper. [vLLM #56456](https://github.com/vllm-project/vllm/pull/56456) adds trailing `num_active_loras` to `prepare_inputs`, keeping dispatch and fast-prefill graph/padding selection consistent. [vLLM #54335](https://github.com/vllm-project/vllm/pull/54335) routes fixed-token scoring through the same unpadded logits path as prompt logprobs, which is incompatible with Ascend lmhead-TP collective row layouts. [vLLM #58149](https://github.com/vllm-project/vllm/pull/58149) makes `ModelState.max_model_len` a model-config-backed property; [vLLM #57913](https://github.com/vllm-project/vllm/pull/57913) moves multimodal capability to `ModelConfig`; [vLLM #56723](https://github.com/vllm-project/vllm/pull/56723) updates DCP state/metadata contracts. | Cover release/main kwargs and PP fields, PD graph reclassification, PCP auxiliary restoration and lmhead scoring guards; update model-state fixtures. | | [`tests/ut/worker/test_model_runner_v2_encoder_decoder.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/worker/test_model_runner_v2_encoder_decoder.py) | [vLLM #58149](https://github.com/vllm-project/vllm/pull/58149) makes `ModelState.max_model_len` a model-config-backed property; [vLLM #57913](https://github.com/vllm-project/vllm/pull/57913) moves multimodal capability to `ModelConfig`; [vLLM #56723](https://github.com/vllm-project/vllm/pull/56723) updates DCP state/metadata contracts. | Update model-state length fixtures through model configuration. | | [`tests/ut/worker/test_model_runner_v2_mamba.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/worker/test_model_runner_v2_mamba.py) | [vLLM #58400](https://github.com/vllm-project/vllm/pull/58400) | Regress prompt-tail masks for FULL/NONE graph modes and zero/three draft tokens; verify the original mask is unchanged. | | [`tests/ut/worker/test_pcp_manager_v2.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/worker/test_pcp_manager_v2.py) | [vLLM #45635](https://github.com/vllm-project/vllm/pull/45635), [vLLM #57980](https://github.com/vllm-project/vllm/pull/57980) | Assert descriptor-based partitioning and hidden/aux restore layout; update AuxOutput fixture configuration. | | [`tests/ut/worker/test_v2_dummy_padding.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/worker/test_v2_dummy_padding.py) | [vLLM #58400](https://github.com/vllm-project/vllm/pull/58400) adds `prefill_runs_as_decode_np` and `decode_graph_eligible` for single-token prompt tails with existing context; `has_prefill` alone no longer determines FULL-decode eligibility. [vLLM #58275](https://github.com/vllm-project/vllm/pull/58275)/[vLLM #57980](https://github.com/vllm-project/vllm/pull/57980) expose `max_query_len`/`is_padding` on dummy batches and an explicit query-length bound for autoregressive graph capture. [vLLM #58149](https://github.com/vllm-project/vllm/pull/58149) makes `ModelState.max_model_len` a model-config-backed property; [vLLM #57913](https://github.com/vllm-project/vllm/pull/57913) moves multimodal capability to `ModelConfig`; [vLLM #56723](https://github.com/vllm-project/vllm/pull/56723) updates DCP state/metadata contracts. | Use the model-config-backed state/dummy descriptor contract and retain graph-eligibility metadata. | | [`tests/ut/worker/v2/test_aclgraph.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/worker/v2/test_aclgraph.py) | [vLLM #57980](https://github.com/vllm-project/vllm/pull/57980), [vLLM #58275](https://github.com/vllm-project/vllm/pull/58275) | Update capture descriptors and dummy-batch keyword expectations. | | [`tests/ut/worker/v2/test_eplb_controller.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/worker/v2/test_eplb_controller.py) | [vLLM #53934](https://github.com/vllm-project/vllm/pull/53934) | Cover release state construction versus main in-place state/communicator reuse. | | [`tests/ut/worker/v2/test_eplb_load_collection_phase.py`](https://github.com/vllm-project/vllm-ascend/blob/23c8b15c2709978463490b0e8531bf5f9f27b9c1/tests/ut/worker/v2/test_eplb_load_collection_phase.py) | [vLLM #53934](https://github.com/vllm-project/vllm/pull/53934) | Match each target's EPLB setup signature in load-collection fixtures. | </details> #### Local A3 nightly validation (2026-10-15; frozen head `444e962`) These four model runs used **`444e962b239c3a8fc56b9255d601818e6e028b9a` + vLLM 0.31.0 (`db9527a`)**, which is also the restored PR head; the deferred connector hardening is not included. Prefill 138 + Decode 136, 16 NPUs per node; original image, weights, datasets, launch and benchmark parameters were retained, with only the second-node address mapped from 139 to 136. All four cases exited pytest 0 on both nodes (one passed JUnit case per node). OTT is output token/s; TPOT/TTFT are ms. | Case | Performance requests | OTT / required minimum | TPOT / TTFT | AIME2025 accuracy | Result | | --- | --- | --- | --- | --- | --- | | Kimi-K2.6 PD | 60/60 | 355.1593 / 337.024075 | 20.5 / 3268.2 | 90%, within 93.33±10 | Performance + accuracy passed | | DeepSeek-V4 Flash PD-prefix | Accuracy only | N/A | N/A | 90%, within 90±10 | Accuracy passed | | GLM-5.1 W8A8C8 128K | 256/256 | 811.7645 / 282.105876 | 39.8 / 32665.6 | No accuracy item | Performance passed | | MiniMax-M2.7 PD-prefix | 160/160 | 711.0518 / 698.497 | 34.8 / 17438.2 | 90%, within 93.33±10 | Performance + accuracy passed | All performance requests above, and the valid controls below, were checked for actual 1024-token outputs, not just success flags. **Conditional controls:** historical throughput differences triggered only the affected performance cases against the same PR base `6d1dd70`, without this PR, + vLLM 0.30.0 (`ced6857`). Kimi: 355.1593 vs 359.8514 OTT (**-1.30%**); the earlier 7.27% loss was not reproduced in this comparison. TPOT was 20.5 vs 19.7ms (+0.8ms/+4.06%), and average request latency +3.48%; a single pair does not establish latency equivalence or a stable regression. GLM: 811.7645 vs 813.3506 OTT (**-0.20%**), TPOT 39.8 vs 39.4ms. Kimi's original random request pacing affects aggregate OTT. No alternating reruns or unaffected-model 0.30 controls were added. Archived evidence: `main2main-nightly-20261013/pr18229-update-444e962-20261015/` (`FINAL_REPORT.md`, per-node logs/JUnit, raw AISBench outputs, token-count audit, controls and diagnostic evidence). Both nodes were released after testing. #### Deferred follow-up: multi-group receive-failure reporting (not included) The request-level receive-failure hardening change has been removed from this upgrade PR; the branch is restored to the nightly-tested head `444e962`. The original four nightlies already passed on this head **without** that fix, so it is not required for these normal-path workloads to pass. The fault-path issue remains real: if a multi-group receive actually fails, the existing connector can report flat block IDs and trigger the vLLM 0.31 scheduler guard, terminating Decode EngineCore. This is handled separately in [#18823](https://github.com/vllm-project/vllm-ascend/pull/18823), retaining ordinary non-HMA block-level recovery semantics. The upgrade PR does not claim fault isolation is fixed. Development fault-injection results belong to the deferred connector implementation, not this restored head. No new NPU run was performed when restoring the exact previously tested code. #### Historical validation for head `23c8b15` - New [E2E workflow](https://github.com/vllm-project/vllm-ascend/actions/runs/38562748688) and [Docs link check](https://github.com/vllm-project/vllm-ascend/actions/runs/38562748395) were triggered after the rebase. Results are pending; previous-head CI does not validate this rebased head. - A targeted [comment command](https://github.com/vllm-project/vllm-ascend/pull/18229#issuecomment-6168149194) requests `test_deepseek_v4_mtp_full_decode_only` against both supported upstream targets. The [command workflow](https://github.com/vllm-project/vllm-ascend/actions/runs/38562831465) is queued; its actual test results remain pending. - Ruff check and format check passed for all 72 changed Python files; Python compilation and diff checks passed. Source-extracted execution of both real CPP routing regressions passed against the required partition signatures of the exact main/release refs. This validates dispatch contracts with lightweight dependency stubs and does not replace full pytest or NPU execution. - Existing source-extracted scheduler regressions passed on both version lanes: eight budget scenarios, two queue paths and empty/fully deferred queue regressions, including negative controls. - Local full pytest/Torch/NPU and `bash format.sh ci` remain unavailable in this Windows environment. They require CI validation. - Previous head `9330488` passed pre-commit, Docs and both CPU UT lanes (main 10782 passed / 114 skipped; release 10784 passed / 112 skipped). Its initial NPU matrix passed 41/42 shards; the remaining main A3 shard failed in unchanged upstream teardown with `psutil.NoSuchProcess`, before the MTP golden acceptance assertion. The consequent gate failure and old-head rerun are historical evidence, not a result for the current head. #### Coverage limits and historical evidence The PR CI selector scans `tests/ut` and `tests/e2e/pull_request`; the new broadcast/strided token-logprob numerical tests under `tests/e2e/nightly` are outside that selection. A5, standalone upstream-worker and coverage-upload jobs were skipped in the inspected run. Skips are not coverage. No dedicated packed-QK RoPE or V1 profiling-randomization standalone UT is added by this final diff. Previous head `15f8a3d` passed [E2E](https://github.com/vllm-project/vllm-ascend/actions/runs/38511018361) and [Docs](https://github.com/vllm-project/vllm-ascend/actions/runs/38511018193), including 42 NPU jobs. Its PR-recorded CPU counts were 10574 passed / 113 skipped for verified main and 10576 passed / 111 skipped for release, with no failures. Those counts belong to that historical head and are not asserted as current-head counts. - vLLM main: https://github.com/vllm-project/vllm/commit/c4df37dfcc7f72ca41cd43801a468526ecf99a2b --------- Signed-off-by: zhao-stack <80399320+zhao-stack@users.noreply.github.com> Co-authored-by: shenzhao <shenzhao9@huawei.com>

mergify Bot added speculative-decoding mrv2 Model Runner V2 specific labels Sep 23, 2026

njhill marked this pull request as ready for review September 23, 2026 20:13

njhill requested review from WoosukKwon and yewentao256 as code owners September 23, 2026 20:13

claude Bot reviewed Sep 23, 2026

View reviewed changes

njhill force-pushed the mrv2-full-cg-one-token-prefill branch from 229e1fa to 967cce3 Compare September 23, 2026 20:52

njhill requested review from NickLucche and aoshen02 September 23, 2026 20:52

njhill force-pushed the mrv2-full-cg-one-token-prefill branch from 967cce3 to 135d447 Compare September 23, 2026 23:50

njhill force-pushed the mrv2-full-cg-one-token-prefill branch from 135d447 to 0af8d3d Compare September 24, 2026 15:21

njhill mentioned this pull request Sep 24, 2026

[Bugfix][MRV2] Treat padded prompt tails as spec-decode rows for hybrid models #58434

Merged

njhill force-pushed the mrv2-full-cg-one-token-prefill branch 3 times, most recently from 0a963e0 to 1cbb155 Compare September 25, 2026 16:22

njhill force-pushed the mrv2-full-cg-one-token-prefill branch from 1cbb155 to 960f1d4 Compare September 25, 2026 18:28

njhill force-pushed the mrv2-full-cg-one-token-prefill branch from 960f1d4 to f7f69e2 Compare September 25, 2026 21:00

njhill force-pushed the mrv2-full-cg-one-token-prefill branch from f7f69e2 to d5a8e22 Compare September 25, 2026 22:18

njhill added the ready ONLY add when PR is ready to merge/full CI is needed label Sep 28, 2026

WoosukKwon approved these changes Sep 28, 2026

View reviewed changes

Merge branch 'main' into mrv2-full-cg-one-token-prefill

0fcfde9

zyongye enabled auto-merge (squash) September 28, 2026 19:36

Merge branch 'main' into mrv2-full-cg-one-token-prefill

be0f2ca

WoosukKwon disabled auto-merge September 28, 2026 19:41

WoosukKwon merged commit e7c9036 into vllm-project:main Sep 28, 2026
8 checks passed

njhill deleted the mrv2-full-cg-one-token-prefill branch September 28, 2026 19:54

NickLucche mentioned this pull request Sep 29, 2026

[Core][PD] Allow FULL decode graphs for one-token prompt continuations on MRv2 #58374

Closed

This was referenced Sep 29, 2026

[Bugfix] Bound GDN speculative decode query widths #55504

Merged

[Bugfix] Route every speculation-capable row through the speculative path so recurrent state stays consistent #56531

Merged

MatthewBonanni mentioned this pull request Oct 5, 2026

[Tracking] HiSparse bugfixes and performance improvements #60072

Open

41 of 64 tasks

atty57 mentioned this pull request Oct 8, 2026

[Bugfix][V1] Keep still-prefilling rows out of full decode graphs #60512

Open

6 tasks done

seongyun1104 mentioned this pull request Oct 10, 2026

[Bugfix][Spec Decode] Ladder cudagraph candidates per uniform query length #56136

Open

iKeybot-code mentioned this pull request Oct 12, 2026

[Misc]: Revert #17128 and #17445 once #17581 (self-padded fine-grained TP) lands vllm-project/vllm-ascend#17626

Open

linsun12 mentioned this pull request Oct 13, 2026

fix(plugin/dsv4): correct DeepSeek-V4 prefix hits, cudagraph replay and MTP on the vLLM plugin ROCm/ATOM#2502

Merged

zhangxinyuehfad mentioned this pull request Oct 14, 2026

[Misc] main2main 0930 c4df37d vllm-project/vllm-ascend#18465

Merged

zhao-stack mentioned this pull request Oct 14, 2026

[Misc][Main2Main] Upgrade release to v0.31.0 with verified main compatibility vllm-project/vllm-ascend#18229

Merged

linsun12 mentioned this pull request Oct 14, 2026

[Bugfix][ROCm][MP] mlock the registered L1 so compaction cannot migrate it LMCache/LMCache#5710

Merged

2 tasks done

Sunbelt Computer Software

PL/B Language Development and Support

Conversation

njhill commented Sep 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Plan

Uh oh!

claude Bot left a comment

Choose a reason for hiding this comment

Claude Code Review

Uh oh!

njhill commented Sep 23, 2026

Pre-PR vs Post-PR

Qwen3-0.6B, P/D TP1/TP1 (mean of 2 reps)

Llama-3-8B-Instruct, P/D TP1/TP1 (mean of 3 reps)

MRV1 vs MRV2, Qwen3-0.6B, P/D TP1/TP1 (Δ vs MRV2 before)

gsm8k (5-shot, strict / flexible)

Uh oh!

njhill commented Sep 23, 2026

Uh oh!

github-actions Bot commented Sep 23, 2026

Uh oh!

njhill commented Sep 23, 2026

Uh oh!

github-actions Bot commented Sep 23, 2026

Uh oh!

njhill commented Sep 24, 2026

Uh oh!

github-actions Bot commented Sep 24, 2026

Uh oh!

seongyun1104 commented Sep 25, 2026

Uh oh!

njhill commented Sep 25, 2026

Uh oh!

github-actions Bot commented Sep 25, 2026

Uh oh!

njhill commented Sep 25, 2026

Uh oh!

github-actions Bot commented Sep 25, 2026

Uh oh!

njhill commented Sep 25, 2026

Uh oh!

njhill commented Sep 25, 2026

Uh oh!

github-actions Bot commented Sep 25, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

njhill commented Sep 23, 2026 •

edited

Loading