{{ message }}
Repository navigation
[Bugfix] Check batch-invariance support for every attention backend - #60271
Merged
yewentao256 merged 3 commits intoOct 8, 2026
Merged
yewentao256 merged 3 commits into
yewentao256 merged 3 commits into
Conversation
With VLLM_BATCH_INVARIANT=1 the attention selector refuses backends that do not declare supports_batch_invariance(). Models that select their own attention backend bypass the selector, so they started anyway and silently returned batch-dependent output (seen with MiniMaxAI/MiniMax-M3). Apply the same check to the backend of every attention layer when the attention groups are built. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: yifanFengg <evanfengff@gmail.com>
yifanFengg
requested review from
WoosukKwon,
njhill and
yewentao256
as code owners
October 6, 2026 17:46
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
yewentao256
reviewed
Oct 7, 2026
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: yifanFengg <evanfengff@gmail.com>
Contributor
Author
yewentao256
approved these changes
Oct 7, 2026
yewentao256
left a comment
yewentao256
left a comment
Member
There was a problem hiding this comment.
LGTM, thanks for the work!
Member
|
/ci run |
|
✅ @yifanFengg, CI is now available for this PR.
|
|
❌ This PR is 101 commits behind upstream |
Member
|
/ci run |
|
✅ Triggered Buildkite CI #93433 for commit |
Collaborator
CI selector (shadow): 127 test steps (163 jobs) instead of 68 (84 jobs)Shadow mode: this changes nothing about what CI runs. It shows what the evidence-based selector would pick for this PR, next to today's rules. How it works. Feedback welcome: reply here if it would skip a step this change needs, or runs something unrelated. Selector would run (127)
Would skip (today's rules run them) (16)
Would add (today's rules do not run them) (75)
AMD mirrors: would skip (10)
AMD mirrors: would add (63)
1 changed files · base |
20 of 29 tasks
michael-gpu
pushed a commit
to practical-inference/vllm-ampere-extended
that referenced
this pull request
Oct 20, 2026
…llm-project#60271) Signed-off-by: yifanFengg <evanfengff@gmail.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
zzzzwwjj
pushed a commit
to vllm-project/vllm-ascend
that referenced
this pull request
Oct 20, 2026
### What this PR does / why we need it?
Bump the verified vLLM main pin from `c4df37df` to `a98247ab` and adapt
vllm-ascend to the upstream changes in that range. The v0.31.0 release
lane is kept intact: each adaptation is gated on
`vllm_version_is("0.31.0")` (or a version-aware helper) so both the
`main` and the `v0.31.0` CPU UT lanes stay green.
The range contains a large breaking KV-cache/attention rework plus a set
of smaller API changes:
- vllm#57169 reworked the KV cache layout: it dropped the MLA
`storage_block_size` override, added a leading `vllm_config` to
`_pool_bytes_per_block`, removed
`kernel_block_sizes`/`blocks_per_kv_block` from `BlockTables`, split
metadata-builder creation out of `initialize_attn_backend` and made
`AttentionGroup` identity-comparable.
- vllm#60335 renamed the spec-decode AR path (`autoregressive` ->
`target_dependent_ar`) and added `num_speculative_steps`/
`num_speculative_tokens` to `propose()` and the decode callbacks.
- vllm#60517 removed the torch.compile sequence-parallelism/AsyncTP
passes `PassConfig.enable_sp`/`fuse_gemm_comms`/`sp_min_token_num`,
`update_sizes_for_sequence_parallelism` and the `tensor_parallel_size`
argument of the cudagraph-size resolvers.
- vllm#60754 renamed the DP-sync state (`dp_sync` -> `dp_sync_state`)
and added `ExecuteModelState.num_spec_tokens_to_schedule`.
- vllm#58411 added `randomize_inputs` to `profile_run`/`_dummy_run`.
- vllm#60271 added a hard batch-invariance support check for every
attention backend.
- vllm#59327 moved the DeepSeek-V4.1 Engram module to `common.engram`.
- vllm#57387 moved the GLM-5-Next config classes into `transformers`
(hence the `transformers` 5.14.1 -> 5.19.0 bump, which main requires
anyway).
- Smaller seams: vllm#59175 added `prefill_end` to KV allocation,
vllm#53175 forwards the layer's spec to
`get_supported_kernel_block_sizes`, vllm#58067 passes `separators` to
xgrammar, vllm#53934 made `CudaGraphManager.dp_size` a property,
vllm#56723 builds draft metadata through the attention group, vllm#57934
makes MRV1 reject DFlash candidate-head drafts.
| Files | Upstream vLLM change (why it breaks vllm-ascend) | vllm-ascend
adaptation |
|-------|--------------------------------------------------|------------------------|
| `.github/vllm-main-verified.commit` | — (main-lane pin only) |
Verified main commit bumped `c4df37df` -> `a98247ab` |
| `pyproject.toml` | vLLM main requires `transformers >= 5.16.1` | Build
requirement bumped `transformers==5.14.1` -> `5.19.0` (satisfies both
lanes) |
| `requirements.txt` | Same transformers floor | Runtime requirement
bumped `transformers==5.14.1` -> `5.19.0` |
| `tests/deepseek_v41_utils.py` |
[vllm#57169](vllm-project/vllm#57169) removed
MLA `storage_block_size` | Gate the `storage_block_size` kwarg of the
DSV4 k-cache and indexer k-cache specs on `vllm_version_is("0.31.0")` |
| `tests/e2e/pull_request/four_card/glm53flash_assets/config.json` |
[vllm#57387](vllm-project/vllm#57387) serves
GLM-5-Next configs from `transformers` 5.19.0, which validates every
field type | Write `swiglu_limit: 10.0` and `gate_lower_bound: -5.0` as
floats |
| `tests/e2e/pull_request/four_card/glm53flash_assets/multimodal.json` |
Same strict dataclass validation | Write `vision_config.swiglu_limit:
10` -> `10.0` |
| `tests/e2e/pull_request/two_card/spec_decode/test_spec_decode.py` |
[vllm#57934](vllm-project/vllm#57934): MRV1
rejects DFlash candidate-head drafts on main | Make the
`test_dflash2_acceptance` skip unconditional instead of main-only |
| `tests/ut/attention/test_batch_invariance_support.py` |
[vllm#60271](vllm-project/vllm#60271) added the
batch-invariance support check | New regression test asserting
`supports_batch_invariance()` on all Ascend backends and the 310P/C8MXFP
subclasses |
| `tests/ut/compilation/test_graph_fusion_pass_manager.py` |
[vllm#60517](vllm-project/vllm#60517) removed
`PassConfig.fuse_gemm_comms` | Only seed the PassConfig flag on v0.31.0
and patch `enable_sp` for the main lane gate |
| `tests/ut/core/test_kv_cache_interface.py` |
[vllm#57169](vllm-project/vllm#57169) removed
`MLAAttentionSpec.storage_block_size` | Add a plain-MLA-subclass case
(`HiddenStateCacheSpec`) resolving `get_storage_block_size()` per lane |
| `tests/ut/distributed/kv_transfer/kv_p2p/test_mooncake_connector.py` |
vLLM main's scheduler calls `get_loaded_kv_cache_group_ids`, backed by
`KVConnectorBase_V1._kv_cache_config` | Seed
`MockKVCacheConfig.transfer_group_ids` and add the main-only
scheduler-hook test (skipped on v0.31.0) |
| `tests/ut/kv_offload/test_mooncake_hybrid_connector.py` | Same new
`get_loaded_kv_cache_group_ids` scheduler hook | Assert the hook is
declared on the class only on main |
| `tests/ut/kv_offload/test_preempt_offload_connector.py` | vLLM main's
remote-KV restore consults `scheduler.prefix_replay_group_ids` | Seed
`prefix_replay_group_ids = ()` in the `__new__`-based scheduler fixture
|
| `tests/ut/models/test_deepseek_v41_cache.py` |
[vllm#57169](vllm-project/vllm#57169) added the
leading `vllm_config` to `_pool_bytes_per_block` and removed MLA
`storage_block_size` | Add a `_pool_bytes_per_block(runtime, groups)`
lane helper, read specs through `get_storage_block_size()` and gate the
one explicit `storage_block_size` kwarg |
| `tests/ut/models/test_engram.py` |
[vllm#59327](vllm-project/vllm#59327) deleted
`deepseek_v41/nvidia/engram.py` and moved it to `common.engram` | Import
`upstream_engram` by lane so collection no longer aborts with
`ImportError` on main |
| `tests/ut/models/test_glm5_next_mtp.py` |
[vllm#57387](vllm-project/vllm#57387) removed
`vllm.transformers_utils.configs.glm5_next` | Import
`Glm5NextTextConfig` from `transformers` on main, keep the vLLM path on
v0.31.0 |
| `tests/ut/ops/test_gdn_attn_builder.py` | Main's GDN metadata builder
resolves the prefill backend from `model_config.hf_text_config` | Give
the `model_config` fixture an `hf_text_config` |
| `tests/ut/patch/platform/test_patch_structured_output.py` | Main gates
the guidance backend on `GuidanceBackend.is_tokenizer_supported` |
Monkeypatch that check to accept the dummy tokenizer on main |
| `tests/ut/patch/platform/test_patch_use_v2_model_runner.py` |
[vllm#60517](vllm-project/vllm#60517) dropped
the `PassConfig.enable_sp` guard, so the V2 runner no longer rejects SP
| Expect `["sequence parallelism"]` only on v0.31.0 |
| `tests/ut/patch/platform/test_prefix_cache_cp_patches.py` |
[vllm#57169](vllm-project/vllm#57169) resolves
the KV cache layout when computing pool bytes per block | Pass
`vllm_config` to the Ascend/upstream `_pool_bytes_per_block` on main and
seed `get_resolved_kv_cache_layout` |
| `tests/ut/spec_decode/test_dflash2_v2.py` |
[vllm#57169](vllm-project/vllm#57169) forwards
the physical block size to the DFlash caller;
[vllm#60754](vllm-project/vllm#60754) renamed
`dp_sync` | Select the `dp_sync`/`dp_sync_state` keyword per lane and
add the main-only kernel-block-size restore test |
| `tests/ut/spec_decode/test_eagle_proposer.py` |
[vllm#60517](vllm-project/vllm#60517) removed
`PassConfig.enable_sp` | Assert `enable_sp` is a PassConfig field only
on v0.31.0 |
| `tests/ut/spec_decode/test_mla_dspark.py` |
[vllm#60754](vllm-project/vllm#60754) renamed
`dp_sync` -> `dp_sync_state` | Pass the lane-specific keyword into
`DSparkSpeculator.propose` |
| `tests/ut/test_platform.py` |
[vllm#60517](vllm-project/vllm#60517) removed
the PassConfig SP fields and `update_sizes_for_sequence_parallelism` |
Assert the SP threshold default only on v0.31.0 and that the removed
method is no longer called on main |
| `tests/ut/worker/test_adaptive_verification.py` |
[vllm#60754](vllm-project/vllm#60754) renamed
the DP-sync argument | Build the execute-model kwargs with
`dp_sync`/`dp_sync_state` per lane |
| `tests/ut/worker/test_attn_utils_v2.py` | vLLM main made
`AttentionGroup` identity-comparable | Compare the preserved fields
(backend, layer names, spec, group id) on main instead of `==` |
| `tests/ut/worker/test_dcp_speculator_metadata_v2.py` |
[vllm#60335](vllm-project/vllm#60335) renamed
the AR speculator module;
[vllm#56723](vllm-project/vllm#56723) builds
draft metadata through the attention group | Import
`TargetDependentARSpeculator` on main and add a recording attention
group exposing `build_metadata` |
| `tests/ut/worker/test_model_runner_v1.py` |
[vllm#57169](vllm-project/vllm#57169) removed
`BlockTables.kernel_block_sizes`; main module lacks
`mamba_utils.postprocess_mamba_all` | Pass `kernel_block_sizes` only on
v0.31.0 and patch with `create=True` |
| `tests/ut/worker/test_model_runner_v2.py` |
[vllm#60517](https://github.com/vllm-project/vllm/pull/60517)/[vllm#58411](https://github.com/vllm-project/vllm/pull/58411):
`max_seq_len_np` and `randomize_inputs` exist on both lanes now | Drop
the per-lane gating and always supply `max_seq_len_np` / expect
`randomize_inputs` |
| `tests/ut/worker/test_model_runner_v2_mamba.py` | vLLM main removed
`MambaHybridModelState._ensure_align_ctx` | Seed `_mamba_ctx` on main
and add two main-only `_ensure_align_ctx` regression tests |
| `tests/ut/worker/test_mtp_pcp_speculator_v2.py` |
[vllm#60335](vllm-project/vllm#60335) renamed
the AR base class;
[vllm#60754](vllm-project/vllm#60754) renamed
`dp_sync` | Import the base class by lane and patch
`_UpstreamAutoRegressiveSpeculator` via the lane-specific keyword |
| `tests/ut/worker/test_pcp_manager_v2.py` | Main declares
`ParallelConfig.pcp_shard_decode_requests`, renamed `dp_sync` and added
`num_spec_tokens_to_schedule` | Assert the field per lane, build
`ExecuteModelState` with the lane keyword and seed
`num_spec_tokens_to_schedule` |
| `tests/ut/worker/test_v2_dummy_padding.py` |
[vllm#60754](vllm-project/vllm#60754) renamed
`dp_sync`; main's dummy context reads `block_tables.block_sizes` | Pass
the lane keyword and provide both `block_sizes` and `kernel_block_sizes`
|
| `tests/ut/worker/test_worker_v1.py` |
[vllm#58411](vllm-project/vllm#58411):
`determine_available_memory` now forwards
`profile_run(randomize_inputs=...)` | Update the stale
`assert_called_once_with()` to expect `randomize_inputs=False` |
| `tests/ut/worker/v2/test_aclgraph.py` |
[vllm#60335](vllm-project/vllm#60335) moved
`SpeculatorCudaGraphManager` to `target_dependent_ar`;
[vllm#53934](vllm-project/vllm#53934) made
`dp_size` a read-only property | Import the manager by lane, back
`dp_size` with `vllm_config` on main and expect the extra
`num_speculative_steps` capture arg |
| `tests/ut/worker/v2/test_speculator.py` |
[vllm#60335](vllm-project/vllm#60335) renamed
the AR speculator and added `num_speculative_steps`;
[vllm#60754](vllm-project/vllm#60754) renamed
`dp_sync` | Import by lane, thread `num_speculative_steps` through the
`_generate_draft`/`_multi_step_decode` assertions and add the main
`dp_sync_state` + `num_speculative_tokens` regression test |
| `tests/ut/worker/v2/test_v2_block_table.py` |
[vllm#57169](vllm-project/vllm#57169) dropped
`kernel_block_sizes` from `BlockTables` | Make the stand-in parent
init's kernel sizes optional and assert the kernel tensor is built by
`AscendBlockTables` on main |
| `vllm_ascend/_310p/sharded_state_loader_310p.py` | vLLM main split
`ShardedStateLoader.DEFAULT_PATTERN` into
`TP_ONLY_PATTERN`/`PP_AND_TP_PATTERN` | Pick the lane's default pattern
name instead of the removed combined one |
| `vllm_ascend/_310p/worker/v2/model_runner.py` |
[vllm#60517](vllm-project/vllm#60517) removed
the `tensor_parallel_size` argument of
`resolve_cudagraph_mode_and_sizes` | Forward `tensor_parallel_size` only
on v0.31.0 |
| `vllm_ascend/_310p/worker/v2/spec_decode/mtp_speculator.py` |
[vllm#60335](vllm-project/vllm#60335) renamed
the AR module and added `num_speculative_steps` to
`_generate_draft`/`_multi_step_decode` | Version-split the two overrides
and the AR import, falling back to `self.num_speculative_steps` when the
capture path omits it |
| `vllm_ascend/ascend_config.py` |
[vllm#60517](vllm-project/vllm#60517) removed
`PassConfig.enable_sp` | Read the flag only on v0.31.0, so
`enable_sp_by_pass` is False (no pass-based SP) on main |
| `vllm_ascend/attention/attention_v1.py` |
[vllm#60271](vllm-project/vllm#60271) rejects
backends that do not support batch invariance | Declare
`supports_batch_invariance()` on main (`not vllm_version_is("0.31.0")`)
|
| `vllm_ascend/attention/dsa_v1.py` |
[vllm#60271](vllm-project/vllm#60271)
batch-invariance check;
[vllm#53175](vllm-project/vllm#53175) forwards
the layer's `KVCacheSpec` into `get_supported_kernel_block_sizes`, so a
rebase had overwritten the six DSA backends back to the zero-arg form
and main raised `TypeError: ... takes 0 positional arguments but 1 was
given` | Re-apply `supports_batch_invariance()` and give all six DSA
backends the per-lane
`get_supported_kernel_block_sizes(kv_cache_spec=None)` signature |
| `vllm_ascend/attention/dsa_v41.py` |
[vllm#60271](vllm-project/vllm#60271)
batch-invariance check | Declare `supports_batch_invariance()` on
`DeepseekV41CacheBackend` for main |
| `vllm_ascend/attention/fa3_v1.py` | Same | Declare
`supports_batch_invariance()` on `AscendFABackend` for main |
| `vllm_ascend/attention/indexer.py` | Same | Declare
`supports_batch_invariance()` on `AscendSFAIndexerBackend` for main |
| `vllm_ascend/attention/indexer_kpool.py` | Same | Declare
`supports_batch_invariance()` on the indexer k-pool and tail backends
for main |
| `vllm_ascend/attention/mla_v1.py` | Same | Declare
`supports_batch_invariance()` on `AscendMLABackend` for main |
| `vllm_ascend/attention/sfa_v1.py` | Same | Declare
`supports_batch_invariance()` on `AscendSFABackend` for main |
| `vllm_ascend/compilation/graph_fusion_pass_manager.py` |
[vllm#60517](vllm-project/vllm#60517) removed
`PassConfig.fuse_gemm_comms` | Gate the MMRS fusion on
`enable_sp(config)` on main, keep reading the PassConfig flag on v0.31.0
|
| `vllm_ascend/core/kv_cache_interface.py` |
[vllm#57169](vllm-project/vllm#57169) removed
the MLA `storage_block_size` override, so a plain MLA subclass crashed
with `AttributeError: 'HiddenStateCacheSpec' object has no attribute
'storage_block_size'` | Read the optional override only on v0.31.0 and
fall back to `block_size` on main |
| `vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_connector.py` |
vLLM main's scheduler calls `get_loaded_kv_cache_group_ids`, which reads
`_kv_cache_config`; the Ascend connector overrode `__init__` without
calling the base and never set it (`AttributeError: 'MooncakeConnector'
object has no attribute '_kv_cache_config'`) | Call
`KVConnectorBase_V1.__init__` on main and declare the main-only
`get_loaded_kv_cache_group_ids` hook |
|
`vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_hybrid_connector.py`
| Same | Same base-init binding and main-only hook |
| `vllm_ascend/models/deepseek_v41/cache_config.py` |
[vllm#57169](vllm-project/vllm#57169) removed
MLA `storage_block_size` | Gate the kwarg in
`make_mla_cache_spec`/`make_index_cache_spec`/`make_folded_index_cache_spec`
on v0.31.0 |
| `vllm_ascend/models/deepseek_v41/engram/parallel.py` |
[vllm#59327](vllm-project/vllm#59327) moved
`engram_head_shard_rank` from `nvidia.engram` to `common.engram` |
Import `engram_head_shard_rank` by lane |
| `vllm_ascend/models/glm5next/attention.py` |
[vllm#57387](vllm-project/vllm#57387) removed
`vllm.transformers_utils.configs.glm5_next`, making the direct import
unresolvable (mypy `import-not-found`) | Import `Glm5NextConfig` from
`transformers` on main, keep the vLLM path on v0.31.0 |
| `vllm_ascend/models/glm5next/kda.py` | Same | Same lane-gated
`Glm5NextConfig` import |
| `vllm_ascend/models/glm5next/model.py` | Same | Same lane-gated
`Glm5NextConfig` import |
| `vllm_ascend/models/kimi_k3.py` | vLLM main made `out_hidden_size` a
required positional argument of `KimiK25MultiModalProjector`
(`TypeError: ... missing 1 required positional argument:
'out_hidden_size'`) | Pass `out_hidden_size = self.hidden_size` on main
only |
| `vllm_ascend/models/minimax_m3/msa_m3.py` |
[vllm#60271](vllm-project/vllm#60271)
batch-invariance check | Declare `supports_batch_invariance()` on the
MiniMax-M3 sparse/indexer backends for main |
| `vllm_ascend/patch/__init__.py` |
[vllm#58067](vllm-project/vllm#58067) xgrammar
`separators`; stale v0.30.0 boundary wording | Document the new xgrammar
patch (why/how/future plan) and rename the boundary to v0.31.0 |
| `vllm_ascend/patch/platform/__init__.py` | Stale release-lane wording
in the Engram adapter comment | `v0.30.0+` -> `v0.31.0+` |
| `vllm_ascend/patch/platform/patch_balance_schedule.py` | Stale
release-lane wording in the block-state snapshot comment | `vLLM
main/v0.30.0` -> `vLLM main/v0.31.0` |
| `vllm_ascend/patch/platform/patch_kv_cache_utils.py` |
[vllm#57169](vllm-project/vllm#57169) added a
leading `vllm_config` argument to `_pool_bytes_per_block` | Define
`_ascend_pool_bytes_per_block` per lane, forwarding `vllm_config` to the
upstream fallback on main |
| `vllm_ascend/patch/platform/patch_mamba_manager.py` |
[vllm#59175](vllm-project/vllm#59175) added
`prefill_end` to `get_num_blocks_to_allocate` | Always forward
`prefill_end` and drop the now-redundant lane gate |
| `vllm_ascend/patch/platform/patch_structured_output.py` |
[vllm#58067](vllm-project/vllm#58067) passes
`separators=(",", ":")` to `GrammarCompiler.compile_json_schema`, which
the shipped xgrammar (0.2.7/0.2.8) mishandles (truncated, invalid JSON
in guided decoding) | Wrap `compile_json_schema` to force
`separators=None` on main only, until xgrammar >= 0.2.9 is pinned |
| `vllm_ascend/patch/worker/__init__.py` |
[vllm#60335](vllm-project/vllm#60335) renamed
the AR speculator module | Import the AR module by lane before the 310P
manager override |
| `vllm_ascend/patch/worker/patch_v2/patch_eagle_speculator.py` | Same
AR module rename | Bind `vllm_speculator_module` from
`target_dependent_ar` on main, `autoregressive` on v0.31.0 |
| `vllm_ascend/patch/worker/patch_v2/patch_model_runner.py` |
[vllm#60517](vllm-project/vllm#60517) removed
the `tensor_parallel_size` argument | Forward it only on v0.31.0 |
| `vllm_ascend/patch/worker/patch_v2/patch_spec_decode_310.py` | Same AR
module rename | Import the AR module by lane before patching the CPU
input helpers |
| `vllm_ascend/platform.py` |
[vllm#60517](vllm-project/vllm#60517) removed
`PassConfig.enable_sp`/`fuse_gemm_comms`/`sp_min_token_num` and
`update_sizes_for_sequence_parallelism` | Gate the SP threshold default
on v0.31.0 and inline the tp-multiple filtering on main |
| `vllm_ascend/worker/model_runner_v1.py` |
[vllm#58411](vllm-project/vllm#58411)
`randomize_inputs`;
[vllm#57169](vllm-project/vllm#57169) split
metadata-builder creation and changed `AttentionGroup`;
[vllm#60517](vllm-project/vllm#60517) removed
the SP pass and the `tensor_parallel_size` argument | Forward
`randomize_inputs` unconditionally, build `AttentionGroup` without
positional builders and call
`initialize_metadata_builders`/`_init_device_metadata_providers` on
main, inline the SP-aware lcm, resolve cudagraph sizes without
`tensor_parallel_size`, treat `update_pass_config` as a no-op on main
and collapse 310P drafter kernel block sizes to an int |
| `vllm_ascend/worker/v2/block_table.py` |
[vllm#57169](vllm-project/vllm#57169) dropped
`kernel_block_sizes`/`blocks_per_kv_block` from `BlockTables` and
stopped expanding block ids | Version-split `__init__` to restore the
kernel-block layout (sizes, `blocks_per_kv_block`, expanded rows,
kernel-size tensor) and override `append_block_ids` on main |
| `vllm_ascend/worker/v2/model_runner.py` |
[vllm#58411](vllm-project/vllm#58411)
`randomize_inputs`; main's rejection sampler takes invalidation state
and needs the new `InputBatch` fields | Forward `randomize_inputs` and
`max_seq_len_np` unconditionally, skip grammar application on empty
logits and pass `grammar_invalid_drafts` output into the main rejection
sampler |
| `vllm_ascend/worker/v2/model_states/mamba_hybrid.py` | vLLM main
removed `MambaHybridModelState._ensure_align_ctx` and moved
align-context creation into `initialize_kv_cache`, so the first forward
aborted with `AttributeError: ... has no attribute '_ensure_align_ctx'`
/ `AssertionError` when the context was absent | Define
`_ensure_align_ctx` per lane: delegate on v0.31.0, lazily (re)create the
align context on main from the forward context |
| `vllm_ascend/worker/v2/pcp_manager.py` | vLLM main made
`shard_decode_requests` a required `PCPManager.__init__` argument and
`maybe_build_pcp_manager` passes it (`TypeError: ... unexpected keyword
argument 'shard_decode_requests'`) | Version-split
`AscendPCPManager.__init__`, accepting and forwarding the flag on main
while keeping the Ascend buffer setup shared |
| `vllm_ascend/worker/v2/spec_decode/autoregressive/aclgraph.py` |
[vllm#60335](vllm-project/vllm#60335) moved the
speculator CUDA-graph manager and added `num_speculative_steps` to the
captured forward | Import `SpeculatorCudaGraphManager` by lane and pass
`num_speculative_steps` on main |
| `vllm_ascend/worker/v2/spec_decode/autoregressive/speculator.py` |
[vllm#60335](vllm-project/vllm#60335) renamed
the AR base class and added `num_speculative_steps`;
[vllm#60754](vllm-project/vllm#60754) renamed
`dp_sync` -> `dp_sync_state` (`TypeError:
AscendAutoRegressiveSpeculator.propose() got an unexpected keyword
argument 'dp_sync_state'`) | Bind the upstream base class by lane and
version-split `propose()`/`_propose_impl()` plus
`_generate_draft`/`_multi_step_decode` so each lane forwards exactly its
arguments |
| `vllm_ascend/worker/v2/spec_decode/dflash/speculator.py` |
[vllm#57169](vllm-project/vllm#57169) makes the
DFlash caller forward the physical block size while the Ascend block
table stays kernel-block addressed (draft KV mismapped, every draft
rejected);
[vllm#60335](https://github.com/vllm-project/vllm/pull/60335)/[vllm#60754](https://github.com/vllm-project/vllm/pull/60754)
speculator contract | Version-split `prepare_dflash_inputs_factory`
(restoring the kernel block size per group on main) and
`propose()`/`_propose_impl()`, keeping the profiling DP-count guard |
| `vllm_ascend/worker/v2/spec_decode/dspark/speculator.py` | Same
speculator contract changes | Same version-split
`propose()`/`_propose_impl()` adaptation |
| `vllm_ascend/worker/worker.py` |
[vllm#58411](vllm-project/vllm#58411)
`profile_run` takes `randomize_inputs` | Compute `randomize_inputs` from
the env flag + DP size and forward it on both `profile_run` calls |
### Does this PR introduce _any_ user-facing change?
No. This is a CI/main2main compatibility update: behavior on vLLM main
is unchanged (the restorations only bring the main lane back to the
v0.31.0 behavior) and the v0.31.0 release lane is preserved.
### How was this patch tested?
- Unit tests updated per lane; `pytest -sv tests/ut` against both the
verified
main commit (`a98247ab`) and the release tag (`v0.31.0`).
- vLLM main:
vllm-project/vllm@c4df37d
---------
Signed-off-by: hfadzxy <starmoon_zhang@163.com>
Signed-off-by: zhangxinyuehfad <starmoon_zhang@163.com>
tangdafu
pushed a commit
to tangdafu/vllm-ascend
that referenced
this pull request
Oct 20, 2026
### What this PR does / why we need it?
Bump the verified vLLM main pin from `c4df37df` to `a98247ab` and adapt
vllm-ascend to the upstream changes in that range. The v0.31.0 release
lane is kept intact: each adaptation is gated on
`vllm_version_is("0.31.0")` (or a version-aware helper) so both the
`main` and the `v0.31.0` CPU UT lanes stay green.
The range contains a large breaking KV-cache/attention rework plus a set
of smaller API changes:
- vllm#57169 reworked the KV cache layout: it dropped the MLA
`storage_block_size` override, added a leading `vllm_config` to
`_pool_bytes_per_block`, removed
`kernel_block_sizes`/`blocks_per_kv_block` from `BlockTables`, split
metadata-builder creation out of `initialize_attn_backend` and made
`AttentionGroup` identity-comparable.
- vllm#60335 renamed the spec-decode AR path (`autoregressive` ->
`target_dependent_ar`) and added `num_speculative_steps`/
`num_speculative_tokens` to `propose()` and the decode callbacks.
- vllm#60517 removed the torch.compile sequence-parallelism/AsyncTP
passes `PassConfig.enable_sp`/`fuse_gemm_comms`/`sp_min_token_num`,
`update_sizes_for_sequence_parallelism` and the `tensor_parallel_size`
argument of the cudagraph-size resolvers.
- vllm#60754 renamed the DP-sync state (`dp_sync` -> `dp_sync_state`)
and added `ExecuteModelState.num_spec_tokens_to_schedule`.
- vllm#58411 added `randomize_inputs` to `profile_run`/`_dummy_run`.
- vllm#60271 added a hard batch-invariance support check for every
attention backend.
- vllm#59327 moved the DeepSeek-V4.1 Engram module to `common.engram`.
- vllm#57387 moved the GLM-5-Next config classes into `transformers`
(hence the `transformers` 5.14.1 -> 5.19.0 bump, which main requires
anyway).
- Smaller seams: vllm#59175 added `prefill_end` to KV allocation,
vllm#53175 forwards the layer's spec to
`get_supported_kernel_block_sizes`, vllm#58067 passes `separators` to
xgrammar, vllm#53934 made `CudaGraphManager.dp_size` a property,
vllm#56723 builds draft metadata through the attention group, vllm#57934
makes MRV1 reject DFlash candidate-head drafts.
| Files | Upstream vLLM change (why it breaks vllm-ascend) | vllm-ascend
adaptation |
|-------|--------------------------------------------------|------------------------|
| `.github/vllm-main-verified.commit` | — (main-lane pin only) |
Verified main commit bumped `c4df37df` -> `a98247ab` |
| `pyproject.toml` | vLLM main requires `transformers >= 5.16.1` | Build
requirement bumped `transformers==5.14.1` -> `5.19.0` (satisfies both
lanes) |
| `requirements.txt` | Same transformers floor | Runtime requirement
bumped `transformers==5.14.1` -> `5.19.0` |
| `tests/deepseek_v41_utils.py` |
[vllm#57169](vllm-project/vllm#57169) removed
MLA `storage_block_size` | Gate the `storage_block_size` kwarg of the
DSV4 k-cache and indexer k-cache specs on `vllm_version_is("0.31.0")` |
| `tests/e2e/pull_request/four_card/glm53flash_assets/config.json` |
[vllm#57387](vllm-project/vllm#57387) serves
GLM-5-Next configs from `transformers` 5.19.0, which validates every
field type | Write `swiglu_limit: 10.0` and `gate_lower_bound: -5.0` as
floats |
| `tests/e2e/pull_request/four_card/glm53flash_assets/multimodal.json` |
Same strict dataclass validation | Write `vision_config.swiglu_limit:
10` -> `10.0` |
| `tests/e2e/pull_request/two_card/spec_decode/test_spec_decode.py` |
[vllm#57934](vllm-project/vllm#57934): MRV1
rejects DFlash candidate-head drafts on main | Make the
`test_dflash2_acceptance` skip unconditional instead of main-only |
| `tests/ut/attention/test_batch_invariance_support.py` |
[vllm#60271](vllm-project/vllm#60271) added the
batch-invariance support check | New regression test asserting
`supports_batch_invariance()` on all Ascend backends and the 310P/C8MXFP
subclasses |
| `tests/ut/compilation/test_graph_fusion_pass_manager.py` |
[vllm#60517](vllm-project/vllm#60517) removed
`PassConfig.fuse_gemm_comms` | Only seed the PassConfig flag on v0.31.0
and patch `enable_sp` for the main lane gate |
| `tests/ut/core/test_kv_cache_interface.py` |
[vllm#57169](vllm-project/vllm#57169) removed
`MLAAttentionSpec.storage_block_size` | Add a plain-MLA-subclass case
(`HiddenStateCacheSpec`) resolving `get_storage_block_size()` per lane |
| `tests/ut/distributed/kv_transfer/kv_p2p/test_mooncake_connector.py` |
vLLM main's scheduler calls `get_loaded_kv_cache_group_ids`, backed by
`KVConnectorBase_V1._kv_cache_config` | Seed
`MockKVCacheConfig.transfer_group_ids` and add the main-only
scheduler-hook test (skipped on v0.31.0) |
| `tests/ut/kv_offload/test_mooncake_hybrid_connector.py` | Same new
`get_loaded_kv_cache_group_ids` scheduler hook | Assert the hook is
declared on the class only on main |
| `tests/ut/kv_offload/test_preempt_offload_connector.py` | vLLM main's
remote-KV restore consults `scheduler.prefix_replay_group_ids` | Seed
`prefix_replay_group_ids = ()` in the `__new__`-based scheduler fixture
|
| `tests/ut/models/test_deepseek_v41_cache.py` |
[vllm#57169](vllm-project/vllm#57169) added the
leading `vllm_config` to `_pool_bytes_per_block` and removed MLA
`storage_block_size` | Add a `_pool_bytes_per_block(runtime, groups)`
lane helper, read specs through `get_storage_block_size()` and gate the
one explicit `storage_block_size` kwarg |
| `tests/ut/models/test_engram.py` |
[vllm#59327](vllm-project/vllm#59327) deleted
`deepseek_v41/nvidia/engram.py` and moved it to `common.engram` | Import
`upstream_engram` by lane so collection no longer aborts with
`ImportError` on main |
| `tests/ut/models/test_glm5_next_mtp.py` |
[vllm#57387](vllm-project/vllm#57387) removed
`vllm.transformers_utils.configs.glm5_next` | Import
`Glm5NextTextConfig` from `transformers` on main, keep the vLLM path on
v0.31.0 |
| `tests/ut/ops/test_gdn_attn_builder.py` | Main's GDN metadata builder
resolves the prefill backend from `model_config.hf_text_config` | Give
the `model_config` fixture an `hf_text_config` |
| `tests/ut/patch/platform/test_patch_structured_output.py` | Main gates
the guidance backend on `GuidanceBackend.is_tokenizer_supported` |
Monkeypatch that check to accept the dummy tokenizer on main |
| `tests/ut/patch/platform/test_patch_use_v2_model_runner.py` |
[vllm#60517](vllm-project/vllm#60517) dropped
the `PassConfig.enable_sp` guard, so the V2 runner no longer rejects SP
| Expect `["sequence parallelism"]` only on v0.31.0 |
| `tests/ut/patch/platform/test_prefix_cache_cp_patches.py` |
[vllm#57169](vllm-project/vllm#57169) resolves
the KV cache layout when computing pool bytes per block | Pass
`vllm_config` to the Ascend/upstream `_pool_bytes_per_block` on main and
seed `get_resolved_kv_cache_layout` |
| `tests/ut/spec_decode/test_dflash2_v2.py` |
[vllm#57169](vllm-project/vllm#57169) forwards
the physical block size to the DFlash caller;
[vllm#60754](vllm-project/vllm#60754) renamed
`dp_sync` | Select the `dp_sync`/`dp_sync_state` keyword per lane and
add the main-only kernel-block-size restore test |
| `tests/ut/spec_decode/test_eagle_proposer.py` |
[vllm#60517](vllm-project/vllm#60517) removed
`PassConfig.enable_sp` | Assert `enable_sp` is a PassConfig field only
on v0.31.0 |
| `tests/ut/spec_decode/test_mla_dspark.py` |
[vllm#60754](vllm-project/vllm#60754) renamed
`dp_sync` -> `dp_sync_state` | Pass the lane-specific keyword into
`DSparkSpeculator.propose` |
| `tests/ut/test_platform.py` |
[vllm#60517](vllm-project/vllm#60517) removed
the PassConfig SP fields and `update_sizes_for_sequence_parallelism` |
Assert the SP threshold default only on v0.31.0 and that the removed
method is no longer called on main |
| `tests/ut/worker/test_adaptive_verification.py` |
[vllm#60754](vllm-project/vllm#60754) renamed
the DP-sync argument | Build the execute-model kwargs with
`dp_sync`/`dp_sync_state` per lane |
| `tests/ut/worker/test_attn_utils_v2.py` | vLLM main made
`AttentionGroup` identity-comparable | Compare the preserved fields
(backend, layer names, spec, group id) on main instead of `==` |
| `tests/ut/worker/test_dcp_speculator_metadata_v2.py` |
[vllm#60335](vllm-project/vllm#60335) renamed
the AR speculator module;
[vllm#56723](vllm-project/vllm#56723) builds
draft metadata through the attention group | Import
`TargetDependentARSpeculator` on main and add a recording attention
group exposing `build_metadata` |
| `tests/ut/worker/test_model_runner_v1.py` |
[vllm#57169](vllm-project/vllm#57169) removed
`BlockTables.kernel_block_sizes`; main module lacks
`mamba_utils.postprocess_mamba_all` | Pass `kernel_block_sizes` only on
v0.31.0 and patch with `create=True` |
| `tests/ut/worker/test_model_runner_v2.py` |
[vllm#60517](https://github.com/vllm-project/vllm/pull/60517)/[vllm#58411](https://github.com/vllm-project/vllm/pull/58411):
`max_seq_len_np` and `randomize_inputs` exist on both lanes now | Drop
the per-lane gating and always supply `max_seq_len_np` / expect
`randomize_inputs` |
| `tests/ut/worker/test_model_runner_v2_mamba.py` | vLLM main removed
`MambaHybridModelState._ensure_align_ctx` | Seed `_mamba_ctx` on main
and add two main-only `_ensure_align_ctx` regression tests |
| `tests/ut/worker/test_mtp_pcp_speculator_v2.py` |
[vllm#60335](vllm-project/vllm#60335) renamed
the AR base class;
[vllm#60754](vllm-project/vllm#60754) renamed
`dp_sync` | Import the base class by lane and patch
`_UpstreamAutoRegressiveSpeculator` via the lane-specific keyword |
| `tests/ut/worker/test_pcp_manager_v2.py` | Main declares
`ParallelConfig.pcp_shard_decode_requests`, renamed `dp_sync` and added
`num_spec_tokens_to_schedule` | Assert the field per lane, build
`ExecuteModelState` with the lane keyword and seed
`num_spec_tokens_to_schedule` |
| `tests/ut/worker/test_v2_dummy_padding.py` |
[vllm#60754](vllm-project/vllm#60754) renamed
`dp_sync`; main's dummy context reads `block_tables.block_sizes` | Pass
the lane keyword and provide both `block_sizes` and `kernel_block_sizes`
|
| `tests/ut/worker/test_worker_v1.py` |
[vllm#58411](vllm-project/vllm#58411):
`determine_available_memory` now forwards
`profile_run(randomize_inputs=...)` | Update the stale
`assert_called_once_with()` to expect `randomize_inputs=False` |
| `tests/ut/worker/v2/test_aclgraph.py` |
[vllm#60335](vllm-project/vllm#60335) moved
`SpeculatorCudaGraphManager` to `target_dependent_ar`;
[vllm#53934](vllm-project/vllm#53934) made
`dp_size` a read-only property | Import the manager by lane, back
`dp_size` with `vllm_config` on main and expect the extra
`num_speculative_steps` capture arg |
| `tests/ut/worker/v2/test_speculator.py` |
[vllm#60335](vllm-project/vllm#60335) renamed
the AR speculator and added `num_speculative_steps`;
[vllm#60754](vllm-project/vllm#60754) renamed
`dp_sync` | Import by lane, thread `num_speculative_steps` through the
`_generate_draft`/`_multi_step_decode` assertions and add the main
`dp_sync_state` + `num_speculative_tokens` regression test |
| `tests/ut/worker/v2/test_v2_block_table.py` |
[vllm#57169](vllm-project/vllm#57169) dropped
`kernel_block_sizes` from `BlockTables` | Make the stand-in parent
init's kernel sizes optional and assert the kernel tensor is built by
`AscendBlockTables` on main |
| `vllm_ascend/_310p/sharded_state_loader_310p.py` | vLLM main split
`ShardedStateLoader.DEFAULT_PATTERN` into
`TP_ONLY_PATTERN`/`PP_AND_TP_PATTERN` | Pick the lane's default pattern
name instead of the removed combined one |
| `vllm_ascend/_310p/worker/v2/model_runner.py` |
[vllm#60517](vllm-project/vllm#60517) removed
the `tensor_parallel_size` argument of
`resolve_cudagraph_mode_and_sizes` | Forward `tensor_parallel_size` only
on v0.31.0 |
| `vllm_ascend/_310p/worker/v2/spec_decode/mtp_speculator.py` |
[vllm#60335](vllm-project/vllm#60335) renamed
the AR module and added `num_speculative_steps` to
`_generate_draft`/`_multi_step_decode` | Version-split the two overrides
and the AR import, falling back to `self.num_speculative_steps` when the
capture path omits it |
| `vllm_ascend/ascend_config.py` |
[vllm#60517](vllm-project/vllm#60517) removed
`PassConfig.enable_sp` | Read the flag only on v0.31.0, so
`enable_sp_by_pass` is False (no pass-based SP) on main |
| `vllm_ascend/attention/attention_v1.py` |
[vllm#60271](vllm-project/vllm#60271) rejects
backends that do not support batch invariance | Declare
`supports_batch_invariance()` on main (`not vllm_version_is("0.31.0")`)
|
| `vllm_ascend/attention/dsa_v1.py` |
[vllm#60271](vllm-project/vllm#60271)
batch-invariance check;
[vllm#53175](vllm-project/vllm#53175) forwards
the layer's `KVCacheSpec` into `get_supported_kernel_block_sizes`, so a
rebase had overwritten the six DSA backends back to the zero-arg form
and main raised `TypeError: ... takes 0 positional arguments but 1 was
given` | Re-apply `supports_batch_invariance()` and give all six DSA
backends the per-lane
`get_supported_kernel_block_sizes(kv_cache_spec=None)` signature |
| `vllm_ascend/attention/dsa_v41.py` |
[vllm#60271](vllm-project/vllm#60271)
batch-invariance check | Declare `supports_batch_invariance()` on
`DeepseekV41CacheBackend` for main |
| `vllm_ascend/attention/fa3_v1.py` | Same | Declare
`supports_batch_invariance()` on `AscendFABackend` for main |
| `vllm_ascend/attention/indexer.py` | Same | Declare
`supports_batch_invariance()` on `AscendSFAIndexerBackend` for main |
| `vllm_ascend/attention/indexer_kpool.py` | Same | Declare
`supports_batch_invariance()` on the indexer k-pool and tail backends
for main |
| `vllm_ascend/attention/mla_v1.py` | Same | Declare
`supports_batch_invariance()` on `AscendMLABackend` for main |
| `vllm_ascend/attention/sfa_v1.py` | Same | Declare
`supports_batch_invariance()` on `AscendSFABackend` for main |
| `vllm_ascend/compilation/graph_fusion_pass_manager.py` |
[vllm#60517](vllm-project/vllm#60517) removed
`PassConfig.fuse_gemm_comms` | Gate the MMRS fusion on
`enable_sp(config)` on main, keep reading the PassConfig flag on v0.31.0
|
| `vllm_ascend/core/kv_cache_interface.py` |
[vllm#57169](vllm-project/vllm#57169) removed
the MLA `storage_block_size` override, so a plain MLA subclass crashed
with `AttributeError: 'HiddenStateCacheSpec' object has no attribute
'storage_block_size'` | Read the optional override only on v0.31.0 and
fall back to `block_size` on main |
| `vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_connector.py` |
vLLM main's scheduler calls `get_loaded_kv_cache_group_ids`, which reads
`_kv_cache_config`; the Ascend connector overrode `__init__` without
calling the base and never set it (`AttributeError: 'MooncakeConnector'
object has no attribute '_kv_cache_config'`) | Call
`KVConnectorBase_V1.__init__` on main and declare the main-only
`get_loaded_kv_cache_group_ids` hook |
|
`vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_hybrid_connector.py`
| Same | Same base-init binding and main-only hook |
| `vllm_ascend/models/deepseek_v41/cache_config.py` |
[vllm#57169](vllm-project/vllm#57169) removed
MLA `storage_block_size` | Gate the kwarg in
`make_mla_cache_spec`/`make_index_cache_spec`/`make_folded_index_cache_spec`
on v0.31.0 |
| `vllm_ascend/models/deepseek_v41/engram/parallel.py` |
[vllm#59327](vllm-project/vllm#59327) moved
`engram_head_shard_rank` from `nvidia.engram` to `common.engram` |
Import `engram_head_shard_rank` by lane |
| `vllm_ascend/models/glm5next/attention.py` |
[vllm#57387](vllm-project/vllm#57387) removed
`vllm.transformers_utils.configs.glm5_next`, making the direct import
unresolvable (mypy `import-not-found`) | Import `Glm5NextConfig` from
`transformers` on main, keep the vLLM path on v0.31.0 |
| `vllm_ascend/models/glm5next/kda.py` | Same | Same lane-gated
`Glm5NextConfig` import |
| `vllm_ascend/models/glm5next/model.py` | Same | Same lane-gated
`Glm5NextConfig` import |
| `vllm_ascend/models/kimi_k3.py` | vLLM main made `out_hidden_size` a
required positional argument of `KimiK25MultiModalProjector`
(`TypeError: ... missing 1 required positional argument:
'out_hidden_size'`) | Pass `out_hidden_size = self.hidden_size` on main
only |
| `vllm_ascend/models/minimax_m3/msa_m3.py` |
[vllm#60271](vllm-project/vllm#60271)
batch-invariance check | Declare `supports_batch_invariance()` on the
MiniMax-M3 sparse/indexer backends for main |
| `vllm_ascend/patch/__init__.py` |
[vllm#58067](vllm-project/vllm#58067) xgrammar
`separators`; stale v0.30.0 boundary wording | Document the new xgrammar
patch (why/how/future plan) and rename the boundary to v0.31.0 |
| `vllm_ascend/patch/platform/__init__.py` | Stale release-lane wording
in the Engram adapter comment | `v0.30.0+` -> `v0.31.0+` |
| `vllm_ascend/patch/platform/patch_balance_schedule.py` | Stale
release-lane wording in the block-state snapshot comment | `vLLM
main/v0.30.0` -> `vLLM main/v0.31.0` |
| `vllm_ascend/patch/platform/patch_kv_cache_utils.py` |
[vllm#57169](vllm-project/vllm#57169) added a
leading `vllm_config` argument to `_pool_bytes_per_block` | Define
`_ascend_pool_bytes_per_block` per lane, forwarding `vllm_config` to the
upstream fallback on main |
| `vllm_ascend/patch/platform/patch_mamba_manager.py` |
[vllm#59175](vllm-project/vllm#59175) added
`prefill_end` to `get_num_blocks_to_allocate` | Always forward
`prefill_end` and drop the now-redundant lane gate |
| `vllm_ascend/patch/platform/patch_structured_output.py` |
[vllm#58067](vllm-project/vllm#58067) passes
`separators=(",", ":")` to `GrammarCompiler.compile_json_schema`, which
the shipped xgrammar (0.2.7/0.2.8) mishandles (truncated, invalid JSON
in guided decoding) | Wrap `compile_json_schema` to force
`separators=None` on main only, until xgrammar >= 0.2.9 is pinned |
| `vllm_ascend/patch/worker/__init__.py` |
[vllm#60335](vllm-project/vllm#60335) renamed
the AR speculator module | Import the AR module by lane before the 310P
manager override |
| `vllm_ascend/patch/worker/patch_v2/patch_eagle_speculator.py` | Same
AR module rename | Bind `vllm_speculator_module` from
`target_dependent_ar` on main, `autoregressive` on v0.31.0 |
| `vllm_ascend/patch/worker/patch_v2/patch_model_runner.py` |
[vllm#60517](vllm-project/vllm#60517) removed
the `tensor_parallel_size` argument | Forward it only on v0.31.0 |
| `vllm_ascend/patch/worker/patch_v2/patch_spec_decode_310.py` | Same AR
module rename | Import the AR module by lane before patching the CPU
input helpers |
| `vllm_ascend/platform.py` |
[vllm#60517](vllm-project/vllm#60517) removed
`PassConfig.enable_sp`/`fuse_gemm_comms`/`sp_min_token_num` and
`update_sizes_for_sequence_parallelism` | Gate the SP threshold default
on v0.31.0 and inline the tp-multiple filtering on main |
| `vllm_ascend/worker/model_runner_v1.py` |
[vllm#58411](vllm-project/vllm#58411)
`randomize_inputs`;
[vllm#57169](vllm-project/vllm#57169) split
metadata-builder creation and changed `AttentionGroup`;
[vllm#60517](vllm-project/vllm#60517) removed
the SP pass and the `tensor_parallel_size` argument | Forward
`randomize_inputs` unconditionally, build `AttentionGroup` without
positional builders and call
`initialize_metadata_builders`/`_init_device_metadata_providers` on
main, inline the SP-aware lcm, resolve cudagraph sizes without
`tensor_parallel_size`, treat `update_pass_config` as a no-op on main
and collapse 310P drafter kernel block sizes to an int |
| `vllm_ascend/worker/v2/block_table.py` |
[vllm#57169](vllm-project/vllm#57169) dropped
`kernel_block_sizes`/`blocks_per_kv_block` from `BlockTables` and
stopped expanding block ids | Version-split `__init__` to restore the
kernel-block layout (sizes, `blocks_per_kv_block`, expanded rows,
kernel-size tensor) and override `append_block_ids` on main |
| `vllm_ascend/worker/v2/model_runner.py` |
[vllm#58411](vllm-project/vllm#58411)
`randomize_inputs`; main's rejection sampler takes invalidation state
and needs the new `InputBatch` fields | Forward `randomize_inputs` and
`max_seq_len_np` unconditionally, skip grammar application on empty
logits and pass `grammar_invalid_drafts` output into the main rejection
sampler |
| `vllm_ascend/worker/v2/model_states/mamba_hybrid.py` | vLLM main
removed `MambaHybridModelState._ensure_align_ctx` and moved
align-context creation into `initialize_kv_cache`, so the first forward
aborted with `AttributeError: ... has no attribute '_ensure_align_ctx'`
/ `AssertionError` when the context was absent | Define
`_ensure_align_ctx` per lane: delegate on v0.31.0, lazily (re)create the
align context on main from the forward context |
| `vllm_ascend/worker/v2/pcp_manager.py` | vLLM main made
`shard_decode_requests` a required `PCPManager.__init__` argument and
`maybe_build_pcp_manager` passes it (`TypeError: ... unexpected keyword
argument 'shard_decode_requests'`) | Version-split
`AscendPCPManager.__init__`, accepting and forwarding the flag on main
while keeping the Ascend buffer setup shared |
| `vllm_ascend/worker/v2/spec_decode/autoregressive/aclgraph.py` |
[vllm#60335](vllm-project/vllm#60335) moved the
speculator CUDA-graph manager and added `num_speculative_steps` to the
captured forward | Import `SpeculatorCudaGraphManager` by lane and pass
`num_speculative_steps` on main |
| `vllm_ascend/worker/v2/spec_decode/autoregressive/speculator.py` |
[vllm#60335](vllm-project/vllm#60335) renamed
the AR base class and added `num_speculative_steps`;
[vllm#60754](vllm-project/vllm#60754) renamed
`dp_sync` -> `dp_sync_state` (`TypeError:
AscendAutoRegressiveSpeculator.propose() got an unexpected keyword
argument 'dp_sync_state'`) | Bind the upstream base class by lane and
version-split `propose()`/`_propose_impl()` plus
`_generate_draft`/`_multi_step_decode` so each lane forwards exactly its
arguments |
| `vllm_ascend/worker/v2/spec_decode/dflash/speculator.py` |
[vllm#57169](vllm-project/vllm#57169) makes the
DFlash caller forward the physical block size while the Ascend block
table stays kernel-block addressed (draft KV mismapped, every draft
rejected);
[vllm#60335](https://github.com/vllm-project/vllm/pull/60335)/[vllm#60754](https://github.com/vllm-project/vllm/pull/60754)
speculator contract | Version-split `prepare_dflash_inputs_factory`
(restoring the kernel block size per group on main) and
`propose()`/`_propose_impl()`, keeping the profiling DP-count guard |
| `vllm_ascend/worker/v2/spec_decode/dspark/speculator.py` | Same
speculator contract changes | Same version-split
`propose()`/`_propose_impl()` adaptation |
| `vllm_ascend/worker/worker.py` |
[vllm#58411](vllm-project/vllm#58411)
`profile_run` takes `randomize_inputs` | Compute `randomize_inputs` from
the env flag + DP size and forward it on both `profile_run` calls |
### Does this PR introduce _any_ user-facing change?
No. This is a CI/main2main compatibility update: behavior on vLLM main
is unchanged (the restorations only bring the main lane back to the
v0.31.0 behavior) and the v0.31.0 release lane is preserved.
### How was this patch tested?
- Unit tests updated per lane; `pytest -sv tests/ut` against both the
verified
main commit (`a98247ab`) and the release tag (`v0.31.0`).
- vLLM main:
vllm-project/vllm@c4df37d
---------
Signed-off-by: hfadzxy <starmoon_zhang@163.com>
Signed-off-by: zhangxinyuehfad <starmoon_zhang@163.com>
chujiangang2006
pushed a commit
to chujiangang2006/vllm-ascend
that referenced
this pull request
Oct 20, 2026
### What this PR does / why we need it?
Bump the verified vLLM main pin from `c4df37df` to `a98247ab` and adapt
vllm-ascend to the upstream changes in that range. The v0.31.0 release
lane is kept intact: each adaptation is gated on
`vllm_version_is("0.31.0")` (or a version-aware helper) so both the
`main` and the `v0.31.0` CPU UT lanes stay green.
The range contains a large breaking KV-cache/attention rework plus a set
of smaller API changes:
- vllm#57169 reworked the KV cache layout: it dropped the MLA
`storage_block_size` override, added a leading `vllm_config` to
`_pool_bytes_per_block`, removed
`kernel_block_sizes`/`blocks_per_kv_block` from `BlockTables`, split
metadata-builder creation out of `initialize_attn_backend` and made
`AttentionGroup` identity-comparable.
- vllm#60335 renamed the spec-decode AR path (`autoregressive` ->
`target_dependent_ar`) and added `num_speculative_steps`/
`num_speculative_tokens` to `propose()` and the decode callbacks.
- vllm#60517 removed the torch.compile sequence-parallelism/AsyncTP
passes `PassConfig.enable_sp`/`fuse_gemm_comms`/`sp_min_token_num`,
`update_sizes_for_sequence_parallelism` and the `tensor_parallel_size`
argument of the cudagraph-size resolvers.
- vllm#60754 renamed the DP-sync state (`dp_sync` -> `dp_sync_state`)
and added `ExecuteModelState.num_spec_tokens_to_schedule`.
- vllm#58411 added `randomize_inputs` to `profile_run`/`_dummy_run`.
- vllm#60271 added a hard batch-invariance support check for every
attention backend.
- vllm#59327 moved the DeepSeek-V4.1 Engram module to `common.engram`.
- vllm#57387 moved the GLM-5-Next config classes into `transformers`
(hence the `transformers` 5.14.1 -> 5.19.0 bump, which main requires
anyway).
- Smaller seams: vllm#59175 added `prefill_end` to KV allocation,
vllm#53175 forwards the layer's spec to
`get_supported_kernel_block_sizes`, vllm#58067 passes `separators` to
xgrammar, vllm#53934 made `CudaGraphManager.dp_size` a property,
vllm#56723 builds draft metadata through the attention group, vllm#57934
makes MRV1 reject DFlash candidate-head drafts.
| Files | Upstream vLLM change (why it breaks vllm-ascend) | vllm-ascend
adaptation |
|-------|--------------------------------------------------|------------------------|
| `.github/vllm-main-verified.commit` | — (main-lane pin only) |
Verified main commit bumped `c4df37df` -> `a98247ab` |
| `pyproject.toml` | vLLM main requires `transformers >= 5.16.1` | Build
requirement bumped `transformers==5.14.1` -> `5.19.0` (satisfies both
lanes) |
| `requirements.txt` | Same transformers floor | Runtime requirement
bumped `transformers==5.14.1` -> `5.19.0` |
| `tests/deepseek_v41_utils.py` |
[vllm#57169](vllm-project/vllm#57169) removed
MLA `storage_block_size` | Gate the `storage_block_size` kwarg of the
DSV4 k-cache and indexer k-cache specs on `vllm_version_is("0.31.0")` |
| `tests/e2e/pull_request/four_card/glm53flash_assets/config.json` |
[vllm#57387](vllm-project/vllm#57387) serves
GLM-5-Next configs from `transformers` 5.19.0, which validates every
field type | Write `swiglu_limit: 10.0` and `gate_lower_bound: -5.0` as
floats |
| `tests/e2e/pull_request/four_card/glm53flash_assets/multimodal.json` |
Same strict dataclass validation | Write `vision_config.swiglu_limit:
10` -> `10.0` |
| `tests/e2e/pull_request/two_card/spec_decode/test_spec_decode.py` |
[vllm#57934](vllm-project/vllm#57934): MRV1
rejects DFlash candidate-head drafts on main | Make the
`test_dflash2_acceptance` skip unconditional instead of main-only |
| `tests/ut/attention/test_batch_invariance_support.py` |
[vllm#60271](vllm-project/vllm#60271) added the
batch-invariance support check | New regression test asserting
`supports_batch_invariance()` on all Ascend backends and the 310P/C8MXFP
subclasses |
| `tests/ut/compilation/test_graph_fusion_pass_manager.py` |
[vllm#60517](vllm-project/vllm#60517) removed
`PassConfig.fuse_gemm_comms` | Only seed the PassConfig flag on v0.31.0
and patch `enable_sp` for the main lane gate |
| `tests/ut/core/test_kv_cache_interface.py` |
[vllm#57169](vllm-project/vllm#57169) removed
`MLAAttentionSpec.storage_block_size` | Add a plain-MLA-subclass case
(`HiddenStateCacheSpec`) resolving `get_storage_block_size()` per lane |
| `tests/ut/distributed/kv_transfer/kv_p2p/test_mooncake_connector.py` |
vLLM main's scheduler calls `get_loaded_kv_cache_group_ids`, backed by
`KVConnectorBase_V1._kv_cache_config` | Seed
`MockKVCacheConfig.transfer_group_ids` and add the main-only
scheduler-hook test (skipped on v0.31.0) |
| `tests/ut/kv_offload/test_mooncake_hybrid_connector.py` | Same new
`get_loaded_kv_cache_group_ids` scheduler hook | Assert the hook is
declared on the class only on main |
| `tests/ut/kv_offload/test_preempt_offload_connector.py` | vLLM main's
remote-KV restore consults `scheduler.prefix_replay_group_ids` | Seed
`prefix_replay_group_ids = ()` in the `__new__`-based scheduler fixture
|
| `tests/ut/models/test_deepseek_v41_cache.py` |
[vllm#57169](vllm-project/vllm#57169) added the
leading `vllm_config` to `_pool_bytes_per_block` and removed MLA
`storage_block_size` | Add a `_pool_bytes_per_block(runtime, groups)`
lane helper, read specs through `get_storage_block_size()` and gate the
one explicit `storage_block_size` kwarg |
| `tests/ut/models/test_engram.py` |
[vllm#59327](vllm-project/vllm#59327) deleted
`deepseek_v41/nvidia/engram.py` and moved it to `common.engram` | Import
`upstream_engram` by lane so collection no longer aborts with
`ImportError` on main |
| `tests/ut/models/test_glm5_next_mtp.py` |
[vllm#57387](vllm-project/vllm#57387) removed
`vllm.transformers_utils.configs.glm5_next` | Import
`Glm5NextTextConfig` from `transformers` on main, keep the vLLM path on
v0.31.0 |
| `tests/ut/ops/test_gdn_attn_builder.py` | Main's GDN metadata builder
resolves the prefill backend from `model_config.hf_text_config` | Give
the `model_config` fixture an `hf_text_config` |
| `tests/ut/patch/platform/test_patch_structured_output.py` | Main gates
the guidance backend on `GuidanceBackend.is_tokenizer_supported` |
Monkeypatch that check to accept the dummy tokenizer on main |
| `tests/ut/patch/platform/test_patch_use_v2_model_runner.py` |
[vllm#60517](vllm-project/vllm#60517) dropped
the `PassConfig.enable_sp` guard, so the V2 runner no longer rejects SP
| Expect `["sequence parallelism"]` only on v0.31.0 |
| `tests/ut/patch/platform/test_prefix_cache_cp_patches.py` |
[vllm#57169](vllm-project/vllm#57169) resolves
the KV cache layout when computing pool bytes per block | Pass
`vllm_config` to the Ascend/upstream `_pool_bytes_per_block` on main and
seed `get_resolved_kv_cache_layout` |
| `tests/ut/spec_decode/test_dflash2_v2.py` |
[vllm#57169](vllm-project/vllm#57169) forwards
the physical block size to the DFlash caller;
[vllm#60754](vllm-project/vllm#60754) renamed
`dp_sync` | Select the `dp_sync`/`dp_sync_state` keyword per lane and
add the main-only kernel-block-size restore test |
| `tests/ut/spec_decode/test_eagle_proposer.py` |
[vllm#60517](vllm-project/vllm#60517) removed
`PassConfig.enable_sp` | Assert `enable_sp` is a PassConfig field only
on v0.31.0 |
| `tests/ut/spec_decode/test_mla_dspark.py` |
[vllm#60754](vllm-project/vllm#60754) renamed
`dp_sync` -> `dp_sync_state` | Pass the lane-specific keyword into
`DSparkSpeculator.propose` |
| `tests/ut/test_platform.py` |
[vllm#60517](vllm-project/vllm#60517) removed
the PassConfig SP fields and `update_sizes_for_sequence_parallelism` |
Assert the SP threshold default only on v0.31.0 and that the removed
method is no longer called on main |
| `tests/ut/worker/test_adaptive_verification.py` |
[vllm#60754](vllm-project/vllm#60754) renamed
the DP-sync argument | Build the execute-model kwargs with
`dp_sync`/`dp_sync_state` per lane |
| `tests/ut/worker/test_attn_utils_v2.py` | vLLM main made
`AttentionGroup` identity-comparable | Compare the preserved fields
(backend, layer names, spec, group id) on main instead of `==` |
| `tests/ut/worker/test_dcp_speculator_metadata_v2.py` |
[vllm#60335](vllm-project/vllm#60335) renamed
the AR speculator module;
[vllm#56723](vllm-project/vllm#56723) builds
draft metadata through the attention group | Import
`TargetDependentARSpeculator` on main and add a recording attention
group exposing `build_metadata` |
| `tests/ut/worker/test_model_runner_v1.py` |
[vllm#57169](vllm-project/vllm#57169) removed
`BlockTables.kernel_block_sizes`; main module lacks
`mamba_utils.postprocess_mamba_all` | Pass `kernel_block_sizes` only on
v0.31.0 and patch with `create=True` |
| `tests/ut/worker/test_model_runner_v2.py` |
[vllm#60517](https://github.com/vllm-project/vllm/pull/60517)/[vllm#58411](https://github.com/vllm-project/vllm/pull/58411):
`max_seq_len_np` and `randomize_inputs` exist on both lanes now | Drop
the per-lane gating and always supply `max_seq_len_np` / expect
`randomize_inputs` |
| `tests/ut/worker/test_model_runner_v2_mamba.py` | vLLM main removed
`MambaHybridModelState._ensure_align_ctx` | Seed `_mamba_ctx` on main
and add two main-only `_ensure_align_ctx` regression tests |
| `tests/ut/worker/test_mtp_pcp_speculator_v2.py` |
[vllm#60335](vllm-project/vllm#60335) renamed
the AR base class;
[vllm#60754](vllm-project/vllm#60754) renamed
`dp_sync` | Import the base class by lane and patch
`_UpstreamAutoRegressiveSpeculator` via the lane-specific keyword |
| `tests/ut/worker/test_pcp_manager_v2.py` | Main declares
`ParallelConfig.pcp_shard_decode_requests`, renamed `dp_sync` and added
`num_spec_tokens_to_schedule` | Assert the field per lane, build
`ExecuteModelState` with the lane keyword and seed
`num_spec_tokens_to_schedule` |
| `tests/ut/worker/test_v2_dummy_padding.py` |
[vllm#60754](vllm-project/vllm#60754) renamed
`dp_sync`; main's dummy context reads `block_tables.block_sizes` | Pass
the lane keyword and provide both `block_sizes` and `kernel_block_sizes`
|
| `tests/ut/worker/test_worker_v1.py` |
[vllm#58411](vllm-project/vllm#58411):
`determine_available_memory` now forwards
`profile_run(randomize_inputs=...)` | Update the stale
`assert_called_once_with()` to expect `randomize_inputs=False` |
| `tests/ut/worker/v2/test_aclgraph.py` |
[vllm#60335](vllm-project/vllm#60335) moved
`SpeculatorCudaGraphManager` to `target_dependent_ar`;
[vllm#53934](vllm-project/vllm#53934) made
`dp_size` a read-only property | Import the manager by lane, back
`dp_size` with `vllm_config` on main and expect the extra
`num_speculative_steps` capture arg |
| `tests/ut/worker/v2/test_speculator.py` |
[vllm#60335](vllm-project/vllm#60335) renamed
the AR speculator and added `num_speculative_steps`;
[vllm#60754](vllm-project/vllm#60754) renamed
`dp_sync` | Import by lane, thread `num_speculative_steps` through the
`_generate_draft`/`_multi_step_decode` assertions and add the main
`dp_sync_state` + `num_speculative_tokens` regression test |
| `tests/ut/worker/v2/test_v2_block_table.py` |
[vllm#57169](vllm-project/vllm#57169) dropped
`kernel_block_sizes` from `BlockTables` | Make the stand-in parent
init's kernel sizes optional and assert the kernel tensor is built by
`AscendBlockTables` on main |
| `vllm_ascend/_310p/sharded_state_loader_310p.py` | vLLM main split
`ShardedStateLoader.DEFAULT_PATTERN` into
`TP_ONLY_PATTERN`/`PP_AND_TP_PATTERN` | Pick the lane's default pattern
name instead of the removed combined one |
| `vllm_ascend/_310p/worker/v2/model_runner.py` |
[vllm#60517](vllm-project/vllm#60517) removed
the `tensor_parallel_size` argument of
`resolve_cudagraph_mode_and_sizes` | Forward `tensor_parallel_size` only
on v0.31.0 |
| `vllm_ascend/_310p/worker/v2/spec_decode/mtp_speculator.py` |
[vllm#60335](vllm-project/vllm#60335) renamed
the AR module and added `num_speculative_steps` to
`_generate_draft`/`_multi_step_decode` | Version-split the two overrides
and the AR import, falling back to `self.num_speculative_steps` when the
capture path omits it |
| `vllm_ascend/ascend_config.py` |
[vllm#60517](vllm-project/vllm#60517) removed
`PassConfig.enable_sp` | Read the flag only on v0.31.0, so
`enable_sp_by_pass` is False (no pass-based SP) on main |
| `vllm_ascend/attention/attention_v1.py` |
[vllm#60271](vllm-project/vllm#60271) rejects
backends that do not support batch invariance | Declare
`supports_batch_invariance()` on main (`not vllm_version_is("0.31.0")`)
|
| `vllm_ascend/attention/dsa_v1.py` |
[vllm#60271](vllm-project/vllm#60271)
batch-invariance check;
[vllm#53175](vllm-project/vllm#53175) forwards
the layer's `KVCacheSpec` into `get_supported_kernel_block_sizes`, so a
rebase had overwritten the six DSA backends back to the zero-arg form
and main raised `TypeError: ... takes 0 positional arguments but 1 was
given` | Re-apply `supports_batch_invariance()` and give all six DSA
backends the per-lane
`get_supported_kernel_block_sizes(kv_cache_spec=None)` signature |
| `vllm_ascend/attention/dsa_v41.py` |
[vllm#60271](vllm-project/vllm#60271)
batch-invariance check | Declare `supports_batch_invariance()` on
`DeepseekV41CacheBackend` for main |
| `vllm_ascend/attention/fa3_v1.py` | Same | Declare
`supports_batch_invariance()` on `AscendFABackend` for main |
| `vllm_ascend/attention/indexer.py` | Same | Declare
`supports_batch_invariance()` on `AscendSFAIndexerBackend` for main |
| `vllm_ascend/attention/indexer_kpool.py` | Same | Declare
`supports_batch_invariance()` on the indexer k-pool and tail backends
for main |
| `vllm_ascend/attention/mla_v1.py` | Same | Declare
`supports_batch_invariance()` on `AscendMLABackend` for main |
| `vllm_ascend/attention/sfa_v1.py` | Same | Declare
`supports_batch_invariance()` on `AscendSFABackend` for main |
| `vllm_ascend/compilation/graph_fusion_pass_manager.py` |
[vllm#60517](vllm-project/vllm#60517) removed
`PassConfig.fuse_gemm_comms` | Gate the MMRS fusion on
`enable_sp(config)` on main, keep reading the PassConfig flag on v0.31.0
|
| `vllm_ascend/core/kv_cache_interface.py` |
[vllm#57169](vllm-project/vllm#57169) removed
the MLA `storage_block_size` override, so a plain MLA subclass crashed
with `AttributeError: 'HiddenStateCacheSpec' object has no attribute
'storage_block_size'` | Read the optional override only on v0.31.0 and
fall back to `block_size` on main |
| `vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_connector.py` |
vLLM main's scheduler calls `get_loaded_kv_cache_group_ids`, which reads
`_kv_cache_config`; the Ascend connector overrode `__init__` without
calling the base and never set it (`AttributeError: 'MooncakeConnector'
object has no attribute '_kv_cache_config'`) | Call
`KVConnectorBase_V1.__init__` on main and declare the main-only
`get_loaded_kv_cache_group_ids` hook |
|
`vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_hybrid_connector.py`
| Same | Same base-init binding and main-only hook |
| `vllm_ascend/models/deepseek_v41/cache_config.py` |
[vllm#57169](vllm-project/vllm#57169) removed
MLA `storage_block_size` | Gate the kwarg in
`make_mla_cache_spec`/`make_index_cache_spec`/`make_folded_index_cache_spec`
on v0.31.0 |
| `vllm_ascend/models/deepseek_v41/engram/parallel.py` |
[vllm#59327](vllm-project/vllm#59327) moved
`engram_head_shard_rank` from `nvidia.engram` to `common.engram` |
Import `engram_head_shard_rank` by lane |
| `vllm_ascend/models/glm5next/attention.py` |
[vllm#57387](vllm-project/vllm#57387) removed
`vllm.transformers_utils.configs.glm5_next`, making the direct import
unresolvable (mypy `import-not-found`) | Import `Glm5NextConfig` from
`transformers` on main, keep the vLLM path on v0.31.0 |
| `vllm_ascend/models/glm5next/kda.py` | Same | Same lane-gated
`Glm5NextConfig` import |
| `vllm_ascend/models/glm5next/model.py` | Same | Same lane-gated
`Glm5NextConfig` import |
| `vllm_ascend/models/kimi_k3.py` | vLLM main made `out_hidden_size` a
required positional argument of `KimiK25MultiModalProjector`
(`TypeError: ... missing 1 required positional argument:
'out_hidden_size'`) | Pass `out_hidden_size = self.hidden_size` on main
only |
| `vllm_ascend/models/minimax_m3/msa_m3.py` |
[vllm#60271](vllm-project/vllm#60271)
batch-invariance check | Declare `supports_batch_invariance()` on the
MiniMax-M3 sparse/indexer backends for main |
| `vllm_ascend/patch/__init__.py` |
[vllm#58067](vllm-project/vllm#58067) xgrammar
`separators`; stale v0.30.0 boundary wording | Document the new xgrammar
patch (why/how/future plan) and rename the boundary to v0.31.0 |
| `vllm_ascend/patch/platform/__init__.py` | Stale release-lane wording
in the Engram adapter comment | `v0.30.0+` -> `v0.31.0+` |
| `vllm_ascend/patch/platform/patch_balance_schedule.py` | Stale
release-lane wording in the block-state snapshot comment | `vLLM
main/v0.30.0` -> `vLLM main/v0.31.0` |
| `vllm_ascend/patch/platform/patch_kv_cache_utils.py` |
[vllm#57169](vllm-project/vllm#57169) added a
leading `vllm_config` argument to `_pool_bytes_per_block` | Define
`_ascend_pool_bytes_per_block` per lane, forwarding `vllm_config` to the
upstream fallback on main |
| `vllm_ascend/patch/platform/patch_mamba_manager.py` |
[vllm#59175](vllm-project/vllm#59175) added
`prefill_end` to `get_num_blocks_to_allocate` | Always forward
`prefill_end` and drop the now-redundant lane gate |
| `vllm_ascend/patch/platform/patch_structured_output.py` |
[vllm#58067](vllm-project/vllm#58067) passes
`separators=(",", ":")` to `GrammarCompiler.compile_json_schema`, which
the shipped xgrammar (0.2.7/0.2.8) mishandles (truncated, invalid JSON
in guided decoding) | Wrap `compile_json_schema` to force
`separators=None` on main only, until xgrammar >= 0.2.9 is pinned |
| `vllm_ascend/patch/worker/__init__.py` |
[vllm#60335](vllm-project/vllm#60335) renamed
the AR speculator module | Import the AR module by lane before the 310P
manager override |
| `vllm_ascend/patch/worker/patch_v2/patch_eagle_speculator.py` | Same
AR module rename | Bind `vllm_speculator_module` from
`target_dependent_ar` on main, `autoregressive` on v0.31.0 |
| `vllm_ascend/patch/worker/patch_v2/patch_model_runner.py` |
[vllm#60517](vllm-project/vllm#60517) removed
the `tensor_parallel_size` argument | Forward it only on v0.31.0 |
| `vllm_ascend/patch/worker/patch_v2/patch_spec_decode_310.py` | Same AR
module rename | Import the AR module by lane before patching the CPU
input helpers |
| `vllm_ascend/platform.py` |
[vllm#60517](vllm-project/vllm#60517) removed
`PassConfig.enable_sp`/`fuse_gemm_comms`/`sp_min_token_num` and
`update_sizes_for_sequence_parallelism` | Gate the SP threshold default
on v0.31.0 and inline the tp-multiple filtering on main |
| `vllm_ascend/worker/model_runner_v1.py` |
[vllm#58411](vllm-project/vllm#58411)
`randomize_inputs`;
[vllm#57169](vllm-project/vllm#57169) split
metadata-builder creation and changed `AttentionGroup`;
[vllm#60517](vllm-project/vllm#60517) removed
the SP pass and the `tensor_parallel_size` argument | Forward
`randomize_inputs` unconditionally, build `AttentionGroup` without
positional builders and call
`initialize_metadata_builders`/`_init_device_metadata_providers` on
main, inline the SP-aware lcm, resolve cudagraph sizes without
`tensor_parallel_size`, treat `update_pass_config` as a no-op on main
and collapse 310P drafter kernel block sizes to an int |
| `vllm_ascend/worker/v2/block_table.py` |
[vllm#57169](vllm-project/vllm#57169) dropped
`kernel_block_sizes`/`blocks_per_kv_block` from `BlockTables` and
stopped expanding block ids | Version-split `__init__` to restore the
kernel-block layout (sizes, `blocks_per_kv_block`, expanded rows,
kernel-size tensor) and override `append_block_ids` on main |
| `vllm_ascend/worker/v2/model_runner.py` |
[vllm#58411](vllm-project/vllm#58411)
`randomize_inputs`; main's rejection sampler takes invalidation state
and needs the new `InputBatch` fields | Forward `randomize_inputs` and
`max_seq_len_np` unconditionally, skip grammar application on empty
logits and pass `grammar_invalid_drafts` output into the main rejection
sampler |
| `vllm_ascend/worker/v2/model_states/mamba_hybrid.py` | vLLM main
removed `MambaHybridModelState._ensure_align_ctx` and moved
align-context creation into `initialize_kv_cache`, so the first forward
aborted with `AttributeError: ... has no attribute '_ensure_align_ctx'`
/ `AssertionError` when the context was absent | Define
`_ensure_align_ctx` per lane: delegate on v0.31.0, lazily (re)create the
align context on main from the forward context |
| `vllm_ascend/worker/v2/pcp_manager.py` | vLLM main made
`shard_decode_requests` a required `PCPManager.__init__` argument and
`maybe_build_pcp_manager` passes it (`TypeError: ... unexpected keyword
argument 'shard_decode_requests'`) | Version-split
`AscendPCPManager.__init__`, accepting and forwarding the flag on main
while keeping the Ascend buffer setup shared |
| `vllm_ascend/worker/v2/spec_decode/autoregressive/aclgraph.py` |
[vllm#60335](vllm-project/vllm#60335) moved the
speculator CUDA-graph manager and added `num_speculative_steps` to the
captured forward | Import `SpeculatorCudaGraphManager` by lane and pass
`num_speculative_steps` on main |
| `vllm_ascend/worker/v2/spec_decode/autoregressive/speculator.py` |
[vllm#60335](vllm-project/vllm#60335) renamed
the AR base class and added `num_speculative_steps`;
[vllm#60754](vllm-project/vllm#60754) renamed
`dp_sync` -> `dp_sync_state` (`TypeError:
AscendAutoRegressiveSpeculator.propose() got an unexpected keyword
argument 'dp_sync_state'`) | Bind the upstream base class by lane and
version-split `propose()`/`_propose_impl()` plus
`_generate_draft`/`_multi_step_decode` so each lane forwards exactly its
arguments |
| `vllm_ascend/worker/v2/spec_decode/dflash/speculator.py` |
[vllm#57169](vllm-project/vllm#57169) makes the
DFlash caller forward the physical block size while the Ascend block
table stays kernel-block addressed (draft KV mismapped, every draft
rejected);
[vllm#60335](https://github.com/vllm-project/vllm/pull/60335)/[vllm#60754](https://github.com/vllm-project/vllm/pull/60754)
speculator contract | Version-split `prepare_dflash_inputs_factory`
(restoring the kernel block size per group on main) and
`propose()`/`_propose_impl()`, keeping the profiling DP-count guard |
| `vllm_ascend/worker/v2/spec_decode/dspark/speculator.py` | Same
speculator contract changes | Same version-split
`propose()`/`_propose_impl()` adaptation |
| `vllm_ascend/worker/worker.py` |
[vllm#58411](vllm-project/vllm#58411)
`profile_run` takes `randomize_inputs` | Compute `randomize_inputs` from
the env flag + DP size and forward it on both `profile_run` calls |
### Does this PR introduce _any_ user-facing change?
No. This is a CI/main2main compatibility update: behavior on vLLM main
is unchanged (the restorations only bring the main lane back to the
v0.31.0 behavior) and the v0.31.0 release lane is preserved.
### How was this patch tested?
- Unit tests updated per lane; `pytest -sv tests/ut` against both the
verified
main commit (`a98247ab`) and the release tag (`v0.31.0`).
- vLLM main:
vllm-project/vllm@c4df37d
---------
Signed-off-by: hfadzxy <starmoon_zhang@163.com>
Signed-off-by: zhangxinyuehfad <starmoon_zhang@163.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
With
VLLM_BATCH_INVARIANT=1, the attention selector refuses backends that do not declaresupports_batch_invariance()("VLLM batch_invariant mode is not supported for ..."). Models that select their own attention backend never go through the selector, so they start anyway and silently return batch-dependent output. This PR applies the same check to every attention layer when the attention groups are built.Claims
MiniMaxAI/MiniMax-M3starts in batch-invariant mode onmainand the determinism tests report violations (for example 32 of 32 prompts). With this PR the engine refuses to start with the existing error message.Qwen/Qwen3-235B-A22B-Thinking-2507-FP8(GB300),Qwen/Qwen3-1.7Banddeepseek-ai/DeepSeek-V2-Lite-Chat(A10). Every model on the tested list indocs/features/batch_invariance.mduses a standard backend.Validation
GB300 runs: 4x NVIDIA GB300 (SM 10.3),
mainat182247257fversus that commit plus this PR's source change. A10 runs: 2x NVIDIA A10 (SM 8.6),mainatd09ff77637plus this PR's source change. One pytest process per test. The blocks are copied from the logs without editing.1.
main: MiniMax-M3 runs in batch-invariant mode and is not batch invariant (tensor-parallel 4)2. This PR: the engine refuses to start for MiniMax-M3 in batch-invariant mode
3. This PR: models on the standard backends still pass
4. New unit test (without and with the source change)
lm_evalandvllm bench servecomparisons are not included because the change is a start-up check only: no kernel or request path is touched, and for the affected model the "after" state is that the engine does not start.Details
Root cause.
supports_batch_invariance()is enforced invllm/v1/attention/selector.py. Attention layers that set their backend themselves (the model packages undervllm/models/, for example MiniMax-M3's sparse attention and indexer) are not created through the selector, and none of those backends declares support, so batch-invariant mode was accepted for them without any guarantee. I found this by running the determinism suite on MiniMax-M3: it fails onmain, and still fails with #59985 applied, so it is not the router issue fixed there.Change.
init_attn_backendalready visits the backend of every attention layer; it now raises the selector's error for a backend that does not declare support whenVLLM_BATCH_INVARIANTis set. Backends that do support it only need to overridesupports_batch_invariance().Trade-offs and limitations.
Pull Request Checklist
I used vLLM's
/pr-checklistskill. (Mandatory for agents, optional for humans).AI assistance was used during the creation of this PR.
Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.
Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.
Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.
Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.
🤖 Generated with Claude Code