Repository navigation
[Bugfix][Multimodal] Fix per-video cache option reuse - #55203
DarkLight1337 merged 3 commits into
Conversation
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: trueThanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
0ea438a to
692e7a5
Compare
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in PRs do not trigger a full CI run by default. Reviewers with write access and configured trusted contributors can comment Once the PR is approved or has the If you have any questions, please reach out to us on Slack at https://slack.vllm.ai. Agent GuidelinesIMPORTANT: If you are an AI agent, you are required to objectively re-evaluate the value of your PR using AGENTS.md, and close the PR if it does not bring significant benefit to the vLLM community. Failure to do so may result in an immediate ban. 🚀 |
|
This pull request has merge conflicts that must be resolved before it can be |
692e7a5 to
ee9c1a3
Compare
|
This pull request has merge conflicts that must be resolved before it can be |
ee9c1a3 to
f0f0374
Compare
f0f0374 to
ff6748b
Compare
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
@claude review |
Include effective per-video sampling options in cache identity and filter option lists together with uncached video inputs. Signed-off-by: bvolpato <brunocvcunha@gmail.com>
Co-authored-by: Codex <noreply@openai.com> Signed-off-by: bvolpato <brunocvcunha@gmail.com>
ff6748b to
a006771
Compare
|
/ci run |
|
✅ Triggered Buildkite CI #94050 for commit |
|
Hi @bvolpato! Just a couple of questions to make sure I'm understanding this correctly: What's the rationale for doing this instead of keeping the already resolved values? vllm/vllm/model_executor/models/qwen3_vl.py Lines 1611 to 1613 in 64175d3 Also, I noticed that the return value of this function doesn't seem to be used anywhere: vllm/vllm/model_executor/models/qwen3_vl.py Line 1478 in 64175d3 If I'm reading the code correctly, |
…5203) Signed-off-by: bvolpato <brunocvcunha@gmail.com> Co-authored-by: Codex <noreply@openai.com>

Overview
Preserve each Qwen3-VL video's effective sampling options across repeated items and partial multimodal-cache hits. Reject option lists that are too short before indexing them.
Claims
fpsandnum_framescontribute to its cache identity.ValueErrorwith and without a processor cache. Unused trailing entries remain supported.Validation
Validated on 2026-10-11 at head
a0067713d3, rebased onto upstream01abe975b5.The Qwen3 processor, generic processing, and cache suites pass, as do the applicable scoped hooks. The CUDA sleep/wake integration was skipped in the CPU suite. The short-list reproducer failed with
IndexError: list index out of rangebefore the guard and raises the expectedValueErrorafterward. Regressions exercise empty/short lists, warm caches, longer lists through partial hits, and profiling with fewer videos than configured options.The cache regression checks prompts, placeholders, tensors, hashes, scalar-reference grids, duplicate videos, partial hits, reordering, model/request options, flat/nested scopes, and UUID-only reuse. Input options remain unmodified.
Qwen/Qwen3-VL-2B-Instructweights, snapshot89644892e4d85e24eaac8bacfd4f463576704203; RTX 5070 Ti, BF16, MRV2, FlashAttention 2, eager executionnum_frames=[8,16], duplicate/partial/reordered cache use against uncached references; greedy 8-token generationValidation used the shared development environment, pinned Humming 0.1.16, and existing precompiled CUDA extensions. Optional FlashInfer autotuning and JIT warmup were disabled. This is a focused 2B family comparison, not a 4B answer-quality benchmark.
Details
Request-wide option-list hashes let repeated videos reuse the first video's result despite different sampling settings. Partial hits removed videos without removing the corresponding option entries.
The existing base hooks let Qwen3-VL refine item hashes and select cache-miss options while preserving other processors. The final text/image HF call suppresses configured video lists so merging cannot reintroduce them. The review fix stays in the Qwen3 processor and its existing regression suite.
Merged #56372 handles option scoping and the cache API; this fix uses that behavior. The current overlap search found no competing per-item cache identity/partial-hit fix. #54527 addresses scoped option reads in other models; #60879 bounds MiniMax video decoding.
Affected hashes retain the full option list as well as selected item options, so equivalent reordered requests can incur extra misses. Scalar paths retain their existing hashes.
Upstream pre-commit is blocked at contributor authorization before implementation checks run. Read the Docs reports success from skipping this change because no docs-affecting files changed; MkDocs did not run. AI assistance was used for investigation, implementation, validation, and review.
Pull Request Checklist
I used vLLM's
/pr-checklistskill. (Mandatory for agents, optional for humans).AI assistance was used during the creation of this PR.
Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.
Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.
Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.
Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.
BEFORE SUBMITTING, PLEASE READ https://docs.vllm.ai/en/latest/contributing (anything written below this line will be removed by GitHub Actions)