iframe-proxy

MatthewCYM · 2026-09-16T09:24:08Z

Purpose

Qwen3-ASR currently inherits Omni's waveform padding and forces a Whisper feature extractor, which can change audio features and placeholder counts relative to the checkpoint's upstream processor.

Following the review feedback, this revision:

Uses the upstream Transformers Qwen3ASRProcessor and removes the bundled processor and its registration.
Requires Transformers >=5.13.0 and updates the corresponding test metadata.
Uses BaseMultiModalProcessor without Omni's waveform padding.
Processes clips independently and packs valid features, preserving parity across differently sized clips and partial cache hits.
Preserves text-only and precomputed-feature inputs.

Both checkpoint formats remain supported: the original checkpoint declares WhisperFeatureExtractor, while the native -hf checkpoint declares Qwen3ASRFeatureExtractor.

No shared Omni/VL implementation, model weights, encoder kernels, or speculative-decoding behavior is changed.

Test Plan

Validated against upstream main 528fa835fd37d0805a824d516e22f88df61a771a
with Transformers 5.13.0 and 5.17.0.

.venv/bin/python -m pytest \
  tests/models/multimodal/processing/test_qwen3_asr.py \
  tests/models/multimodal/processing/test_qwen3_omni.py \
  tests/multimodal/test_audio.py \
  tests/models/multimodal/processing/test_common.py \
  tests/model_executor/test_qwen3_asr_mrope.py \
  tests/entrypoints/speech_to_text/transcription/test_qwen3_asr_sanitize_prompt.py \
  tests/entrypoints/speech_to_text/transcription/test_transcription_inter_chunk_spacing.py \
  -k 'not transformers and (Qwen3-ASR or test_qwen3_asr or test_qwen3_omni or test_audio.py or ForcedAligner or test_transcription_inter_chunk_spacing)' \
  -q

152 repository tests passed per Transformers version, including all 50 new regression cases.
Supplementary local validation passed another 63 pytest cases and four realtime-processor comparisons per version. These supplementary checks are not all included in this PR.
Applicable repository pre-commit hooks, the scoped manual Python 3.12 mypy hook, and whitespace checks passed.

Model validation

Tested both Qwen/Qwen3-ASR-0.6B and Qwen/Qwen3-ASR-0.6B-hf with both Transformers versions.

Each run compared raw audio against independently computed upstream features in the same vLLM engine, using 32 LibriSpeech test-clean utterances plus 32 near-hop-boundary variants. Settings: single H100, BF16, greedy decoding, eager execution, max_num_seqs=1, and processor/prefix caches disabled.

All four model/version combinations matched prompt IDs and generated token IDs on 64/64 clips. Raw/reference WER was 2.8846% in every run. This is a bounded correctness evaluation, not a full-split accuracy or speed benchmark.

Environment: Python 3.13.15, PyTorch 2.13.0+cu130, and NumPy 2.4.6. Tests used this checkout's Python source with existing vLLM 0.29 CUDA extensions, not a fresh CUDA build. Full upstream CI remains necessary. Realtime and forced-aligner checks cover preprocessing, not complete streaming or timestamp-accuracy evaluation.

Related work and disclosure

This uses the processor hook introduced by #53610; that refactor does not implement this ASR preprocessing fix. #57076 addresses the separate post-expansion truncation issue.

AI assistance was used for implementation, investigation, and test preparation.

claude

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

github-actions · 2026-09-16T09:31:16Z

DarkLight1337 · 2026-09-16T09:38:02Z

@ywang96 @Isotr0py would it be better to drop the native processor implementation and simply require transformeres>=5.13?

MatthewCYM · 2026-09-16T09:53:45Z

Hi @DarkLight1337, I retained the bundled processor to preserve compatibility with Transformers 5.10.4 and existing checkpoints. Happy to switch to the upstream Transformers processor if requiring >=5.13 is preferred.

Aatman09 · 2026-09-17T04:03:35Z

Why didn't you use the Qwen3ASRFeatureExtractor and instead used SequentialFeatureExtractor

Isotr0py · 2026-09-17T04:06:34Z

would it be better to drop the native processor implementation and simply require transformeres>=5.13

Sure, we should remove the native ones, we only port them because of missing transfomers support at day0