Overview

Ensure the Rust /v1/completions stream includes token_ids on every choice when return_token_ids=true.

Claims

• Prompt chunks serialize token_ids: [] alongside prompt_token_ids.
• Generated-token chunks preserve their actual token IDs.
• Terminal chunks serialize token_ids: [].
• Requests with return_token_ids=false are unchanged.

Validation

• Manually verified the affected streaming behavior with return_token_ids=true.

• Added completion_stream_serializes_requested_token_ids_on_every_choice, which serializes the prompt, generated-token, and terminal chunks and asserts their exact token_ids values.

• The regression test is covered by the existing Rust Frontend Cargo CI for changes under rust/.

• Focused pre-commit checks passed:

  pre-commit run --files rust/src/server/src/routes/openai/completions.rs

• Rust formatting passed locally in a Rust 1.95 container:

  cargo fmt --manifest-path rust/Cargo.toml --all -- --check

• The targeted regression test passed locally in a Rust 1.95 container:

  cargo test --manifest-path rust/Cargo.toml -p vllm-server \
    completion_stream_serializes_requested_token_ids_on_every_choice

Details

CompletionStreamChoice skips fields whose value is None. Generated deltas already populated token_ids, but prompt and terminal choices left the field as None, so it disappeared from serialized responses even when requested.

The Python frontend already enforces this contract in OpenAIServingCompletion.completion_stream_generator: every emitted CompletionResponseStreamChoice sets token_ids to as_list(output.token_ids) whenever return_token_ids is enabled. This explicitly supplies a list, including an empty list, instead of leaving the optional field unset. The Rust frontend only set the field on generated-token chunks, so prompt and terminal choices omitted it.

The existing stream-yield path now fills only missing token_ids values with an empty list when return_token_ids is enabled. Centralizing this at emission covers every completion choice while preserving real IDs already attached to generated-token chunks. The only payload change is an empty list on opt-in prompt and terminal chunks; no dependencies or public request fields change.