feat: add shared vLLM Responses event contract by franciscojavierarceo · Pull Request #382 · vllm-project/agentic-api · GitHub
Skip to content

feat: add shared vLLM Responses event contract - #382

Draft
franciscojavierarceo wants to merge 1 commit into
vllm-project:mainfrom
franciscojavierarceo:codex/shared-vllm-inference
Draft

franciscojavierarceo wants to merge 1 commit into
vllm-project:mainfrom
franciscojavierarceo:codex/shared-vllm-inference

Conversation

@franciscojavierarceo

@franciscojavierarceo franciscojavierarceo commented Sep 25, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Introduces the published vllm-responses crate as the first shared boundary for Agentic API and the vLLM Rust frontend.

  • Define a transport-independent, typed inference-event contract with no JSON or HTTP types at its public boundary.
  • Adapt Agentic response assembly through the same strict lifecycle machinery used by HTTP/SSE.
  • Preserve response IDs, output identity, function calls, reasoning, usage including cached prompt tokens, failure/incomplete state, and terminal invariants.
  • Publish vllm-responses before agentic-server-core, with dependency-aware release validation and documentation.
  • Keep native tests in #[cfg(test)] modules so production server artifacts do not include test code.

Scope and non-duplication

This implements the initial contract/ingestion milestone for #359 and #361. There is no open Agentic API PR for this shared crate or typed native ingestion; the currently open #350 concerns provider-aware reasoning replay.

It deliberately does not add the vLLM ChatLlm adapter (#362), gRPC/llm-d transport, persistence, or gateway tool execution. vLLM endpoint work remains coordinated with vllm-project/vllm#53380 rather than duplicated here.

Test plan

  • cargo fmt --all -- --check
  • cargo test -p vllm-responses -p agentic-server-core --lib (779 passed, 5 ignored, after rebasing onto current main)
  • cargo clippy -p vllm-responses -p agentic-server-core --all-targets -- -D warnings
  • cargo test -p agentic-server --test responses_test test_prompt_cache_key_matches_pass_through_and_typed_sse_execution -- --exact
  • cargo publish --dry-run --allow-dirty --locked for vllm-responses, core, and server with the staged local dependency overrides
  • uv run --no-project --with tree-sitter --with tree-sitter-rust python scripts/check_rust_file_sizes.py

No model-output or model-accuracy behavior changes in this PR, so model evaluation is not applicable. The vLLM adapter is intentionally not present yet.

AI assistance

This PR was developed with Codex assistance. The shared contract, lifecycle policy, release order, and test results were reviewed by the human submitter; Claude Fable also performed a read-only Rust review whose actionable findings were addressed.

Co-authored-by: Codex <codex@openai.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant