chore: add native streaming placement qualification by LOGO127 · Pull Request #447 · vllm-project/agentic-api · GitHub
Skip to content

chore: add native streaming placement qualification - #447

Draft
LOGO127 wants to merge 1 commit into
vllm-project:mainfrom
LOGO127:test/streaming-placement-245-0b98-20261014
Draft

LOGO127 wants to merge 1 commit into
vllm-project:mainfrom
LOGO127:test/streaming-placement-245-0b98-20261014

Conversation

@LOGO127

@LOGO127 LOGO127 commented Oct 15, 2026

Copy link
Copy Markdown
Contributor

Summary

Related to #245. Draft qualification infrastructure, not a completed placement decision or a production offloading change.

  • Exercise the existing private RoundIngestion and native StreamRelay in equivalent inline and bounded-worker placements without widening the core API or adding another parser, folder or emitter.
  • Reuse named cassette response inputs and native HTTP/SSE framing; keep complete payload/wire gates separate from timing. Cover bounded entries/logical bytes, concurrent feed/drain, cancellation, disconnects, primary errors and observed joins.
  • Add explicit opt-in matrix/resource/queue qualification with shared-origin replay schedules, independent replica windows and unknown quantiles preserved. HTTP measurement deadlines cancel and await their batch, including warmups.

Production ingestion remains inline. Please advise whether this crate-internal opt-in arrangement and review/integration split are the preferred boundary. The earlier #245 scope question remains open. This is a substantial test-only candidate, not a request to infer readiness from its size or test count.

Test Plan

Current local candidate tree: 3ca01a413ca062bddd72c00064935cb740d56c8f, based on 0b98f4baa8963e4e51d2ae816813266bc651fcf0.

cargo +1.98.1 test --locked -p agentic-server-core --lib -- --test-threads=2
cargo +1.98.1 clippy --locked -p agentic-server-core --lib --tests -- -D warnings
cargo +1.98.1 fmt --all -- --check
  • Final core library: 1,172 passed, 0 failed; 18 tests remained explicitly ignored. Strict Clippy, workspace formatting and all applicable candidate-file pre-commit hooks passed.
  • The HTTP teardown regression covers inline/worker owners. A controlled old-behavior mutation failed the joined-client assertion; the fix was restored and verified. This is mutation evidence, not a pristine upstream baseline comparison.
  • Earlier source 82b56831... had completed full-output gates and separately retained inline/fresh-worker records. Those remain historical evidence on that source pin, not this new tree's performance results. The interrupted prefix is excluded; the available arms are cross-session, not one continuous causal experiment.

The new tree's opt-in release/full timing was not rerun. No real-model/GPU performance, complete gateway transport/tool-loop/storage benchmark, exact OS thread peak or placement recommendation is claimed. Input/output queue bounds are currently coupled, not a complete independent asymmetric sweep. These remaining measurement/decision boundaries must be agreed before #245 can be considered complete.

Signed-off-by: luozijian <luozijian0924@gmail.com>
@LOGO127

LOGO127 commented Oct 21, 2026

Copy link
Copy Markdown
Contributor Author

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant