a community oriented 1:1, vLLM-alike (Continuous batching, paged KV) engine in C++ with additional features (GGUF, RadixAttention, Cache-aware scheduling, ...)
-
Updated
Sep 6, 2026 - C++
{{ message }}
#
continous-batching
Here are 3 public repositories matching this topic...
Benchmarking seven LLM optimization techniques across memory, latency, throughput, batching, and distributed-inference trade-offs.
transformers pytorch quantization knowledge-distillation pipeline-parallelism kv-cache tensor-parallelism llm qwen speculative-decoding llm-optimization continous-batching
-
Updated
Sep 3, 2026 - Jupyter Notebook
A minimalist, educational deep-dive into the vLLM architecture.
-
Updated
Feb 20, 2026 - Python
Add this topic to your repo
To associate your repository with the continous-batching topic, visit your repo's landing page and select "manage topics."