Release v0.31.1rc0: [Metrics] Expose cached prompt tokens by cache tier (#56318) · vllm-project/vllm · GitHub
Skip to content

v0.31.1rc0

v0.31.1rc0: [Metrics] Expose cached prompt tokens by cache tier (#56318)
@cquil11 cquil11 tagged this 06 Oct 18:52 
Signed-off-by: Cam Quilici <cjquilici@gmail.com>
Signed-off-by: Cam Quilici <cameron@semianalysis.com>
Co-authored-by: Cam Quilici <cameron@semianalysis.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Assets 2
Loading