Pinned Loading

1. vllm-project/vllm vllm-project/vllm Public

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python 94.6k 23.7k

2. vllm-project/llm-compressor vllm-project/llm-compressor Public

   State-of-the-art LLM compression, built for production inference with vLLM

   Python 3.9k 703

3. llm-d/llm-d llm-d/llm-d Public

   Achieve state of the art inference performance with modern accelerators on Kubernetes

   Python 4.9k 869

4. neuralmagic/deepsparse neuralmagic/deepsparse Public archive

   Sparsity-aware deep learning inference runtime for CPUs

   Python 3.2k 193

5. advos advos Public

   RISC-V OS in Rust with hardware support for SiFive's HiFive1 board

   Rust 1 1

6. torch_bitmask torch_bitmask Public

   Implementations of bitmask compression for weight sparsity in PyTorch

   Python 5 2