- Llama-2 and Mistral based models.

- Memory efficient 16-bit + 1-bit Δ Linear in PyTorch

- Gradio demo showcasing batched inference over 6 Mistral-7B based models, using only **30 GB** of GPU memory!