LLM Router with RL Bandit

Kubernetes-native LLM routing service using Thompson Sampling bandit for intelligent model selection.

Features

• ✅ Reinforcement Learning - Multi-armed bandit learns optimal model routing
• ✅ Kubernetes Native - Deployments, Services, ConfigMaps, HPA, Ingress
• ✅ Feedback Loop - Every request logged for analysis
• ✅ Health Checks - Liveness/readiness probes
• ✅ Horizontal Scaling - Auto-scales based on CPU/memory
• ✅ Multi-Model Support - GPT-4, Llama (via Ollama), extensible

Quick Start

Local Development (Docker Compose)

make local