TensorZero · open-source LLM infrastructure
TensorZero Logo

TensorZero

GITHUB TRENDING
#1 Repository of the Week
1
TensorZero Autopilot is an automated AI engineer that analyzes LLM observability data, sets up evals, optimizes prompts and models, and runs A/B tests.
It's powered by our open-source LLMOps platform that unifies an LLM gateway, observability, evaluation, optimization, and experimentation.

TensorZero is used by companies ranging from frontier AI startups to the Fortune 10 and fuels ~1% of global LLM API spend today.

What is TensorZero Autopilot?

Think of it like Claude Code for LLM engineering.

It dramatically improves the performance of LLM agents across diverse tasks:

Bar chart showing baseline vs. optimized scores across diverse LLM tasks

For example, it can:

  • Analyze millions of inferences to surface error patterns and optimization opportunities
  • Set up evaluations, prevent regressions, and align LLM judges to real-world scenarios
  • Recommend models and inference strategies to improve quality, cost, and latency
  • Generate and refine prompts based on human feedback, metrics, and evaluations
  • Drive optimization workflows like fine-tuning, reinforcement learning, and distillation
  • Run A/B tests to validate changes, identify winners, and close the feedback loop

Learn more →  Schedule a demo →

What is the TensorZero Stack?

TensorZero Stack is an 11.2Kopen-source LLMOps platform that unifies:

  • Gateway: access every LLM provider through a unified API (<1ms p99 latency)
  • Observability: monitor your LLM systems, programmatically or with a UI
  • Evaluation: benchmark individual inferences or end-to-end workflows
  • Optimization: optimize your prompts, models, and inference strategies
  • Experimentation: deploy with built-in A/B testing, fallbacks, etc.

You can take what you need, adopt incrementally, and complement with other tools. It plays nicely with the OpenAI SDK, OpenTelemetry, and every major LLM provider.

Our Quick Start shows how to set up a production-ready LLM application with observability and fine-tuning in just 5 minutes.

Read the docs →

How can I ask questions or share feedback?

Reach out on Slack, Discord, or GitHub. Alternatively, send us an email.

Who is building TensorZero?

  • Aaron Hill Aaron Hill: Rust compiler maintainer, OSS contributor (Rust, Lean), Svix, AWS
  • Alan Mishler Alan Mishler: VP at J.P. Morgan AI Research, CMU PhD (stats), 1.3k+ citations
  • Andrew Jesson Andrew Jesson: Columbia postdoc and Oxford PhD (LLMs), 4k+ citations
  • Antoine Toussaint Antoine Toussaint: staff SWE, quant, Stanford math professor, Princeton PhD
  • Gabriel Bianconi Gabriel Bianconi (CEO): CPO at Ondo Finance (DeFi decacorn), Stanford BS & MS
  • Michelle Hui Michelle Hui: ML + product + community (Wing / Alphabet, UN), Cornell BS & MS
  • Viraj Mehta Viraj Mehta (CTO): CMU PhD (reinforcement learning), Stanford BS & MS

We’re backed by FirstMark, Bessemer, Bedrock, and dozens of angels. See our $7.3M seed round announcement and coverage from VentureBeat.

We’re also hiring in NYC.