Installation – Axolotl

Installation

This guide covers all the ways you can install and set up Axolotl for your environment.

1 Requirements

  • NVIDIA GPU (Ampere architecture or newer for bf16 and Flash Attention) or AMD GPU
  • Python ≥3.12
  • PyTorch ≥2.13.0 (2.13.0 recommended)

2 Installation

Important

For Blackwell GPUs, please use PyTorch ≥2.11.0 and CUDA 13.0. CUDA 12.8 cannot compile for sm_103a (B300), and using CUDA 13.0 also avoids first-launch JIT overhead on B100 / B200 / RTX 50-series.

2.1 Quick Install

Axolotl uses uv as its package manager. uv is a fast, reliable Python package installer and resolver built in Rust.

Install uv if not already installed:

curl -LsSf https://astral.sh/uv/install.sh | sh
source $HOME/.local/bin/env

Choose your CUDA version (e.g. cu128, cu130), create a venv, and install:

export UV_TORCH_BACKEND=cu130  # or cu128
uv venv
source .venv/bin/activate
uv pip install --no-build-isolation axolotl[deepspeed]

2.2 Offline or Air-Gapped Install

For a machine without internet access, including Slurm compute nodes where only the login node is online, build a bundle on a connected host and install from it. See Offline and Air-Gapped Deployment.

2.3 Edge/Development Build

For the latest features between releases:

git clone https://github.com/axolotl-ai-cloud/axolotl.git
cd axolotl
export UV_TORCH_BACKEND=cu130  # or cu128
uv venv
source .venv/bin/activate
uv pip install --no-build-isolation -e '.[deepspeed]'

2.4 Docker

docker run --gpus '"all"' --rm -it --ipc=host axolotlai/axolotl-uv:main-latest

For development with Docker:

docker compose up -d
TipAdvanced Docker Configuration 
docker run --privileged --gpus '"all"' --shm-size 10g --rm -it \
  --name axolotl --ipc=host \
  --ulimit memlock=-1 --ulimit stack=67108864 \
  --mount type=bind,src="${PWD}",target=/workspace/axolotl \
  -v ${HOME}/.cache/huggingface:/root/.cache/huggingface \
  axolotlai/axolotl-uv:main-latest
Important

For Blackwell GPUs, please use axolotlai/axolotl-uv:main-py3.12-cu130-2.13.0 or the cloud variant axolotlai/axolotl-cloud-uv:main-py3.12-cu130-2.13.0. For expert-parallel MoE training on B200 (DeepEP, flash-attn-4, SonicMoE), use axolotlai/axolotl-cloud-uv:main-py3.12-cu130-2.14.1-blackwell, which ships those kernels prebuilt; see the Docker documentation.

Please refer to the Docker documentation for more information on the different Docker images that are available.

3 Cloud Environments

3.1 Cloud GPU Providers

For providers supporting Docker:

3.2 SkyPilot

To launch Axolotl on your own cloud accounts (AWS, GCP, Azure, etc.) or Kubernetes cluster with a single command, see the SkyPilot guide.

3.3 Google Colab

4 Platform-Specific Instructions

4.1 macOS

uv pip install --no-build-isolation -e '.'

See Section 7 for Mac-specific issues.

4.2 Windows

Important

We recommend using WSL2 (Windows Subsystem for Linux) or Docker.

5 Migrating from pip to uv

If you have an existing pip-based Axolotl installation, you can migrate to uv:

# Install uv
curl -LsSf https://astral.sh/uv/install.sh | sh
source $HOME/.local/bin/env

# Create a fresh venv (recommended for a clean start)
export UV_TORCH_BACKEND=cu130  # or cu128
uv venv
source .venv/bin/activate

# Reinstall axolotl
uv pip install --no-build-isolation axolotl[deepspeed]

6 Using pip (Alternative)

If you are unable to install uv, you can still use pip directly.

Important

Please make sure to have PyTorch installed before installing Axolotl with pip.

Follow the instructions at: https://pytorch.org/get-started/locally/

pip3 install -U packaging setuptools wheel ninja
pip3 install --no-build-isolation axolotl[deepspeed]

For editable/development installs:

pip3 install -U packaging setuptools wheel ninja
pip3 install --no-build-isolation -e '.[deepspeed]'

7 Troubleshooting

If you encounter installation issues, see our FAQ and Debugging Guide.