@codex review

@codex review

Codex Review: Didn't find any major issues. Delightful!

Reviewed commit: c8a2e85d9c

Codex Review: Didn't find any major issues. Bravo.

Reviewed commit: fbd785d72d

The gate treats "no visible GPU's SM is in supported_sms" as "no kernel image exists" (studio/backend/core/inference/llama_cpp.py:6066, raised at :15926-15928). That equivalence only holds in one direction, so the gate refuses launches that work.

The CUDA bundles are built with plain arch numbers, not -real, so every fatbin carries PTX: archs:"75 80 86 89" for cuda13-older and "50-virtual 61-virtual" for cuda12-legacy in unslothai/llama.cpp/.github/workflows/unsloth-prebuilt.yml:392-396, fed to -DCMAKE_CUDA_ARCHITECTURES at unsloth-prebuilt-cuda.yml:185. cuobjdump on the published b10360-mix-87da1a2 assets confirms it:

• linux-x64-cuda13-older: 146 cubins and 146 PTX images each for sm_75/80/86/89.
• linux-x64-cuda12-legacy: 0 cubins, 146 PTX images each for compute_50 and compute_61.
• linux-x64-cuda13-newer: lowest image of either kind is compute_86.

Host-side kernel selection is clamped by ggml_cuda_highest_compiled_arch(cc) (ggml/src/ggml-cuda/common.cuh:165), so a newer card running older PTX does not reach a NO_DEVICE_CODE trap.

Measured on a host with an RTX 6000 Ada (sm_89) and an RTX 3090 (sm_86), driver 580.173.02. The cuda12-legacy bundle, whose published marker is supported_sms = ["50","52","60","61"] and therefore has zero overlap with either card, runs full GPU inference:

ggml_cuda_init: found 2 CUDA devices
  Device 0: NVIDIA RTX 6000 Ada Generation, compute capability 8.9
  Device 1: NVIDIA GeForce RTX 3090, compute capability 8.6
| llama ?B all F32 | CUDA | ngl 99 | pp8 | 11875.36 t/s |
| llama ?B all F32 | CUDA | ngl 99 | tg8 |  4246.14 t/s |

_cuda_sm_gate_error on that same host, with that bundle's real marker values, returns:

The installed llama.cpp build only has GPU code for sm_50-sm_61, but GPU 0 is sm_89,
GPU 1 is sm_86 -- it was likely installed on a machine with a different GPU, so run
`unsloth studio update` to reinstall the llama.cpp runtime.

The refusal is raised before the spawn and outside the --fit fallback, so a working install becomes an unconditional failure. It reaches offline hosts and LLAMA_SERVER_PATH-pinned installs that cannot be updated.

The direction that genuinely aborts is the reverse one: a bundle newer than the GPU, such as cuda13-newer (lowest image compute_86) on an sm_75 host, where there is neither a cubin nor back-compatible PTX. host_sm < min(supported_sms) selects that case and leaves the working one alone.

studio/backend/tests/test_cuda_sm_gate.py:114-123 encodes the upgrade case, cuda13-older on an sm_90 host, as "the incident shape". That bundle ships compute_89 PTX, so the test asserts a refusal for a configuration that runs.

On JIT cost, with an isolated CUDA_CACHE_PATH the cold cuda12-legacy run took 35s with default lazy module loading and 235s under CUDA_MODULE_LOADING=EAGER (368 MB cache), both inside the 600s health wait at llama_cpp.py:16140.

The rest of the change reads correctly: the marker plumbing at install_llama_prebuilt.py:6077 and :6212-6214, the fingerprint-guarded reuse backfill, the CUDA_DEVICE_ORDER fail-open, and the fail-open paths for non-CUDA, ROCm, CPU and source installs. The coverage predicate is the blocker.

Codex Review: Didn't find any major issues. More of your lovely PRs please.

Reviewed commit: cd7998bc47

Reviewed this one closely because it is the only PR here that can refuse a launch that currently works. Verdict: the change is sound and worth landing, but the PR description argues the wrong direction and should be corrected before merge.

Before: the prebuilt marker recorded no CUDA SM coverage. A bundle baked for one GPU could reach llama-server on a host it had no kernels for, and the child aborted on every launch with a generic error plus retries.

After: supported_sms is written on install and backfilled on reuse, and a pre-spawn gate refuses the launch with an actionable message when every visible GPU is older than the oldest arch the bundle was compiled for.

The description is backwards. It gives a T4-baked bundle failing on an H100 as the motivating case. That is the direction that works: the bundles compile plain arch numbers, so each fatbin carries PTX beside its cubins and the driver JITs forward onto a newer card, exactly as ggml_cuda_highest_compiled_arch expects. The genuinely broken direction is the opposite one, newer-baked onto an older GPU, which is what the code now gates on after cd7998bc. The implementation is right; please fix the example so nobody "corrects" the code back to exact-SM matching later.

Simulation. Isolated uv venv, Python 3.11, no NVIDIA hardware: nvidia-smi, the marker and the visibility masks are faked in the shapes the installer really writes.

• New studio/backend/tests/test_cuda_sm_gate_os_matrix.py, the sibling of test_gpu_arch_gate_os_matrix_7624.py that the ROCm gate already has. [Windows, Linux, WSL, macOS] x [NVIDIA, AMD, CPU-only], spying the marker reader and the nvidia-smi probe so that "inert" is actually proven. A gate that ran and passed returns None exactly like one that never ran, so matching output alone shows nothing.
• Floor decision, including sm_121 PTX not JITing down to sm_120 and sm_120 to sm_121 being fine, a mixed host passing on its newest card, and unreadable or failing probes failing open.
• Visibility masks: a numeric mask without CUDA_DEVICE_ORDER=PCI_BUS_ID fails open, an empty mask hides everything, and a UUID or MIG mask is dropped so the whole host is weighed.
• Two installer invariants that nothing checked and that the design depends on: supported_sms must stay out of expected_install_fingerprint, or every install predating the field compares stale and reinstalls on upgrade, and a non-CUDA bundle must record [], which is the only reason the call site needs no is_vulkan_backend guard.

440 tests pass across the gate and installer suites.

Old installs. A marker without supported_sms reads as unknown coverage and fails open, so nothing that works today starts refusing after an upgrade. The gate is also correctly placed after argv and env are resolved, so a CPU-only zero-offload launch, which masks the GPUs anyway, is exempt.

Pushed the matrix and the two invariant tests to this branch. No behaviour change from me.

One thing left as a note rather than a change: _cuda_compute_caps shells out to nvidia-smi with a 10s timeout on every load and is not memoised. That is a per-load subprocess on the critical path, worth a cache if load latency ever matters.

On UI evidence for this one: there is a visible effect, since the gate surfaces as a load error in place of a repeated failed spawn, but I could not produce an honest before/after pair on this host and did not want to post a misleading one.

The blocker is that the effect needs a bundle that genuinely lacks kernels for the host GPU. This box is sm_100 and its installed bundle covers it correctly, so the gate is silent here by design. I can doctor supported_sms in the marker to make the gate fire, but that produces the wrong picture: the BEFORE side ignores the marker and loads fine, so the pair would read as "this PR refuses a model that used to work" and would be showing my edited metadata rather than the incompatibility the PR exists for. Faking the failure this way is worse than having no picture.

The decision itself is pinned by tests instead, which is the more useful artefact here anyway since the claim is a decision table rather than an appearance: 91 tests across test_cuda_sm_gate.py and the new test_cuda_sm_gate_os_matrix.py, covering [Windows, Linux, WSL, macOS] x [NVIDIA, AMD, CPU-only] with the marker reader and the nvidia-smi probe spied so that "inert" is proven rather than inferred, plus the floor decision, the visibility masks, and both directions of sm_120 against sm_121.

@codex review

@codex review

@codex review

@codex review

Codex Review: Didn't find any major issues. 👍

Reviewed commit: f3d61a3124

One correction to the description, the code itself is fine.

The description says a bundle baked for an older arch (T4) fails on a newer GPU (H100). That is the wrong way round. Plain -arch=sm_XX numbers embed PTX alongside the SASS, and PTX JITs forward, so an older-baked bundle generally still runs on a newer card. The direction that actually breaks is the reverse: a bundle baked only for newer arches has no SASS and no usable PTX for an older card, which is exactly what this gate catches.

Worth fixing the wording so the next reader does not take the gate to be guarding the opposite case.

@codex review

Codex Review: Didn't find any major issues. Keep it up!

Reviewed commit: 9d1dcfe58a