Extended Results
Evaluated with mini-SWE-agent · 200 tasks · Updated Aug. 16, 2026
Click row to see model details · Sorting: Resolved → Almost resolved → Avg. pass rate (more)
Score vs. average cost / turns
Hover a point for details · The line marks the Pareto frontier (best score per cost) · Click a point to see model details
Score by Model × Task
All models × 200 tasks
0%
100%
Hover for details · Click to open task
Behavioral Test Pass Rate Distribution
Double click legend item (show only this model) · Click (hide model)
Model Comparison
Each dot is one task instance · Hover for details · Click to view task