Greg Kamradt

@GregKamradt

President @arcprize, builder/engineer

San Francisco, CA

agnts.sh/greg

Joined January 2011

1,039 Following

51.3K Followers

Replies Reposts Media Articles

Pinned
Greg Kamradt
@GregKamradt
Mar 25
Today we're launching ARC-AGI-3 135 Novel Environments (nearly 1K levels) we build by hand It is the only unsaturated agent benchmark in the world Each game is 100% human solvable, AI scores <1% This gap between human and AI performance proves we do not have AGI Agents today
98
Greg Kamradt
@GregKamradt
10h
Abstractions find a way
Elon Musk
@elonmusk
12h
Replying to @willdepue
Bitstream in, bitstream out
1
Greg Kamradt
@GregKamradt
16h
This was fun - thanks for having @arcprize and me @kaggle
Kaggle
@kaggle
17h
@zapier, one of our Bench Summit partners, led a panel discussion on the realities of managing highly influential evaluation signals. The panel, moderated by Lukas Bergstrom (@lukasb), included insights from Robin Salimans (@SalimansRobin) of Zapier, Greg Kamradt (@GregKamradt)
1
Greg Kamradt
@GregKamradt
22h
Im still trying to understand why some data points break away from the expected v2/3 trend Argon low seems to do a lot better on v3 than its v2 would suggest it should
ARC Prize
@arcprize
22h
Replying to @arcprize
On ARC-AGI-3, there is a similar relationship: models that score higher on ARC-AGI-2 tend to also score higher on ARC-AGI-3. One notable outlier is Gemini 4 Argon at low reasoning, which scored only 42.1% on ARC-AGI-2 but 97.5% on ARC-AGI-3 with the provider adapter harness.
1
Greg Kamradt
@GregKamradt
22h
Plotting v1 vs v2 scores show a smooth progression V2 was able to extend the usefulness of the original arc agi format at higher model levels In fact we could probably get a difficulty per task at this point with all the data we have If we did a v2.5 (we have no plans to) we
ARC Prize
@arcprize
22h
Plotting ARC-AGI model scores reveals a strong relationship across all three benchmarks. For example, the power curve v2 = v1^4.53 explains about 98% of the variation in ARC-AGI-2 scores. Models tied on ARC-AGI-1 can still differ by over 20 percentage points on ARC-AGI-2.
1

Sunbelt Computer Software

PL/B Language Development and Support

Greg Kamradt