Commit history

1. Commits on Oct 26, 2026

   • fix: LongMemEval repeated sessions get their own document id; retry unparseable JSON and transient 5xx; env knobs for observations and concurrency (#66)

     nicoloboschi

     authored
     3d1514cView commit details

     Copy full SHA for 3d1514c

     Browse repository at this point
2. Commits on Oct 21, 2026

   • sdebench: run Codex on the Codex subscription, and pick the Codex/Claude model by env (#63)

     Show description for 16fcebb
     nicoloboschi

     authored
     16fcebbView commit details

     Copy full SHA for 16fcebb

     Browse repository at this point
3. Commits on Oct 20, 2026

   • sdebench: Hermes Agent as a coding agent (--agent hermes) (#62)

     Show description for 9e8ad7d
     nicoloboschi

     authored
     9e8ad7dView commit details

     Copy full SHA for 9e8ad7d

     Browse repository at this point
4. Commits on Oct 14, 2026

   • fix: --query-id loads only its unit's documents; hindsight-coding waits up to an hour for a bank to settle (#60)

     nicoloboschi

     authored
     aef3702View commit details

     Copy full SHA for aef3702

     Browse repository at this point

   • retract: drop the three rag-mode hindsight results with no backing artifact (#54)

     Show description for e9d5730
     nicoloboschi

     authored
     e9d5730View commit details

     Copy full SHA for e9d5730

     Browse repository at this point
5. Commits on Oct 12, 2026

   • fix(results): label Honcho's BEAM scores as Honcho, not Hindsight RAG (#59)

     nicoloboschi

     authored
     20751faView commit details

     Copy full SHA for 20751fa

     Browse repository at this point
6. Commits on Oct 9, 2026

   • beam: keep both sides in the mission, drop answer hints, load 10M as dated sessions (#58)

     Show description for d97d496
     nicoloboschi

     authored
     d97d496View commit details

     Copy full SHA for d97d496

     Browse repository at this point
7. Commits on Oct 7, 2026

   • fix(hindsight): stop discarding facts that share a chunk with a better-ranked one (#56)

     Show description for 8ea4c83
     nicoloboschi

     authored
     8ea4c83View commit details

     Copy full SHA for 8ea4c83

     Browse repository at this point
8. Commits on Oct 6, 2026

   • harness(beam): one answer prompt for every query, and pin the judge and the seed (#55)

     Show description for 5ad91b0
     nicoloboschi

     authored
     5ad91b0View commit details

     Copy full SHA for 5ad91b0

     Browse repository at this point

   • feat(sdebench): run the plugin as installed, record attribution per task, add pi (#53)

     Show description for 6e21ebc
     nicoloboschi

     authored
     6e21ebcView commit details

     Copy full SHA for 6e21ebc

     Browse repository at this point
9. Commits on Sep 30, 2026

   • fix(sdebench): run the coding arm correctly against a remote Hindsight; feat(ui): corrections vs cost/time chart (#49)

     Show description for f618ed7
     nicoloboschi

     authored
     f618ed7View commit details

     Copy full SHA for f618ed7

     Browse repository at this point
10. Commits on Sep 22, 2026

    • fix(sdebench): stop redaction crashing runs, read the plugin's diagnostics again (#47)

      Show description for 03c1d0f
      nicoloboschi

      authored
      03c1d0fView commit details

      Copy full SHA for 03c1d0f

      Browse repository at this point
11. Commits on Sep 16, 2026

    • fix(dataset): accept user_ids in LoComo and PersonaMem load_documents (#45)

      Show description for 309e006
      nicoloboschi

      authored
      309e006View commit details

      Copy full SHA for 309e006

      Browse repository at this point

    • chore: pin the interpreter to 3.12 (#44)

      Show description for 1b49f7c
      nicoloboschi

      authored
      1b49f7cView commit details

      Copy full SHA for 1b49f7c

      Browse repository at this point
12. Commits on Sep 2, 2026

    • results: publish precisionmembench manifest and blob-backed dataset export

      Show description for 347f551
      nicoloboschi

      committed
      347f551View commit details

      Copy full SHA for 347f551

      Browse repository at this point

    • feat(precisionmembench): port the benchmark, and publish a Hindsight run

      Show description for 50b30a7
      nicoloboschi

      committed
      50b30a7View commit details

      Copy full SHA for 50b30a7

      Browse repository at this point
13. Commits on Aug 31, 2026

    • results: drop beam hindsight rag (leak) runs from manifests

      Show description for f6547b9
      nicoloboschi

      committed
      f6547b9View commit details

      Copy full SHA for f6547b9

      Browse repository at this point
14. Commits on Aug 3, 2026

    • blob-manifest: republish beam rag runs (100k/1m/500k 404ed in prod — blobs missing/stale)

      nicoloboschi

      committed
      62364d7View commit details

      Copy full SHA for 62364d7

      Browse repository at this point

    • blob-manifest: official sdebench runs published to Vercel Blob (run pages 404ed in prod — the serverless fn serves results from Blob, and the campaign runs were committed but never uploaded)

      nicoloboschi

      committed
      0a03c84View commit details

      Copy full SHA for 0a03c84

      Browse repository at this point

    • sdebench: dataset submodule bump (amb CLI docs, provider names)

      nicoloboschi

      committed
      259da95View commit details

      Copy full SHA for 259da95

      Browse repository at this point

    • chore: rename the CLI omb -> amb (#28)

      Show description for a49e450
      nicoloboschi

      authored
      a49e450View commit details

      Copy full SHA for a49e450

      Browse repository at this point

    • drop swebench_cl, ama-bench, memsim, membench

      Show description for 5595363
      nicoloboschi

      committed
      5595363View commit details

      Copy full SHA for 5595363

      Browse repository at this point

    • sdebench: does memory help a coding agent? (61-task boltons benchmark + official tri-agent results)

      Show description for 7acca55
      nicoloboschi

      authored
      7acca55View commit details

      Copy full SHA for 7acca55

      Browse repository at this point
15. Commits on Jun 23, 2026

    • prompt optimizations (#20)

      Show description for aa9273a
      nicoloboschi

      authored
      aa9273aView commit details

      Copy full SHA for aa9273a

      Browse repository at this point
16. Commits on May 8, 2026

    • Update memory provider to use Gemini 2.5 Flash (#10)

      Show description for 45fa380
      z4y4ts

      authored
      45fa380View commit details

      Copy full SHA for 45fa380

      Browse repository at this point
17. Commits on Apr 13, 2026

    • feat: add Ogham memory provider (#9)

      Show description for 84c3f96
      kevin-burns

      authored
      84c3f96View commit details

      Copy full SHA for 84c3f96

      Browse repository at this point
18. Commits on Apr 6, 2026

    • fix(ui): improve contrast and readability of small labels

      Show description for c85ad25
      cdbartholomew

      committed
      c85ad25View commit details

      Copy full SHA for c85ad25

      Browse repository at this point

    • fix: point non-beam manifest entries at real rag/ blob paths

      Show description for e708c1d
      cdbartholomew

      committed
      e708c1dView commit details

      Copy full SHA for e708c1d

      Browse repository at this point
19. Commits on Apr 1, 2026

    • feat: add scoring notes for all datasets

      Show description for 3e92541
      nicoloboschi

      committed
      3e92541View commit details

      Copy full SHA for 3e92541

      Browse repository at this point

    • chore: remove debug endpoint and logging

      nicoloboschi

      committed
      fb63307View commit details

      Copy full SHA for fb63307

      Browse repository at this point

    • fix: sync blob-manifest.json with beam entries

      nicoloboschi

      committed
      d8db0a2View commit details

      Copy full SHA for d8db0a2

      Browse repository at this point

    • debug: add blob manifest debug endpoint

      nicoloboschi

      committed
      2acd80cView commit details

      Copy full SHA for 2acd80c

      Browse repository at this point

    • debug: add logging to blob URL resolution

      nicoloboschi

      committed
      b621e4aView commit details

      Copy full SHA for b621e4a

      Browse repository at this point

    • fix: remove stale hindsight-beam entry from results manifest

      nicoloboschi

      committed
      8c8b832View commit details

      Copy full SHA for 8c8b832

      Browse repository at this point

    • fix: remove duplicate rag mode entries from results manifest

      nicoloboschi

      committed
      2aab8beView commit details

      Copy full SHA for 2aab8be

      Browse repository at this point