You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Dream Cycle 2026-10-12] darwin-evolution: Tier-1 mock-sandbox's memoryThreshold extracted but never consumed (fitness-blind surface) + weight-eft,learn scan #396
Last ledger row on main: 2026-10-09 (host-adapters/kernel, PR #386, ACCEPT, merged). 13 dream/* PRs currently open (#360-#392); only 1 merge (#386) in the last 14 nights. Oldest open darwin-evolution PR unrelated to tonight's finding: #371 (2026-10-02, safety.ts optional-chaining bypass, ACCEPT) and #379 (2026-10-07, Tier-2 toolPolicy ordering measurement-blindness, ACCEPT) — both reviewed for duplicate-direction risk before selecting tonight's candidate (see below). Oldest overall open PR (#360, 2026-09-27) is now 15 days old, past the 14-day STALE threshold — restated once per STEP 1.1 precedent (already flagged repeatedly 09-15 through 10-09), not re-escalated.
Prior gist self-score (2026-10-09, host-adapters/kernel): 10/10 — benchmark evidence grade A (real concurrency stress repro), 5+ competitor rows, 5 specific recommendations, valid witness, under 1500 words, genuinely novel (independent duplication of the same bug into a scaffold-template copy).
Deep Dive Findings
Darwin Mode's Tier-1 "mock" sandbox (packages/darwin-mode/src/mock-sandbox.ts, ADR-101/102) — the deterministic, LLM-free evaluator used by the evolve:dry script every generated harness ships, and the web-UI's default sandbox: 'mock' config — extracts a memoryThreshold param (from a mutated memory_policy.ts surface file) that the existing DeterministicMutator really does perturb in production (its generic numeric-threshold edit rule matches memory_policy.ts's threshold = 0.5 literal). The pre-fix simulateAgentLoop() never consumed memoryThreshold anywhere: every memoryPolicy mutation scored byte-identically to its parent, 100% of the time. pickSurface() samples all 7 ADR-071 surfaces uniformly, so ~5/7 of Tier-1's generated mutations (memoryPolicy, toolPolicy, reviewer, scorePolicy, and planner's cosmetic-only planSteps) were fitness-invisible pre-fix. ADR-101's original spec named toolPolicy as the outcome-driving param; ADR-102 (as-built) silently swapped it for memoryPolicy — tonight's fix closes the gap between ADR-102's own documented promise and the code. At least the 4th instance of "signal computed then discarded" in this family (ADR-249 cost seam 08-17; Tier-2 gate 09-02; traceQuality seam 09-22; Tier-2 toolPolicy ordering 10-07/#379).
Full research (5 roles, external sourcing graded A/B/C, competitor comparison) in the gist, committed at docs/dream-cycle/2026-10-12-gist.md on this issue's branch (gist-hosting tool unavailable this session — no authenticated gh CLI, no MCP gist tool — committed-file convention per 2026-09-09/#299 precedent).
Hypothesis
Given a Tier-1 (mock) Darwin-mode evolution run, when a child variant mutates only the memoryPolicy surface, then simulateAgentLoop()'s solved/durationMs (and scoreBenchmark()'s finalScore) should change relative to the parent once the mutated memoryThreshold crosses 0.5 (pre-fix: provably byte-identical 100% of the time), subject to: (a) baseline threshold=0.5 stays a byte-for-byte no-op; (b) zero new dependency/LLM calls; (c) mock-sandbox.test.ts and all dependent darwin-mode suites stay 100% green with only additive assertions.
Evaluation Receipt
evaluated: accepted
Baseline (git stash on source only): 2 of 4 new tests fail non-vacuously (strict/lenient memoryThreshold producing byte-identical solved/attemptsUsed to baseline).
Candidate: mock-sandbox.test.ts 8→12/12 (+4, 0 regressions). Full @metaharness/darwin suite: 669/683 (14 pre-existing skips, 0 regressions, 74 files). tsc --noEmit clean. No cross-package consumer of mock-sandbox.ts exists outside its own test file.
Backward-compatible by construction: memoryPenalty(0.5)=0, matching the template's literal default.
Darwin Results
DARWIN=not_applicable — internal fix to Tier-1's own evaluator, not a mutation to one of the 7 ADR-071-approved harness surfaces (same reasoning as 2026-10-09's kernel fix, #386).
Flywheel Evidence
Not applicable — not a flywheel-promotion candidate. Durable record is this issue + the PR + the committed gist + the ledger row.
Reward Hack Check
Clear. No test/benchmark weakened (diff is purely additive). Independent critic (round 1) found one real defect — a 4th test claimed to exercise a Math.max(0,…) floor clamp that never actually engaged (the clamp is provably inert for any input since attempt never goes negative) — fixed round 2 same night: removed the dead clamp, rewrote the test to exercise genuine saturation behavior. No sign/magnitude/boundary gaming found in the surviving mechanism.
Security Review
Not security-sensitive — touches only an internal deterministic simulation function; no tool/MCP authority, credential, or prompt-injection surface involved.
Scan Findings
weight-eft: assertTunableSize() (packages/weight-eft/src/train.ts:114-122, fed by cli.ts:87) has a live NaN bypass of its 7–14B safety gate: Number('bad-input') → NaN, and both NaN < 1 and NaN > 14 evaluate false, so the size-cap check never throws — a model of any size proceeds unblocked. Confirmed still live by direct code read tonight; no existing test exercises a non-numeric --params-b. Not fixed tonight (scan surface, kept to a single conceptual change on the DEEP surface) — good candidate for a near-future night. The "7 of 8 bench/** instances still unpatched" claim from 2026-10-07's PR #379 notes could not be re-located this session; reporting unconfirmed rather than re-asserting it.
learn: no new disclosed-but-unfixed gap found; the 2026-09-17 dry-run/ADR-citation fixes (#324) hold. learn.mjs/gepa-loop.mjs's cost-budget parsing fails closed (not open) on bad --max-cost input — opposite direction from weight-eft's bug, not a live-spend gap.
Correction for future nights: mcp-scan/threat-model/sbom/secrets are real, tested CLI subcommands — dispatched from the separate harness bin (packages/create-agent-harness/src/subcommands.ts), not the metaharness scaffolder bin whose --help doesn't list them. Future security-adversarial nights should probe harness --help, not just metaharness --help.
Competitors Reviewed
AlphaEvolve, OpenEvolve, Darwin Gödel Machine, AFlow/ADAS/EvoFlow, DSPy/GEPA, LangGraph/AutoGen/CrewAI/OpenAI Agents SDK/Claude Agent SDK, plus SWE-bench harness-vs-model ablation literature (AHE, Gorinova et al. position paper). Full grading in the gist.
Gist
Committed: docs/dream-cycle/2026-10-12-gist.md (this branch). Gist-hosting tool unavailable this session (gh CLI auth invalid for both GH_TOKEN/GITHUB_TOKEN; no MCP gist-creation tool) — GitHub issue/PR operations used the GitHub MCP server instead, which authenticated fine (get_me → ruvnet).
Activity