{{ message }}
Repository navigation
Tags: ruvnet/metaharness
Tags
feat(darwin): independent contrastive confirmation with public pilot … …evidence Add frozen paired holdout validation, exact statistical gates, total deadline enforcement, conservative budgets and bound receipts. Include 99 confirmation tests and a pinned CLINC150 experiment. The live candidate tied baseline at 38/40 and cost more; both superiority gates rejected. 830 tests passed, 19 existing skips; package build, fresh consumer acceptance and production audit passed. Validated ruOS source tree matches local commit 1eb56a8d55f2ff430a4095cc1f10a3f266a04f49 exactly. Rust CI blocker tracked in #408.
feat(darwin): bounded contrastive development API v0.11.0 Includes source, built distribution, scoped memory, live editor adapter, adversarial tests, synthetic benchmark, and release evidence. Validated on ruOS. Published tree exactly matches local reviewed tree 70a8e0f (local source commit 7f9455f, evidence commit d92e932).
chore(release): metaharness 0.4.17 + patch releases for changed packa… …ges (#356) Ships the merged dream-cycle fixes: writeAtomic path traversal and host adapter injection (#300), .env guard (#327), MCP detection (#298), grade masking (#337), toolSafety (#304), reward-hack tool content (#293), reviewer isolation (#352) and the flywheel/darwin correctness fixes.
ADR-240: AGNTCY identity, OASF export, observability, CASA compiler (#… …155) * docs(adr): ADR-237 — AGNTCY identity, OASF export, semantic observability Adopts Cisco Outshift's AGNTCY/Internet-of-Cognition ecosystem as complementary build-time infrastructure for generated harnesses: signed AGNTCY identity in the harness manifest (layered on the existing ADR-011 witness manifest), OASF capability-record export to the AGNTCY Directory (a projection of existing score/genome/mcp-scan output, not new evaluation logic), AGNTCY OTel semantic-convention mapping for existing telemetry, and the CASA intent-to-authority- envelope compiler (translation may use an LLM; enforcement, owned by companion ruflo ADR-324, never does). Companion to ruflo ADR-324, which owns the runtime half of this integration (SLIM transport, CASA enforcement, IOC Layer 9 events). * feat(agntcy): ADR-237 — identity, OASF export, observability, CASA compiler Implements the buildable-now portion of ADR-237 (AGNTCY identity, OASF export, semantic observability) as the optional @metaharness/agntcy sibling package (mirrors @metaharness/darwin/redblue/flywheel — never a kernel dependency). No public AGNTCY npm packages exist under any plausible name (verified live) — identity-provider and Directory network calls are honest, config-gated stubs, never faked successes. - src/identity/: AgntcyIdentity schema (hand-rolled shape-gate, no Zod — this repo has zero existing Zod usage), deriveBadgesFromMcpScan() reading the real ScanReport/ThreatModel shapes (not guessed ones), and a documented signing integration point citing the real existing Rust signer (crates/kernel/src/witness.rs::sign_manifest) rather than inventing a second signature scheme — no JS/wasm binding for it exists yet, so this is a TODO with the exact wiring documented, not a silent parallel signer. - src/oasf/: OasfRecord projected from real harness-score/-genome/ -mcp-scan output; fails closed (returns {incomplete, missingFields}) on fields with no real internal source (e.g. pricingMeteringClass) rather than fabricating a value; publishToDirectory() always reports {published: false, reason: ...} since no Directory endpoint exists publicly yet. - src/observability/: the 8 AGNTCY OTel attributes this package owns, mapped from real @metaharness/router / memory-tier / flywheel scorer output. - src/casa/: the intent-to-authority-envelope compiler (deterministic, no LLM call), schema field-for-field identical to ruflo's mirrored copy for cross-repo interoperability. This package only compiles envelopes — enforcement is owned by companion ruflo ADR-324. - src/index.ts: wired to re-export all four subsystems as namespaces (identity/oasf/observability/casa) — previously landed empty as a placeholder while sibling work was in flight. Adversarial security review (ADR-237 §4's own requirement before this compiler is trusted) found and this commit fixes a real bug: the RULE_TABLE matched bare "push"/"publish"/"release"/"deploy" with no context, false-permissively granting git.push/deployment.create for objectives like "push notification integration" or "review the release notes for security issues". Patterns now require a co-occurring contextual noun. Fixed identically in the mirrored ruflo copy (plugins/ruflo-agntcy/src/casa/compile.ts) to keep the two repos' envelopes interoperable, per the module's own sync requirement. Companion: ruflo ADR-324 (runtime half — SLIM transport, CASA enforcement, IOC Layer 9 events). * docs(adr): renumber ADR-237 to ADR-240 — collided with real merged ADRs origin/main already had real, Accepted/Proposed content at ADR-237 (evals-math-live-gsm8k-fast-domain), ADR-238 (workspace-lens-jacobian- interpretability), and ADR-239 (workspace-probe) — a numbering collision from an inaccurate original check (read a concurrent session's dirty local files instead of origin/main's actual git tree). Renumbered to ADR-240, the next free slot. Content unchanged, only the number/filename and internal references moved. * docs(agntcy): fix stale companion-ADR cross-reference (ruflo ADR-324 -> ADR-380) Ruflo's own numbering collided too (ADR-324 was already taken by agentic-policy-engine-codex-swarm on their main) and got renumbered to ADR-380 — this package's references to it were stale. * fix(agntcy): real, live-tested AGNTCY Directory integration (was: false "no SDK" claim) Corrects a wrong claim from the initial scaffold: it said no AGNTCY package existed on npm under any plausible name. That check only guessed scoped names (@agntcy/dir, @agntcy/slim) and got 404s — the real Directory SDK is published unscoped as `agntcy-dir` (v1.5.0), and SLIM's real Node bindings are `@agntcy/slim-bindings` (handled in the ruflo companion PR). oasf/publish.ts now pushes and publishes real OASF records via the real SDK, verified end-to-end against a live local Directory server (Go apiserver + zot OCI registry + postgres, built and run from agntcy/dir's own install/docker/docker-compose.yml) — push, publish, and an independent lookup all exercised for real, not mocked. Two real, non-obvious things needed to get this working: - The real Directory schema REQUIRES a non-empty `skills` array with a numeric AGNTCY taxonomy id. This repo's internal capability tracking is name-based (harness-genome agent names), so a small honest mapping table (KNOWN_AGENT_SKILLS) translates known agent kinds to real taxonomy ids (empirically derived + cross-checked against agntcy/oasf's schema files, documented as such in code comments) with a real category-level fallback for anything unmapped — every capability is still recorded verbatim in annotations regardless, so nothing is lost to the fallback. - The live server's schema validator rejects `skills[].name` in every format tried (dotted taxonomy path, dot-separated, bare leaf, human caption) — including the taxonomy's own real names — and only accepts a bare numeric `id`. This looks like a real upstream bug, not a mapping error; filing an issue against agntcy/dir with the reproduction. Also fixed: package.json now declares the real `agntcy-dir` dependency (previously claimed "dependency-free"); added the required `.npmrc` Buf Schema Registry scope (one of the SDK's transitive deps is Buf-generated, not on the default npm registry); corrected README.md's "Status" section end to end. AGNTCY Identity (github.com/agntcy/identity) really is Go-only with no JS/TS SDK — that specific stub (identity/sign.ts's witness-signing TODO) was correct as originally written and is unchanged. * docs(agntcy): link the filed upstream issue (agntcy/dir#1943) Root-caused first: agntcy/oasf-sdk's Go client is just an HTTP client against a remote hosted validation service (schema.oasf.outshift.com) -- no local source to patch, so a PR isn't possible here, only a report. * docs(adr): ADR-240 update section + finish the ADR-324->380 cross-ref sync The cross-reference fix (ruflo's ADR-324 renumbered to ADR-380) was made earlier but the docs/adrs/ files were never committed, only packages/agntcy/ — completing that now, plus the ADR-240 update section documenting the real Directory integration and the agntcy/dir#1943 upstream bug filed. * fix(ci): move Buf registry .npmrc to repo root; regenerate lockfile The .npmrc I added lived inside packages/agntcy/, but this repo's CI runs npm install/npm ci from the repo root (npm workspaces) — a subdirectory .npmrc is never picked up for a root-level install, so every Node CI job (npm-audit, sbom, real-tools, the full Node 20/22 matrix) started failing with a 404 on @buf/agntcy_dir.bufbuild_es right after the Directory-fix push. Moved the registry scope to the repo root .npmrc (the correct place for a real, repo-wide dependency now that agntcy-dir is one) and regenerated package-lock.json via 'npm install --package-lock-only' (the same command CI's own npm-audit job runs, and one that never touches node_modules — verified the shared workspace node_modules symlink is still intact after running it). 'npm audit --omit=dev --audit-level=high' (CI's exact gate) -> 0 vulnerabilities. * fix(test): real bug in the Directory live-server reachability probe — surfaced by real CI, not local review Real CI (Node 20 macOS, then 22 macOS) caught what local runs against my already-running local Directory server never could: the connectivity probe used client.lookup() on a bogus CID as its reachability signal, wrapped in its own .catch(() => undefined) — swallowing ANY rejection (including "nothing listening") and unconditionally setting serverReachable = true afterward regardless of whether the call actually succeeded. In CI, with no Directory docker stack running, this made every live-network assertion attempt to run anyway and fail for real. First fix attempt (remove the inner .catch(), let a real connection failure propagate to the outer catch) was ALSO wrong, just in the other direction: a real, reachable server's lookup() rejects for "record not found" too (a normal, expected response for a synthetic probe CID), which made a genuinely reachable server look identical to an unreachable one — verified this by re-running locally with the live server still up and watching the live test start skipping. Replaced both with a raw TCP connect() check against host:port — "is anything listening" has no AGNTCY-protocol-level ambiguity to get wrong. Verified against both real scenarios directly: a live server (4/4 tests run, including the real push/publish/lookup) and a genuinely unreachable port (3 pass, 2 skip cleanly). Also added a production security note to publish.ts: the plain `new Config(serverAddress)` construction is the SDK's insecure default (no TLS, no client auth) — fine for the local dev server this test suite targets, not safe against a production/public Directory server without the SDK's x509/JWT auth modes. * fix(ci): give pack+install smoke test the Buf registry .npmrc scripts/install-all.mjs installs packed tarballs into a throwaway project under os.tmpdir(), outside the repo tree, so it never saw the repo-root .npmrc's @buf:registry mapping. @metaharness/agntcy pulls in agntcy-dir's @buf/agntcy_dir.bufbuild_es transitive dep, which only exists on the Buf Schema Registry — npm 404'd against the default registry in every pack+install CI job (macos/ubuntu/windows). Copy the repo's .npmrc into the throwaway project before installing. Verified locally: pack-all + install-all now pass all 33 tarballs.
fix(ci): unblock npm publish — add napi package.json + non-blocking n… …ative matrix (#18) The publish workflow's `gcp-auth-and-publish` job `needs: [build-native, ...]`, but `napi build` failed for every target with "package.json not found at crates/kernel-napi/package.json", cancelling the matrix and skipping the npm publish entirely (the last 3 release runs never published). - Add crates/kernel-napi/package.json with the napi-rs v3 config (binaryName + targets). Validated locally: `napi build --platform --release` now produces kernel.linux-x64-gnu.node (exit 0). - build-native: continue-on-error + fail-fast=false. The per-platform .node artifacts are not yet consumed by the publish job (native publish is a TODO), so a flaky cross-compile target must not gate the real npm publish.