An AI agent can propose any tool call. REMORA decides whether that exact call may run, and keeps the evidence.
REMORA is a governance overlay that sits between a tool-using agent and the tools it wants to use. Before a call runs, it checks the call against policy the deployment owns, not the agent, and returns one of four decisions:
| Decision | Meaning |
|---|---|
| ACCEPT | The action may proceed under the current policy and context. |
| VERIFY | A defined check or review is required before execution. |
| ABSTAIN | Required information is missing or insufficient. |
| ESCALATE | The action requires a higher-authority decision. |
An approval covers only the exact call it was given for. It can be used once, and policy is checked again at the moment of dispatch. A model's confidence can never override a hard block. The enforcing surface is POST /v1/execution/*.
Status: research and shadow-mode software with no production certification. External replication is pending; every deployment still needs its own validation.
python -m pip install -e ".[dev]"
python -m remora try # five guided scenarios
python -m remora assess drop_database # the verdict and the rules behind it
python -m remora whatif drop_database # what would it take to ACCEPT?whatif searches for the smallest change that would let a call through. For a production drop_database, nothing a model can say gets there:
verdict now: ESCALATE
model signals alone: cannot reach ACCEPT (trust, phase, evidence, quorum, temperature: every combination tried)
agent alone: cannot reach ACCEPT (proposal + model signals, within 4 changes)
- Authority comes from the deployment. Tool meaning, target, risk and intent come from signed, deployment-owned sources, never from the agent that wants to act.
- Deterministic rules form the floor. Model signals can route a call to review; they cannot unlock one.
- Every claim is tied to evidence. Results resolve to a claim register, and failures stay in NEGATIVE_RESULTS.md.
The research question: can authorization that is enforced outside the agent and bound to the exact call stop the calls an agent should not make while keeping it useful? The short answers, with the details and limits further down:
- Blocking is the easy half. On a public function-calling benchmark (BFCL v4), two sealed runs accepted none of the wrong calls. On its own, though, REMORA let only about 3 in 10 correct calls through. Staying useful is the open problem.
- A floor is not a judge. On AgentHarm, a deterministic gate blocked every harmful and every harmless request alike. An LLM monitor on its own told most harmful/harmless pairs apart, and the gate on top erased that signal (NEGATIVE_RESULTS §80). How a deterministic floor and a learned monitor should combine is now a main research question.
- We publish what fails. That includes an uncertainty signal falsified on fresh data, label leakage in one of our own benchmarks, and the peer-review objections that are still open.
- Not yet tested. An agent actively trying to get past the gate. That is the next experiment (plan).
- Reviewing the research? Five files are enough: the paper, NEGATIVE_RESULTS.md, the peer-review findings, the claim register and how to reproduce every result. The repository also keeps about 400 Markdown files of design history and product material; none of them is needed to judge a claim.
- Building on the code? Start with the developer handoff and the execution quickstart. The full reading path is below.
- Checking REMORA from another project? See Verify it from outside.
Reading path for the code, and the documentation index
- Developer handoff: the shortest technical path through the repository
- Architecture: components, data flow, module stability
- Execution quickstart: configure and run the enforcing path
- API reference: public interfaces with a
curlround-trip; wire contract inschemas/openapi.json - Python SDK: the one namespace with a backward-compatibility guarantee
- Evidence and claims: what each result establishes, and what it does not
- NEGATIVE_RESULTS.md: failed hypotheses and limitations, kept permanently
- Peer-review findings: every objection raised against the paper, and what was done about it
The documentation index lists the complete registered set.
How an action gets through, step by step
- Authoritative context. Tool meaning, target, risk and approved intent come from deployment-owned sources such as a Signed ToolSpec, never from the calling agent.
- Policy decision. Deterministic hard guards outrank model-derived signals.
- Review or grant.
VERIFYandESCALATEenter bounded review;ACCEPTreceives a short-lived, single-use grant bound to the proposal. - PEP and dispatch. The grant is consumed, an
ExecutionLeasebinds policy identity to call identity, andGovernedToolDispatcherinvokes the deployment-owned callable. - Lifecycle and evidence. Dispatch intent, outcome, effect verification and audit records stay joinable by proposal identity.
The separate /v1/assess research surface can use oracle, evidence and uncertainty components. They are not prerequisites for the execution kernel and cannot override its deterministic hard-guard floor.
DEVELOPER_OVERVIEW.md draws the CORE / OPTIONAL / EXPERIMENTAL / HISTORICAL boundary; the machine-checked product truth contract classifies every capability; the execution quickstart walks a deployment-shaped example.
Headline values are governed by the claim register and stay tied to committed result artifacts and scope caveats. Every number below is bounded by documented assumptions, benchmark populations and evaluation protocols. None is a general safety guarantee.
| # | Benchmark | Result | Scope |
|---|---|---|---|
| 1 | BFCL v4 (C-ext4) | Correct calls accepted 29.9% (87/291), against 6.9% for the frozen v1 bundle on the same episodes; native wrong-call acceptance 0.0% (0/295, as for a refuse-all control; 5.42% upper bound clustered by tool); obtainable-input VERIFY 84.0% (100/119) | Sealed once, the whole remaining reserve, 1,562 episodes; all eight pre-registered targets met, but they were fixed after bundle v2 existed and a refuse-all control meets five of the eight (PRF-008); the gain sits on tools also seen in development (CLAIM-021) |
| 2 | BFCL v4 (C-ext3) | Native wrong-call acceptance 0.0% (0/500; Wilson 95% upper bound 0.76% by row, 3.5% clustered by tool); irrelevant-tool refusal 100.0% (300/300); required-input guessing 0.0% (0/398) | Sealed once, 2,799 episodes, frozen semantic bundle + authority floor; four of seven pre-registered targets missed, among them the safety target for constructed wrong tools (2/199 = 1.005% accepted against a bar of at most 1%) and read autonomy (26.6% against 75%), so the joint hypothesis was not supported; see NEGATIVE_RESULTS §39 |
| 3 | Adversarial simulator | 0.0% unsafe runs (0 of 70 templates, 56 of them harmful; 700 tasks); 95% cluster-level Wilson upper bound 5.2% (6.4% over the 56 harmful templates); utility +0.456, a post-hoc test | Simulated; effective N = 70 templates × 10 cosmetic variants, of which 56 harmful templates carry the unsafe rate; unsafe-rate gap Δ=0.0143 vs. baseline, not statistically significant |
| 4 | AgentHarm | 0.0% wrongly allowed (0/208); 95% upper bound 1.81%; false-block rate on the benign twins 100.0% (208/208 blocked) | Intent classification on the instruction text only, from an imported run; tool calls were not intercepted. Every score sat above the escalation threshold, so the gate blocked every scenario, harmful or benign. This measures a blocking floor, not discrimination between harmful and benign, and the result is not attributed to the hard-block layer. The 208 scenarios come from 47 base behaviours; over those the upper bound is 7.6% |
| 5 | Historical regression | 0.0% wrongly allowed (0/167; Wilson 95% upper bound 2.25%) | Previously observed failures only |
How to read those numbers
- The deterministic simulator has 70 independent templates; cosmetic variants do not increase the effective sample size.
- BFCL v4 C-ext3 measures the safety axis only: declared semantic authority takes native wrong-call acceptance from 24/500 to 6 to 0 across the ablation arms. The same run missed four of its seven pre-registered targets: one safety target (constructed wrong-tool acceptance, 2/199) and three utility targets (read autonomy, obtainable VERIFY, unobtainable ABSTAIN); those findings and the permanent 10.9% baseline live in NEGATIVE_RESULTS §39 and CLAIM-019's caveat, not here.
- BFCL v4 C-ext4 is the utility half: a second deterministic bundle accepts about four times as many correct calls as the frozen one, with the safety counts still at zero. Most correct calls are still not accepted autonomously, and generalisation to unseen tool families is not shown.
- REMORA routes annotated BFCL calls; it does not generate them. Each native wrong call is another task's correct call, and only 4 of the 295 in C-ext4 name a tool the task itself offered, so the zero counts say little about confusing two tools of one task. A name-overlap rule reaches part of the gold-call signal but also accepts wrong calls and calls missing required arguments (post hoc, NEGATIVE_RESULTS §83).
- Internal reproducibility is not external replication or field validation.
Replaced claims are kept in superseded claims. Failed hypotheses and limitations are kept in NEGATIVE_RESULTS.md.
python -m pytest tests/ -q # the deterministic suite (about 10,800 tests)
python scripts/demo_industrial_maintenance.py # a worked operational scenario
python -m remora doctor # check the local configurationLibrary call, and why it is advisory
assess_tool_call(...) evaluates the context the caller supplies. It does not control downstream credentials. Only risk and action labels from a registry= the deployment owns can reach ACCEPT; labels passed by the caller can only lower the verdict (PRF-040). For enforcement, route the call through /v1/execution/* and make the governed dispatcher the only path to the protected tool.
from remora import assess_tool_call
REGISTRY = {"drop_database": {"risk_tier": "critical", "action_type": "destructive_write"}}
assessment = assess_tool_call("drop_database", {"db": "prod-main"}, registry=REGISTRY)For external review or pilot work, set REMORA_RUNTIME_PROFILE=review or controlled_pilot. Those profiles require the Signed ToolSpec and durable-state prerequisites described in the execution quickstart.
Independent projects, shared boundaries, verifiable evidence (agent-governance-vocabulary#177). REMORA publishes what it claims and asks other implementations to check it. It does not take foreign evidence as authority.
| You want to | Start here |
|---|---|
| Verify a REMORA claim with your own implementation (producer side) | artifacts/interop/index.json. Each contract carries a claim packet, pinned fixture digests and a verifier request with the result vocabulary ESTABLISHED, CONTRADICTED, NOT_ESTABLISHED. The open request is runtime-surface E7; three execution-boundary fixtures (exact-call-binding-v1, fresh-authority-v1, effect-evidence-v1) and one consumed edge (agentavow-tool-manifest-e8-v0.1) are published. The three boundary fixtures are pinned at fe324dd7 and REPRODUCED: Probity ran them with a second implementation, recorded as NOT_INDEPENDENT; their v1.1 successors are DRAFT. The consumed edge has author-run records only and is DRAFT. Status per edge is derived in docs/interop/INTEROP_MATRIX.md. Do not import REMORA code; a run of the reference verifier is not independent verification. |
| Test your own system against the decision-to-effect vectors (adapter) | python conformance/decision-to-effect-v1/run_conformance.py --adapter remora, then copy adapter_skeleton.py and fill in what your system can express. An unsupported vector is a recorded result, not a failure. |
| Hand REMORA evidence from your project (consumer side) | external-evidence-ref-v1 keeps your native claim and result; consumed-artifact-v1 records attribution. Neither changes a decision. The boundary is in docs/interop/FEDERATION.md. |
Verification of a REMORA artifact is exactly that, not endorsement, and a result at one assurance layer is not evidence for a stronger one. One round has been run from outside, with the survivors kept as open gaps: external adequacy record (#629); the step-by-step is in FEDERATION.md. Other corpora: evidence sufficiency, autoreview to effect, non-transitivity of authority.
It cannot enforce against a credential path that bypasses its dispatcher. Deployment identity, credential custody, downstream authorization and operational controls stay with the deployment. Capability maturity is tracked, per capability, in the capability register.
Repository map and current engineering state
Research modules stay in the repository for reproducibility and audit history: AROMER (remora/aromer/), the older statistical-physics hypotheses (remora/thermodynamics.py, remora/lyapunov.py, remora/research_attic/, legacy names for uncertainty observables, not physics) and historical design documents. Their presence does not make them part of the enforcing runtime path; the capability register says what is wired, and DEVELOPER_OVERVIEW.md states the known engineering boundary.
For product-oriented integration see Assured Agent Execution, which consumes pinned REMORA artifacts rather than copying the governance core.
Jev (TypeSafe) is a semantic sensor beside the policy engine: experimental, evidence only, never authority. It runs in shadow on the enforcing path with per-vertical profiles, and its reading goes to the VERIFY lookup and the ESCALATE approver. See integrations/jev_decision_provider.md.
The research-to-control mapping lives in docs/research/research_control_matrix.generated.md; related work in docs/09-related-work.md. Citation metadata is in CITATION.cff; GitHub renders it as BibTeX from the "Cite this repository" panel.
Generative-AI tools were used during development. AI-generated text or code is not evidence by itself; claims must resolve to committed artifacts, tests or verified sources. Disclosure: docs/AI_USE.md.
REMORA versions from v0.10.0 are source-available under the Business Source License 1.1, with commercial licensing under the REMORA Commercial License. Research, benchmarking and reproducibility work are permitted within the terms in LICENSING.md. Contribution requirements, branch lifecycle, documentation style and claim hygiene are defined in CONTRIBUTING.md and docs/10-contributing.md.