The Cognitive Pentad for Autonomous AI Agents — Local-first Model Context Protocol (MCP) infrastructure providing persistent state memory, perceptual vision caching, 3D/2D spatial world models, strategic BDI reasoning, and a "System One" fast decision layer.
-brightgreen.svg?style=flat-square){kind=icon}
PuterVision engineers high-performance, zero-telemetry, local-first infrastructure and Model Context Protocol (MCP) servers for autonomous AI agents, coding assistants, and high-frequency browser/game automation runtimes.
Modern frontier LLMs possess remarkable semantic reasoning but suffer from fundamental architectural bottlenecks when interacting with continuous, real-time environments:
- Stateless Ephemeral Contexts: "Agent amnesia" across tool steps, compaction cycles, and session restarts.
- Visual Token Bloat & Hallucination: Re-ingesting raw screen pixels burns thousands of tokens per step, induces latency, and degrades reasoning fidelity.
- Absence of Spatial & Object Permanence: Inability to track unobserved entities, physical collision bounds, or 3D view projections.
- Uncalibrated Deliberation: Ad-hoc tool execution without formal multi-objective utility scoring or quantitative risk calculation.
- Sluggish, Non-Deterministic Action Loops: Multi-second LLM roundtrips incapable of handling reactive events or ~60Hz browser/game controls.
The PuterVision Cognitive Pentad resolves these bottlenecks through a modular, synergistic architecture. Backed by local SQLite databases in WAL mode, sub-millisecond retrieval, SHA-256 Merkle audit trails, and zero external network calls, each server handles a distinct cognitive tier while seamlessly interoperating across shared data structures.
Inspired by dual-process cognitive science and high-frequency agent architectures—specifically the typed System 1 pattern pioneered by TypeSafe's Jev (evaluating typed Choice, Score, and Noul primitives over compact state without token generation)—PuterVision introduces a formalized "System One" Fast Decision Layer.
Where TypeSafe's Jev operates as a hosted neural decision model (~70–500ms), PuterVision implements that same architectural shape locally via in-memory LRU caches, deterministic heuristics, and vector centroid matching for sub-millisecond execution (<1ms) without calling external APIs:
-
System 1 (Fast, Intuitive, Non-Generative):
- Operates in
$<1\text{ms}$ using local in-memory LRU caches, heuristic decision trees, and vector centroid matching. - Evaluates typed, non-generative primitives: categorical labeling (
classify), Boolean propositions (ask_noul), discrete alternatives (ask_choice), scalar scoring (ask_score), and blast-radius security gating (gate_intention). - Strictly abstains when features are sparse or margins are ambiguous (
$\Delta u < 0.25$ ), triggering clean escalation without hallucinations.
- Operates in
-
System 2 (Slow, Deliberative, BDI Cognition):
- Operates on longer horizons (
$\sim 1\text{s} - 5\text{s}$ ) for multi-step strategic planning. - Decomposes high-level objectives into dependency DAGs, updates beliefs with exponential decay, computes expected multi-attribute utilities ((E[U] = \sum w_i u_i)), and replans upon structural blockers.
- Operates on longer horizons (
flowchart TD
subgraph Slices ["Compact Multi-Modal Slices (<2KB)"]
VM["👁️ vision-memory-mcp (v1.4.0)<br/>Compact Visual Slice & Centroids"]
WM["🌐 world-model-mcp (v0.6.0)<br/>Compact Spatial Slice & Nearest Entities"]
SM["💾 state-memory-mcp (v1.4.0)<br/>Compact TaskSlice & Active Blockers"]
end
subgraph StatePackAssembly ["Canonical Contract (§10.1)"]
SP["📦 Canonical StatePack<br/>Deterministic UTF-8 JSON + SHA-256 Merkle Hash"]
end
subgraph System2 ["System 2: Strategic Deliberation (1s - 5s)"]
BDI["🧭 BDI Cognition Engine<br/>Goal DAGs • Beliefs • Utility E[U] • Replanning"]
end
subgraph System1 ["System 1: Fast Decision Layer (<1ms)"]
DE["⚡ DecisionEngine (L1 / L2)<br/>classify • ask_noul • ask_choice • ask_score"]
IG["🛡️ gate_intention<br/>Blast-Radius Audit & HMAC-SHA256 Token Issuance"]
end
subgraph Runtime ["High-Frequency Execution (~60Hz)"]
BM["⚡ behavior-mcp (v0.4.0)<br/>Synchronous semantic_check • Reactive Triggers • HMAC Verify"]
end
VM --> SP
WM --> SP
SM --> SP
SP --> BDI
SP --> DE
BDI -->|Strategic Directives| IG
DE -->|Fast Verdicts| IG
IG -->|HMAC Dispatch Token| BM
BM -->|Execution Telemetry & Significant Decisions| SM
The Pentad exposes a unified surface of 68 specialized Model Context Protocol tools across 5 local-first servers. All tool descriptions adhere strictly to Glama Tool Description Quality Score (TDQS) standards and are indexed on the official Model Context Protocol Registry (registry.modelcontextprotocol.io):
To keep this overview concise and readable, deep architectural and reference specifications have been organized into specialized sub-documents:
- 🔒 100% Local-First & Zero Telemetry: All tasks, decisions, visual caches, 3D entity models, and reasoning beliefs reside strictly in local SQLite and LanceDB files on your machine. Zero cloud dependencies, zero external analytics collection, and zero data leakage.
- ⚡ Sub-Millisecond Retrieval & Execution: Eliminates massive context window token burn. High-frequency indexing, perceptual hashing, and local graph traversal deliver deterministic responses in
<1ms. - 🛡️ Fail-Closed Safety & Cryptographic Gating: Engineered with 5-layer safety stacks, watchdog circuit breakers, constant-time HMAC-SHA256 intention dispatch token verification, and SHA-256 Merkle audit chains for reproducible, tamper-evident agent trajectories.
- 🧪 Exhaustive Automated Verification: Backed by 1,463 automated unit, integration, and live synergy tests (100% pass rate across 310 test files) with zero version drift verified across all packages, manifests, and documentation.
- 🔌 Universal MCP Compatibility: Natively supported by Claude Code, Cursor, Gemini CLI / Antigravity, Windsurf, VS Code, and any Model Context Protocol compliant client.
Add the Pentad servers directly to your AI assistant configuration (claude.json, .cursor/mcp.json, mcp_config.json):
{
"mcpServers": {
"state-memory-mcp": {
"command": "npx",
"args": ["-y", "@putervision/state-memory-mcp"]
},
"vision-memory-mcp": {
"command": "npx",
"args": ["-y", "@putervision/vision-memory-mcp"]
},
"world-model-mcp": {
"command": "npx",
"args": ["-y", "@putervision/world-model-mcp"]
},
"agent-reasoning-mcp": {
"command": "npx",
"args": ["-y", "@putervision/agent-reasoning-mcp"],
"env": {
"PENTAD_HMAC_SECRET": "your-secure-32-byte-secret-key-here"
}
},
"behavior-mcp": {
"command": "npx",
"args": ["-y", "@putervision/behavior-mcp"],
"env": {
"PENTAD_HMAC_SECRET": "your-secure-32-byte-secret-key-here"
}
}
}
}Install the servers globally for direct CLI commands and local background daemons:
npm install -g \
@putervision/state-memory-mcp \
@putervision/vision-memory-mcp \
@putervision/world-model-mcp \
@putervision/agent-reasoning-mcp \
@putervision/behavior-mcpInitialize your workspace databases and scaffold IDE agent instruction rules:
state-memory-mcp init && vision-memory-mcp init && world-model-mcp init && agent-reasoning-mcp init && behavior-mcp initFor full client configuration templates (Cursor, Claude Code, Windsurf, Antigravity, VS Code), environment variables, and Docker setups, see the Getting Started Guide (
GETTING_STARTED.md).
- 🌐 Official Website: putervision.com
- 📦 npm Organization: @putervision
- 🐙 GitHub Organization: github.com/putervision
- 📜 License: MIT Open Source
