🪐 Pluto: Multi-Agent Document Intelligence Pipeline

Pluto is an advanced, production-ready document question-answering system designed for complex research and technical documents. Moving beyond naive single-prompt RAG architectures, Pluto deploys a Multi-Agent Orchestration Pipeline that separates document routing, extraction, synthesis, and evidence verification.

By intelligently switching models based on task complexity (e.g., using lightweight 8B models for extraction and massive 70B+ models for deep reasoning and synthesis), Pluto significantly reduces API costs while drastically cutting down LLM hallucinations through mathematical token-overlap verification.

🚀 Key Features

• Multi-Agent Debate: A built-in "Critic" agent actively challenges the "Synthesizer" agent to find missing gaps in the generated answer before presenting it to the user.
• Deterministic Evidence Checking: Bypasses expensive LLM calls by validating claims against source texts using mathematical token-overlap heuristics (Threshold $\geq$ 0.72).
• Conversational Memory: Uses PostgreSQL (with local JSON fallback) to track session history, automatically injecting key findings and detecting semantic rephrasing across follow-up queries.
• Dynamic Model Routing: Automatically falls back across NVIDIA NIM, Groq, and Mistral APIs to ensure high availability and prevent rate-limit crashes.
• Real-time Streaming UI: A sleek Vue.js dashboard powered by Server-Sent Events (SSE) that displays agent communication, pipeline tracing, and exact quote mapping live.

---

🧠 System Architecture

graph TD
    subgraph Frontend [Vue.js Dashboard]
        UI[User Interface]
        SSE[SSE Live Stream]
    end

    subgraph Backend [FastAPI Server]
        API[REST Endpoints]
        Bus[Agent Message Bus]
        DB[(PostgreSQL / JSON Memory)]
    end

    subgraph MultiAgentPipeline [Pluto Orchestration]
        S0[S0: Router Agent]
        S1[S1: Extractor Agent]
        S2[S2: Synthesizer Agent]
        Critic[Critic Agent]
        S3[S3: Evidence Verifier]
    end

    subgraph Infrastructure [Data & Models]
        Corpus[Document Corpus]
        Cache[LRU Extraction Cache]
        LLM[NVIDIA NIM / Groq / Mistral]
    end

    %% Flow logic
    UI -- HTTP POST Query --> API
    API -- Initializes --> MultiAgentPipeline
    MultiAgentPipeline -- Publishes State --> Bus
    Bus -- Streams --> SSE
    SSE -- Updates --> UI

    %% Pipeline Logic
    S0 -- Selects Top-K Chunks --> Corpus
    S0 --> S1
    S1 <--> Cache
    S1 -- Extracts Claims --> S2
    S2 <--> Critic
    S2 -- Outline --> S3
    S3 -- Verifies Quotes --> Corpus
    
    MultiAgentPipeline <--> DB
    MultiAgentPipeline <--> LLM

Loading

---

⚙️ The Four-Stage Workflow

Pluto operates across four distinct, isolated stages to ensure maximum accuracy and traceability:

S0: Route & Retrieve

Instead of passing full documents, S0 uses BM25 & Cross-Encoder logic to find the most relevant chunks of text. It assigns a processing mode (MODE_QUICK, MODE_REASONING, etc.) based on the density of the chunk.

S1: Extract

S1 spins up an 8-worker thread pool to concurrently parse the chosen chunks. It generates structured JSON claims and maps them to exact locations (e.g., "Paragraph 2, Table 1"). Extractions are cached via LRU cache to prevent redundant processing on follow-up queries.

S2: Merge & Synthesize

S2 aggregates the extracted claims into a cohesive answer outline. Before moving on, the Critic Agent evaluates the outline against the original query to detect missing nuances or irrelevant points, forcing the Synthesizer to revise if flaws are found.

S3: Evidence Check

S3 acts as the final firewall against hallucinations. It cross-references every generated claim against the original source text. If the token overlap is $\geq$ 0.72, the claim is approved. Otherwise, it triggers a secondary LLM logic check. Unsupported claims are stripped from the final answer.

---

💻 Tech Stack

• Backend: Python 3.10+, FastAPI, Uvicorn, Pydantic, PostgreSQL
• Frontend: Vue.js 3 (CDN), TailwindCSS, Vanilla JS
• Document Parsing: pdfplumber, PyPDF2
• AI/ML: OpenAI Python Client (Standardized interface for NVIDIA/Groq), HuggingFace Transformers
• DevOps: Docker, Pytest

---

🛠️ Local Installation & Setup

1. Clone the Repository

   git clone https://github.com/your-github-link-here/pluto.git
   cd pluto

2. Install Dependencies

   pip install -r mp1/requirements.txt

3. Configure Environment Variables Rename .env.example to .env and add your API keys:

   NVIDIA_API_KEY=your_key_here
   # Fallbacks (Optional but recommended)
   GROQ_API_KEY=your_key_here
   MISTRAL_API_KEY=your_key_here

4. Run the Application

   python mp1/main.py --serve --port 8000

   Navigate to http://127.0.0.1:8000 in your browser.

---

📊 Benchmarking

Pluto includes a built-in benchmarking suite to compare the Multi-Agent architecture against a traditional naive Single-Model baseline.

To run a direct comparison to measure latency, confidence, and verifiable evidence counts:

python mp1/benchmark/compare.py