When an incident fires, on-call engineers get fifteen alerts that are all symptoms of one root cause. The first ten minutes is manual correlation: which service, which deploy, what changed. This agent does that in seconds. The interesting part is not the speed. It's that the agent tells you what it doesn't know.
Live demo: https://oncall.siddhantkhare.com Password-protected. Contact Siddhant on Twitter or by email for access.
The agent calls five tools in sequence against a simulated observability stack:
| Tool | Purpose |
|---|---|
get_alerts |
Pull the full alert stream across all services |
get_recent_deploys |
Correlate alert onset with code changes |
get_service_dependencies |
Assess blast radius |
get_runbook |
Get structured remediation guidance |
verify_hypothesis |
Run a targeted check to confirm or rule out the lead |
It produces an Incident Brief: ranked hypotheses with confidence scores, supporting and contradicting evidence per hypothesis, a timeline, suggested actions ordered by urgency, and explicit open questions stating what evidence would close the gap.
The agent maintains a ranked hypothesis list, not a single root cause. It shows the evidence trail and what would change its mind. When the evidence is genuinely ambiguous, it says so and tells you the single check that would break the tie.
Follow-up chat keeps the full investigation context. Ask a follow-up question and the agent answers in prose, grounded in the tool results from the original run. If the answer shifts confidence in any hypothesis, a visual diff appears under the response showing the old and new scores and why they changed.
Eight parameterized scenarios, each generated fresh on every run via a seeded RNG. Each has an observability context tooltip in the UI explaining which metrics and span patterns to watch in a real environment.
flowchart TD
User(["Browser\nNext.js · Vercel"])
subgraph Vercel["Vercel (static)"]
UI["Next.js UI\nScenario selector\nAgent activity feed\nIncident brief\nFollow-up chat"]
end
subgraph Fly["Fly.io (no timeout)"]
API["Hono API"]
Agent["OpenAI Agents SDK"]
Tools["Tools\nget_alerts\nget_recent_deploys\nget_service_dependencies\nget_runbook\nverify_hypothesis"]
end
LLM(["gpt-5.4\nOpenAI"])
User -->|"GET /"| UI
UI -->|"POST /investigate\nSSE stream"| API
API --> Agent
Agent <-->|"tool calls"| Tools
Agent <-->|"inference"| LLM
API -->|"SSE events\ntool_call · tool_result\nbrief_complete"| UI
Vercel serves static HTML and JS only. The browser opens an SSE connection directly to the Fly.io backend, which runs the agent and streams every tool call and result back as events. The live step panel in the UI is fed by those events in real time.
The backend enforces server-side origin checks on /investigate, /followup, and /scenarios. Requests without a recognized Origin header get a 403. CORS alone is browser-enforced; this blocks direct script abuse.
# Backend
cd backend
cp .env.example .env # add OPENAI_API_KEY
npm install
npm run dev # http://localhost:8080
# Frontend (separate terminal)
cd frontend
cp .env.example .env # NEXT_PUBLIC_API_URL=http://localhost:8080
npm install
npm run dev # http://localhost:3000Backend (Fly.io)
cd backend
fly apps create oncall-agent
fly secrets set OPENAI_API_KEY=sk-...
fly deployFrontend (Vercel)
cd frontend
vercel link
vercel env add NEXT_PUBLIC_API_URL production # https://oncall-agent.fly.dev
vercel env add NEXT_PUBLIC_SITE_PASSWORD production
vercel --prodAfter deploying the frontend, add its URL to ALLOWED_ORIGINS in fly.toml and redeploy the backend.
Ranked hypotheses, not a single root cause. The obvious design is one answer plus one confidence score. A ranked list is more honest. Incident triage works by maintaining multiple hypotheses and eliminating them as evidence comes in. The agent reflects that.
Hypothesis confidence diff on follow-up. When a follow-up question changes the picture, the agent says so explicitly. A second lightweight agent runs after each follow-up and checks whether the discussion shifted any hypothesis confidence. If it did, a visual diff appears: old score struck through, new score with a signed delta, reason in plain text. This makes the agent's belief revision visible rather than implicit.
Single agent. Kept deliberately. Coordination overhead in multi-agent systems often outweighs the benefit for a sequential triage workflow. One coherent reasoning thread is easier to audit and debug. Hypothesis-parallel subagents (one per signal source) would make sense at production scale.
Simulated data. Fixtures keep the demo self-contained and reproducible. The tool implementations are the only thing that changes in a real deployment. The agent, the streaming protocol, and the UI are identical.
SSE over WebSockets. One-directional stream from server to browser. Simpler to deploy, no persistent connection management, works through Vercel's edge network without configuration.
Confidence is explicit. The agent states what it doesn't know. Open questions are required when top hypothesis confidence is below 0.9. Suggested actions are ordered by urgency, and the first action is always safe (observe or verify) unless confidence exceeds 0.85.
Eight scenarios, not three. The first five cover the common cases. The last three cover the cases that trip up naive agents: config changes with no deploy in the window, load-induced failures that look like regressions, and flapping alerts caused by threshold misconfiguration rather than a real outage. Each scenario is a parameterized generator with a seeded RNG, so every run produces varied but realistic data.
- Backend: Hono on Node.js, deployed to Fly.io
- Frontend: Next.js 15 (static export), deployed to Vercel
- Agent: OpenAI Agents SDK (
@openai/agents) - Streaming: Server-Sent Events
- Types: Zod schemas for structured agent output