Just A Rather Very Intelligent System — A fully local AI assistant with modular skill architecture.
Built by Sami Porokka / Poro-IT OÜ
I Built a Local AI Operating System Over Easter Weekend — dev.to
- Multi-Model Routing — Queries routed to the best local model (fast / reason / code / deep / cloud)
- ReAct Agent Loop — Think → Tool → Observe → Repeat until task complete
- Modular Skills — 20 plug-and-play skill modules with 35+ tools, enable/disable via config
- Voice I/O — Wake word "Hey JARVIS", Whisper STT, Orpheus TTS with 5.1 center-channel output
- Persistent Memory — MemPalace vector DB + Obsidian vault (2000+ memories)
- Stark Industries HUD — Next.js holographic dashboard with GPU monitor, live system log
- Home Automation — Denon AVR, NVIDIA Shield, LG TV, Panasonic Blu-ray, Philips Hue, internet radio
- Cloud LLMs — Claude, GPT-4, Gemini, Groq, Mistral, OpenRouter via unified API skill
- AI Image Gen — FLUX text-to-image with Qwen3 prompt enhancement + VRAM auto-swap
- Phone Assistant — Answer calls, take messages, send SMS, read/send email via Twilio
- 34 Claude Skills — Frontend design, SEO, Playwright, React, and more loaded on demand
JARVIS uses a modular skill system. Each skill is a self-contained Python module in skills/ that registers its own tools, keywords, and executors. Drop a file in → restart → it's live.
| Skill | Tools | Description | Docs |
|---|---|---|---|
| denon | denon_input denon_volume denon_preset denon_surround denon_power |
Denon AVR-X4100W receiver control | Guide |
| shield | room_command scan_network |
NVIDIA Shield TV per-room control | Guide |
| radio | play_radio |
Internet radio streaming via mpv | Guide |
| volume | set_volume |
Windows system volume control | Guide |
| timer | set_timer |
Countdown timers with voice alerts | Guide |
| memory | memory_search memory_add memory_status |
MemPalace long-term memory | Guide |
| vault | read_vault_file list_vault_dir |
Obsidian vault file access | Guide |
| web | web_search open_url |
DuckDuckGo search + browser | Guide |
| shell | shell_command read_file |
Safe shell execution + file reading | Guide |
| claude_skills | list_skills use_skill |
Browse/load 34 Claude Code skills | Guide |
| lg_tv | lg_tv |
LG webOS TV — power, volume, inputs, apps | Guide |
| panasonic_bd | bluray |
Panasonic UB9000 4K Blu-ray — play, pause, chapters | Guide |
| network | scan_network |
Network scan with device identification + topology map | Guide |
| git | git |
Git repo management — status, diff, commit, push, pull, branch | Guide |
| flux | flux |
FLUX AI image generation with prompt enhancement | Guide |
| plex | plex |
Plex Media Server — browse, search, playback control | Guide |
| hue | hue |
Philips Hue lighting — on/off, brightness, colors, scenes | Guide |
| cloud_llm | cloud_llm |
Cloud LLM APIs — Claude, GPT-4, Gemini, Groq, Mistral | Guide |
| phone | phone |
Phone calls via Twilio — make/receive calls, voicemail | Guide |
| sms | sms |
SMS text messages via Twilio | Guide |
email |
Email — send, inbox, search via SMTP/IMAP | Guide |
Full skill system documentation: docs/SKILLS.md
| Slot | Model | Size | Use Case |
|---|---|---|---|
| Fast | qwen3:8b | 5 GB | Casual chat, quick answers |
| Reason | qwen3:30b-a3b | 18 GB | Analysis, research, tool use |
| Code | qwen3-coder:30b | 18 GB | Code tasks, debugging |
| Deep | qwen3:30b-a3b | 18 GB | Strategy, deep analysis |
| Cloud | Claude Code | API | Complex code tasks |
- NVIDIA GPU with CUDA (tested on RTX 3090 24GB)
- Node.js 20+
- Python 3.12+
- Ollama
git clone https://github.com/porokka/jarvis-os.git
cd jarvis-os
.\install-windows.ps1git clone https://github.com/porokka/jarvis-os.git
cd jarvis-os
bash install-linux.shBoth installers handle: system packages, Python deps, Ollama + model pulls, Next.js app, MemPalace, vault setup.
# Start backend
bash jarvis.sh start
# Start HUD (separate terminal)
cd app && npm run dev
# Open http://localhost:3000bash jarvis.sh start # Boot everything
bash jarvis.sh stop # Shut down
bash jarvis.sh status # Health check
bash jarvis.sh restart # Restart all servicespython3 scripts/voice_capture.py # Always-on mode
python3 scripts/voice_capture.py --wake # Wake word mode ("Hey JARVIS")python3 tts/setup.py # Download model (~3GB VRAM)
python3 tts/server.py # Start on :5100| Interface | URL | Description |
|---|---|---|
| Stark HUD | http://localhost:3000 | Next.js holographic dashboard |
| Browser UI | http://localhost:4000 | Simple browser voice UI |
| ReAct API | http://localhost:7900 | Tool-augmented Ollama proxy |
| Orpheus TTS | http://localhost:5100 | Local TTS server (optional) |
POST /api/chat ReAct loop with tools (Ollama-compatible)
GET /api/health Health check
GET /api/skills List loaded skills and their tools
GET /api/timers Active countdown timers
jarvis-os/
├── jarvis.sh # Start/stop/restart/status
├── JARVIS.md # Personality file
├── scripts/
│ ├── watcher.sh # Router + orchestrator
│ ├── react_server.py # ReAct loop + skill loader (315 lines)
│ ├── server.py # Browser bridge HTTP server
│ ├── voice_capture.py # Whisper STT with wake word
│ └── system_api.py # System-level APIs
├── skills/ # ← Modular skill modules
│ ├── __init__.py
│ ├── loader.py # Dynamic discovery + import
│ ├── denon.py # Denon AVR receiver
│ ├── shield.py # NVIDIA Shield rooms
│ ├── radio.py # Internet radio
│ ├── volume.py # System volume
│ ├── timer.py # Countdown timers
│ ├── memory.py # MemPalace memory
│ ├── vault.py # Obsidian vault
│ ├── web.py # Web search + URLs
│ ├── shell.py # Shell + file reading
│ └── claude_skills.py # Claude skill browser
├── config/
│ ├── skills.json # Enable/disable skills
│ └── denon.json # Denon AVR device config
├── app/ # Next.js Stark Industries HUD
│ ├── app/
│ │ ├── page.tsx # Main HUD dashboard
│ │ ├── components/ # GPU monitor, system log, timers, etc.
│ │ └── api/ # Next.js API routes
│ └── lib/
│ └── bridge.ts # Claude/Ollama bridge
├── tts/ # Orpheus 3B TTS server
│ ├── server.py
│ ├── engine.py
│ └── setup.py
├── docs/
│ ├── ARCHITECTURE.md # Full system architecture
│ ├── SKILLS.md # Skill system guide
│ └── skills/ # Per-skill documentation
└── vault/ # Embedded vault context
"Play Nova radio"
"Switch Denon to PC"
"Put on headphones mode"
"Play Metallica on Spotify in the living room"
"Set a timer for 10 minutes — pasta is ready"
"Search for latest Next.js 15 features"
"What do you remember about StockWatch?"
"Turn the volume to 40%"
"Set surround mode to stereo"
"Scan the network for devices"
MIT License — see LICENSE for details.