🌐 Production URL: https://your-domain.com
A comprehensive web application for comparing AI language model responses side-by-side. Send your prompts to multiple AI models simultaneously — GPT-4.1, Claude 3.7, Gemini 2.0, Grok — and get instant comparisons, quality scores, AI-powered consensus analysis, and detailed performance metrics.
- Multi-Model Comparison — Run a single prompt across up to 20 models simultaneously (tier-dependent) and view all responses side-by-side.
- Quality Scoring — AI-powered evaluation of response quality, relevance, and accuracy with per-model scores.
- Consensus Analysis — Automated synthesis of key insights and areas of agreement or disagreement across all responses.
- Response Timing & Metrics — Track latency, token counts, and estimated cost per model in a dedicated Metrics tab.
- Diff View — Visual highlighting of where model responses agree or diverge.
- Cached Results — Prompts already tested by the community return instantly from cache; no credits are deducted. A green badge confirms when a cached result is served.
- Shareable Results — Generate a public share link for any comparison result. Shared pages display the original prompt and all responses.
- Share to X — One-click tweet with the prompt text and a link to the shared result.
- Personal Library — Save, organize, and reuse your own prompts.
- Prompt Marketplace — Discover and share high-quality prompts with the community. Browse by category, popularity, or persona.
- For You Feed — Personalized prompt recommendations based on your persona and feedback history.
- 👍 Like, 👎 Dislike, or ✕ Dismiss each card to train your feed.
- "Why this?" popover explains each recommendation.
- Feed auto-refreshes after 3 feedback actions.
- Tag Categorization — 30 predefined categories (Code Generation, Creative Writing, Data Analysis, Legal, Medical, and more).
- Bookmarking & Likes — Save and like marketplace prompts.
- Prompt Quality Analyzer (Pro/Mega) — Real-time scoring on clarity, specificity, context, structure, and actionability, with AI-powered rewrite suggestions.
- Prompt Optimizer — Automatically compress prompts to reduce token usage before sending.
- Create & Publish Skills — Package a system prompt as a reusable "skill" and share it to the marketplace.
- Test Skills on Compare — Launch a skill-test session directly from any skill card; results are saved to the skill's test history.
- Security Scanner — Skills are scanned for shell commands, eval, and injection patterns before publishing.
- DMCA / Report Abuse — Users can report marketplace content; admins manage a moderation queue.
- Query History — Full history of all your comparisons with the original prompt, model responses, and scores.
- Cost Dashboard — Per-query cost breakdown, monthly spending totals, and budget alerts.
- Milestone Achievements — Achievement toasts at 1, 5, 10, 25, 50, and 100 total queries.
- Saved Comparison Profiles — Save and reload named model selection presets.
- Personal API Keys — Add your own OpenAI, Anthropic, Google, or xAI API keys to bypass rate limits.
- Magic Link Login — Passwordless email login with a one-time code.
- OAuth Authentication — Secure login via Platform OAuth.
- Onboarding Flow — Guided welcome flow with persona selection, example prompts, and a first comparison walkthrough.
Ten role-specific landing pages showcase relevant use cases:
| Path | Persona |
|---|---|
/for/developers |
Code review, debugging, architecture |
/for/teachers |
Lesson plans, quiz generation |
/for/lawyers |
Contract review, legal research |
/for/doctors |
Clinical summaries, patient education |
/for/marketers |
Copywriting, campaign ideas |
/for/writers |
Story development, editing |
/for/students |
Essay help, study guides |
/for/researchers |
Literature review, hypothesis testing |
/for/entrepreneurs |
Business plans, pitch decks |
/for/hr |
Job descriptions, interview questions |
- API Key Management — Add, edit, and delete personal AI provider keys.
- System Prompt Customization — Override the default system prompt per-session.
- For You Preferences — View liked/disliked/dismissed prompt counts and reset your feed preferences.
- Notification Preferences — Control which in-app notifications you receive.
- Billing & Subscription — Manage your plan via Stripe.
The admin panel uses a left sidebar with four grouped sections:
People
- User Management — View all users, change roles (Trial / Pro / Mega / Admin), toggle email verification.
- Invitations — Manage user invitation codes.
- Contacts — View and export contact submissions.
Content
- Feedback — Review user feedback submissions.
- Prompt Viewer — Browse all prompts in the system.
- Skills — Moderate marketplace skills; view stats and delete content.
- Changelog — Create, edit, publish, and notify users about release notes.
Analytics
- Statistics — Summary stats (queries, active users, avg response time, errors).
- Optimization Analytics — Token savings, user adoption rate, top users by savings.
- Tag Analytics — Tag usage, popular categories, marketplace trends.
System
- Error Logs — Searchable error log with AI-powered "Explain this error" feature.
- System API Keys — Configure provider API keys used by trial users.
- AI Models — Add, edit, toggle, and delete available models.
- Payments — Stripe subscription and payment management.
- Event Log — Full audit trail of system events.
- Magic Link Log — History of all magic link login attempts.
- Notifications — Send in-app broadcasts to users by tier.
- Model Health — Live health check across all configured models.
- Extension Signups — Alpha tester email list with copy-all export.
- React 19 with TypeScript
- Tailwind CSS 4 for styling
- shadcn/ui component library
- Wouter for client-side routing
- TanStack Query (via tRPC) for data fetching and caching
- Express 4.22 with TypeScript
- tRPC 11.17 for end-to-end type-safe API
- Drizzle ORM 0.45 for database management
- Superjson for transparent Date/BigInt serialization
- Axios 1.16 for outbound HTTP requests
- MySQL 8.0+ or MariaDB 10.6+
- TiDB compatible (used in production)
- Node.js 22.x
- pnpm package manager
- Vitest for unit testing (341 tests)
- Zero high CVEs in direct dependencies (as of v1.8.x)
- pnpm overrides for
rollup,tar,qs,lodash,socket.io-parser - Express 4 pinned at
4.x— Express 5 is incompatible withpath-to-regexpwildcard routes
ai-model-comparator/
├── client/ # Frontend React application
│ ├── public/ # Static assets
│ └── src/
│ ├── components/ # Reusable UI components
│ │ ├── AdminChangelog.tsx
│ │ ├── AdminEventLog.tsx
│ │ ├── AdminMagicLinkLogs.tsx
│ │ ├── AdminNotifications.tsx
│ │ ├── AdminPromptViewer.tsx
│ │ ├── ContactManagement.tsx
│ │ ├── DashboardLayout.tsx
│ │ ├── ErrorLogsViewer.tsx
│ │ ├── FeedbackManagement.tsx
│ │ ├── InvitationManager.tsx
│ │ ├── MilestoneToast.tsx
│ │ ├── StripeAdminPanel.tsx
│ │ ├── TagAnalytics.tsx
│ │ └── ui/ # shadcn/ui primitives
│ ├── pages/ # Page-level components
│ │ ├── Admin.tsx # Admin dashboard (left sidebar)
│ │ ├── Billing.tsx # Stripe billing
│ │ ├── Changelog.tsx # Public changelog
│ │ ├── Compare.tsx # Main comparison page
│ │ ├── CostDashboard.tsx # Spending analytics
│ │ ├── Docs.tsx # Documentation
│ │ ├── Extension.tsx # Browser extension info
│ │ ├── History.tsx # Query history
│ │ ├── Home.tsx # Landing page
│ │ ├── Library.tsx # Personal prompt library
│ │ ├── Marketplace.tsx # Community marketplace + For You
│ │ ├── PersonaLanding.tsx # /for/:persona pages
│ │ ├── Profile.tsx # User profile
│ │ ├── Settings.tsx # Settings + Preferences
│ │ ├── SharedResult.tsx # Public shared result view
│ │ ├── SkillsLibrary.tsx # Skills marketplace
│ │ ├── Stats.tsx # Public stats dashboard
│ │ └── WelcomeOnboarding.tsx
│ ├── contexts/ # React contexts (auth, theme)
│ ├── hooks/ # Custom hooks
│ ├── lib/trpc.ts # tRPC client binding
│ ├── App.tsx # Routes and layout
│ └── main.tsx # Providers
├── server/ # Backend Express application
│ ├── _core/ # Framework plumbing (auth, context, vite bridge)
│ ├── db.ts # Database query helpers
│ ├── email.ts # SendGrid email helpers (incl. extension signup)
│ ├── routers.ts # Root tRPC router (30+ sub-routers)
│ ├── routersAnalytics.ts
│ ├── routersChangelog.ts
│ ├── routersMarketplace.ts # Marketplace + For You feed
│ ├── routersSkills.ts
│ ├── routersStripe.ts
│ ├── skillScanner.ts # Security scanner for skill content
│ └── *.test.ts # Vitest test files
├── drizzle/
│ └── schema.ts # 50+ database tables
├── scripts/
│ ├── seed-changelog-178.mjs # Changelog seed (v1.8.0–1.8.5)
│ └── seed-data.mjs
├── shared/ # Shared constants and types
├── storage/ # S3 storage helpers
├── package.json
└── README.md
| Table | Purpose |
|---|---|
users |
Accounts, roles (trial/pro/mega/admin), OAuth identity |
queries |
Comparison runs — prompt, models, promptHash for caching |
responses |
Per-model results — response text, tokens, duration, cost |
aiModels |
Configured AI models with pricing |
userAiApiKeys |
User-provided provider API keys |
marketplacePrompts |
Community-published prompts |
promptTags |
30 category tags |
promptLikes / promptBookmarks |
Engagement signals |
forYouFeedback |
Like/dislike/dismiss signals for personalized feed |
userCategoryWeights |
Per-user category boost weights for For You |
userSkills / marketplaceSkills |
Skills library |
skillTestResults |
Results from skill-test sessions |
contentReports |
DMCA / abuse reports |
changelog |
Published release notes |
eventLogs |
Full audit trail |
magicLinkCodes |
Passwordless login tokens |
subscriptions / paymentHistory |
Stripe billing |
extensionSignups |
Browser extension alpha list (with referral tracking) |
errorLogs |
Server-side error capture |
visitorSessions / pageViews |
Anonymous analytics |
The application exposes ~30 tRPC sub-routers. Key procedures:
query.submit— Submit a prompt to one or more models; returns a query ID for polling.query.getResults— Poll for results by query ID or public share token.query.getHistory— Paginated query history for the current user.query.share— Generate a public share token.
marketplace.getMarketplacePrompts— Paginated list with tag/search filters.marketplace.getForYou— Personalized recommendations based on persona and feedback.marketplace.submitForYouFeedback— Like/dislike/dismiss a For You card.marketplace.getForYouFeedback— Fetch the current user's feedback map.marketplace.resetForYouFeedback— Clear all feedback and category weights.
skills.listMarketplace— Browse published skills.skills.create/skills.update— Author a skill (with security scan).skills.publishToMarketplace— Submit for community use.skills.listAdmin/skills.adminDeleteSkill— Admin moderation.
changelog.list— Public list of published entries.changelog.adminCreate/adminUpdate/adminTogglePublish— Admin authoring.
admin.getUsers/updateUserRole/updateEmailVerificationadmin.getSummaryStats/getOptimizationStatsadmin.getSystemApiKeys/setSystemApiKey/deleteSystemApiKeyadmin.getModels/createModel/updateModel/toggleModelStatusadmin.explainError— AI-powered error explanation.
auth.me— Current user.auth.logoutmagicAuth.sendCode/verifyCode— Magic link login.account.updateProfile/changePassword
git clone https://github.com/your-github-username/ai-model-comparator.git
cd ai-model-comparatorpnpm installcp .env.example .envRequired variables:
# Database (MySQL / TiDB)
DATABASE_URL=mysql://user:password@host:3306/ai_comparator
# Auth
JWT_SECRET=your-secure-random-secret
OAUTH_SERVER_URL=https://your-oauth-server.example.com
VITE_OAUTH_PORTAL_URL=https://your-oauth-portal.example.com
VITE_APP_ID=your-manus-app-id
OWNER_OPEN_ID=your-open-id
OWNER_NAME=Your Name
# Manus built-in APIs (LLM, storage, notifications)
BUILT_IN_FORGE_API_URL=https://your-forge-api.example.com
BUILT_IN_FORGE_API_KEY=your-server-api-key
VITE_FRONTEND_FORGE_API_KEY=your-frontend-api-key
VITE_FRONTEND_FORGE_API_URL=https://your-forge-api.example.com
# Email (SendGrid)
SENDGRID_API_KEY=your-sendgrid-api-key
SENDGRID_FROM_EMAIL=noreply@yourdomain.com
# Stripe (optional)
STRIPE_SECRET_KEY=sk_...
STRIPE_WEBHOOK_SECRET=whsec_...
VITE_STRIPE_PUBLISHABLE_KEY=pk_...
# Analytics (optional)
VITE_ANALYTICS_ENDPOINT=https://analytics.yourdomain.com
VITE_ANALYTICS_WEBSITE_ID=your-website-id
# Branding
VITE_APP_TITLE=AI Model Comparator
VITE_APP_LOGO=/logo.png
VITE_APP_DOMAIN=yourdomain.com# Create database
mysql -u root -p -e "CREATE DATABASE ai_comparator CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;"
# Push schema
pnpm db:pushpnpm dev
# App available at http://localhost:3000pnpm build
node server/_core/index.ts# Run all tests (341 tests across 27 files)
pnpm test
# Run a specific file
pnpm test server/marketplace.features.test.ts
# Watch mode
pnpm test --watchTest files cover: auth, marketplace, For You feedback, prompt analyzer, skills, rate limiting, cost tracking, analysis, and more.
- Express 4 is pinned at
^4.22.x. Do not upgrade to Express 5 — it usespath-to-regexp8.x which breaks wildcard route matching (app.use("(.*)", ...)). - All direct dependencies are free of high CVEs as of v1.8.x.
- pnpm overrides patch transitive vulnerabilities in
rollup,tar,qs,lodash,lodash-es, andsocket.io-parser.
See the live Changelog page or the Admin → Changelog panel.
Proprietary software. All rights reserved.
Version: 1.8.5 Last Updated: May 2026