feat(alerting): custom alert rule builder (plan 32) by duyet · Pull Request #2257 · chmonitor/chmonitor · GitHub
Skip to content

feat(alerting): custom alert rule builder (plan 32) - #2257

Merged
duyet merged 1 commit into
mainfrom
advisor/32-custom-alert-rule-builder
Jul 4, 2026
Merged

feat(alerting): custom alert rule builder (plan 32)#2257
duyet merged 1 commit into
mainfrom
advisor/32-custom-alert-rule-builder

Conversation

@duyet

@duyet duyet commented Jul 3, 2026

Copy link
Copy Markdown
Member

Summary

Implements plan 32: a no-code builder for numeric-threshold alert rules — pick
a whitelisted metric (a fixed catalog key mapped server-side to one vetted
read-only SQL template), an operator, and warning/critical thresholds.
No free-form SQL field exists anywhere in the API or UI.

  • lib/health/rule-builder-schema.tsMETRIC_CATALOG (10 curated read-only
    queries: active/failed mutations, max parts per partition, readonly
    replicas, replication lag/queue, disk usage %, running/long-running
    queries, stuck merges), zod validation, pure compileCustomRule, and
    assertReadOnlySql — a deny-list guard checked at compile time, again
    before persisting, and again before registering into the sweep. Plan 33
    (assertReadOnlyAction) isn't merged yet, so this ports the same intent
    locally rather than depending on it.
  • lib/health/custom-rules-store.ts + migration 0014_custom_alert_rules.sql
    — owner-scoped D1 CRUD (mirrors events/subscription-store.ts), fails open
    on any D1/query failure (loadCustomRulesIntoRegistry) so built-ins keep
    running untouched.
  • lib/health/custom-rules-auth.ts — owner resolution that stays auth-gated
    when Clerk is configured (signed-out request → 401, preserves cloud
    per-user isolation) but falls back to a fixed single-tenant owner id when
    Clerk is not configured. Custom rules are a core monitoring feature per
    root CLAUDE.md ("self-hosted stays whole"), so this had to work without
    Clerk — unlike webhook subscriptions (plan 44), which are Clerk-only.
  • API: routes/api/v1/health/custom-rules.ts (GET list / POST create, +
    ?catalog=1 for the builder dropdown), custom-rules/$id.ts (DELETE),
    custom-rules/test.ts (read-only "test" preview against a host).
  • components/health/rule-builder.tsx — new "Custom Rules" tab in Health
    Settings: metric/op/threshold form, live SQL-free preview, save/list/
    delete/test. Shows an explicit "not available" message (like the sibling
    webhook panel) when no D1 binding is configured, instead of a form that
    would 501 on save.
  • lib/alerting/rule-registry.ts — added an optional classify field to
    AlertRuleDef so </<= operators ("lower is worse", e.g. a cache-hit
    ratio) work without changing classifyValue's default higher-is-worse
    semantics that every built-in rule relies on. The sweep's unregister loop
    keys off id.startsWith('custom:'), not type === 'custom' — so it never
    touches the pre-existing built-in fatal-log-entries rule, which already
    used type: 'custom' for taxonomy purposes.
  • lib/health/server-sweep.ts — re-syncs custom rules every sweep tick:
    unregisters stale custom:* ids first, then loads whatever is currently
    enabled in D1 (cron is a single global process, not scoped to a signed-in
    visitor — see open question 3 below).

Coordination note (plan 31 — compound alert rules, parallel work)

Plan 31 (compound alert rules) is being implemented in parallel on
advisor/31-compound-alert-rules, but that branch has zero commits as of
this PR (still even with origin/main), so there is no code overlap today.
Both plans touch the concept of a "rule" but at different layers: this PR
extends rule-registry.ts/AlertRuleDef with one small additive optional
field (classify) and never restructures the rule shape. If plan 31
introduces a type: 'compound' variant or a different composition mechanism,
it should compose cleanly with this — a compound rule's sub-conditions could
themselves reference custom rules by id. Flagging for the reviewer to
reconcile if 31 lands with conflicting assumptions about the rule model.

Answers to the plan's open questions

  1. Registration lifetime — per-sweep (recommended option): unregister
    custom:* ids, then reload enabled rows from D1 every tick.
  2. Operator direction — supported both directions via the new optional
    AlertRuleDef.classify field (see above) rather than inverting SQL or
    negating thresholds.
  3. Per-owner isolation in the sweep — the sweep loads all enabled
    custom rules across every owner (listAllEnabledCustomRules, no owner
    filter). The cron sweep is a single global process, mirroring how
    HEALTH_ALERT_WEBHOOK_URL is already a single env-wide destination today.
    True per-owner alert routing in a multi-tenant cloud deployment is a
    documented follow-up, not attempted here.
  4. Catalog size — shipped with 10 curated metrics; adding one is a
    deliberate code change in METRIC_CATALOG.
  5. Free vs paid — free/OSS; plan-enforcement.ts does not list custom
    rules as a paid capability.

Test plan

  • cd apps/dashboard && bun run type-check — 0 errors
  • cd apps/dashboard && bun run build — vite build + tsc --noEmit, exit 0
  • cd apps/dashboard && bun test src/lib/health/rule-builder-schema.test.ts --isolate — 18 pass, 0 fail (off-catalog metric rejected, non-numeric/non-finite threshold rejected, compiled SQL === catalog template exactly, deny-list passes/fails as expected, </<= classification direction)
  • cd apps/dashboard && bun test src/lib/health/ --isolate — 192 pass, 0 fail (includes the new custom-rules-store.sql.test.ts, which proves the DELETE SQL is ownership-guarded by running it against real bun:sqlite, mirroring subscription-store.sql.test.ts's pattern for the same IDOR class)
  • bun run lint (root — apps/dashboard has no standalone lint script; the plan's literal cd apps/dashboard && bun run lint fails with "Script not found", a pre-existing plan/package.json mismatch, not something introduced here) — clean, no fixes needed
  • Owner isolation beyond the DELETE-guard sql.test (e.g. cross-owner GET/POST) is not separately covered — flagging per the "should-add" note rather than silently implying full coverage
  • Not manually exercised against a live D1-backed deployment (no D1 binding in this sandbox)

https://claude.ai/code/session_01XpfaNdJZh4Rtr6cQm4DYqa

@github-actions github-actions Bot added the app: dashboard Changes to the apps/dashboard TanStack Start app label Jul 3, 2026
@duyet
duyet enabled auto-merge (squash) July 3, 2026 17:29

duyet commented Jul 4, 2026

Copy link
Copy Markdown
Member Author

Lets users build a numeric-threshold alert rule from whitelisted building
blocks (a known metric -> a vetted read-only SQL template, an operator,
warning/critical thresholds) instead of editing TypeScript. Compiles to an
AlertRuleDef, persists in D1 (custom_alert_rules), and the sweep re-syncs
enabled custom rules every tick alongside the built-ins.

- rule-builder-schema.ts: fixed METRIC_CATALOG (10 vetted read-only queries),
  zod validation, pure compileCustomRule, assertReadOnlySql deny-list
  (defense-in-depth; no SQL is ever built from free text)
- custom-rules-store.ts + 0014 migration: owner-scoped D1 CRUD, fails open
  on D1/query failure so built-ins keep running
- custom-rules-auth.ts: owner resolution that stays auth-gated when Clerk is
  configured but falls back to a fixed single-tenant owner when it is not,
  so this works self-hosted without Clerk (core feature, not cloud-only)
- custom-rules.ts / $id.ts / test.ts API routes: GET/POST/DELETE + a
  read-only "test" endpoint for the builder's live preview
- rule-builder.tsx + Custom Rules tab in health settings: no free-form SQL
  field anywhere
- rule-registry.ts: optional AlertRuleDef.classify so "<"/"<=" ops work
  without changing classifyValue's default higher-is-worse semantics

Co-Authored-By: duyetbot <bot@duyet.net>
@duyet
duyet force-pushed the advisor/32-custom-alert-rule-builder branch from 10aaac1 to 1aac6e3 Compare July 4, 2026 06:04
@github-actions

github-actions Bot commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

☁️ Cloudflare Preview Deployment

App Preview URL
Dashboard https://preview.dash.chmonitor.dev
MCP https://preview.dash.chmonitor.dev/api/mcp
Landing https://preview.chmonitor.dev
Docs https://preview.docs.chmonitor.dev
Property Value
Commit 1aac6e3
Deployed at 2026-07-04T06:07:25.370Z

Previews are automatically updated on every push to this PR.
Unchanged apps keep their previous preview (path-filtered deploys).

@duyet
duyet merged commit 8401a50 into main Jul 4, 2026
18 of 19 checks passed
@duyet
duyet deleted the advisor/32-custom-alert-rule-builder branch July 4, 2026 06:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

app: dashboard Changes to the apps/dashboard TanStack Start app

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant