Next release roadmap

Deliver three reproducible application paths: ticket routing, browser forms/search, and desktop document/save workflows. Each path needs a validated model/backend combination, an independently checked outcome, a runnable recipe, benchmark records and a matching application + System1-Agents + System1-Omni demo.

Keep this issue open as the live task tracker. Use the existing linked issues and PRs for implementation; unchecked tasks remain open even when part of their implementation has merged. The Agents capability matrix and final release evidence are tracked here.

Status checked on 2026-10-08 against Agents 5d89f85 and Omni 4a79980. Checked items describe their stated merged foundation, not validation of all model/scenario/device combinations.

Application scenario support

• Ship the recipe index/template and application + agents + Omni demo requirements:
  PR #43,
  PR #45.
• Finish the reference ticket-routing recipe with independently checked routes, a served-model trace,
  and a video from that recipe's run. The served-Laya adapter and its demo are merged; the recipe still needs
  its own evidence. Related: PR #35,
  PR #43, recipe.
• Add browser form submission, date selection and search fixtures with fixed starting states,
  reset procedures and independent completion checks:
  issue #13.
• Complete desktop typing, keyboard, scrolling and document/file-dialog workflows; verify the saved result
  and state which operating systems were actually tested:
  issue #15,
  RFC #26,
  PR #27.
• Complete a screenshot-dependent desktop task through Omni, starting with the alternating Save/Cancel
  fixture and its independently checked file changes:
  issue #44.
• Qualify bounded recovery on browser and desktop tasks: verified recovery or a clear stop within the
  attempt/time budget, with permissions retained:
  issue #16,
  PR #31.
• Save the final browser page as an inspection artifact alongside the independent task verdict:
  issue #41,
  PR #42.

Benchmarks

• Integrate the public231 decision-service benchmark, including raw request/response evidence,
  accuracy, validity, calibration and per-round p50/p95 latency:
  PR #52.
• Establish the repeatable application baseline using the existing eval runner and record formats.
  Count failures, timeouts and unsupported tasks in completion denominators; report total time, decision
  latency, model cost and failure reason:
  issue #13.
• Compare recovery enabled/disabled on identical tasks and budgets; record completion, additional
  model calls, added time and wasted actions:
  issue #16,
  PR #31.
• Publish reproducible application comparisons with both source SHAs, checkpoint revisions,
  hardware/settings, all attempts, observed variability and raw artifact links. Separate preparation,
  readiness, inference, environment execution and total task time:
  issue #13,
  PR #52.

Decision quality and application completion answer different questions. Kernel/engine optimization and
its measurements belong in System1-Omni; application reports should link the exact engine configuration
and evidence. Before comparisons, declare the hypothesis, isolated variable, controls, success criterion
and run budget. A demo is evidence of one run, not a completion rate or speedup. See
the eval protocol and
Omni's engine benchmark reports.

Model backend support

• Ship served Laya through the existing decision-model interface, CLI and MCP:
  PR #35.
• Ship optional in-process Cua-S1 4B text/multimodal inference and screenshot target selection.
  This does not complete the Omni screenshot client:
  PR #32,
  issue #44.
• Publish an Agents capability matrix for model/checkpoint, input modality, question types, scenario,
  serving path and tested hardware. Track implementation and real validation separately; mark untested
  combinations explicitly. This roadmap tracker owns the Agents matrix; reconcile it with
  Omni's model tracker #83 and
  supported-models documentation.
• Qualify Open-Jev 9B/27B on the text reference path and browser suite through the existing local
  System One endpoint configuration; record the actual served checkpoint rather than inferring it from
  the adapter name:
  PR #36,
  PR #52,
  issue #13.
• Require cold-load timing for new in-process backends, separately from warm inference and episodes:
  PR #51,
  issue #29.

Start with Laya for ticket routing, evaluate Open-Jev for text/browser decisions, and Cua-S1 4B for
screenshots. Engine support alone does not establish fitness for a scenario. Client support for choice
and noul also does not establish score support merely because a worker offers it.

Integration with System1-Omni

• Complete the desktop screenshot serving bridge: current capture/task/candidates to Omni Cua-S1 4B,
  validated answers back to the agent, capture/window binding through execution, and failure/stale-capture
  coverage. Keep image bytes out of serialized observation logs:
  issue #44,
  issue #14,
  PR #32.
• Agree and test the shared /v1/systemone contract, with model-specific capabilities and mappings,
  readiness, bounded deadlines, errors and response provenance. Reuse the existing decision-model seam:
  Omni RFC #61,
  served-Laya design.
• Qualify slow local-server deadline handling, including transport/server evidence and non-finite
  timeout rejection:
  PR #36,
  PR #53.
• Publish one reproducible setup/run/verify path and matching video/trace for each release application.
  Validate the first inference after readiness; identify both repository revisions, checkpoint, worker/frontend,
  device/settings and requests from the same run:
  PR #35,
  issue #13,
  issue #44,
  PR #45.

System1-Agents owns observations, offered actions, execution, recovery and application evaluation.
System1-Omni owns model loading, warmup, serving, accelerator execution and engine optimization.
Record chat-model work separately from decision-model inference.

Delivery order and release gate

1. Publish the capability matrix and complete the ticket-routing text path; integrate the decision benchmark.
2. Establish the browser suite and use it to qualify model choices and bounded recovery.
3. Complete the desktop input/save path and the served screenshot bridge.
4. Publish recipes, benchmark reports, capability status and matching demos for the three application paths.

The release gate is three reproducible application paths, each with a validated model/backend
combination and an independently checked outcome. Required evidence gaps remain open even when an
implementation PR has merged. Follow the recipe guide and
video requirements.

Follow-up model and scenario work

These extend coverage after the reference paths; they are not additional gates for the next release.

• Integrate and evaluate CLM with a real encoder; distinguish the merged Omni stub-contract recipe
  from measured decision quality:
  issue #24,
  PR #46.
• Integrate and evaluate Julia-1 on a suitable labelled task:
  issue #25.
• Evaluate the proposed OmniJev vision browser adapter on the repeatable suite, and verify the
  actual inference path before claiming an Omni integration:
  PR #33.
• Complete the public injection-guard labelled set and runtime quarantine evidence:
  issue #3,
  PR #23,
  PR #6.
• Ship the Snake recipe/game client as an additional application example:
  PR #48.
• Evaluate the draft Sokoban addition against the System 1 task-fit rule; deduction-heavy levels
  require a separate justification and measured baseline:
  PR #34.

Roadmap document

Documentation PR #55 updates the repository roadmap and README navigation; it is pending merge. Read the proposed roadmap at commit 8ca07e0.