test(snapshot): Add structured JSON fixture targets (#349) * ✨ feat: Add domain result output architecture with structured JSON envelope Introduce a new type layer that separates domain results from presentation, enabling tools to produce structured JSON output alongside existing text/JSONL rendering. Migrates simulator-list as the pilot tool to validate the end-to-end flow. - Add ToolDomainResultBase and 26 schema-derived domain result types - Add ProgressEvent, StructuredOutputEnvelope, ToolExecutionContext types - Add PipelineEvent adapter for backward-compatible rendering - Add DefaultToolExecutionContext bridging new and existing execution models - Wire CLI --output json to produce StructuredOutputEnvelope when available - Migrate simulator list tool to new ToolExecutor pattern as proof-of-concept - Replace old STRUCTURED_JSON_OUTPUT_PLAN with comprehensive architecture doc * ♻️ refactor(tools): Migrate 31 tools to ToolExecutor domain result pattern Apply the domain result architecture to all non-build tool families so each tool produces a typed domain result and emits progress events, while preserving existing PipelineEvent rendering via the adapter. Migrated tool families: - device: list, app-path, install, launch, stop - macos: app-path, launch, stop - project-discovery: discover, schemes, bundle-id (sim+mac), build-settings - session-management: clear, set, show, use-profile - simulator: boot, open, app-path, install, launch, stop - simulator-management: erase, location, appearance, statusbar - swift-package: list, run, stop Each tool now exposes a createXxxExecutor() returning a ToolExecutor and sets structuredOutput on the handler context for CLI JSON envelope output. Existing text/MCP behavior is preserved through the DefaultToolExecutionContext + PipelineEvent adapter bridge. * ♻️ refactor(tools): Migrate build/test, debugging, UI automation, coverage, scaffolding, and capture tools Complete Phase 2 of the domain result architecture by migrating all remaining tool families to the ToolExecutor pattern, covering build/test streaming, debugger integration, UI automation, coverage, scaffolding, and video capture. Migrated tool families: - build/build-run/test: simulator, device, macos, swift-package (11 tools) - debugging: attach, detach, continue, breakpoints, lldb, stack, variables (8) - ui-automation: tap, swipe, button, gesture, key, type, screenshot, snapshot (11) - coverage: report, file-coverage (2) - project-scaffolding: iOS, macOS (2) - capture: record-video (1) - swift-package: clean (1) New shared infrastructure: - xcodebuild-domain-results.ts: streaming xcodebuild to domain result mapping - tool-execution-compat.ts: handler-backed execution context helper - ui-automation/shared/domain-result.ts: UI action result helpers - project-scaffolding/domain-result.ts: scaffold result helpers Only intentionally skipped: xcode-ide bridge tools, doctor, clean, manage-workflows, active-processes (no matching domain result types). * ♻️ refactor(tools): Migrate final 9 tools and add new structured output schemas Complete Phase 2 by migrating all remaining tool handlers: doctor, clean, manage-workflows, sync-xcode-defaults, and the 5 xcode-ide bridge tools. Adds 6 new structured output schemas and aligns clean and sync-xcode-defaults with their existing JSON fixture expectations. New domain result types and schemas: - doctor-report, workflow-selection (new schemas) - xcode-bridge-status, xcode-bridge-sync, xcode-bridge-tool-list, xcode-bridge-call-result (new schemas) - clean reuses build-result schema (per fixture) - sync-xcode-defaults reuses session-defaults schema (per fixture) Bridge layer updated to carry typed payloads so tools translate real data instead of parsing rendered text. Every tool handler in the codebase now uses the ToolExecutor pattern with structured output. Only 2 internal utilities remain without it (device/build-settings, swift-package/active-processes). * ✨ feat(cli): Complete Phase 3 output routing with jsonl mode and daemon streaming Implement the final output routing layer so each CLI mode draws from the correct canonical source: json from domain results, jsonl from streamed ProgressEvents, and text from PipelineEvent-adapted progress. Phase 3 changes: - Add --output jsonl CLI mode streaming ProgressEvents as NDJSON - Add ToolHandlerContext.emitProgress for canonical progress delivery - Add InvokeOptions.onProgress/onStructuredOutput hooks - Wire direct invocation path to forward progress hooks - Upgrade daemon tool.invoke to v3 streaming protocol: progress frames + terminal result frame replacing buffered PipelineEvent[] - Update daemon server, client, and tool-invoker for streaming - Add DefaultToolExecutionContext.progressSink and getResult() - Remove dead cli-json render strategy - Remove unused DaemonToolMessage type - Forward progressSink in compat helpers for stateful tools Daemon protocol bumped to v3 for tool.invoke only; xcode-ide and other unary methods remain unchanged. * 🔥 refactor: Remove PipelineEvent and complete domain result architecture Fully remove PipelineEvent as the canonical data model, completing Phase 5 of the domain result output architecture. ProgressEvent is now the sole event contract across rendering, daemon transport, and tool execution. Core changes: - Expand ProgressEvent with rendering-capable variants (header, section, detail-tree, file-ref, next-steps, build-stage, compiler diagnostics, test progress/failure/discovery) - Build progress-native text renderer replacing PipelineEvent-based rendering while preserving byte-identical text output - Migrate xcodebuild parser pipeline to emit ProgressEvent directly - Migrate all tool handlers, test helpers, bridge integrations, and daemon transport off PipelineEvent - Remove RenderSession PipelineEvent dependency Deleted: - src/types/pipeline-events.ts - src/utils/domain-result-adapter.ts (adapter no longer needed) Added: - src/utils/renderers/domain-result-text.ts (structured result to text rendering) All text snapshot fixtures pass unchanged. Zero PipelineEvent references remain in src/. * 🐛 fix(snapshot): Fix rendering regressions and add JSON fixture parity tests Fix text and structured output regressions exposed by running the full snapshot test suite end-to-end after the PipelineEvent removal. No fixture files were modified — all fixes are in tool and renderer code to match the existing hand-crafted fixtures. Text rendering fixes: - Eliminate duplicate header/error emission in tool handlers that were emitting both legacy events and structured result text - Fix spacing, error prefix, and section ordering regressions across device, simulator, project-discovery, scaffolding, and session tools - Restore canonical transcript shape for build/test/clean tools JSON fixture parity infrastructure: - Add json-fixture-parity.snapshot.test.ts validating structured output against all hand-crafted JSON fixtures - Add json-harness.ts for running tools and capturing structured output - Add json-normalize.ts for deterministic comparison of tool output against fixture expectations Validated: 1453 unit tests pass, 523 snapshot tests pass (37 skipped for environment reasons), 141 JSON schema validations pass. * ♻️ refactor(tools): Phase 5a — non-streaming tools render from domain result only Eliminate dual-path rendering for non-streaming tools. The domain result is now the sole source of text output — no progress events are emitted during execution. This removes the architectural issue where both the live progress stream and renderDomainResultTextItems could produce the same text, causing duplication bugs. Changes: - Simplify DefaultToolExecutionContext.emitResult() to store only - Expand renderDomainResultTextItems with complete rendering for all 20+ non-streaming result kinds - Update shouldRenderStructuredOutput to render structured output for all non-streaming results by default - Remove ctx.emit()/ctx.emitProgress() from all non-streaming tool handlers (~65 tools across all families) - Update architecture doc to reflect the two-category model: non-streaming (domain result only) vs streaming (live progress) - Update unit tests for new rendering contract Streaming tools (build, build-run, test) are unchanged — they still emit progress events for CLI text/jsonl modes. * 🐛 fix(rendering): Fix snapshot regressions and implement Phase 5b-d streaming Restore durable headers via ctx.emit() for non-streaming tools that need frontmatter in their text output. Implement explicit liveProgressEnabled gating so streaming only activates for CLI text/jsonl. Make xcodebuild tools emit complete live tails (summary, build log path). Remove filterStructuredOutputItems and dual-path coordination logic from cli-text-renderer. * 🐛 fix(snapshot): Fix all 48 snapshot test rendering regressions Fix every non-streaming tool snapshot failure caused by Phase 5a migration. Domain result text rendering now matches all CLI and MCP fixtures byte-for-byte. Fixes: - Restore durable ctx.emit(header) for tools needing frontmatter - Fix app-path error summary to use "Query failed." not full message - Fix app-path success to include duration in simulator variant - Add MCP build-and-run intermediate status lines to domain renderer - Fix discover-projs path normalization for workspace root - Fix swift-package run error suppressing extra build log tail - Fix snapshot-ui header title, JSON alignment, tips formatting - Fix snapshot-ui error details section rendering - Fix build settings empty-value formatting preserving "key =" - Update cli-text-renderer to skip structured replay when live header was already emitted All 544 snapshot tests pass (0 failures, 16 skipped). All 1467 unit tests pass. * 🐛 fix(snapshot): Un-skip all snapshot tests and fix JSON parity Remove all it.skip and conditional skip patterns added during the refactor. Fix JSON fixture parity for simulator, device, session, and resource tests. Add deterministic simulator state setup for list snapshot tests. Normalize JSON device state strings. Unit tests: 1467/1467 passed Snapshot tests: 0 skipped (was 16) * 🐛 fix(snapshot): Fix swift-package list stale daemon process leak Stop the daemon before swift-package list tests to clear stale process records from prior run tests. Also kill any system-level spm processes not tracked by the daemon. * 🐛 fix(snapshot): Remove multi-simulator orchestration and restore test isolation Remove ensureNamedSimulatorStates and ensureSimulatorState functions that were added during the refactor. These booted up to 5 real named simulators with no teardown, causing resource exhaustion, Launch Services errors, device test stalls, and fixture mismatches. Restore pre-refactor approach: - Only ensureSimulatorBooted for primary test simulator - createTemporarySimulator for throwaway tests - Environment-dependent list/resource tests marked as it.skip - Suite-level daemon/process cleanup for swift-package isolation * 🐛 fix(snapshot): Fix last 2 JSON parity failures Fix simulator-management open test to expect success (no longer fails with Launch Services error now that simulators aren't exhausted). Update simulator test failure JSON fixture to match real intentional test failures instead of system launch denial that only occurred under simulator resource exhaustion. * fix(snapshot): Un-skip all tests, add state normalization, boot iPhone 17 Pro - Remove all it.skip patterns from snapshot suites - Boot iPhone 17 Pro as primary test simulator (not iPhone 17) - Add normalizer for simulator state markers ([STATUS], SIM_STATE) - Add normalizer for floating-point frame coordinates in UI hierarchy - Add JSON normalizer for simulator state fields - Update list/resource fixtures for single-booted-sim environment - Update device list fixtures with STATUS placeholders - Add test execution rules to CLAUDE.md and AGENTS.md - Add investigation report for snapshot failures Pre-existing failures (verified on pre-refactor base): - snapshot-ui success: accessibility hierarchy content drift - smoke tests: 4 pre-existing failures * fix(snapshot): Normalize volatile UI hierarchy and simulator state - Normalize accessibility hierarchy JSON block to stable placeholder (content varies with simulator state/app foreground) - Normalize simulator state in JSON fixtures (Booted/Shutdown -> ) - Update snapshot-ui fixtures with correct tips text and normalized hierarchy - Fix simulator build fixture sim name (iPhone 17, not iPhone 17 Pro) - Boot iPhone 17 Pro as primary sim, pass iPhone 17 as build param name * fix(snapshot): Use schema-valid minimal hierarchy in JSON fixture * fix(device): Disable post-failure devicectl diagnose on test runs Pass `-collect-test-diagnostics never` to xcodebuild for all build and test actions. After a failing physical-device `xcodebuild test`, Xcode invokes `devicectl diagnose`, which can prompt for a macOS password on a TTY and wedge automated runs until they time out. Opting out of post-failure diagnostics collection makes the real error surface immediately instead of hanging. Co-Authored-By: Claude * ref(clean): Route clean through executeXcodeBuildCommand Replace the hand-rolled command construction and error extraction in the clean tool with a call to the shared `executeXcodeBuildCommand` helper. This consolidates xcodebuild invocation behind one path, so clean automatically inherits project/workspace resolution, destination handling, and any shared flags (including the new `-collect-test-diagnostics never`) without duplicating logic. Stderr is captured and filtered for a minimal set of diagnostic noise patterns so failure messages remain useful without dragging in xcodebuild banner lines. Co-Authored-By: Claude * test(snapshot): Restore 120s timeouts and normalize Xcode running line Drop CLI and MCP snapshot command timeouts from 300s back to 120s now that the device diagnostics hang is prevented upstream. Longer timeouts only masked the wedge and slowed the suite. Also normalize the doctor resource's "Xcode running" line so the fixture no longer depends on whether Xcode happens to be open during a run, and add a `test:snapshot:device` script for targeted reruns of the device snapshot file. Co-Authored-By: Claude * docs: Add device snapshot hang RCA and investigation notes Capture the root-cause analysis for the physical-device snapshot hang triggered by `devicectl diagnose` interactive auth, along with two broader investigation logs covering device snapshot regressions and CLI/MCP snapshot parity. Add a pointer in AGENTS.md so future runs that hit the same symptom can jump straight to the RCA. Co-Authored-By: Claude * ref: Replace ProgressEvent with DomainFragment streaming model Retire the ProgressEvent union in favor of DomainFragment types that share the `kind` discriminator with their corresponding final domain result, so streaming partial updates and final results are part of the same model family and no longer require a separate UI event model at the tool layer. Introduce `src/types/domain-fragments.ts` for tool-level fragments (invocation, build/test/run stages, compiler diagnostics, etc.) and `src/types/runtime-status.ts` for infrastructure-level status fragments. Renderers translate fragments to presentation output, keeping tools free of rendering concerns. Add a `raw` render strategy plus `src/rendering/render-items.ts` to stream transcript-kind fragments (process-command, process-line) straight to stderr while still rendering the final domain result on stdout. Extract app lifecycle and app-query domain shapes into dedicated modules (`utils/app-lifecycle-results.ts`, `utils/app-query-results.ts`) and add `utils/transcript-context.ts` for shared transcript hooks. Migrate every MCP tool, daemon path, CLI renderer, and test fixture to the fragment model, rename `getEvents`/`renderEvents` to `getFragments`/`renderFragments`, and drop the now-unused `tool-error-handling.ts`, `tool-event-builders.ts`, and `types/progress-events.ts`. Remove the outdated `DOMAIN_RESULT_OUTPUT_ARCHITECTURE.md` doc and the two smoke tests (`e2e-mcp-doctor`, `e2e-mcp-simulator`) superseded by snapshot coverage. * test(runtime): Align rebased tests with fragment contracts Update the rebased test suite to match the current fragment and runtime-status APIs. Replace stale progress-event expectations with fragment-based assertions, move infrastructure status helpers onto the runtime-only status type, and repair the small broken test helpers surfaced by typecheck after the rebase. This keeps the branch working on top of main without reintroducing old ProgressEvent assumptions. Co-Authored-By: OpenAI Codex * fix(review): Address daemon and screenshot feedback Fix the daemon-backed JSONL path so streamed fragments are only recorded once in the render session while still being serialized once to NDJSON. Also preserve screenshot dimensions when the base64 fallback keeps the original PNG after optimization fails, and remove the dead imports and helper parameter surfaced in review. Co-Authored-By: OpenAI Codex * fix(schema): Deduplicate bridge status defs Move the shared xcode bridge status schema fragment into the common structured-output defs file so the status and sync schemas cannot drift independently. Also replace the hardcoded physical device UUID in the repo-maintained agent docs with a portable placeholder for other contributors. Co-Authored-By: OpenAI Codex * fix(cli): Return JSON envelope for missing structured output Keep --output json machine-readable when a tool fails to populate structured output. Instead of throwing and printing a plain stderr message, write a parseable JSON error envelope and exit with a non-zero code. Update the CLI test to assert the envelope contract so this failure mode stays stable for automation consumers. Co-Authored-By: OpenAI Codex * ref(output): Standardize tool output model Unify structured tool results around canonical envelope fields, diagnostics, and rendering paths. Update tool manifests, schemas, documentation, and tests so streaming tools use streaming-only context while unary tools return structured results. Co-Authored-By: Codex * test(snapshot): Update structured output fixtures Refresh CLI, MCP, and JSON snapshot fixtures for the standardized output model and diagnostics rendering. Keep fixture churn separate from the source changes for review. Co-Authored-By: Codex * test(snapshot): Improve fixture diff formatting Render fixture mismatches with a real line diff so consecutive changes are grouped as removals followed by additions. Keep the output plain ASCII so Vitest error messages remain aligned and readable. Co-Authored-By: Codex * test(snapshot): Normalize build settings PATH Replace volatile PATH build setting values with a stable placeholder while keeping the build settings entry in the fixture output. Co-Authored-By: Codex * test(snapshot): Refresh recorded fixture values Update recorded snapshot fixture values that changed during the latest focused runs. Keep these separate from harness and normalisation logic for easier review. Co-Authored-By: Codex * ref(cli): Flatten JSONL event envelope Project domain fragments directly onto the public `--output jsonl` wire shape instead of wrapping them in `{type:"fragment", fragment}`. The event discriminator is derived mechanically from the fragment's `kind` and `fragment` fields (lowercased `.`) and all remaining fields pass through unchanged, so new fragments surface automatically without mapping work. * ref(runtime-status): Centralize infrastructureStatus helper Export infrastructureStatus from src/types/runtime-status.ts and import it in cli-tool-catalog and typed-tool-factory instead of redefining the same helper locally in each module. Prevents drift if RuntimeStatusFragment's shape or level set ever changes. * fix(tests): Update simulator/debugger failure assertions to new output format The output standardization refactor (b6bde243) strips known prefixes (e.g. 'Install app in simulator operation failed:') inside buildInstall/ Launch/StopFailure and renders the header as 'Failed to install/launch/ stop app.' with the raw diagnostic under 'Errors (N):'. Update the stale assertions in install/launch/stop_app_sim and debug_attach_sim tests to match the rendered text, keeping the diagnostic-bullet assertions intact. * fix(list_schemes): Label project paths correctly in structured output buildSchemeList{Result,ErrorResult} previously always stored the input path under artifacts.workspacePath, so a projectPath input surfaced in the rendered output as 'Workspace: /.../Foo.xcodeproj'. Union the scheme-list artifacts shape, its JSON schema, and the domain-result renderer so projects populate artifacts.projectPath and render with the 'Project' label, while workspaces continue to populate workspacePath. Adds coverage for both input kinds in list_schemes.test.ts. --------- Co-authored-by: Claude Co-authored-by: OpenAI Codex

Sunbelt Computer Software

PL/B Language Development and Support

Uh oh!