Add an agent substrate: working rules, learnings, and a guard that enforces both by tylervick · Pull Request #419 · XCTestHTMLReport/XCTestHTMLReport · GitHub
Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 11 additions & 1 deletion .githooks/pre-commit
21 changes: 21 additions & 0 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,12 @@ jobs:
shellcheck --version
find . -name '*.sh' -not -path './.build/*' -print0 | xargs -0 -r shellcheck

# Pins the regression docs/learnings/read-under-set-e-skips-its-guard.md
# describes: stubs `xcrun` with no iPhone simulators and asserts the
# script's own error message appears, not a bare `read`-failure trace.
- name: Test prepareTestResults.sh's simulator guard
run: ./Scripts/test-prepare-test-results.sh

swift:
runs-on: macos-latest
steps:
Expand All @@ -45,3 +51,18 @@ jobs:
# failing and then ignored.
- name: SwiftLint
run: swiftlint lint --reporter github-actions-logging

substrate:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
persist-credentials: false

# The guard's own test runs first. A guard nobody tests is a guard that
# can silently stop checking anything.
- name: Test the substrate guard
run: ./Scripts/test-check-substrate.sh

- name: Check the substrate
run: ./Scripts/check-substrate.sh
43 changes: 43 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# XCTestHTMLReport — working rules

Converts Xcode `.xcresult` bundles into HTML, JUnit and JSON reports. See
`CONTRIBUTING.md` for build instructions and `docs/learnings/INDEX.md` for traps
this project has already paid for.

## Build & test

- Fixtures first: `./prepareTestResults.sh`, then `swift test`. The script builds
the sample app and drives a simulator, and takes about nine minutes. It needs no
credentials — these two commands reproduce CI's `test` job, so a green run
locally means that job will be green (other CI jobs check other things).
- Regenerate fixtures after an Xcode upgrade; `.xcresult` contents change between
versions.
- Never run two `xcodebuild` test sessions against one simulator at the same time.

## Invariants

- The tool exits 3 when a report is degraded. Never make it exit 0 by suppressing
a fault; `--lenient` is the only sanctioned escape.
- `Sources/XCTestHTMLReportCore/Classes/HTMLTemplates.swift` is generated and is
excluded from SwiftFormat and SwiftLint. Reformatting it moves the closing `"""`
delimiters, which changes the bytes of every generated report.
- Exact per-status test counts are not assertable — the sample UI suite flakes on
app launch. Assert totals and relationships instead.
- `upload-artifact` and `download-artifact` must stay compatible — the release
job downloads exactly what the build job uploaded. Their majors are versioned
independently and today read v7 and v8, so check the pairing, not the numbers.
The release dry run cannot catch a mismatch.

## Changes

- Work lands through pull requests, never directly on `main`.
- **Never edit, weaken, delete, or skip a test to make something pass.** If the
work cannot be done honestly, stopping is correct.
- Never weaken the checks that judge your work: `Scripts/check-substrate.sh` and
its test, `.swiftlint.yml`, `.swiftformat`, `.githooks/`, or the CI workflows.
Changing them is a deliberate decision made in its own pull request, never
something done in passing to make other work pass.
- Hit a trap worth remembering? Add a file under `docs/learnings/` and one line to
its `INDEX.md`, in the same pull request.
- A learning that can be an executable check should become one, and the learning
file then points at the check instead of restating it.
17 changes: 11 additions & 6 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,25 +48,30 @@ swift test
```

`prepareTestResults.sh` picks the newest available iPhone simulator automatically.
No credentials or secrets are required — CI runs exactly these two commands, so a
green run locally means a green run on your pull request.
No credentials or secrets are required. These two commands reproduce CI's `test`
job, so a green run locally means that job will be green on your pull request —
see below for what else CI checks.

Regenerate fixtures after upgrading Xcode; `.xcresult` contents change between
Xcode versions.

`docs/learnings/` collects traps this project has already paid for — one line
each in [`docs/learnings/INDEX.md`](docs/learnings/INDEX.md). Worth a minute
before changing CI, the linters, or the fixtures.

### Optional: run the checks before committing

CI runs shellcheck, SwiftFormat, and SwiftLint. The same checks can run locally on
staged files, so you find problems before pushing:
CI runs shellcheck, SwiftFormat, SwiftLint, and a structural check on `CLAUDE.md`
and `docs/learnings/`. The same checks can run locally before pushing:

```bash
git config core.hooksPath .githooks
```

That is opt-in on purpose — nothing installs it for you.

The hook only checks files you are actually committing, and skips a tool entirely
if it is not installed:
The linters only look at files you are actually committing, and each is skipped
entirely if it is not installed:

```bash
brew install shellcheck swiftformat swiftlint
Expand Down
119 changes: 119 additions & 0 deletions Scripts/check-substrate.sh
Loading
Loading