Keep the cli tests out of the home directory, and give the fence a release path by bogatyrjov1 · Pull Request #183 · backthread/backthread · GitHub
Skip to content

Keep the cli tests out of the home directory, and give the fence a release path - #183

Merged
bogatyrjov1 merged 1 commit into
mainfrom
arp-1764-1758-oss-hygiene
Sep 13, 2026
Merged

bogatyrjov1 merged 1 commit into
mainfrom
arp-1764-1758-oss-hygiene

Conversation

@bogatyrjov1

Copy link
Copy Markdown
Contributor

What

1. The cli test suite no longer writes into the home directory — and CI now checks that it doesn't.

Measured on main by running every test file under a fresh $HOME: two tests created ~/.backthread/first-run.json — capture.test.ts ("runCapture never throws even when a dep throws synchronously") and captureCommand.test.ts ("end-to-end: manual capture drives the real runCapture with all seams mocked"). Both build their deps by hand with a bare env: {}, so the real trust gate ran and persisted its flag into the developer's own state dir. backfill.test.ts and sweep.test.ts were already isolated (#178) and are clean.

Worse, the capture.test.ts case lands on the no-auth path with the real ensureAuth still wired: on any home without a token — CI, every run — it opened a login session against the production app and tried to launch a browser (the run printed a live cli-auth?session=… URL). On a developer machine the existing token short-circuited it, which is why nobody saw it.

Both tests now point BACKTHREAD_CONFIG_DIR at a temp dir and the first also stubs ensureAuthImpl.

scripts/check-home-isolation.mjs re-runs the cli suite under a fresh empty home with BACKTHREAD_CONFIG_DIR cleared, then fails if .backthread/ appeared under it or the real ~/.backthread changed (entries, sizes, mtimes). --self-test runs a probe that does write ~/.backthread/probe.json through the same harness and requires the check to report it, and a probe that writes nothing and requires a pass. Its own CI job (home-isolation) — the test-matrix guard rightly refuses an if: step inside test.

Mutation-proven locally: reverting either test fix turns the guard red naming .backthread/first-run.json; with both fixes it is green and the self-test passes.

2. @backthread/redact gets a release path, and the CLI release refuses to ship a fence npm doesn't serve.

packages/redact/package.json says 0.1.6; npm serves 0.1.2 (npm view @backthread/redact versions). The CLI bundle inlines the local version and stamps it on every request as x-backthread-redact-version — so the published CLI has been telling every server "the fence that redacted this was redact@0.1.6", naming four rounds of hardening (#174, #175, #178, #179, #180) that nobody outside this repo can read. The package is public precisely so that claim is auditable.

  • .github/workflows/release-redact.yml — on a redact-v* tag, same OIDC trusted-publishing shape as the extractor. Needs a one-time Trusted Publisher on npm (documented in RELEASING.md).
  • packages/redact/CHANGELOG.md — 0.1.0 → 0.1.6 reconstructed from the PR history; ships in the tarball (files).
  • scripts/check-published-version.mjs <workspace> — asks npm for the served versions and fails closed when the local version is not among them (or npm cannot answer). Wired into release.yml before npm publish, with a --self-test that re-runs the guard against a stub npm on PATH: a list without the version must be red, a list with it must be green, an unparseable answer must be red.
  • Run live against this checkout it fails today, naming 0.1.6 vs 0.1.2 — the real defect, which is the point.

Why

  • Decision: publish 0.1.6 rather than move the header back to 0.1.2. The header is a description of the code the bundle actually runs; lowering it would make the CLI claim to redact with a fence it no longer contains. The only honest fix is to make the code readable at the number it names.
  • Decision: guard the CLI release, not PR CI. A redact bump PR is unpublished by definition until its tag is pushed; a PR-time check would be red on every legitimate bump. The invariant that matters is "no CLI ships stamping an unserved fence", and that is a release-time property.
  • Decision: the home-isolation guard judges only .backthread/. doctor.test.ts and npm.test.ts spawn the real npm binary, which writes ~/.npm under whatever home it is given; that is a network seam, reported as a note, not a fail — so a red here always means the cli itself wrote state.
  • Trade-off: the guard runs the cli suite a second time in CI (~1 min). A single-run version would have to instrument the suite itself; the second run keeps the guard independent of what the tests choose to stub.
  • Limitation: the real-dir fingerprint check can trip on a developer machine where a live Backthread hook touches ~/.backthread mid-run; it is load-bearing in CI, where no such directory exists.

Other unisolated seams found (enumerated, not fixed)

  • doctor.test.ts — two tests reach the real runNpm(['view','backthread','version']) (registry network call + ~/.npm cache/log writes).
  • npm.test.ts — spawns the real npm with a bogus subcommand (no network, but ~/.npm/_logs is written).
  • capture.test.ts (fixed here, but the shape recurs): any hand-built CaptureDeps that omits ensureAuthImpl reaches the real login on a token-less home. Only deps()-built cases are safe by construction.
  • Tests read the real ~/.backthread/config.json whenever env lacks BACKTHREAD_CONFIG_DIR and readConfigImpl is not injected — which is what masked the login on a developer machine.

Gate

npm run typecheck clean; npm test — backthread 954 / extractor 2201 / ci 363 passing, floors held; committed bundle in sync; check-no-internal-refs, check-test-matrix, check-provenance all OK; check-home-isolation + self-test OK; check-published-version --self-test OK.

🤖 Generated with Claude Code

https://claude.ai/code/session_01PeY7VEztPfBPEWeXwkLhdP

…lease path

Two cli tests built their deps by hand with a bare `env: {}`, so the real trust
gate ran and persisted first-run.json into the developer's own ~/.backthread on
every `npm test`. One of them also landed on the no-auth path with the real
ensureAuth still wired, which on any home without a token — CI — opened a login
session against the production app. Both now point the config dir at a temp
dir and stub the login. A new guard re-runs the cli suite under a fresh empty
home, fails if `.backthread/` appears there or the real one changes, and proves
on a probe that does write that it would have said so; it has its own CI job.

@backthread/redact sat at 0.1.6 in package.json while npm served 0.1.2: four
rounds of fence hardening shipped inlined in the CLI bundle, stamped on every
request as `x-backthread-redact-version`, with no tarball anyone could read.
The fence now has a release workflow on `redact-v*` tags (same OIDC shape as
the extractor), a changelog that ships in the tarball, and the CLI release
refuses to publish while the redact version it stamps is not on npm — a guard
that is self-tested against a stub registry in the same run.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PeY7VEztPfBPEWeXwkLhdP
@bogatyrjov1
bogatyrjov1 merged commit f613e70 into main Sep 13, 2026
7 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants