fix(security): fail-closed signature check in the launchers; replace checksum CI jobs [DEVA11Y-474] by Crash0v3rrid3 · Pull Request #47 · browserstack/AccessibilityDevTools · GitHub
Skip to content

fix(security): fail-closed signature check in the launchers; replace checksum CI jobs [DEVA11Y-474] - #47

Merged
Crash0v3rrid3 merged 2 commits into
fix/DEVA11Y-473-fail-closed-checksumfrom
fix/DEVA11Y-474-launcher-signature-verification
Oct 21, 2026
Merged

Crash0v3rrid3 merged 2 commits into
fix/DEVA11Y-473-fail-closed-checksumfrom
fix/DEVA11Y-474-launcher-signature-verification

Conversation

@Crash0v3rrid3

Copy link
Copy Markdown
Collaborator

Stacked on #46. This PR targets #46's branch so its diff shows only the launcher changes. GitHub retargets it to main automatically once #46 merges.

Why

The bash/zsh/fish launchers (scripts/*/cli.sh) verified the CLI against a <asset>.sha256 checksum file that the CDN never publishes. When that file was missing or malformed they ran the binary unverified: the same fail-open behaviour Security reopened DEVA11Y-473 over. Security's condition 3 on DEVA11Y-473 (comment 2471198) requires the launchers to get the same signature checks as the plugin.

Two CI jobs in spm-smoke-test.yml encoded the old design:

  • "CLI binary integrity check (functional matrix)" asserted that a missing or malformed checksum file still lets the binary run.
  • "CLI checksum sidecar published (advisory…)" was waiting to become a hard check once checksum files shipped, which they never will.

What

Launchers. verify_binary_integrity is replaced by verify_binary_signature, identical in all three copies:

  • macOS: codesign --verify --strict with the same Developer ID requirement as fix(security): fail-closed CLI signature verification + https/browserstack.com redirect allowlist [DEVA11Y-473] #46, pinned to Team YQ5FZQ855D. It runs on the extracted binary before chmod/publish.
    • The launchers re-extract on every run (even on a 304), so this covers cached downloads too.
    • On failure the launcher refuses to run and deletes the cached zip, so the next run downloads a fresh copy instead of re-extracting the rejected one.
  • Linux: the build is unsigned, so the launcher proceeds (the risk Security accepted on DEVA11Y-473).
  • Any other OS: refused.
  • The launchers' self-update .sha256 files are regenerated, so verify-selfupdate-checksums stays green.

CI. Both jobs are replaced with tests/launcher-signature/run_tests.sh (runnable locally), on macos-14:

  • Launcher signature verification (functional matrix): lifts verify_binary_signature verbatim from each launcher. It must accept the real CLI and refuse flipped-byte, ad-hoc-signed, unsigned and missing binaries; Linux must pass through and an unknown OS must be refused.
  • Published CLI signed by BrowserStack (Team YQ5FZQ855D): the latest macOS arm64 and x64 releases must satisfy the Team-pinned requirement. If a release is ever signed differently, this check goes red before every user's launcher starts refusing.

Neither job is a required check (main requires semgrep/ci and Claude Code Review), so the renames don't block merging.

README: the sentence #46 adds ("launcher scripts do not yet verify…") now describes the launcher behaviour.

Testing (macOS arm64, real network)

Test Result
run_tests.sh matrix ✅ 21/21 (7 cases × bash/zsh/fish)
run_tests.sh published ✅ arm64 and x64 latest pass
Requirement pinned to a wrong Team ID, genuine binary ❌ rejected (code failed to satisfy specified code requirement(s)), so the Team pin does the work
Launcher end-to-end, empty cache ✅ downloads, verifies, publishes, runs the CLI
Launcher end-to-end, cached zip replaced by an ad-hoc-signed binary (server answers 304) ✅ exit 2 BrowserStack CLI signature verification failed; zip deleted; previously good binary untouched (same SHA-256)
Next run after the refusal ✅ re-downloads, verifies and runs
bash -n on all launchers; self-update checksum files ✅

The end-to-end runs used a copy of the launcher pointed at a scratch cache directory.

🤖 Generated with Claude Code

…checksum CI jobs [DEVA11Y-474]

The bash/zsh/fish launchers verified the CLI against a <asset>.sha256 sidecar
that the CDN never publishes, and proceeded unverified when it was missing or
malformed: the same bypass Security reopened DEVA11Y-473 over.

Replace verify_binary_integrity with verify_binary_signature, mirroring the
SwiftPM plugin (#46): on macOS, `codesign --verify --strict` against a
Developer ID requirement pinned to Team YQ5FZQ855D, run on the extracted
binary before chmod/publish. Failure refuses to run and drops the cached zip
so the next run re-downloads instead of re-extracting the same payload after
a 304. Linux is unsigned and passes through (accepted risk, DEVA11Y-473);
any other OS is refused.

CI: replace the two checksum jobs, which asserted the fail-open behaviour and
waited on sidecars that will never ship, with
tests/launcher-signature/run_tests.sh:
- matrix: each launcher's function accepts the real CLI and refuses
  flipped-byte, ad-hoc-signed, unsigned and missing binaries;
- published: the latest macOS arm64/x64 releases satisfy the Team-pinned
  requirement, so a re-signed release turns CI red first.

Self-update .sha256 files regenerated for the edited launchers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Crash0v3rrid3
Crash0v3rrid3 requested a review from a team as a code owner October 21, 2026 10:27
@coderabbitai

coderabbitai Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

…ate case [DEVA11Y-474]

The suite lifted verify_binary_integrity by name, which no longer exists, and
drove download_binary with OS=macos and unsigned fixtures that the new
signature check refuses before the guard's outcome is observable.

Run the guard cases with OS=linux, where the real verify_binary_signature
passes through by design, so they still execute only real code. Add case 9
(macOS): the real codesign check must refuse an unsigned payload with exit 2,
remove the staged binary and the zip, and leave an existing cached binary
intact. 75/75 locally.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Crash0v3rrid3
Crash0v3rrid3 merged commit 445fa50 into fix/DEVA11Y-473-fail-closed-checksum Oct 21, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant