You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
toolchain-drift/test fixture verify gate accepts stub bundles — assert non-empty test data #454
Found during the PR #453 review. A CI runner's simulator vanished mid-prepareTestResults.sh (Unable to find a device matching the provided destination specifier), the script's || true swallowed the failure, and the resulting stub .xcresult (zero testNodes; neither backend can render it) passed the fixture verify step, which checks only Info.plist presence. The test job then failed downstream with a confusing error instead of failing fast at generation.
Fix: harden the verify step (shared by test.yml and toolchain-drift.yml via the fixture cache path) to assert each bundle actually contains test data — e.g. xcrun xcresulttool get test-results summary reporting a non-zero totalTestCount, per bundle. That converts a runner-infrastructure flake into an immediate, attributable failure and keeps a stub from ever being SAVED to the fixture cache (the near-miss scenario: a stub cached under a valid key would poison every subsequent hit until the key rotates).
Two smaller notes from the same review, fine to fold in here or fix opportunistically: scope the 'no other fixture test produces Expected Failure' doc comment in the sample app to TestResults.xcresult (RetryTests.testInUnknownState already records one in RetryResults), and consider pinning renderer scale=1 for the programmatic PNG attachment so it is truly 8×8.
Activity
Reject stub fixture bundles at the verify gate (fixes #454)
Route the last two fixture gates through the shared script (#470 review)
Watch the next Xcode from a beta drift leg that can never redden the …