fix: Ignore payload-less attachments during validation by mvanhorn · Pull Request #432 · XCTestHTMLReport/XCTestHTMLReport · GitHub
Skip to content

fix: Ignore payload-less attachments during validation - #432

Closed
mvanhorn wants to merge 1 commit into
XCTestHTMLReport:mainfrom
mvanhorn:fix/387-ignore-payloadless-attachments
Closed

mvanhorn wants to merge 1 commit into
XCTestHTMLReport:mainfrom
mvanhorn:fix/387-ignore-payloadless-attachments

Conversation

@mvanhorn

@mvanhorn mvanhorn commented Aug 11, 2026 •

Copy link
Copy Markdown

Summary by CodeRabbit

Tighten the post-condition in Summary.validate() so .none is fault-worthy only when the attachment also has a payload identifier, preserving the existing idempotent fault deduplication and genuine unresolved-payload behavior. Add a passing sample UI test that records an attachment with XCTest's payload-removing lifetime so prepareTestResults.sh produces a real attachment entry without a payloadRef; keep fixture generation and production parsing unchanged. Strengthen FaultReportingTests by first proving the generated fixture contains that payload-less attachment, then asserting validation does not report it as unresolved, so the regression test cannot pass vacuously.

Verification

  • Generate TestResults.xcresult from the sample project and verify the fixture contains at least one attachment whose payloadId is absent and whose rendered content is .none.
  • Validate that fixture and verify the payload-less attachment produces no unresolvedAttachment fault; running the CLI against the same fixture must continue to exit 0 without a degradation warning.
  • Verify attachments with a non-nil payload reference and successfully exported content remain fault-free in both linking and downsized fixture paths.
  • Preserve the existing failure rule and idempotency: an attachment that has a payload reference but resolves to .none is still reported as unresolved, and repeated validation does not duplicate that fault or disturb previously collected faults.

Why

Summary.validate() currently records an unresolvedAttachment fault whenever an attachment's rendering content is .none. Attachment also uses .none for attachments that never had a payloadRef, so a valid payload-less attachment is misclassified as failed resolution and can make the CLI exit 3. The model already retains Attachment.payloadId, which distinguishes an attempted payload resolution from the absence of a payload without adding a new state or test-only abstraction. The current generated fixtures contain no payload-less attachment, leaving this false-positive path untested.

Closes #387

Summary by CodeRabbit

  • Bug Fixes

    • Improved attachment validation so attachments without payload references are ignored rather than incorrectly reported as unresolved.
    • Clarified diagnostics for attachments whose payloads cannot be resolved.
  • Tests

    • Added coverage for attachments configured to be deleted after successful test completion.
    • Updated test result expectations and validation checks to reflect the expanded sample test coverage.

@coderabbitai

coderabbitai Bot commented Aug 11, 2026 •

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
Tests/XCTestHTMLReportTests/FaultReportingTests.swift (1)

53-72: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Add coverage for payload-bearing unresolved attachments.

This test verifies only the payload-less case. It does not verify that a non-nil payloadId with .none content still creates an .unresolvedAttachment fault. The test would pass if Summary.validate() skipped all .none attachments. Add or retain a fixture case for the required positive behavior.

This matches the PR objective to continue reporting attachments with payload references that resolve to .none.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@Tests/XCTestHTMLReportTests/FaultReportingTests.swift` around lines 53 - 72,
Extend the test around summary.validate() to include an attachment with a
non-nil payloadId and .none content, then assert that validation reports it as
an .unresolvedAttachment fault. Preserve the existing assertion that
payload-less attachments are not reported, and ensure the fixture covers both
behaviors.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@Tests/XCTestHTMLReportTests/FaultReportingTests.swift`:
- Around line 53-72: Extend the test around summary.validate() to include an
attachment with a non-nil payloadId and .none content, then assert that
validation reports it as an .unresolvedAttachment fault. Preserve the existing
assertion that payload-less attachments are not reported, and ensure the fixture
covers both behaviors.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: a7afa8f5-29c5-4a69-9297-72476560133a

📥 Commits

Reviewing files that changed from the base of the PR and between f717a1d and eb5fd0a.

📒 Files selected for processing (4)
  • Sources/XCTestHTMLReportCore/Classes/Models/Summary.swift
  • Tests/XCTestHTMLReportTests/CoreTests.swift
  • Tests/XCTestHTMLReportTests/FaultReportingTests.swift
  • XCTestHTMLReportSampleApp/SampleAppUITests/FirstSuite.swift

@tylervick

Copy link
Copy Markdown
Member

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

validate() false-positives on attachments with no payloadRef

2 participants