{{ message }}
Repository navigation
fix(utils): fail tests for severe test hook errors - #16110
Open
carrerasdarren-cell wants to merge 2 commits into
Open
carrerasdarren-cell wants to merge 2 commits into
carrerasdarren-cell wants to merge 2 commits into
Conversation
Contributor
Comment on lines
+220
to
+221
| if (type === 'Test' && !error && !skip) { | ||
| error = getSevereError(afterResults) |
Contributor
There was a problem hiding this comment.
Comment on lines
+945
to
+955
Contributor
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Proposed changes
Fixes #14314.
A
SevereServiceErrorfrom Mocha or JasminebeforeTest/afterTestcurrently logs an error while the affected test passes. Fail that test through the existing shared test wrapper. A severebeforeTesterror prevents its body from running and still reachesafterTestwith the failure; a severeafterTesterror fails an otherwise passing test. Ordinary hook errors, existing body failures, skips, later tests, and Mocha native retries retain their behavior. The check scans all hook results, so an earlier ordinary error cannot hide a later severe error.This is limited to Test wrappers; Cucumber's separate scenario/step hooks and the global hook executor are unchanged. Document the test-level behavior and add a dedicated named mock-driver smoke suite.
How you tested
Compiled clean main
795fa22a0ade22cdb84dfbece5c57926fd981203before the baseline. Actual Mocha/Jasmine Launcher runs and JSON reports reproduced four false-green severe cases among ten baseline cases; all ten meet the intended behavior after the change. Six ordinary/no-error controls retain their results.Added durable regressions before editing product code: two fail on the original behavior; the focused file then passes all 21 tests.
pnpm run test:package wdio-utils,wdio-mocha-framework, andwdio-jasmine-framework: 725 tests pass across 33 files, with no type errors.pnpm run test:smoke severeTestHooks: 18 case assertions pass with actual Launcher/reporter events, covering ordinary and severe errors, mixed hook arrays, subsequent tests, original body/expectation failures, and successful one-shot Mocha native retries. These use the repository's mock driver, not a real browser.Review follow-up: two additional regressions fail on the initially submitted revision and the focused file now passes 25 tests. A newly severe
afterTestfailure pauses the debug agent once after that hook; local broker/CLI checks cover resume and close for both frameworks and preserve the existing body-failure pause. All six runtime cases pass with the mock driver. Jasmine retains its existing adapter-provided bundled spec identity; no identity metadata repair is claimed. The smoke suite also removes only its own temporary directory, verified on success and an intentional failure path while preserving prior artifacts.Existing named smokes pass:
mochaHooksTestrunner,jasmineHooksTestrunner,jasmineAfterHookArgsValidation,severeErrorTest, andcucumberTestrunner.pnpm run test:typings:mocha,pnpm run test:typings:jasmine, andpnpm run test:oxlintpass.pnpm run docs:generate:enandpnpm --filter=@wdio/docs run check --skip-build-checkspass. The fulldocs:checkrequires an absentwebsite/build; no production site build was run.Unit tests for the touched package(s)
Type definition tests
Smoke or e2e
Types of changes
Checklist
Backport Request
This change targets current main/v10. No v9 backport is included.
Further comments
Codex assisted with implementation, review, and the local validation listed above. Mocha native framework retries are tested; this does not claim that WDIO's body-only
repeatTestretries rerun test hooks.