fix(@wdio/cucumber-framework): report the scenarios after a failFast stop in parallel mode as Cucumber does by plum117 · Pull Request #16162 · webdriverio/webdriverio · GitHub
Skip to content

fix(@wdio/cucumber-framework): report the scenarios after a failFast stop in parallel mode as Cucumber does - #16162

Closed
plum117 wants to merge 8 commits into
webdriverio:mainfrom
plum117:fix/cucumber-parallel-failfast-skip
Closed

plum117 wants to merge 8 commits into
webdriverio:mainfrom
plum117:fix/cucumber-parallel-failfast-skip

Conversation

@plum117

@plum117 plum117 commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

Stacked on #16161. Only the last commit (7b584f013) is this PR; the others are #16161.

Summary

With cucumberOpts.parallelMode and failFast, the scenarios that did not run after the stop were reported as plain skipped steps. Cucumber runs them in skip mode, so a sequential run reports more:

Unrun scenario after a failFast stop Parallel before Now (same as sequential)
Before / After hooks no events hook:start / hook:end with state SKIPPED
Undefined step skipped failed, Step "…" is not defined. …
Ambiguous step skipped, no reason skipped with the matching step definitions as pendingReason
Window none none (skip mode runs no code)
 runParallelCucumber, after the slot pool
   if failFast stopped the run
-    emitSkippedScenario(pickle)           # plain skipped steps
+    runScenario(pickle, no allocation)    # TestCaseRunner with skip: true, as Cucumber

Evidence

Real browser (headless Chrome, BiDi), failFast: true, 2 slots, the same specs run sequential and with parallelMode: 'contexts':

Before After
spec 1 failing, 17 skipped (sequential: 2 failing, 16 skipped) same as sequential
dot, concise, JUnit, JSON different same as sequential
Framework events of the 8 scenarios 6 different 8 the same

Also the same as sequential with scenarioLevelReporter: true, and no change for the base, dryRun cases.

  • Unit: 202 tests in @wdio/cucumber-framework; new: reports their hooks as skipped and does not run them, keeps an undefined step failed, as Cucumber does, keeps the reason of an ambiguous step (each failed before the fix). The 2 older failFast tests now use a defined failing step, because skip mode keeps an undefined step failed.
  • Smoke: the 16 Cucumber suites pass.

Merge Danger

Door: two-way

Blast Radius: Cucumber parallel mode with failFast only. The run result does not change (the run already failed).

plum117 and others added 8 commits October 20, 2026 20:09
…al and parallel runs

Move the status rules of the formatter's afterTest (undefined, failed,
ambiguous, the pending reason and the undefined step suffix) and the
feature stack lines into getStepResult() and getFeatureStack() in
utils.ts, so the parallel step path can use the same rules.

No change in the sequential output: the formatter tests and their
snapshots pass without edits.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… steps in parallel mode as in sequential mode

With parallelMode, the step path only converted the Cucumber status:

- an undefined step was reported as passed, with no error
- an ambiguous step was skipped with no reason, and
  failAmbiguousDefinitions was ignored for steps
- a failed step had the whole Cucumber message as error message, so
  reporters showed `Error: Error: ...` and internal stack lines

The parallel step path now uses getStepResult(), as the sequential
formatter does, and lists the matching step definitions of an ambiguous
step from Cucumber's support code messages.

Real-browser check (Chrome, sequential vs parallel run of the same
specs): the spec, dot, concise, JUnit and JSON reporters now give the
same step results, also with ignoreUndefinedDefinitions and
failAmbiguousDefinitions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Parallel mode ran the scenarios in file order and ignored
cucumberOpts.order. It now orders them with Cucumber's own
orderPickles, as Cucumber does in a sequential run: `random:<seed>`
gives the same order as a sequential run, `random` logs its seed, and
an unknown order fails before any window opens.

Real-browser check (Chrome, 2 feature files, order random:7, 1 slot):
sequential and parallel runs start the scenarios in the same order.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…itle in parallel mode

In parallel mode a Before/After hook had an empty title. Sequential
mode titles it with the hook name and tag expression, for example
`@beforeFails`, which the spec reporter shows for a failing hook and
the JSON reporter writes as the hook title.

The hook titles now come from the same support code messages as the
step definitions.

Real-browser check (Chrome, sequential vs parallel run of the same
specs): the spec, dot, concise, JUnit and JSON reporters give the same
result.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…os in parallel mode as in sequential mode

With scenarioLevelReporter, the parallel scenario result kept its own
rules and collected an ambiguous step result before its message was
added:

- with failAmbiguousDefinitions, an ambiguous scenario failed with no
  error, and when skipped it had no pendingReason
- with ignoreUndefinedDefinitions, the scenario title had no
  ` (undefined step)` suffix
- the undefined scenario message had quotes and no scenario location

The scenario result now uses getStepResult(), as the sequential
formatter does, with the ambiguous message added first.

Real-browser check (Chrome, scenarioLevelReporter, also with
failAmbiguousDefinitions and ignoreUndefinedDefinitions): the spec,
dot, concise and JSON reporters give the same result in sequential and
parallel runs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rror that shows it

getStepResult() took the feature stack as a string, so every step
result scanned the feature children, also a passing step. It now takes
a function and calls it only for the undefined and ambiguous failures,
as the formatter did before the shared rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n parallel mode

With tagsInTitle, a parallel feature suite had no tags in its title
(`Statuses in parallel` instead of `@feat: Statuses in parallel`), and
its start had no description and keyword. Reporters such as JUnit and
spec then named the feature differently than in a sequential run.

Real-browser check (Chrome, tagsInTitle, step and scenario level): the
JUnit testsuite names and the spec failure titles are the same as in a
sequential run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…stop as Cucumber does

After a failFast stop, parallel mode reported each scenario that did
not run as plain skipped steps. Cucumber runs these scenarios in skip
mode: their hooks are reported as skipped, an undefined step stays
failed, and an ambiguous step keeps its reason.

They now run through Cucumber's test case runner with `skip: true`,
with no window, so the reporters get the same events as in a
sequential run. The failFast tests use a defined failing step, because
in skip mode an undefined step stays failed.

Real-browser check (Chrome, failFast, step and scenario level, 2
slots): the spec, dot, concise, JUnit and JSON reporters and the
framework events are the same as in a sequential run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@plum117 plum117 added the PR: Bug Fix 🐛 PRs that contain bug fixes label Oct 21, 2026
@pkg-pr-new

pkg-pr-new Bot commented Oct 21, 2026

Copy link
Copy Markdown

Open in StackBlitz

create-wdio

npm i https://pkg.pr.new/webdriverio/webdriverio/create-wdio@16162

eslint-plugin-wdio

npm i https://pkg.pr.new/webdriverio/webdriverio/eslint-plugin-wdio@16162

wdio

npm i https://pkg.pr.new/webdriverio/webdriverio/wdio@16162

@wdio/ai-service

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/ai-service@16162

@wdio/allure-reporter

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/allure-reporter@16162

@wdio/appium-service

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/appium-service@16162

@wdio/browser-runner

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/browser-runner@16162

@wdio/cli

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/cli@16162

@wdio/concise-reporter

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/concise-reporter@16162

@wdio/config

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/config@16162

@wdio/cucumber-framework

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/cucumber-framework@16162

@wdio/cucumberjs-json-reporter

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/cucumberjs-json-reporter@16162

@wdio/display-server

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/display-server@16162

@wdio/dot-reporter

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/dot-reporter@16162

@wdio/firefox-profile-service

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/firefox-profile-service@16162

@wdio/globals

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/globals@16162

@wdio/jasmine-framework

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/jasmine-framework@16162

@wdio/json-reporter

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/json-reporter@16162

@wdio/junit-reporter

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/junit-reporter@16162

@wdio/lighthouse-service

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/lighthouse-service@16162

@wdio/local-runner

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/local-runner@16162

@wdio/logger

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/logger@16162

@wdio/mocha-framework

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/mocha-framework@16162

@wdio/protocols

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/protocols@16162

@wdio/repl

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/repl@16162

@wdio/reporter

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/reporter@16162

@wdio/runner

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/runner@16162

@wdio/sauce-service

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/sauce-service@16162

@wdio/session

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/session@16162

@wdio/shared-store-service

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/shared-store-service@16162

@wdio/smoke-test-cjs-service

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/smoke-test-cjs-service@16162

@wdio/smoke-test-reporter

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/smoke-test-reporter@16162

@wdio/smoke-test-service

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/smoke-test-service@16162

@wdio/snapshot

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/snapshot@16162

@wdio/spec-reporter

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/spec-reporter@16162

@wdio/static-server-service

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/static-server-service@16162

@wdio/sumologic-reporter

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/sumologic-reporter@16162

@wdio/testingbot-service

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/testingbot-service@16162

@wdio/types

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/types@16162

@wdio/utils

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/utils@16162

@wdio/webdriver-mock-service

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/webdriver-mock-service@16162

webdriver

npm i https://pkg.pr.new/webdriverio/webdriverio/webdriver@16162

webdriverio

npm i https://pkg.pr.new/webdriverio/webdriverio@16162

commit: 7b584f0

@plum117

plum117 commented Oct 21, 2026

Copy link
Copy Markdown
Contributor Author

@plum117 plum117 closed this Oct 21, 2026
@plum117
plum117 deleted the fix/cucumber-parallel-failfast-skip branch October 21, 2026 11:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

PR: Bug Fix 🐛 PRs that contain bug fixes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant