{{ message }}
Stub the two helpers the sliced harnesses started importing - #9342
Merged
Conversation
Backend CI's "Repo tests (CPU)" has been red on main since 588405d, with 111 failures in two files. Both are the same defect, and neither is about what the tests claim to be testing. These harnesses replay real studio source with its import block stripped, so every name the sliced region calls has to be defined in the harness. Two landed without one: refresh-context-usage.ts findLatestUserVideoBase64 (#9056, decline to price a prompt carrying video) chat-runtime-store.ts resolvePreserveThinkingOnLoad Both product changes are right. The first is the more dangerous shape: with no stub the replayed body throws a ReferenceError, the effect bails, `counts` stays 0, and the assertion reads "the empty New Chat view must be priced exactly once" -- a pricing bug that does not exist. 41 tests failed that way, and bisecting main was the only way to see it, because the message points at the wrong thing. test_chat_autoload_failure_gate.py already guards this and said exactly what was missing, which is why its 70 failures named resolvePreserveThinkingOnLoad outright. test_new_chat_context_recount.py had no such guard; it does now, built the same way -- parse the real import list, assert the harness defines each name. Mutation-tested: dropping the video stub turns it red with the name in the message. The preserve-thinking stub is the real resolver's rule verbatim, since no scenario here sets a stored preference. tests/studio: 0 failures, from 111.
With the harness stubs in place, Repo tests (CPU) is down to one failure, and it is the last of the same kind: correct product code reported as a defect by a scan that reads the file too loosely. test_no_torch_backend_auto_outside_fallback allows --torch-backend=auto only inside install.sh's "GPU detection failed" branch, and located the end of that branch as the first line equal to `fi` after the comment. #8670 put a `case` into the branch to choose the desktop install spec, with an `if`/`fi` inside one of its arms. That `fi` is now the first one, so the block ended four lines early and the branch's own install call -- line 5280, the one the test exists to permit -- was reported as a primary path using the flag. The end is now found by counting nesting. `case` counts too: it closes with `esac`, so an `if`/`fi` inside a case arm would still throw off a counter that only tracked `if`. Added test_the_fallback_range_reaches_the_end_of_the_branch, because the bug here was in the range and not in what the range was used for, so nothing was checking the range itself. It pins both ends: the block contains the branch's first `if` and both of its --torch-backend=auto calls, and does not run past its closing `fi`. Mutation-tested both ways. Adding --torch-backend=auto outside the branch still fails the original assertion; reverting the nesting count to the first-`fi` behaviour fails the new one. tests/python/test_cross_platform_parity.py 78 passed.
Member
Author
|
Second commit. With the harness stubs in place
Mutation-tested in both directions. |
for more information, see https://pre-commit.ci
Member
Author
danielhanchen
added a commit
that referenced
this pull request
Aug 20, 2026
The branch was 60 commits behind main, which is the only reason the stack was red: main has since fixed the four failure families this stack's CI was inheriting (#9342 for the sliced chat-adapter harness and the install.sh fallback range, #9348 for the two backend guards on the 3.13 leg, #9335 for the 3.9 union floor).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backend CI's Repo tests (CPU) has been red on main since
588405dce, with 111 failures across two files. Both are the same defect, and neither is about what the tests claim to be testing.Bisected:
f3b042534is the last green (124 passed),588405dcethe first red (41 failed).The defect
These harnesses replay real studio source with its import block stripped, so every name the sliced region calls has to be defined in the harness. Two landed without one:
refresh-context-usage.tsfindLatestUserVideoBase64chat-runtime-store.tsresolvePreserveThinkingOnLoadBoth product changes are right. The first is the more dangerous shape: with no stub the replayed body throws a
ReferenceError, the effect bails,countsstays 0, and the assertion readswhich is a pricing bug that does not exist. 41 tests failed that way. Bisecting main was the only way to find it, because the failure message points somewhere else entirely.
Why one file diagnosed itself and the other did not
test_chat_autoload_failure_gate.pyalready guards this, and said exactly what was missing:That is why its 70 failures named the cause outright while the other 41 did not.
test_new_chat_context_recount.pyhad no such guard; it does now, built the same way - parse the real import list, assert the harness defines each name. Mutation-tested: dropping the video stub turns it red namingfindLatestUserVideoBase64.The preserve-thinking stub is the real resolver's rule verbatim (
Boolean(supports_preserve_thinking && preserve_thinking_default)), since no scenario in that file sets a stored preference.Result
tests/studio: 0 failures, from 111.