Ask the repo, not sys.path, whether a stubbed name is third-party by danielhanchen · Pull Request #9133 · unslothai/unsloth · GitHub
Skip to content

Ask the repo, not sys.path, whether a stubbed name is third-party - #9133

Merged
danielhanchen merged 2 commits into
mainfrom
guard-ignores-repo-modules
Aug 18, 2026
Merged

Ask the repo, not sys.path, whether a stubbed name is third-party#9133
danielhanchen merged 2 commits into
mainfrom
guard-ignores-repo-modules

Conversation

@danielhanchen

Copy link
Copy Markdown
Member

The stub-shadowing guard added in #9095 is failing Repo tests (CPU) on main:

assert not {'test_llama_cpp_wait_for_vram_settle.py': ['loggers']}

loggers is the backend's own module. Stubbing it shadows nothing, and the guard exists for third-party libraries.

The bug is the reasoning in its own docstring

It said an in-repo name resolves only with studio/backend on sys.path, "which this test does not have and should not add". That is untrue in the job that runs it: under pytest tests/ -n 4 from the repo root, studio/backend does end up on sys.path, find_spec("loggers") resolves, and the guard fires on a stub doing nothing wrong.

It passed locally, where the path happens to differ. That is the worst shape a CI-only assertion can have, and it is why this reached main.

The fix

Ask the tree, which is the same everywhere: if studio/backend defines the name, it is repo code, and a stub for it is substitution rather than shadowing. importlib is consulted only for names the repo does not define.

Reproduced both ways under the runner's path:

PYTHONPATH=studio/backend pytest ...::test_an_isolated_file_never_shadows_an_installed_library_with_a_stub
  before: AssertionError: ... {'test_llama_cpp_wait_for_vram_settle.py': ['loggers']}
  after:  1 passed

danielhanchen and others added 2 commits August 18, 2026 01:03
The stub-shadowing guard I added is failing Repo tests (CPU) on main:

    assert not {'test_llama_cpp_wait_for_vram_settle.py': ['loggers']}

loggers is the backend's OWN module. Stubbing it shadows nothing, and the guard
is about third-party libraries.

The bug is the reasoning in its own docstring, which said an in-repo name
resolves only with studio/backend on sys.path, "which this test does not have and
should not add". That is untrue in the job that runs it: under pytest tests/ -n 4
from the repo root, studio/backend does end up on sys.path, find_spec("loggers")
resolves, and the guard fires on a stub that is doing nothing wrong. It passed
locally, where the path differs, which is the worst shape a CI-only assertion can
have and the reason it reached main.

So the question is answered from the tree, which is the same everywhere: if
studio/backend defines the name, it is repo code and a stub for it is
substitution rather than shadowing. importlib is consulted only for the rest.

Reproduced both ways under the runner's path: PYTHONPATH=studio/backend fails on
the old code with exactly the CI message and passes on the new.
@danielhanchen
danielhanchen merged commit 019a481 into main Aug 18, 2026
43 of 49 checks passed
@danielhanchen
danielhanchen deleted the guard-ignores-repo-modules branch August 18, 2026 01:43
danielhanchen added a commit that referenced this pull request Aug 20, 2026
* Studio: normalize replayed tool-call ids for external providers (#8913)

* Map foreign tool-call ids to Mistral's 9-char alphanumeric format

* Keep replayed tool-call ids unique when stripped bases collide

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Guard the loggers stub so it never shadows the real package

* Probe for the real loggers package without importing it

The loggers stub guard added an `import loggers` purely to test that the real
package resolves, which fails the source lint: verify_import_hoist.py reads a
newly added module-level import that nothing loads as a botched hoist, and the
build blocks on it.

The import is also the wrong probe. loggers/handlers.py imports structlog at
module scope and the structlog stub is not installed until further down the
file, so in the one environment these stubs exist for, no real structlog, the
probe raises on the missing transitive dep and the except branch stubs over the
real in-repo package. That is the shadowing the guard was added to prevent.

find_spec answers "is it resolvable" without executing the module, so a missing
transitive dep cannot skew the answer, and it leaves no unused binding. Same
except clause as _is_installed() in test_backend_ci_parallel_isolation.py:
find_spec raises for a missing parent and ValueError when a prior test already
put a bare ModuleType in sys.modules, where the stub is present anyway.

The stub-shadowing scan matched a literal `import name` only, so it required the
spelling the hoist lint rejects and the two rules could not both be satisfied.
It now credits a find_spec probe as well, with a test covering both spellings.

* Keep replayed tool-call ids inside Anthropic's charset

Anthropic validates tool ids and states the rule in the 400 it returns:
"tool_use.id: String should match pattern '^[a-zA-Z0-9_-]+$'". A colon is not
in that set, and two of the shapes we store carry one into a replay.

The duplicate-base fallback is the first: when two responses both start their
ids at call_0, both keep the whole stored "call_0:<uuid>" rather than the
stripped base, and at 43 chars that is under the 64-char branch, so it went out
verbatim. The frontend's confirmation-scoped "<sandbox>:<thread>:<approval>"
ids are the second, with no uuid suffix to strip. Anthropic does not originate
either, since its own tool ids are server-tool ids that _filter_tool_calls
drops; the path is an existing chat switched onto an Anthropic model, which is
the most commonly reported form of this failure in other clients.

Sanitizing alone is not enough, because "a:b" and "a_b" would both become
"a_b", and pairing the wrong result with the wrong call is a silent wrong
answer rather than an error. The sha256 tail is taken over the unsanitized
value, so the map stays injective, and the readable prefix survives for logs.
31 + 1 + 32 is 64, matching the shortening scheme alongside it.

Only ids Anthropic would already have rejected change, so a chat that works
today replays byte-identical ids. Tests cover the charset, injectivity across
the sanitize collision, the untouched legal ids, and idempotence per provider,
since a normalized id that drifted on the next turn would unpair the call from
its result.

* Drop the find_spec arm of the stub-shadowing scan, superseded on main

#9133 landed the same fix from a better angle: _is_installed now asks
_is_repo_module first, so an in-repo name like loggers is never treated as a
third-party library the stub could shadow, and the scan no longer cares which
way the file proves the package exists.

That makes the find_spec arm added here redundant, and it edits the same block
#9133 rewrote, which is the conflict against main. Reverting it leaves the
find_spec probe in test_llama_cpp_wait_for_vram_settle.py, which is still needed
on its own terms: it is what keeps the unused import out of the source lint and
what stops the missing structlog from being read as a missing loggers.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Seed the tool-call id ledger from the replayed history

Normalizing a replayed id down to its bare base gave the loop an id it can
mint again. The healer's counter is per request, so the first turn of every
request mints call_0, and a history where exactly one stored id claims that
base now replays as a bare call_0 too. Both land in one upstream body: two
tool_use blocks under one id on Anthropic, a "Duplicate tool call id"
rejection from mistral-common's request validator, or a silently mispaired
result on the older tokenizers that pair positionally.

used_call_ids is the ledger that already stops a provider from reusing an id
across turns of one run, and calls() already renames a repeat. It just never
saw the history, because it started empty. Seed it from the conversation the
run begins with.

Renaming once is then not enough on its own. The id the rename mints is
stored by the client and comes back as history on the next request, so the
request after a healed turn replays both call_0 and call_0_1_0, and a fresh
call_0 would be renamed straight onto the call_0_1_0 already in the body.
That is the same collision one request later rather than a fix for it, so
the rename counts up until the id is actually free. The first attempt is
unchanged, so an id that was already unique keeps the name it has today, and
a history that does not claim the minted id is left alone.

* Tighten the comments added by this PR

Collapse the multi-line explanations around the replay id map, the tool-call
id ledger and the loggers find_spec probe to their load-bearing reasons.
Comments, docstrings and whitespace only.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant