One interpreter leg on a pull request, and a floor lint that reads more than syntax by danielhanchen · Pull Request #9100 · unslothai/unsloth · GitHub
Skip to content

One interpreter leg on a pull request, and a floor lint that reads more than syntax - #9100

Merged
danielhanchen merged 14 commits into
mainfrom
single-leg-matrix-plus-vermin
Aug 18, 2026
Merged

One interpreter leg on a pull request, and a floor lint that reads more than syntax#9100
danielhanchen merged 14 commits into
mainfrom
single-leg-matrix-plus-vermin

Conversation

@danielhanchen

Copy link
Copy Markdown
Member

A pull request ran Backend CI on 3.10 and 3.13. It now runs 3.13 only. Main still runs all four, so anything that needs a real run is caught at merge rather than never.

Backend CI is 26% of all CI minutes, and the org queue is currently 688 jobs deep against ~28 running. This halves the interpreter cost of every pull request.

Paying for the leg that goes

The floor leg was worth something, and I did not want to drop it and write ast.parse in the gap, because that answers the wrong question. What a dropped floor leg actually stops catching is reaching for a stdlib name that does not exist yet. core/research_runs.py already uses anext, which is 3.10. That parses perfectly on 3.9 and fails only when the line runs, so the existing feature_version check would never have seen it.

scripts/lint_backend_python_floor.py asks vermin, which reads syntax and stdlib API availability:

$ python3 scripts/lint_backend_python_floor.py
[floor] backend source must run on Python 3.10 (oldest leg in the matrix)
  'itertools.batched' member requires !2, 3.12
::error title=Backend needs a newer Python than the matrix floor::...

It takes its target from the workflow's own matrix rather than a number written in the script, so raising the floor in one place moves both. It runs from workflow-trigger-lint.yml, which carries no paths filter, so it sees the pull requests that touch only backend source, which are the ones that most need it now. Seconds, not minutes.

The single leg has to be the newest

Removals and deprecations land on the newest interpreter first and on the oldest never, so running only the oldest would be the wrong single choice. The guard asserts which end it is, rather than only that there is one.

What is genuinely given up

Stated plainly, and kept as a test rather than deleted with the old guard. The backend has sys.version_info branches at the 3.10, 3.11 and 3.12 boundaries, and a pull request no longer takes both sides of any of them. Nothing static covers that — a parse reads both sides and runs neither. test_the_boundaries_the_subset_stops_executing_are_still_run_on_main lists exactly which boundaries are affected and fails if Backend CI ever stops running on push to main, at which point this stops being a trade and becomes a straight loss.

Mutation tested

mutation result
single leg made the floor rather than the ceiling fails
lint invocation removed from the trigger-lint job fails
vermin removed from the install line fails
Backend CI taken off push-to-main fails 2
itertools.batched added to backend source lint exits 1

The vermin check needed a second pass, which is worth recording: the first version looked for the string anywhere in the workflow and was satisfied by a comment that mentioned it.

…re than syntax

A pull request ran 3.10 and 3.13. It now runs 3.13 only. Main still runs all four, so
anything that needs a real run is caught at merge rather than never.

The leg that goes is worth something, so this pays for it rather than dropping it. What a
dropped floor leg actually stops catching is not syntax: it is reaching for a stdlib name
that does not exist yet. core/research_runs.py already uses anext, which is 3.10, and that
parses perfectly on every version and fails only when the line runs, so the existing
ast.parse floor check would not have seen it. scripts/lint_backend_python_floor.py asks
vermin instead, which reads syntax AND stdlib API availability, and takes its target from
the workflow's own matrix rather than a number written in the script. Adding a call to
itertools.batched, which is 3.12, fails it in seconds.

It runs from workflow-trigger-lint.yml, which carries no paths filter, so it sees the pull
requests that touch only backend source -- the ones that most need it now.

The single leg has to be the NEWEST. Removals and deprecations land on the newest
interpreter first and on the oldest never, so running only the oldest would be the wrong
single choice; the guard asserts which end it is.

What is genuinely given up, kept visible rather than deleted along with the old guard: the
backend has version_info branches at 3.10, 3.11 and 3.12 boundaries, and a pull request no
longer takes both sides of any of them. Nothing static covers that -- a parse reads both
sides and runs neither. test_the_boundaries_the_subset_stops_executing_are_still_run_on_main
lists them and fails if main ever stops running the full matrix, at which point this stops
being a trade and becomes a straight loss.

Mutation tested: making the single leg the floor fails one test, dropping the lint
invocation fails another, removing vermin from the install fails it too, and taking Backend
CI off push-to-main fails two. That vermin check needed a second pass: the first version
looked for the string anywhere in the workflow and was satisfied by a comment mentioning
it.
@danielhanchen

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b3102a5351

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread scripts/lint_backend_python_floor.py Outdated

# Where the backend's own source lives. Tests are excluded deliberately: they are allowed
# to use whatever the interpreter running them provides, and they are not shipped.
SOURCES = ("core", "utils", "routes", "loggers.py", "main.py")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Scan all shipped backend modules

This allowlist omits 115 shipped Python files under auth, hub, models, plugins, picker, state, and storage, along with top-level modules such as _platform_compat.py; it also names loggers.py even though loggers is a directory. These are active code paths—main.py directly imports _platform_compat, hub, picker, and auth, while routes import the other omitted packages—so a PR can use a Python 3.12+ stdlib API in any of them and pass this new floor check. With the 3.10 PR leg removed, that incompatibility is then discovered only after merge. Build the targets by scanning the backend tree and excluding only tests/vendor/cache directories instead of allowlisting selected modules.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, and this was the worst possible shape for that lint: it looked like coverage while missing most of the tree, at exactly the moment it became the only thing checking the floor before a merge. Counted it -- 116 files, not far off your 115 -- and loggers.py matching nothing because loggers is a directory is the detail that shows the list was never verified against the tree.

Fixed in 2e210c1. It scans studio/backend and excludes only tests and vendored code, taking it from 307 files to 422. Verified rather than asserted: an itertools.batched call, which is 3.12, put into each of hub, auth, picker, state, storage, models, plugins and _platform_compat.py in turn. All eight are caught now; none of them was before.

Widening it immediately found something real, which I think is the strongest argument for your point: locale.getencoding is 3.11 and the floor is 3.10. It turns out to be correctly guarded, in a try/except AttributeError whose fallback is locale.getpreferredencoding(False), commented Python < 3.11. vermin reads names rather than control flow, so it cannot tell a guarded attribute lookup from an unguarded one. That single file is exempt with its reason printed on every run, and an exemption naming a path that no longer exists fails the lint, so it cannot outlive the guard it was written for.

The new guard test counts what the lint would hand to vermin against what is on disk, so narrowing the input back to a package list fails rather than quietly shrinking coverage. Reverting it to core only fails that test.

pre-commit-ci Bot and others added 3 commits August 17, 2026 12:51
The floor lint named core, utils and routes, and silently missed 116 shipped files: all
of hub, plugins, models, storage, auth, picker and state, plus _platform_compat.py, which
main.py imports directly. It also named "loggers.py", which is a directory, so that entry
matched nothing at all. With the 3.10 leg dropped this lint is the only thing looking at
the floor before a merge, and a check that covers most of a tree reads exactly like one
that covers all of it.

It now scans studio/backend and excludes only tests and vendored code, which takes it from
307 files to 422. Verified by putting an itertools.batched call, which is 3.12, into each
of hub, auth, picker, state, storage, models, plugins and _platform_compat.py in turn:
every one is caught now, and none of them was before.

Widening it immediately found something real, which is the point: locale.getencoding is
3.11 and the floor is 3.10. It turns out to be correctly guarded, in a try/except
AttributeError whose fallback is locale.getpreferredencoding(False), commented "Python <
3.11". vermin reads names rather than control flow, so a guarded attribute lookup is
indistinguishable from an unguarded one. That file is exempt with its reason printed on
every run, and an exemption naming a path that no longer exists fails the lint, so it
cannot outlive the guard it was written for.

The guard test counts what the lint would hand to vermin against what is on disk, so
narrowing the input back to a package list fails rather than quietly shrinking coverage.
@danielhanchen

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 55fe666d1f

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +80 to +81
- name: Backend still runs on the oldest interpreter the matrix claims
run: python3 scripts/lint_backend_python_floor.py

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Run the test that protects this lint step

If a PR edits only workflow-trigger-lint.yml and removes this step, the floor check disappears without failing CI: Backend CI's path filter includes only its own workflow file, and this workflow runs three other guard tests but not test_backend_ci_matrix.py::test_the_floor_is_linted_on_every_pull_request. Therefore the newly added assertion that this invocation remains present is never collected for the exact workflow-only change it is meant to reject; invoke that guard from this unfiltered workflow as is already done for the adjacent workflow guards.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, and this is the third time this repo's paths filters have hidden a guard from the change it exists to reject. Backend CI's paths cover its own YAML and studio/**, not .github/workflows/**, so a pull request editing only workflow-trigger-lint.yml could delete the floor lint step and nothing would collect the assertion protecting it.

It runs from that unfiltered workflow now, next to the three guards already there for the same reason. Fixed in the same commit.

Comment thread scripts/lint_backend_python_floor.py Outdated
Comment on lines +99 to +100
if not any(part in EXCLUDE_PARTS for part in path.relative_to(BACKEND).parts)
and str(path.relative_to(BACKEND)) not in GUARDED

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep guarded files in the Vermin scan

When a PR adds an unguarded post-3.10 API anywhere else in state_store.py, this predicate omits the entire file and the new floor lint passes even though the PR runs only Python 3.13. GUARDED documents only the single protected locale.getencoding() call, so excluding at file granularity creates a permanent unrestricted hole, and the coverage test explicitly permits it. The fresh evidence beyond the earlier allowlist issue is this new whole-file exclusion; suppress or sanitize only the known guarded occurrence while continuing to scan the rest of the module.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, and it is the same mistake as the package allowlist one level down: I replaced a too-narrow input with a too-broad exemption. A new unguarded 3.12 call anywhere else in that module would have passed, on a pull request that runs only 3.13.

Fixed in 5d1ec33 by suppressing the call rather than the file. vermin honours a # novermin annotation, so comment parsing is on now and the single guarded call carries it with a note that the except below IS the guard and that vermin reads names rather than control flow. The file is back in the scan -- 422 files again rather than 421 -- and I checked your exact scenario: an unguarded itertools.batched call added elsewhere in state_store.py now fails the lint.

The coverage test no longer permits any file-level exemption at all, rather than permitting a recorded one, so reintroducing the exclusion fails it.

pre-commit-ci Bot and others added 3 commits August 17, 2026 13:09
Excluding state_store.py wholesale left everything else in it permanently unchecked, which
is the package-allowlist mistake from the previous commit one level down: a new unguarded
3.12 call anywhere in that module would have passed the floor lint on a pull request that
runs only 3.13.

The suppression moves to the site. vermin honours a `# novermin` annotation, so comment
parsing is on now and the one guarded call carries the annotation with a note saying the
except below IS the guard and that vermin reads names rather than control flow. The file
is back in the scan, which is 422 files again rather than 421, and adding an unguarded
itertools.batched call elsewhere in it now fails.

The coverage test no longer permits any file-level exemption at all, rather than permitting
a recorded one, so reintroducing the exclusion fails it.

Separately: the assertion that the lint step exists was only collected by Backend CI, whose
paths cover its own YAML and studio/**, not .github/workflows/**. A pull request editing
only workflow-trigger-lint.yml could therefore delete the step without failing anything,
which is the one change the assertion exists to reject. It runs from that unfiltered
workflow now, alongside the three guards already there for the same reason.
@danielhanchen

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5d1ec33e2a

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread scripts/lint_backend_python_floor.py Outdated
Comment on lines +88 to +89
for path in sorted(BACKEND.rglob("*.py"))
if not any(part in EXCLUDE_PARTS for part in path.relative_to(BACKEND).parts)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Include exercised CLI modules in the floor scan

A pull request touching unsloth_cli/** previously ran the backend suite on Python 3.10, and collected tests such as test_cli_mlx_trainer_activates_before_importing_trainer, whose subprocess imports and exercises unsloth_cli.commands.train. The replacement lint scans only studio/backend, so introducing a post-3.10 stdlib API on that CLI path now passes both the 3.13 PR leg and this lint, only failing the restored 3.10 run after merge. Include the shipped CLI modules exercised by this matrix in the Vermin targets.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, and I checked the mechanism rather than the claim: studio-backend-ci lists unsloth_cli/** in its own paths filter and runs pytest unsloth_cli/tests as a step on every leg, so the 3.10 leg this replaces was executing shipped CLI code on the floor interpreter. Aiming the lint at studio/backend alone covers part of what was removed while reading like it covers all of it, which is the package-allowlist shape from the earlier round one level up.

Fixed in 375b6db. ROOTS is both trees and targets() walks each, 441 files rather than 422, still clean at 3.10. The guard asserts on what the lint would actually hand to vermin rather than on its source text, and dropping unsloth_cli back out fails it.

studio-backend-ci lists unsloth_cli/** in its own paths filter and runs pytest
unsloth_cli/tests as a step on every leg, so the 3.10 leg this replaces was
executing shipped CLI code on the floor interpreter, not only backend code. A
lint aimed at studio/backend alone covers part of that while reading like it
covers all of it, which is the same shape as the package allowlist the previous
round removed, one level up.

ROOTS is now both trees and targets() walks each of them, 441 files rather than
422, and the run is still clean at 3.10. The guard asserts on what the lint would
actually hand to vermin rather than on its source, and dropping unsloth_cli back
out fails it.
@danielhanchen

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 375b6db766

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread scripts/lint_backend_python_floor.py Outdated
# So the tree is the input and only these come out:
# tests -- run by whatever interpreter runs them, and not shipped
# vendor -- third party, pinned to its own support range
EXCLUDE_PARTS = ("tests", "vendor", "node_modules", "__pycache__", ".venv")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Scan matrix-run backend tests at the floor

When a PR uses a Python 3.11+ API such as tomllib anywhere under studio/backend/tests, this exclusion removes the file from the Vermin targets. However, the pytest matrix job runs python -m pytest tests/ from studio/backend on every matrix leg; after this change the PR's only leg is 3.13, so both it and the floor lint pass, while the 3.10 run fails only after merge. Test code must also execute on the matrix floor even though it is not shipped, so include these matrix-run tests in the scan or lint them separately.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, and my exclusion comment had the wrong reason written into it: it said tests are not shipped, when shipping is not the question and execution is. pytest tests/ runs from studio/backend on every leg, so a 3.11 API in a test file is executed by the 3.10 leg exactly as one in a shipped module is, and with the pull request down to a single 3.13 leg both it and the lint would pass while main took the failure.

Fixed in 935af05. Only vendored code is excluded now, 1093 files rather than 441, and both test trees are already clean at 3.10, so this closes the hole at no cost today. Putting tests back into EXCLUDE_PARTS fails the new guard.

The first version dropped tests on the theory that they are not shipped. Shipping
is not the question, execution is: studio-backend-ci runs pytest tests/ from
studio/backend on every leg, so a 3.11 API in a test file is executed by the 3.10
leg exactly as one in a shipped module is. With the pull request down to a single
3.13 leg, that leg and this lint would both pass and the failure would arrive on
the push to main, which is the gap this exists to close.

Only vendored code comes out now, pinned to its own support range. 1093 files
rather than 441, still clean at 3.10, so this costs nothing today and closes the
hole. Putting tests back into EXCLUDE_PARTS fails the new guard.
@danielhanchen

Copy link
Copy Markdown
Member Author

@codex review

The 3.10, 3.11 and 3.12 legs are gone from Backend CI, on pull requests and on
main alike. Measured on one runner over the same tree, the four legs collected
the same 26,320 tests and differed by exactly one: the >= 3.12 gate on
test_demonstrates_the_underlying_stdlib_regression. 3.10 and 3.11 reported 26193
passed / 127 skipped, 3.12 and 3.13 reported 26194 / 126. That is 97
runner-minutes per push to run one identical suite four times and learn the value
of a single skip marker, into a queue that has been observed 195 deep, and queue
depth is wall-clock for every other workflow in the repo.

What the older legs were really defending is that nothing reaches for a symbol
newer than the floor, which is static. scripts/lint_backend_python_floor.py now
checks exactly that, on every pull request, in seconds, across 1093 shipped and
executed files, reading stdlib API availability rather than syntax alone.

The floor is DECLARED, as PYTHON_FLOOR in the workflow, next to where the legs
used to be. Deriving it from the matrix was right while the matrix ran several
interpreters and becomes self-defeating with one: a 3.13-only matrix would move
the floor to 3.13 and leave the lint asserting that code written for 3.13 runs on
3.13.

3.10 rather than the 3.9 pyproject.toml declares, because 3.9 is not true today.
unsloth/models/_utils.py already uses dataclasses.dataclass(kw_only) and
tempfile.TemporaryDirectory(ignore_cleanup_errors), both 3.10, so a 3.9 target
fails on the tree as it stands. Either the declaration or those two call sites
has to give, and that is worth its own change; this lint is what made the
mismatch visible rather than what hides it.

The cost is stated rather than buried. A static check does not run anything, so
the sys.version_info branches in sitecustomize.py, native_path_leases.py,
third_party_source.py and worker.py are now covered by reading and by the lint's
view of the names they use, not by execution. The guard that used to assert main
still ran them asserts instead that every file carrying such a branch is inside
the lint's scan, since that is the only check left on them.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3aebdda8d6

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread .github/workflows/studio-backend-ci.yml Outdated
# each is never executed. Those branches are covered by reading, by the floor
# lint's view of the names they use, and by test_python39_compatibility.py's
# parse, not by execution.
python: ['3.13']

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Restore the older interpreter legs on main

When code breaks a supported 3.10–3.12 runtime path without using a newer stdlib name—for example one of the explicitly listed sys.version_info branches—the Vermin scan passes, and this literal matrix also applies to push, so the break is never exercised after merge either. The fresh evidence beyond the earlier comments, which assumed failures would surface on a restored 3.10 main run, is that the final workflow now runs only 3.13 “on pull requests and on main alike”; retain the reduced PR matrix if desired, but restore the older legs for pushes to main.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right that a break in an older runtime path using no newer stdlib name would have been executed nowhere, so I counted the branches rather than argue about the size of the hole.

Seven backend files carry a sys.version_info comparison, at 3.10, 3.12 and 3.14. The 3.10 ones were never straddled even by the old matrix, whose oldest leg WAS 3.10, so every leg took the same side of them and dropping legs loses nothing there. 3.14 is above every leg there has ever been. What a 3.13-only matrix genuinely stops executing is the pre-3.12 side of three files.

Rather than restore the legs, that side keeps a leg of its own: 3.11, the newest version that still takes it, running those three files and nothing else. 57 tests in under seven seconds, on both pull requests and pushes, beside the full leg rather than in front of it. The four full duplicate suites stay gone, which is the point of the change, and the branch coverage you are defending is back. In 8ac4452, with a guard that fails if that leg disappears or is moved to a version on the >= 3.12 side.

Comment thread tests/studio/test_backend_ci_matrix.py Outdated
Comment on lines +82 to +83
floor = _declared_floor()
assert _version(legs[0]) > floor, (

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Assert that the sole leg is the actual ceiling

If a later workflow edit changes the only leg from 3.13 to 3.11 or 3.12, this guard still passes because those versions are merely greater than the 3.10 floor, even though the test is specifically intended to preserve the newest-interpreter coverage for removals and deprecations. Assert the known ceiling (currently 3.13), rather than only asserting that the leg is above the floor.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, and the gap is exactly the one that matters: 3.11 and 3.12 are both above the floor while giving up the removals-and-deprecations coverage that is the entire reason the single leg is the newest one. Fixed in 8ac4452 by writing the ceiling down and comparing by name, so moving it is a decision made and defended in the same change rather than something that follows silently from an edit elsewhere. Pointing the full leg at 3.12 now fails the guard.

Two from review.

The first is the honest objection to a 3.13-only matrix on push as well as on
pull requests: a break in a supported older runtime path that uses no newer
stdlib name passes the lint and is then executed nowhere. So the branches were
counted rather than argued about. Seven backend files carry a sys.version_info
comparison, at 3.10, 3.12 and 3.14. The 3.10 ones were never straddled even by
the old matrix, whose oldest leg WAS 3.10, so every leg took the same side of
them and dropping legs loses nothing there. 3.14 is above every leg there has
ever been. What is genuinely lost is the pre-3.12 side of three files, and that
is small enough to keep running: a second matrix entry on 3.11, the newest
version that still takes that side, running those three files and nothing else.
57 tests in under seven seconds, beside the full leg rather than in front of it,
so the critical path is the full leg either way. It is not a second copy of the
suite, and the four legs it replaces are still gone.

The second is that asserting the sole leg is merely above the floor let 3.11 or
3.12 satisfy it, which would give up the removals-and-deprecations coverage that
is the entire reason the single leg is the newest one. The ceiling is now written
down and compared by name, so moving it is a decision somebody makes and defends
in the same change.

Both new assertions fail when mutated: pointing the full leg at 3.12 fails the
ceiling test, and pointing the spot-check leg at 3.13 fails the pre-3.12 test
because it would then re-test what the full leg already covers.
@danielhanchen

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

💡 Codex Review


P2 Badge Exercise the pre-3.11 sandbox branch on Python 3.10

The 3.11 spot-check cannot cover the supported-3.10 path in core/inference/sandbox_site/sitecustomize.py: pathlib._NormalAccessor is absent starting in 3.11, while tests/test_sandbox_sitecustomize.py::test_path_write_read_text_remap_convention_path explicitly guards the 3.10-only accessor patch that keeps Path.write_text() and read_text() inside the sandbox working directory. The former 3.10 full leg executed this test, but the new 3.11 leg neither has that accessor nor includes this test, and Vermin cannot validate the runtime monkeypatch. Fresh evidence beyond the earlier general matrix comment is this capability-based branch, which the new sys.version_info inventory misses; run this sandbox test on a 3.10 spot leg.

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

💡 Codex Review


P2 Badge Keep the spot-check synchronized with version branches

When a later backend change adds a pre-3.12 branch outside these three test files, or moves an existing branch, neither guard requires this hardcoded list to be updated: test_the_pre_312_branches_are_still_executed_somewhere only checks that some pre-3.12 leg exists, while _boundaries() only checks that the source file is scanned by Vermin. The PR can therefore pass the 3.13 suite, static lint, and guard tests while the older branch runs nowhere on either PRs or main. Fresh evidence beyond the earlier matrix finding is this new fixed test list without a coverage assertion tying it to the discovered version branches.

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@danielhanchen
danielhanchen merged commit 2748b15 into main Aug 18, 2026
37 of 39 checks passed
@danielhanchen
danielhanchen deleted the single-leg-matrix-plus-vermin branch August 18, 2026 01:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant