{{ message }}
Take two ubuntu jobs off their own runners: absorb one, delete the other - #9360
Merged
Conversation
Over 400 completed main push runs, 25 of 96 job types execute in under 120s: 1116s of work spread across 25 runners, each queuing for about three hours. Two of them are dealt with here. Security audit :: pytest tests/security 72s exec, 11096s queue Unsloth export capability :: capability (ubuntu-latest) 67s exec, 10642s queue Different problems, so different treatments. The security suite MOVED, onto the Workflow trigger lint runner ------------------------------------------------------------------------ Same argument that put the lockfile and load-orchestrator lanes into Lint CI in #9176: work with a narrow trigger, moved into a job that was going to occupy a runner on this commit anyway, can only reduce the slots a commit takes. Here the trigger widens too, since this host has no paths filter and security-audit.yml's pull_request does, so the suite now runs on every pull request rather than on the ones that touch its paths. This host and NOT Lint CI, where the other lanes went, and that is the whole decision. Lint CI installs shellcheck from apt, so its harden-runner has to permit escalation and an apt mirror; a security gate moved there would run under a policy weaker than the one it has today. Workflow trigger lint's harden-runner block is byte-for-byte identical to the one the job carried in security-audit.yml (block, disable-sudo, the same six endpoints), so nothing about its isolation changes. harden-runner binds per runner, not per step, which is what makes that the deciding constraint rather than a detail. Folded into the existing pytest invocation rather than added as a step of its own. I wrote it as a separate step first, so a security regression would not be reported as a workflow-guard failure, and test_the_guards_run_in_one_pytest_invocation rejected it: one step per module costs about 15s of interpreter and conftest startup, measured in this repo at 53.9s as one invocation against 300.8s as one each. The guard is right and the attribution preference is not worth 15s. pytest and PyYAML are now pinned here to the versions security-audit.yml pinned them to. That suite runs scripts/lint_workflow_triggers.py as a SUBPROCESS and asserts on its exit semantics, so a pytest or PyYAML that resolves differently changes what it is asserting against. The capability ubuntu leg was DELETED, because it was already duplicated ------------------------------------------------------------------------ That workflow's own comment already explains why it has no macOS leg: every test in tests/test_export_capability.py goes through _patch(), which monkeypatches _has_torch, get_device and is_apple_silicon, so a real Mac proves nothing a Linux runner does not -- and studio-backend-ci.yml runs the same file on ubuntu-latest as part of `pytest tests/`. That argument reaches one step further than it was taken. If Backend CI covers the file on Linux, the ubuntu leg HERE is the duplicate too. Checked: the file is not in that job's --ignore list. The import-safety test does not need a torch-free image either; it installs its own builtins.__import__ blocker and drops preloaded torch/unsloth from sys.modules, so it proves the same thing inside Backend CI's fully installed environment. Windows stays. Nothing else in CI runs that file there, and _has_torch's import probe is the per-OS behaviour the job exists for. The guard ------------------------------------------------------------------------ tests/studio/test_short_job_absorption.py, wired into the unfiltered job. Both changes fail silently rather than loudly if they regress, which is what it is for: - the suite still runs somewhere, and no longer runs twice - the absorbing job's harden-runner has not widened past the six endpoints the suite came with, since "the policy is identical" is the entire justification for this host - the absorbing job has not gained a paths filter - capability still has its Windows leg - studio-backend-ci.yml still runs the whole tests/ tree and does not name test_export_capability.py, because one line added to that --ignore list would remove the coverage the ubuntu leg was deleted for, and nothing would turn red Mutation-tested, each failing exactly one test: drop tests/security from the invocation; add one endpoint to the allowlist; add --ignore=tests/test_export_capability.py to Backend CI; remove the Windows leg. Also corrected a docstring in tests/security/test_scan_packages.py that named tests-security and what it installs. It was about to become false. Verification ------------------------------------------------------------------------ tests/security under the host's own -n 4: 409 passed, 6 skipped. scripts/lint_workflow_triggers.py: OK across 41 workflow files. All three workflows still parse; security-audit.yml keeps its other 4 jobs. Net: 7 short ubuntu slots per commit, down to 5. A note for whoever extends this. The census that found these 15 candidates was partly stale and I nearly acted on it: the lockfile and load-orchestrator rows were already absorbed by #9176 and their samples were pre-merge tails, and the two Local Agent Guides rows show ~0s because they are if-gated to schedule and dispatch, which is a skip and not a fast job. Read the trigger before ranking by duration. Of the remaining candidates, Scorecard is blocked by its job-level id-token: write, the Kaggle gate by a downstream needs:, npm-provenance by an audit egress policy plus registry.npmjs.org, and the notransport clean-install lane by a container that asserts several common tools are absent.
Member
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Take two ubuntu jobs off their own runners: absorb one, delete the other
Over 400 completed main push runs, 25 of 96 job types execute in under 120s:
1116s of work spread across 25 runners, each queuing for about three hours. Two
of them are dealt with here.
Security audit :: pytest tests/security 72s exec, 11096s queue
Unsloth export capability :: capability (ubuntu-latest) 67s exec, 10642s queue
Different problems, so different treatments.
The security suite MOVED, onto the Workflow trigger lint runner
Same argument that put the lockfile and load-orchestrator lanes into Lint CI in
#9176: work with a narrow trigger, moved into a job that was going to occupy a
runner on this commit anyway, can only reduce the slots a commit takes. Here the
trigger widens too, since this host has no paths filter and security-audit.yml's
pull_request does, so the suite now runs on every pull request rather than on the
ones that touch its paths.
This host and NOT Lint CI, where the other lanes went, and that is the whole
decision. Lint CI installs shellcheck from apt, so its harden-runner has to
permit escalation and an apt mirror; a security gate moved there would run under
a policy weaker than the one it has today. Workflow trigger lint's harden-runner
block is byte-for-byte identical to the one the job carried in security-audit.yml
(block, disable-sudo, the same six endpoints), so nothing about its isolation
changes. harden-runner binds per runner, not per step, which is what makes that
the deciding constraint rather than a detail.
Folded into the existing pytest invocation rather than added as a step of its
own. I wrote it as a separate step first, so a security regression would not be
reported as a workflow-guard failure, and
test_the_guards_run_in_one_pytest_invocation rejected it: one step per module
costs about 15s of interpreter and conftest startup, measured in this repo at
53.9s as one invocation against 300.8s as one each. The guard is right and the
attribution preference is not worth 15s.
pytest and PyYAML are now pinned here to the versions security-audit.yml pinned
them to. That suite runs scripts/lint_workflow_triggers.py as a SUBPROCESS and
asserts on its exit semantics, so a pytest or PyYAML that resolves differently
changes what it is asserting against.
The capability ubuntu leg was DELETED, because it was already duplicated
That workflow's own comment already explains why it has no macOS leg: every test
in tests/test_export_capability.py goes through _patch(), which monkeypatches
_has_torch, get_device and is_apple_silicon, so a real Mac proves nothing a Linux
runner does not -- and studio-backend-ci.yml runs the same file on ubuntu-latest
as part of
pytest tests/.That argument reaches one step further than it was taken. If Backend CI covers
the file on Linux, the ubuntu leg HERE is the duplicate too. Checked: the file is
not in that job's --ignore list. The import-safety test does not need a
torch-free image either; it installs its own builtins.import blocker and
drops preloaded torch/unsloth from sys.modules, so it proves the same thing
inside Backend CI's fully installed environment.
Windows stays. Nothing else in CI runs that file there, and _has_torch's import
probe is the per-OS behaviour the job exists for.
The guard
tests/studio/test_short_job_absorption.py, wired into the unfiltered job. Both
changes fail silently rather than loudly if they regress, which is what it is
for:
suite came with, since "the policy is identical" is the entire justification
for this host
test_export_capability.py, because one line added to that --ignore list
would remove the coverage the ubuntu leg was deleted for, and nothing would
turn red
Mutation-tested, each failing exactly one test: drop tests/security from the
invocation; add one endpoint to the allowlist; add
--ignore=tests/test_export_capability.py to Backend CI; remove the Windows leg.
Also corrected a docstring in tests/security/test_scan_packages.py that named
tests-security and what it installs. It was about to become false.
Verification
tests/security under the host's own -n 4: 409 passed, 6 skipped.
scripts/lint_workflow_triggers.py: OK across 41 workflow files.
All three workflows still parse; security-audit.yml keeps its other 4 jobs.
Net: 7 short ubuntu slots per commit, down to 5.
A note for whoever extends this. The census that found these 15 candidates was
partly stale and I nearly acted on it: the lockfile and load-orchestrator rows
were already absorbed by #9176 and their samples were pre-merge tails, and the
two Local Agent Guides rows show ~0s because they are if-gated to schedule and
dispatch, which is a skip and not a fast job. Read the trigger before ranking by
duration. Of the remaining candidates, Scorecard is blocked by its job-level
id-token: write, the Kaggle gate by a downstream needs:, npm-provenance by an
audit egress policy plus registry.npmjs.org, and the notransport clean-install
lane by a container that asserts several common tools are absent.