{{ message }}
Move the two version-incidental macOS jobs onto the emptier pool - #9277
Merged
Conversation
Rust unit tests (macos) is this repo's last finisher. The census behind that
found minutes of work sitting behind hours of queue, and no execution-time work
touches it, so the runner pool is the only lever left.
studio-mac-install-matrix runs macos-15 and macos-26 from ONE matrix, which
makes them comparable on identical commits and identical triggers rather than
across different jobs at different times. Over 163 jobs per leg:
exec med exec p90 queue p90
macos-15 160s 713s 20667s
macos-26 176s 597s 3866s
The medians say these are the same class of machine, which they are: both free
Apple Silicon standard runners. The queue p90 says they are not the same pool.
macos-15 is 5.3x worse in the tail, five and a half hours against one.
Worth being precise about what the tail means here, because the median is zero
on both images and hides all of it. macOS queueing is not slow on average; it
is occasionally catastrophic, and the occasional case is what sets the wall
clock for the commit it lands on.
The likely cause is the macos-14 retirement. Everything migrated onto macos-15,
this job included, and macos-26 was left comparatively empty. That makes this a
fact about today's pool rather than a property of the image, so the comment
records the query as much as the result: it is one read of the install matrix
and can be re-run whenever the queue looks wrong again.
Two jobs, not all of them. These two test our Rust crate and our MLX dispatch,
where the OS version is incidental. The install matrices are deliberately
testing across OS versions and are left alone, and studio-mac-ui-smoke stays
too: it documents a dependency on the frameworks the image ships for Chromium,
which is a claim about macos-15 specifically and wants checking before it moves.
The guard is for the next retirement rather than this one. macos-14's is
recorded in three comments in this repo, each explaining why some job moved off
it, and comments do not fail -- so the next one gets discovered the way this one
was, by a pool nobody is watching. LIVE_MACOS_IMAGES cannot know GitHub's
roadmap, but it forces the retirement to be written down once and then names
every job that has to move. Mutation-tested by putting macos-14 back.
for more information, see https://pre-commit.ci
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Rust unit tests (macos) is this repo's last finisher. The census behind that
found minutes of work sitting behind hours of queue, and no execution-time work
touches it, so the runner pool is the only lever left.
studio-mac-install-matrix runs macos-15 and macos-26 from ONE matrix, which
makes them comparable on identical commits and identical triggers rather than
across different jobs at different times. Over 163 jobs per leg:
The medians say these are the same class of machine, which they are: both free
Apple Silicon standard runners. The queue p90 says they are not the same pool.
macos-15 is 5.3x worse in the tail, five and a half hours against one.
Worth being precise about what the tail means here, because the median is zero
on both images and hides all of it. macOS queueing is not slow on average; it
is occasionally catastrophic, and the occasional case is what sets the wall
clock for the commit it lands on.
The likely cause is the macos-14 retirement. Everything migrated onto macos-15,
this job included, and macos-26 was left comparatively empty. That makes this a
fact about today's pool rather than a property of the image, so the comment
records the query as much as the result: it is one read of the install matrix
and can be re-run whenever the queue looks wrong again.
Two jobs, not all of them. These two test our Rust crate and our MLX dispatch,
where the OS version is incidental. The install matrices are deliberately
testing across OS versions and are left alone, and studio-mac-ui-smoke stays
too: it documents a dependency on the frameworks the image ships for Chromium,
which is a claim about macos-15 specifically and wants checking before it moves.
The guard is for the next retirement rather than this one. macos-14's is
recorded in three comments in this repo, each explaining why some job moved off
it, and comments do not fail -- so the next one gets discovered the way this one
was, by a pool nobody is watching. LIVE_MACOS_IMAGES cannot know GitHub's
roadmap, but it forces the retirement to be written down once and then names
every job that has to move. Mutation-tested by putting macos-14 back.