{{ message }}
Repository navigation
Conversation
Replace the Muse Spark Open B300 SGLang AgentX search space with 11 per-concurrency points: TP8/EP8 with GPU-resident KV at concurrency 1, 4, 8 and 14; TP4/EP4 with GPU-resident KV at concurrency 4; TP4/EP4 with DRAM HiCache at concurrency 10, 16, 20, 28, 36 and 40. 将 Muse Spark Open B300 SGLang AgentX 搜索空间替换为 11 个按并发配置的点： TP8/EP8 仅 GPU KV（并发 1、4、8、14）；TP4/EP4 仅 GPU KV（并发 4）； TP4/EP4 DRAM HiCache（并发 10、16、20、28、36、40）。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
shyeh25
requested review from
Ankur-singh,
kedarpotdar-nv,
nvpohanh and
wzhao18
as code owners
October 20, 2026 07:07
Contributor
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
I reviewed this PR and didn't find any bugs. It's a per-concurrency performance retune of files with code owners, so a look from one of the NVIDIA config owners would still be worthwhile.
What was reviewed:
- The 11
override_tp{N}_c{conc}blocks against the three nvidia-master search-space entries. TP, EP, KV offloading and CONC line up one-to-one: TP8 none at 1/4/8/14, TP4 none at 4, and TP4 dram at 10/16/20/28/36/40. - The base move to 4 GPUs with TP4/EP4. Every TP8 block overrides gpus and TP/EP back to 8, and only c1 keeps the fa4 draft attention backend.
- The tuning choices an owner may want to confirm: at TP8 c4/c8 and TP4 c4,
max-running-requests(2) is below CONC, and at TP4 c28/c36/c40 it is above CONC (56/72/80). - The changelog entry is appended at the tail, in English, and covers the single affected config key. Its
pr-linkis still thepull/XXXplaceholder.
Extended reasoning...
The change rewrites the Muse Spark Open B300 SGLang agentic DFlash srt-slurm recipe into 11 per-concurrency overrides, updates the matching nvidia-master.yaml search space and appends one perf-changelog entry. It touches no security-sensitive surface: only benchmark server flags, env vars and sweep points. The recipe overrides and the master search space match on TP/EP, KV offloading and concurrency. I deferred instead of approving because CODEOWNERS assigns both nvidia-master.yaml and the b300 srt-slurm recipes to named NVIDIA config owners. The per-point admission and memory settings are also measured tuning choices that the owners should confirm.
shyeh25
added
the
full-sweep-fail-fast
Contributor
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=39385088584 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=39385088584 |
Collaborator
|
[by Claude Code] The SGLang cookbook integration of this PR's AgentX config ( This PR has no authorized |
Collaborator
|
[by Claude Code] sgl-project/sglang#45513 has been merged. This PR's AgentX config ( This PR has no authorized |
Collaborator
|
/use 39385088584 |
Contributor
|
@nvpohanh staging run 39385088584. Existing staged runs will be preserved; a completion link will be posted here. |
Collaborator
|
As a PR reviewer and CODEOWNER, I have reviewed this and have:
Additional detail section:
Signed: |
nvpohanh
approved these changes
Oct 21, 2026
Contributor
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replace the Muse Spark Open B300 SGLang AgentX search space (TP8/EP8 at concurrency 1–128 with one shared server config) with 11 per-concurrency points, each with its own
override_tp{N}_c{conc}section:Per point, the recipe sets request admission and decode graph batch limits, memory fraction, SWA/full ratio, prefill-decode interval and chunked prefill; the HiCache points enable layernorm SP with MegaMoE, and concurrency 16 and above use shortest-prefill-first. TP8 concurrency 2 and 16–128 are no longer swept. Except at concurrency 1, the explicit draft attention backend is removed, so SGLang uses the image default. The image, checkpoint and DFlash2 draft settings are unchanged.
AI model disclosure
PR checklist and change type
Type of Change
Checklist
python -m infx.matrix.changed --base origin/mainlists affected config keys, I have appended one new entry covering them to the physical end ofinferencex-e2e/perf-changelog.yaml; if it lists none, I have added none. I have not edited historical entriesinferencex-e2e/perf-changelog.yamlentry, it carries exactly one primary sweep label (a maintainer applies it on fork PRs):full-sweep-fail-fast(recommended),full-sweep-enabled, ornon-canary-full-sweep-enabled. Optional modifiersall-evals,evals-only, andagentx-fastrequire a primary label; the last two block reuse while applied.OWNER/MEMBER/COLLABORATOR) has commented/use <run_id>(or the legacy/reuse-sweep-run) on this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the primary sweep label will no longer automatically kick off new sweeps. Remove and re-add the primary sweep label to force a new sweep.中文
将 Muse Spark Open B300 SGLang AgentX 的搜索空间（TP8/EP8、并发 1–128、共用同一套服务器配置）替换为 11 个按并发配置的点，每个点有独立的
override_tp{N}_c{conc}段：每个点分别设置请求准入与 decode graph 批大小上限、显存占用比例、SWA/full 比例、prefill-decode interval 和 chunked prefill；HiCache 点启用 layernorm SP 与 MegaMoE，并发 16 及以上使用 shortest-prefill-first。TP8 不再压测并发 2 以及 16–128。除并发 1 外，移除显式指定的草稿注意力后端，改用镜像默认值。镜像、模型检查点和 DFlash2 草稿模型设置保持不变。
AI 模型使用说明：运行环境提供的模型标识为 claude-opus-5-5。工作内容包括执行调优（编排代理及其委派的任务代理均使用 claude-opus-5-5）、根据实测结果生成配方、主配置和性能变更记录的改动、在本地检查，以及起草本 PR 说明。
🤖 Generated with Claude Code