[Klaud Cold] Update DSR1 FP4 MI355X ATOM image to v0.1.8 / [Klaud Cold] 将 DSR1 FP4 MI355X ATOM 镜像更新至 v0.1.8 by Klaud-Cold · Pull Request #4358 · SemiAnalysisAI/InferenceX · GitHub
Skip to content

[Klaud Cold] Update DSR1 FP4 MI355X ATOM image to v0.1.8 / [Klaud Cold] 将 DSR1 FP4 MI355X ATOM 镜像更新至 v0.1.8 - #4358

Open
Klaud-Cold wants to merge 2 commits into
mainfrom
klaud/auto-95e9112c71e2a8a9-379a286edcbbdd8c
Open

[Klaud Cold] Update DSR1 FP4 MI355X ATOM image to v0.1.8 / [Klaud Cold] 将 DSR1 FP4 MI355X ATOM 镜像更新至 v0.1.8#4358
Klaud-Cold wants to merge 2 commits into
mainfrom
klaud/auto-95e9112c71e2a8a9-379a286edcbbdd8c

Conversation

@Klaud-Cold

@Klaud-Cold Klaud-Cold commented Oct 21, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Goal: Update ATOM image from rocm/atom-dev:nightly_202610080328 (ATOM v0.1.7) to rocm/atom-dev:nightly_202610190554 (ATOM v0.1.8).
Baseline: 2026-10-20 · rocm/atom-dev:nightly_202610080328@sha256:9c3070bb7d2600d2e1bcfd03cf5f71e4dc0941f3e1d20d3a5845056aae2334eb
Mean latency · Sources: API 1, API 2, API 3

Point Total tok/s/GPU ↑ Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
8k/1k c4 TP4 EP1 efae51 809.12 90.01 357.61 10.47
8k/1k c4 TP8 EP1 3bc487 479.73 53.37 251.86 8.87
8k/1k c8 TP4 EP1 1093a6 1,403.44 157.77 445.14 11.98
8k/1k c16 TP4 EP1 d8c84b 2,033.15 225.29 569.53 16.65
8k/1k c32 TP4 EP1 e312ad 2,692.01 301.04 838.15 25.09
8k/1k c64 TP4 EP1 2c6047 3,577.47 396.89 1,286.07 38.3
8k/1k c128 TP4 EP1 702801 4,305.96 476.95 2,129.36 63.94
8k/1k c256 TP4 EP1 3fcd1f 4,944.23 549.52 3,976.74 110.96
Eval Score ↑ Samples
gsm8k/em_strict · c64 95.3% 1,319
gsm8k/em_strict · c256 95.91% 1,319

Stop: Add the klaud-abort label at any time to stop this candidate now; Klaud cancels its runs, closes this PR and does not retry it. Remove the label to let the planner pick it up again.

中文

**目标：**将 ATOM 镜像从 rocm/atom-dev:nightly_202610080328（ATOM v0.1.7）更新至 rocm/atom-dev:nightly_202610190554（ATOM v0.1.8）。
**基线：**2026-10-20 · rocm/atom-dev:nightly_202610080328@sha256:9c3070bb7d2600d2e1bcfd03cf5f71e4dc0941f3e1d20d3a5845056aae2334eb
平均延迟 · 来源： API 1, API 2, API 3；数值及异常说明见上表。
**停止：**随时添加 klaud-abort 标签即可立即停止该候选：Klaud 会取消其运行、关闭此 PR 且不再重试。移除该标签后，规划器会在下次扫描时重新考虑它。

@github-actions @claude
chore(amd): bump dsr1-fp4-mi355x-atom to ATOM v0.1.8 image
173d4b2 
Pin rocm/atom-dev:nightly_202610190554 by digest; its OCI revision
label is the ATOM v0.1.8 release commit daa4f1b.

将 dsr1-fp4-mi355x-atom 更新为 ATOM v0.1.8 镜像：按摘要固定
rocm/atom-dev:nightly_202610190554，其 OCI revision 标签为 ATOM v0.1.8
发布提交 daa4f1b。

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献！

  • **审阅：**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER，请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，再通过 Slack 联系核心维护者。必须严格遵循模板，并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have，才能触发签核验证。
  • **PR 验证：**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目，并且恰好带有一个主标签时，才会运行扫描：full-sweep-fail-fast（强烈推荐；canary 加逐矩阵 fail-fast）、full-sweep-enabled（有 canary；矩阵任务在失败后继续运行）或 non-canary-full-sweep-enabled（无 canary，也无 fail-fast）。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
  • **合并后：**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复；参见重新运行失败任务的说明。

@Klaud-Cold

Klaud-Cold commented Oct 21, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator Author

Initial attempt · Passed · Run 39524129060 / attempt 1 · 2026-10-21 01:20 UTC
rocm/atom-dev:nightly_202610190554@sha256:9765777e2f56c5c17fefc9e141b37c51f1ab2dea009b158bcf1823f10030cea3 · 6046414d2d1a · Mean latency
Change: Measured ref 173d4b2: image-only bump to ATOM v0.1.8 (image OCI revision daa4f1b = v0.1.8 tag, https://github.com/ROCm/ATOM/releases/tag/v0.1.8; ROCm/ATOM@a526f0d...daa4f1b). The used flags (-tp, --kv_cache_dtype, --max-model-len, --block-size) are unchanged in atom/model_engine/arg_utils.py; the new options default off or apply only to DeepSeek-V4.1. Uncertainty: MLA shuffled-KV and MoE changes touch the DeepSeek-R1 path, and Triton moves 3.7 to 3.8. No recipe or engine patches.

Point Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
8k/1k c4 TP8 EP1 3bc487 53.73 (+0.7%) 250.56 (-0.5%) 8.8 (-0.7%)
8k/1k c4 TP4 EP1 efae51 92.28 (+2.5%) 354.53 (-0.9%) 10.2 (-2.5%)

Note: All rows: request errors unavailable.

Eval Score ↑ Samples
gsm8k/em_strict · c64 95.75% (+0.45 pp) 1,319 each

Next: Append the perf-changelog entry and run the full sweep with full-sweep-fail-fast.

中文

初次尝试 · 已通过 · Run 39524129060 / attempt 1 · 2026-10-21 01:20 UTC
rocm/atom-dev:nightly_202610190554@sha256:9765777e2f56c5c17fefc9e141b37c51f1ab2dea009b158bcf1823f10030cea3 · 6046414d2d1a · 平均延迟
**变更：**测量提交 173d4b2：仅将镜像更新至 ATOM v0.1.8（镜像 OCI revision daa4f1b 即 v0.1.8 标签，https://github.com/ROCm/ATOM/releases/tag/v0.1.8；https://github.com/ROCm/ATOM/compare/a526f0d...daa4f1b）。所用参数（-tp、--kv_cache_dtype、--max-model-len、--block-size）在 atom/model_engine/arg_utils.py 中未变；新增选项默认关闭或仅作用于 DeepSeek-V4.1。不确定性：MLA shuffled-KV 与 MoE 改动涉及 DeepSeek-R1 路径，Triton 由 3.7 升至 3.8。无 recipe 或引擎补丁。实测数值及异常说明见上表。
**下一步：**追加 perf-changelog 条目，并以 full-sweep-fail-fast 运行完整扫描。

@github-actions @claude
chore(changelog): add dsr1-fp4-mi355x-atom ATOM v0.1.8 entry
44aaab1 
为 dsr1-fp4-mi355x-atom 的 ATOM v0.1.8 镜像更新追加 perf-changelog 条目。

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Klaud-Cold Klaud-Cold added the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Oct 21, 2026
@Klaud-Cold

Klaud-Cold commented Oct 21, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator Author

Final full sweep · Passed · Run 39527054825 / attempt 1 · 2026-10-21 02:00 UTC
rocm/atom-dev:nightly_202610190554@sha256:9765777e2f56c5c17fefc9e141b37c51f1ab2dea009b158bcf1823f10030cea3 · 44aaab1d0071 · Mean latency
Change: Validate the complete updated-image family at this head.

Point Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
8k/1k c4 TP4 EP1 efae51 91.75 (+1.9%) 380.65 (+6.4%) 10.24 (-2.1%)
8k/1k c4 TP8 EP1 3bc487 53.97 (+1.1%) 244.52 (-2.9%) 8.77 (-1.1%)
8k/1k c8 TP4 EP1 1093a6 160.73 (+1.9%) 462.82 (+4%) 11.74 (-2%)
8k/1k c16 TP4 EP1 d8c84b 233.44 (+3.6%) 559.86 (-1.7%) 16.04 (-3.6%)
8k/1k c32 TP4 EP1 e312ad 305.92 (+1.6%) 798.26 (-4.8%) 24.73 (-1.4%)
8k/1k c64 TP4 EP1 2c6047 405.37 (+2.1%) 1,286.27 (+0%) 37.47 (-2.2%)
8k/1k c128 TP4 EP1 702801 480.44 (+0.7%) 2,102.62 (-1.3%) 63.51 (-0.7%)
8k/1k c256 TP4 EP1 3fcd1f 562.9 (+2.4%) 3,931.37 (-1.1%) 108.25 (-2.4%)

Note: All rows: request errors unavailable.

Eval Score ↑ Samples
gsm8k/em_strict · c64 95.68% (+0.38 pp) 1,319 each
gsm8k/em_strict · c256 95.53% (-0.38 pp) 1,319 each

Next: Mark ready for maintainer review.

中文

最终完整 sweep · 已通过 · Run 39527054825 / attempt 1 · 2026-10-21 02:00 UTC
rocm/atom-dev:nightly_202610190554@sha256:9765777e2f56c5c17fefc9e141b37c51f1ab2dea009b158bcf1823f10030cea3 · 44aaab1d0071 · 平均延迟
**变更：**验证此提交更新镜像后的完整配置族。实测数值及异常说明见上表。
**下一步：**标记为就绪，等待维护者审查。

@github-actions

github-actions Bot commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=39527054825

View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=39527054825

@Klaud-Cold
Klaud-Cold marked this pull request as ready for review October 21, 2026 02:00
@Klaud-Cold
Klaud-Cold requested a review from a team October 21, 2026 02:00
@Klaud-Cold

Klaud-Cold commented Oct 21, 2026

Copy link
Copy Markdown
Collaborator Author

validated · Repairs: 0 · Runs: 39524129060, 39527017978, 39527054825
All owned runs ended. Full sweep verified; ready for review.

中文

validated · 修复次数：0 · 运行：39524129060, 39527017978, 39527054825
所有自有运行均已结束。完整 sweep 已验证；已就绪，等待审查。

@Klaud-Cold

Klaud-Cold commented Oct 21, 2026

Copy link
Copy Markdown
Collaborator Author

/use 39527054825

@claude

claude Bot commented Oct 21, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Claude finished @Klaud-Cold's task in 14s —— View job

LGTM - no blocking issues found

@github-actions

github-actions Bot commented Oct 21, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

claude[bot]
claude Bot reviewed Oct 21, 2026
View reviewed changes

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review found no issues

No high-confidence issues detected in this change.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@claude claude[bot] claude[bot] left review comments
@billishyahao billishyahao Awaiting requested review from billishyahao billishyahao is a code owner
@chunfangamd chunfangamd Awaiting requested review from chunfangamd chunfangamd is a code owner
@seungrokj seungrokj Awaiting requested review from seungrokj seungrokj is a code owner
@1am9trash 1am9trash Awaiting requested review from 1am9trash 1am9trash is a code owner
@zijiecode zijiecode Awaiting requested review from zijiecode zijiecode is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended)

Projects

InferenceX Board
Status: No status

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@Klaud-Cold