[Klaud Cold] Update DSR1 FP8 H200 TRT-LLM image to 1.4.0rc1 / [Klaud Cold] 将 DSR1 FP8 H200 TRT-LLM 镜像更新至 1.4.0rc1 by Klaud-Cold · Pull Request #4359 · SemiAnalysisAI/InferenceX · GitHub
Skip to content

[Klaud Cold] Update DSR1 FP8 H200 TRT-LLM image to 1.4.0rc1 / [Klaud Cold] 将 DSR1 FP8 H200 TRT-LLM 镜像更新至 1.4.0rc1 - #4359

Merged
adibarra merged 4 commits into
mainfrom
klaud/auto-6874bba2f5590b1c-26ea5eb9a6709fef
Oct 22, 2026
Merged

[Klaud Cold] Update DSR1 FP8 H200 TRT-LLM image to 1.4.0rc1 / [Klaud Cold] 将 DSR1 FP8 H200 TRT-LLM 镜像更新至 1.4.0rc1#4359
adibarra merged 4 commits into
mainfrom
klaud/auto-6874bba2f5590b1c-26ea5eb9a6709fef

Conversation

@Klaud-Cold

@Klaud-Cold Klaud-Cold commented Oct 21, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Goal: Update TensorRT-LLM image from nvcr.io/nvidia/tensorrt-llm/release:1.4.0rc0@sha256:cab8318458b3dd92cc74c53709a30001d90d38928e5bfeb9fb84280ce00369f5 to nvcr.io/nvidia/tensorrt-llm/release:1.4.0rc1@sha256:3bedbe0321e7eb5430f3177d88894a3cf2dc2f6a087d0e85c13fc882667c75c0.
Baseline: 2026-10-20 · nvcr.io/nvidia/tensorrt-llm/release:1.4.0rc0@sha256:cab8318458b3dd92cc74c53709a30001d90d38928e5bfeb9fb84280ce00369f5
8k/1k · TP8/EP8 · Mean latency · Sources: API 1, API 2, API 3

Concurrency Total tok/s/GPU ↑ Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
4 340.57 37.89 396.09 12.49
8 529.47 59.52 478.11 16.03
16 766.51 84.93 627.77 22.28
32 1,065.23 119.12 892.36 31.91
64 1,551.93 172.17 3,977.4 41.14
Eval Score ↑ Samples
gsm8k/em_strict · c32 95.75% 1,319
gsm8k/em_strict · c64 95.3% 1,319

Stop: Add the klaud-abort label at any time to stop this candidate now; Klaud cancels its runs, closes this PR and does not retry it. Remove the label to let the planner pick it up again.

中文

**目标：**将 TensorRT-LLM 镜像从 nvcr.io/nvidia/tensorrt-llm/release:1.4.0rc0@sha256:cab8318458b3dd92cc74c53709a30001d90d38928e5bfeb9fb84280ce00369f5 更新至 nvcr.io/nvidia/tensorrt-llm/release:1.4.0rc1@sha256:3bedbe0321e7eb5430f3177d88894a3cf2dc2f6a087d0e85c13fc882667c75c0。
**基线：**2026-10-20 · nvcr.io/nvidia/tensorrt-llm/release:1.4.0rc0@sha256:cab8318458b3dd92cc74c53709a30001d90d38928e5bfeb9fb84280ce00369f5
8k/1k · TP8/EP8 · 平均延迟 · 来源： API 1, API 2, API 3；数值及异常说明见上表。
**停止：**随时添加 klaud-abort 标签即可立即停止该候选：Klaud 会取消其运行、关闭此 PR 且不再重试。移除该标签后，规划器会在下次扫描时重新考虑它。

@github-actions @claude
Update dsr1-fp8-h200-trt TensorRT-LLM image to 1.4.0rc1
21a8c04 
将 dsr1-fp8-h200-trt 的 TensorRT-LLM 镜像更新至 1.4.0rc1。

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献！

  • **审阅：**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER，请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，再通过 Slack 联系核心维护者。必须严格遵循模板，并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have，才能触发签核验证。
  • **PR 验证：**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目，并且恰好带有一个主标签时，才会运行扫描：full-sweep-fail-fast（强烈推荐；canary 加逐矩阵 fail-fast）、full-sweep-enabled（有 canary；矩阵任务在失败后继续运行）或 non-canary-full-sweep-enabled（无 canary，也无 fail-fast）。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
  • **合并后：**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复；参见重新运行失败任务的说明。

@Klaud-Cold

Klaud-Cold commented Oct 21, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator Author

Initial attempt · Failed · Run 39524192317 / attempt 1 · 2026-10-21 01:35 UTC
nvcr.io/nvidia/tensorrt-llm/release:1.4.0rc1@sha256:3bedbe0321e7eb5430f3177d88894a3cf2dc2f6a087d0e85c13fc882667c75c0 · 6046414d2d1a
Change: Bump only the master image from TensorRT-LLM 1.4.0rc0 (cc63e66) to 1.4.0rc1 (0372799), with recipe flags unchanged, measured at branch commit 21a8c04.

Eval Score ↑ Samples
gsm8k/em_strict · c32 95.6% (-0.15 pp) 1,319 each
gsm8k/em_strict · c64 95.22% (-0.08 pp) 1,319 each

Next: Rerun the failed and cancelled benchmark jobs on run 39524192317 as infrastructure retry 1 at the same head.

中文

初次尝试 · 失败 · Run 39524192317 / attempt 1 · 2026-10-21 01:35 UTC
nvcr.io/nvidia/tensorrt-llm/release:1.4.0rc1@sha256:3bedbe0321e7eb5430f3177d88894a3cf2dc2f6a087d0e85c13fc882667c75c0 · 6046414d2d1a
**变更：**仅将主配置镜像从 TensorRT-LLM 1.4.0rc0 (cc63e66) 更新至 1.4.0rc1 (0372799)，recipe 参数保持不变，在分支提交 21a8c04 上测量。实测数值及异常说明见上表。
**下一步：**在相同提交上将运行 39524192317 中失败和被取消的基准测试任务作为第 1 次基础设施重试重跑。

@Klaud-Cold

Klaud-Cold commented Oct 21, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator Author

Infrastructure retry 1 · Passed · Run 39524192317 / attempt 2 · 2026-10-21 01:59 UTC
nvcr.io/nvidia/tensorrt-llm/release:1.4.0rc1@sha256:3bedbe0321e7eb5430f3177d88894a3cf2dc2f6a087d0e85c13fc882667c75c0 · 6046414d2d1a · 8k/1k · TP8/EP8 · Mean latency
Change: Rerun the failed c4 and cancelled c64 smoke benchmarks unchanged after a node-local pyxis image-cache failure.

Concurrency Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
4 37.82 (-0.2%) 397.79 (+0.4%) 12.51 (+0.2%)
64 175.02 (+1.7%) 3,919.81 (-1.4%) 40.55 (-1.4%)

Note: All rows: request errors unavailable.

Eval Score ↑ Samples
gsm8k/em_strict · c32 95.6% (-0.15 pp) 1,319 each
gsm8k/em_strict · c64 95.22% (-0.08 pp) 1,319 each

Next: Append the perf-changelog entry and run the full sweep with full-sweep-fail-fast.

中文

基础设施重试 1 · 已通过 · Run 39524192317 / attempt 2 · 2026-10-21 01:59 UTC
nvcr.io/nvidia/tensorrt-llm/release:1.4.0rc1@sha256:3bedbe0321e7eb5430f3177d88894a3cf2dc2f6a087d0e85c13fc882667c75c0 · 6046414d2d1a · 8k/1k · TP8/EP8 · 平均延迟
**变更：**在节点本地 pyxis 镜像缓存故障后，原样重跑失败的 c4 和被取消的 c64 冒烟基准测试。实测数值及异常说明见上表。
**下一步：**追加 perf-changelog 条目，并使用 full-sweep-fail-fast 运行完整扫描。

@github-actions @claude
Add perf-changelog entry for dsr1-fp8-h200-trt TensorRT-LLM 1.4.0rc1
36e0ceb 
为 dsr1-fp8-h200-trt 的 TensorRT-LLM 1.4.0rc1 更新添加 perf-changelog 条目。

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Klaud-Cold Klaud-Cold added the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Oct 21, 2026
@Klaud-Cold

Klaud-Cold commented Oct 21, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator Author

Final full sweep · Passed · Run 39530755141 / attempt 1 · 2026-10-21 02:36 UTC
nvcr.io/nvidia/tensorrt-llm/release:1.4.0rc1@sha256:3bedbe0321e7eb5430f3177d88894a3cf2dc2f6a087d0e85c13fc882667c75c0 · 36e0cebb2fb7 · 8k/1k · TP8/EP8 · Mean latency
Change: Validate the complete updated-image family at this head.

Concurrency Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
4 37.82 (-0.2%) 400.38 (+1.1%) 12.5 (+0.2%)
8 59.24 (-0.5%) 486.99 (+1.9%) 16.1 (+0.4%)
16 85.06 (+0.1%) 619.66 (-1.3%) 22.26 (-0.1%)
32 118.58 (-0.5%) 902.19 (+1.1%) 32.07 (+0.5%)
64 175.98 (+2.2%) 3,809.99 (-4.2%) 40.48 (-1.6%)

Note: All rows: request errors unavailable.

Eval Score ↑ Samples
gsm8k/em_strict · c32 95.45% (-0.30 pp) 1,319 each
gsm8k/em_strict · c64 95.83% (+0.53 pp) 1,319 each

Next: Mark ready for maintainer review.

中文

最终完整 sweep · 已通过 · Run 39530755141 / attempt 1 · 2026-10-21 02:36 UTC
nvcr.io/nvidia/tensorrt-llm/release:1.4.0rc1@sha256:3bedbe0321e7eb5430f3177d88894a3cf2dc2f6a087d0e85c13fc882667c75c0 · 36e0cebb2fb7 · 8k/1k · TP8/EP8 · 平均延迟
**变更：**验证此提交更新镜像后的完整配置族。实测数值及异常说明见上表。
**下一步：**标记为就绪，等待维护者审查。

@github-actions

github-actions Bot commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=39530755141

View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=39530755141

@Klaud-Cold
Klaud-Cold marked this pull request as ready for review October 21, 2026 02:36
@Klaud-Cold
Klaud-Cold requested a review from a team October 21, 2026 02:36
@Klaud-Cold

Klaud-Cold commented Oct 21, 2026

Copy link
Copy Markdown
Collaborator Author

validated · Repairs: 0 · Runs: 39524192317, 39530705169, 39530755141
All owned runs ended. Full sweep verified; ready for review.

中文

validated · 修复次数：0 · 运行：39524192317, 39530705169, 39530755141
所有自有运行均已结束。完整 sweep 已验证；已就绪，等待审查。

@Klaud-Cold

Klaud-Cold commented Oct 21, 2026

Copy link
Copy Markdown
Collaborator Author

/use 39530755141

@claude

claude Bot commented Oct 21, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Claude finished @Klaud-Cold's task in 19s —— View job

LGTM, no blocking issues found.

This PR bumps the dsr1-fp8-h200-trt image to nvcr.io/nvidia/tensorrt-llm/release:1.4.0rc1, which is a public NGC image pinned by digest. It also adds a matching perf-changelog.yaml entry at the end of the file that covers that config key.

@github-actions

github-actions Bot commented Oct 21, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

@Klaud-Cold staged run 39530755141.

This run remains available across future /stage-results requests. Staging the same run ID again updates its staged data. Staging workflow

claude[bot]
claude Bot reviewed Oct 21, 2026
View reviewed changes

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review found no issues

No high-confidence issues detected in this change.

@alec-semi
Merge origin/main [skip-sweep]
c526bf8 
Resolve conflicts: take main's refactored recipes and re-apply this PR's non-image edits; take main's perf-changelog.yaml and re-append this PR's entry.

中文：合并 origin/main；配方采用 main 重构后的版本并重新应用本 PR 的非镜像改动，perf-changelog.yaml 采用 main 版本并在末尾重新追加本 PR 的条目。
@adibarra

adibarra commented Oct 22, 2026

Copy link
Copy Markdown
Collaborator

@adibarra
adibarra merged commit 55749d2 into main Oct 22, 2026
26 checks passed
@adibarra
adibarra deleted the klaud/auto-6874bba2f5590b1c-26ea5eb9a6709fef branch October 22, 2026 02:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@claude claude[bot] claude[bot] left review comments
@Ankur-singh Ankur-singh Awaiting requested review from Ankur-singh Ankur-singh is a code owner
@kedarpotdar-nv kedarpotdar-nv Awaiting requested review from kedarpotdar-nv kedarpotdar-nv is a code owner
@wzhao18 wzhao18 Awaiting requested review from wzhao18 wzhao18 is a code owner
@nvpohanh nvpohanh Awaiting requested review from nvpohanh nvpohanh is a code owner
@RohitNagraj RohitNagraj Awaiting requested review from RohitNagraj RohitNagraj is a code owner

Assignees

No one assigned

Labels

full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended)

Projects

InferenceX Board
Status: Done

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Klaud-Cold @adibarra @alec-semi