{{ message }}
Repository navigation
Conversation
Recipes can now set the aiperf timeslice width through benchmark.env.AIPERF_SLICE_DURATION, which becomes `agentx profile --slice-duration <value>`. Unset leaves the scenario default and the command line unchanged. 中文：新增 AIPERF_SLICE_DURATION，并将其传递给 agentx profile。配方可通过 benchmark.env.AIPERF_SLICE_DURATION 设置 aiperf 时间切片宽度，对应 `agentx profile --slice-duration <value>`。未设置时保留场景默认值，命令行不变。
7 of 11 tasks
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
AgentX replay runs cannot choose aiperf's timeslice width.
agentx-harness1.0.11's AgentX scenario preset defaultsslice_durationto 1.0 s, andinfx.bench.agentichas no way to pass--slice-duration, so every run computes per-second request and server-metric timeslices in the final export. Nothing in InferenceX reads them: result processing uses only run-widestats(total,sum,max,avg), nevertimeslicesor the slice-derivedrate_*fields.This adds an optional
AIPERF_SLICE_DURATIONinput. When a recipe setsbenchmark.env.AIPERF_SLICE_DURATION,replay_argv()appends--slice-duration <value>toagentx profile. When it is unset, the command line is unchanged. A recipe can use it to coarsen slicing (removing most of the export cost) or, if the preset default changes upstream, to turn per-second slicing back on for by-hand analysis.No benchmark point changes, so there is no
perf-changelog.yamlentry.Companion change: SemiAnalysisAI/agentx-harness#55 removes the 1 s default from the
agentxpreset, so timeslicing becomes opt-in. Until a release with it is pinned here, recipes can setAIPERF_SLICE_DURATIONto avoid the cost; afterwards the variable is how a recipe turns per-window slicing back on.On DSV4.1-Flash FP4 16P1D at c13312 (GB300), a 3600 s slice cut aiperf's post-profiling processing from 1 h 38 m to 23 m and the job from 3:41 to 2:23 of the 4 h limit; serving metrics and the processed result fields were unchanged.
Validation details
Matched srt-slurm runs, same recipe (tachometer off),
agentx-harness==1.0.11, differing only inAIPERF_SLICE_DURATION:AIPERF_SLICE_DURATION=3600All results receivedserver_metrics_export.jsonbenchmark_command.txtcontained--slice-duration 3600, and the exportedinput_config.artifacts.slice_durationwas3600.0. Both processed result JSONs have the same 169 fields and 26 nulls.AI model disclosure
claude-opus-5-5(Anthropic), as exposed by the Hermes runtimePR checklist and change type
Type of Change
Checklist
python -m infx.matrix.changed --base origin/mainlists affected config keys, I have appended one new entry covering them to the physical end ofinferencex-e2e/perf-changelog.yaml; if it lists none, I have added none. I have not edited historical entriesinferencex-e2e/perf-changelog.yamlentry, it carries exactly one primary sweep label (a maintainer applies it on fork PRs):full-sweep-fail-fast(recommended),full-sweep-enabled, ornon-canary-full-sweep-enabled. Optional modifiersall-evals,evals-only, andagentx-fastrequire a primary label; the last two block reuse while applied.OWNER/MEMBER/COLLABORATOR) has commented/use <run_id>(or the legacy/reuse-sweep-run) on this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the primary sweep label will no longer automatically kick off new sweeps. Remove and re-add the primary sweep label to force a new sweep.中文
AgentX 回放运行无法设置 aiperf 的时间切片宽度。
agentx-harness1.0.11 的 AgentX 场景预设将slice_duration默认设为 1.0 秒，而infx.bench.agentic无法传递--slice-duration，因此每次运行都会在最终导出时计算逐秒的请求指标和服务端指标时间切片。InferenceX 并不读取这些数据：结果处理只使用全程stats（total、sum、max、avg），从不使用timeslices或基于切片计算的rate_*字段。本 PR 新增可选输入
AIPERF_SLICE_DURATION。配方设置benchmark.env.AIPERF_SLICE_DURATION时，replay_argv()会向agentx profile追加--slice-duration <value>；未设置时命令行不变。配方可以用它放宽切片宽度（去掉大部分导出开销），或者在上游更改预设默认值后，重新开启逐秒切片以便手动分析。本 PR 不改变任何基准测试点，因此不新增
perf-changelog.yaml条目。配套改动：SemiAnalysisAI/agentx-harness#55 移除
agentx预设中 1 秒的默认切片宽度，使时间切片改为按需开启。在本仓库固定到包含该改动的版本之前，配方可以设置AIPERF_SLICE_DURATION来避免这部分开销；之后，配方可通过该变量重新开启按时间窗口切片。在 GB300 上的 DSV4.1-Flash FP4 16P1D、并发 13312 配置中，3600 秒切片使 aiperf 在 profiling 结束后的处理时间从 1 小时 38 分降至 23 分，作业总时长从 3:41 降至 2:23（上限 4 小时）；服务指标和处理后的结果字段均未改变。验证数据见上方 “Validation details” 表格。
AI 模型使用说明：使用
claude-opus-5-5（Anthropic，由 Hermes 运行环境提供的标识）排查处理耗时、实现改动和测试、分析 A/B 基准测试结果并撰写本说明。