fix(ci): make curl benchmark deterministic by YOMXXX · Pull Request #2596 · rtk-ai/rtk · GitHub
Skip to content

fix(ci): make curl benchmark deterministic#2596

Open
YOMXXX wants to merge 1 commit into
rtk-ai:developfrom
YOMXXX:fix/benchmark-curl-local-fixtures
Open

fix(ci): make curl benchmark deterministic#2596
YOMXXX wants to merge 1 commit into
rtk-ai:developfrom
YOMXXX:fix/benchmark-curl-local-fixtures

Conversation

@YOMXXX

@YOMXXX YOMXXX commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Replace the benchmark curl/httpbin text and JSON calls with local file:// fixtures.
  • Avoids comparing two independent external network responses in the benchmark job.
  • Keeps the benchmark exercising the rtk curl path while removing the flaky curl text negative-token failure seen across open PRs.

Verification

  • bash -n scripts/benchmark.sh
  • git diff --check
  • cargo +1.93.0 build --release
  • Manual curl -s file://... vs target/release/rtk curl file://... check for both JSON and text fixtures

Note: a full local CI=1 scripts/benchmark.sh run still fails in this worktree because local warm-cache commands like cargo build produce tiny raw output and become existing unrelated negative cases. The curl rows specifically now report equal token counts instead of a negative result.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant