build(nix): repackage publint for nix-update and auto-merge update PRs by ryoppippi · Pull Request #1317 · ccusage/ccusage · GitHub
Skip to content

build(nix): repackage publint for nix-update and auto-merge update PRs#1317

Open
ryoppippi wants to merge 6 commits into
mainfrom
nix-update
Open

build(nix): repackage publint for nix-update and auto-merge update PRs#1317
ryoppippi wants to merge 6 commits into
mainfrom
nix-update

Conversation

@ryoppippi

@ryoppippi ryoppippi commented Jun 14, 2026

Copy link
Copy Markdown
Member

Summary

Repackages the pinned publint CLI so it can be updated by nix-update,
adds a one-command update path, and rewires the scheduled update workflow
so its PRs actually run CI and auto-merge.

What changed

publint packaging

  • Replace the hand-written inlined pnpm-lock.yaml in nix/publint.nix
    with buildNpmPackage + a committed nix/publint-package-lock.json
    generated from a thin wrapper package. The runtime closure (publint +
    its 5 deps) is unchanged; version is now the single source of truth.
  • Exclude the generated lockfile from oxfmt in nix/treefmt.nix:
    npmDepsHash is computed against the lockfile bytes, so reformatting it
    breaks the buildNpmPackage consistency check.
  • Add nix-update to the dev shell and a just update-publint [<version>]
    recipe that regenerates the lockfile and bumps version + npmDepsHash.

update automation

  • Add a setup-git-bot composite action that mints a token from the
    ccusage org updater GitHub App.
  • Add a update-publint job to the update pricing workflow.
  • Drive the litellm / models.dev / publint jobs with the org App token
    (CCUSAGE_UPDATER_APP_ID / _PRIVATE_KEY) instead of the default
    GITHUB_TOKEN, and enable squash auto-merge.

Why

nix-update alone cannot regenerate the npm lockfile through the
runCommand indirection, so the recipe wraps both steps into one entry
point shared by maintainers and CI. PRs opened with the default
GITHUB_TOKEN do not trigger on: pull_request workflows, so the
automation PRs never ran CI and could not merge hands-off; an App token
fixes that.

Follow-up required before the App jobs work

  1. Create the ccusage org updater GitHub App (Contents + Pull requests:
    read & write) and install it on ccusage/ccusage.
  2. Add repo secrets CCUSAGE_UPDATER_APP_ID and
    CCUSAGE_UPDATER_APP_PRIVATE_KEY.
  3. Add branch protection with required status checks on main so
    auto-merge waits for green.

Testing

  • nix build .#publintpublint, 0.3.12
  • just update-publint 0.3.21 → builds; reverted to 0.3.12
  • nix flake check (treefmt / publint check), actionlint, typos

View with Codesmith
Need help on this PR? Tag /codesmith with what you need. Autofix is enabled.


Summary by cubic

Repackages the pinned publint CLI with buildNpmPackage so it can be updated via nix-update, and hardens the scheduled updaters to use an App token, run CI, and auto-merge with proper error handling.

  • Refactors

    • Switched publint to buildNpmPackage with a committed nix/publint-package-lock.json; version is the single source of truth and the runtime closure is unchanged.
    • Added just update-publint [<version>] and included nix-update in the dev shell to regenerate the lockfile and bump version + npmDepsHash.
    • Excluded nix/publint-package-lock.json from oxfmt to keep npmDepsHash stable.
    • Hardened updater workflows: scoped the bot token to checkout steps, set the authenticated push URL only when creating PRs, and fail on auth/API/branch-protection errors (only warn when auto-merge isn’t enabled).
  • Migration

    • Create and install the org GitHub App on the repo (Contents + Pull requests: read/write).
    • Add repo secrets CCUSAGE_UPDATER_APP_ID and CCUSAGE_UPDATER_APP_PRIVATE_KEY.
    • Enable required status checks on main so squash auto-merge can trigger.

Written for commit 1d06090. Summary will update on new commits.

Review in cubic

Summary by CodeRabbit

  • Chores
    • Updated CI automation to use a dedicated bot identity/token for making and merging update PRs.
    • Added an automated publint update workflow job with PR creation/refresh and auto-merge attempt.
    • Introduced a new update-publint task to bump publint and regenerate the pinned lockfile/flake inputs.
    • Enhanced the dev environment with additional build tooling (nix-update).
    • Switched publint packaging to be driven by the npm lockfile and excluded that lockfile from formatter changes.

The previous derivation embedded a hand-written pnpm-lock.yaml inside a
runCommand, so updating publint meant editing the inlined lockfile by
hand and it was not consumable by nix-update.

Repackage it with buildNpmPackage and a committed package-lock.json
generated from a thin wrapper package that depends on the pinned
release. The whole runtime closure now lives in one lockfile plus a
single npmDepsHash, and `version` is the single source of truth that
feeds the wrapper's dependency entry.

Exclude the generated lockfile from oxfmt: npmDepsHash is computed
against the lockfile bytes, and reformatting it to tabs breaks the
buildNpmPackage consistency check. The CLI closure (publint plus its
five runtime deps) is unchanged.
Add nix-update to the dev shell and a `just update-publint [<version>]`
recipe that regenerates the committed lockfile, then bumps the version
and npmDepsHash in nix/publint.nix via nix-update.

nix-update alone cannot regenerate the npm lockfile through the
runCommand indirection, so the recipe wraps both steps into one entry
point shared by maintainers and CI.
Add an update-publint job to the update-pricing workflow, chained after
the models.dev job and mirroring the existing litellm/models.dev jobs:
run `just update-publint`, skip when unchanged, validate with
`just check`, and open or update a PR on automation/publint.
Add a composite action that mints an installation token from the
ccusage org updater GitHub App and configures the matching bot git
identity. Ported from the dotfiles updater so automated jobs can act as
the App rather than the default GITHUB_TOKEN.
Drive the litellm, models.dev and publint update jobs with the ccusage
org App token (CCUSAGE_UPDATER_APP_ID / _PRIVATE_KEY) instead of the
default GITHUB_TOKEN, and enable squash auto-merge on each PR.

PRs opened with the default GITHUB_TOKEN do not trigger on:pull_request
workflows, so the automation PRs never ran CI and could not be merged
hands-off. Using an App token makes CI run on the PR; auto-merge then
lands it once required checks pass (configure branch protection on main
for the gate to take effect).
@pullfrog

pullfrog Bot commented Jun 14, 2026

Copy link
Copy Markdown
Contributor

@coderabbitai

coderabbitai Bot commented Jun 14, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: d629569d-2e9c-4794-9f4b-7e98d390577c

📥 Commits

Reviewing files that changed from the base of the PR and between ce4dfb6 and 1d06090.

📒 Files selected for processing (1)
  • .github/workflows/update-pricing.yaml
🚧 Files skipped from review as they are similar to previous changes (1)
  • .github/workflows/update-pricing.yaml

📝 Walkthrough

Walkthrough

Migrates the publint Nix derivation from pnpm to an npm lockfile–based buildNpmPackage wrapper, introduces a committed nix/publint-package-lock.json, and adds a just update-publint recipe. A new reusable setup-git-bot GitHub Actions composite action generates a bot token; existing pricing automation jobs and a new update-publint CI job are wired to use it with auto-merge support.

Changes

publint Nix derivation migration and local tooling

Layer / File(s) Summary
publint derivation: pnpm → npm lockfile
nix/publint.nix, nix/publint-package-lock.json
Replaces fetchPnpmDeps/pnpm_11/stdenvNoCC parameters with buildNpmPackage; source now generates a minimal package.json and copies the committed lockfile; npmDepsHash updated; install and wrapper phase retained. Lockfile pins publint@0.3.12 and transitive deps with integrity hashes.
Dev shell, formatter exclusion, and update recipe
nix/dev-shell.nix, nix/treefmt.nix, justfile
Adds nix-update to dev shell buildInputs. Excludes nix/publint-package-lock.json from the oxfmt formatter with explanatory comments. Adds update-publint just recipe that pins a version, regenerates the npm lockfile in a temp dir, copies it into nix/, runs nix-update, and formats via nix fmt.

GitHub Actions bot token and update-publint CI job

Layer / File(s) Summary
setup-git-bot composite action
.github/actions/setup-git-bot/action.yml
New composite action with app-id and private-key required inputs. Generates an installation token via actions/create-github-app-token, configures git user.name/user.email from gh api, and exposes the token as an output.
Bot token wiring in existing pricing jobs
.github/workflows/update-pricing.yaml
update-pricing and update-models-dev-pricing jobs add a pre-checkout for local actions, run setup-git-bot, re-checkout using the bot token output, switch GH_TOKEN on PR creation steps to the bot token, and add gh pr merge --auto --squash with a warning fallback when auto-merge is unavailable.
New update-publint CI job
.github/workflows/update-pricing.yaml
New job following the same bot-token pattern: runs just update-publint, skips PR creation if nix/publint.nix and nix/publint-package-lock.json are unchanged, validates via just check, creates/updates a PR on automation/publint, and attempts auto-merge.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

  • ccusage/ccusage#1312: Adds publint as a Nix flake check and Nix-provided tooling — directly connected to this PR's migration of nix/publint.nix from pnpm to npm and the new automated update workflow.

Poem

🐇 Hoppity-hop, from pnpm I flee,
npm lockfiles now set publint free!
A bot gets its token, its name, and its key,
Auto-merging PRs while I sip on some tea.
The rabbit types fast — let CI agree! 🎉

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately captures the main changes: repackaging publint for Nix with nix-update support and enabling auto-merge for update PRs.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch nix-update

Comment @coderabbitai help to get the list of available commands and usage tips.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jun 14, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
ccusage-guide 1d06090 Commit Preview URL

Branch Preview URL
Jun 14 2026, 07:02 PM

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/update-pricing.yaml:
- Line 86: The `gh pr merge` command is using a blanket `|| echo
"::warning::..."` that suppresses all failures including authentication errors,
API errors, and branch protection mismatches, allowing the job to pass despite
broken automation. Replace the blanket error suppression with conditional logic
that only suppresses the warning when auto-merge is specifically not enabled as
a feature, while allowing other error conditions (auth failures, API errors,
etc.) to fail the job as they should. This fix needs to be applied at all three
locations where the `gh pr merge` command with auto-merge is called.
- Around line 27-31: The checkout action is persisting the bot token in
`.git/config` which unnecessarily expands the token exposure surface. Remove the
token parameter from the checkout action that uses the bot token output, which
will prevent storing credentials in git config. Instead, authenticate the push
operations only where needed by passing the token directly to the git push
commands in each "Create pull request" step, limiting token exposure to only the
operations that actually require authentication.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: a31b293b-dd3d-4a93-a257-dac2fb847a46

📥 Commits

Reviewing files that changed from the base of the PR and between 92c518d and ce4dfb6.

📒 Files selected for processing (7)
  • .github/actions/setup-git-bot/action.yml
  • .github/workflows/update-pricing.yaml
  • justfile
  • nix/dev-shell.nix
  • nix/publint-package-lock.json
  • nix/publint.nix
  • nix/treefmt.nix

Comment thread .github/workflows/update-pricing.yaml
Comment thread .github/workflows/update-pricing.yaml Outdated
@pkg-pr-new

pkg-pr-new Bot commented Jun 14, 2026

Copy link
Copy Markdown

Open in StackBlitz

ccusage

npx https://pkg.pr.new/ccusage@1317

@ccusage/ccusage-darwin-arm64

npx https://pkg.pr.new/@ccusage/ccusage-darwin-arm64@1317

@ccusage/ccusage-darwin-x64

npx https://pkg.pr.new/@ccusage/ccusage-darwin-x64@1317

@ccusage/ccusage-linux-arm64

npx https://pkg.pr.new/@ccusage/ccusage-linux-arm64@1317

@ccusage/ccusage-linux-x64

npx https://pkg.pr.new/@ccusage/ccusage-linux-x64@1317

@ccusage/ccusage-win32-x64

npx https://pkg.pr.new/@ccusage/ccusage-win32-x64@1317

commit: 1d06090

@github-actions

Copy link
Copy Markdown
Contributor

ccusage performance comparison

PR SHA: ce4dfb637029
Base SHA: 92c518dfed4e

This compares the Rust PR release binary against the configured base package on the same CI runner.

Package runtime diagnostics

Compares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2597 files)
All rows run --offline --json, measured by hyperfine with 0 warmups and 1 runs. This isolates wrapper overhead from the installed native optional dependency and the workspace release binary built on the runner.

Command Runtime Input Median Throughput Samples
claude --offline --json Package wrapper 1.01 GiB 255.3ms 3.94 GiB/s 1
claude --offline --json Installed native binary 1.01 GiB 225.9ms 4.46 GiB/s 1
codex --offline --json Package wrapper 1.01 GiB 101.2ms 9.95 GiB/s 1
codex --offline --json Installed native binary 1.01 GiB 82.6ms 12.19 GiB/s 1

Committed fixture performance

Committed small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage.

Fixtures: Claude apps/ccusage/test/fixtures/claude (0.00 MiB, 2 files), Codex apps/ccusage/test/fixtures/codex (0.00 MiB, 1 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published native ccusage binary from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 2 warmups and 7 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude daily --offline --json 0.00 MiB 23.6ms 4.0ms 5.84x 53.75 MiB 10.19 MiB 0.19x 0.07 MiB/s 0.38 MiB/s
claude session --offline --json 0.00 MiB 25.2ms 3.6ms 7.03x 53.75 MiB 10.20 MiB 0.19x 0.06 MiB/s 0.43 MiB/s
codex daily --offline --json 0.00 MiB 25.8ms 2.1ms 12.17x 53.50 MiB 8.18 MiB 0.15x 0.03 MiB/s 0.40 MiB/s
codex session --offline --json 0.00 MiB 25.1ms 2.1ms 11.99x 53.75 MiB 8.18 MiB 0.15x 0.03 MiB/s 0.41 MiB/s

Large real-world-shaped fixture performance

Generated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2597 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published native ccusage binary from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude --offline --json 1.01 GiB 268.8ms 234.4ms 1.15x 946.32 MiB 948.32 MiB 1.00x 3.75 GiB/s 4.30 GiB/s
codex --offline --json 1.01 GiB 101.4ms 82.2ms 1.23x 419.03 MiB 427.04 MiB 1.02x 9.93 GiB/s 12.25 GiB/s

Artifact size

Artifact Base PR Delta Ratio
packed ccusage-*.tgz 18.08 KiB 18.08 KiB -0.00 KiB 1.00x
installed native package binary 3851.06 KiB 3851.06 KiB +0.00 KiB 1.00x

Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees.

@github-actions

Copy link
Copy Markdown
Contributor

ccusage performance comparison

PR SHA: ce4dfb637029
Base SHA: 92c518dfed4e

This compares the PR package against the configured base package on the same CI runner.

Package runtime diagnostics

Compares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2597 files)
All rows run --offline --json, measured by hyperfine with 0 warmups and 1 runs. This isolates wrapper overhead from the installed native optional dependency and the workspace release binary built on the runner.

Command Runtime Input Median Throughput Samples
claude --offline --json Package wrapper 1.01 GiB 242.9ms 4.15 GiB/s 1
claude --offline --json Installed native binary 1.01 GiB 220.5ms 4.57 GiB/s 1
codex --offline --json Package wrapper 1.01 GiB 102.8ms 9.80 GiB/s 1
codex --offline --json Installed native binary 1.01 GiB 99.4ms 10.13 GiB/s 1

Committed fixture performance

Committed small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage.

Fixtures: Claude apps/ccusage/test/fixtures/claude (0.00 MiB, 2 files), Codex apps/ccusage/test/fixtures/codex (0.00 MiB, 1 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published ccusage package from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 2 warmups and 7 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude daily --offline --json 0.00 MiB 25.7ms 27.9ms 0.92x 53.75 MiB 53.50 MiB 1.00x 0.06 MiB/s 0.06 MiB/s
claude session --offline --json 0.00 MiB 26.1ms 23.1ms 1.13x 53.50 MiB 53.75 MiB 1.00x 0.06 MiB/s 0.07 MiB/s
codex daily --offline --json 0.00 MiB 24.0ms 23.8ms 1.01x 54.00 MiB 53.50 MiB 0.99x 0.04 MiB/s 0.04 MiB/s
codex session --offline --json 0.00 MiB 22.7ms 22.9ms 0.99x 53.50 MiB 54.00 MiB 1.01x 0.04 MiB/s 0.04 MiB/s

Large real-world-shaped fixture performance

Generated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2597 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published ccusage package from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude --offline --json 1.01 GiB 262.1ms 245.3ms 1.07x 944.33 MiB 934.32 MiB 0.99x 3.84 GiB/s 4.10 GiB/s
codex --offline --json 1.01 GiB 106.4ms 104.3ms 1.02x 415.04 MiB 429.04 MiB 1.03x 9.46 GiB/s 9.65 GiB/s

Artifact size

Artifact Base PR Delta Ratio
packed ccusage-*.tgz 18.08 KiB 18.08 KiB -0.00 KiB 1.00x
installed native package binary 3851.06 KiB 3851.06 KiB +0.00 KiB 1.00x

Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 issues found across 7 files

Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic

Comment thread .github/workflows/update-pricing.yaml Outdated
Comment thread .github/workflows/update-pricing.yaml
Add persist-credentials: false to the bot-token checkouts so the token is
not left in .git/config across later steps, and set the authenticated
push URL explicitly inside each 'Create pull request' step where it is
actually needed.

Replace the blanket 'gh pr merge ... || echo' with a check that only
emits the auto-merge warning when the failure message mentions
auto-merge; auth, API, and branch-protection errors now fail the job
instead of being silently dropped.

Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown
Contributor

ccusage performance comparison

PR SHA: 1d0609083029
Base SHA: 92c518dfed4e

This compares the Rust PR release binary against the configured base package on the same CI runner.

Package runtime diagnostics

Compares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2597 files)
All rows run --offline --json, measured by hyperfine with 0 warmups and 1 runs. This isolates wrapper overhead from the installed native optional dependency and the workspace release binary built on the runner.

Command Runtime Input Median Throughput Samples
claude --offline --json Package wrapper 1.01 GiB 238.2ms 4.23 GiB/s 1
claude --offline --json Installed native binary 1.01 GiB 221.0ms 4.56 GiB/s 1
codex --offline --json Package wrapper 1.01 GiB 102.1ms 9.86 GiB/s 1
codex --offline --json Installed native binary 1.01 GiB 78.8ms 12.78 GiB/s 1

Committed fixture performance

Committed small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage.

Fixtures: Claude apps/ccusage/test/fixtures/claude (0.00 MiB, 2 files), Codex apps/ccusage/test/fixtures/codex (0.00 MiB, 1 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published native ccusage binary from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 2 warmups and 7 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude daily --offline --json 0.00 MiB 27.1ms 3.9ms 7.02x 54.00 MiB 10.20 MiB 0.19x 0.06 MiB/s 0.40 MiB/s
claude session --offline --json 0.00 MiB 24.5ms 2.5ms 9.64x 53.50 MiB 10.20 MiB 0.19x 0.06 MiB/s 0.61 MiB/s
codex daily --offline --json 0.00 MiB 24.7ms 2.1ms 11.96x 53.50 MiB 8.18 MiB 0.15x 0.03 MiB/s 0.42 MiB/s
codex session --offline --json 0.00 MiB 23.4ms 2.1ms 11.07x 54.00 MiB 8.18 MiB 0.15x 0.04 MiB/s 0.40 MiB/s

Large real-world-shaped fixture performance

Generated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2597 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published native ccusage binary from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude --offline --json 1.01 GiB 269.8ms 223.7ms 1.21x 956.32 MiB 942.32 MiB 0.99x 3.73 GiB/s 4.50 GiB/s
codex --offline --json 1.01 GiB 101.4ms 80.1ms 1.27x 431.04 MiB 425.04 MiB 0.99x 9.93 GiB/s 12.56 GiB/s

Artifact size

Artifact Base PR Delta Ratio
packed ccusage-*.tgz 18.08 KiB 18.08 KiB -0.00 KiB 1.00x
installed native package binary 3851.06 KiB 3851.00 KiB -0.06 KiB 1.00x

Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees.

@github-actions

Copy link
Copy Markdown
Contributor

ccusage performance comparison

PR SHA: 1d0609083029
Base SHA: 92c518dfed4e

This compares the PR package against the configured base package on the same CI runner.

Package runtime diagnostics

Compares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2597 files)
All rows run --offline --json, measured by hyperfine with 0 warmups and 1 runs. This isolates wrapper overhead from the installed native optional dependency and the workspace release binary built on the runner.

Command Runtime Input Median Throughput Samples
claude --offline --json Package wrapper 1.01 GiB 251.4ms 4.00 GiB/s 1
claude --offline --json Installed native binary 1.01 GiB 231.4ms 4.35 GiB/s 1
codex --offline --json Package wrapper 1.01 GiB 105.1ms 9.58 GiB/s 1
codex --offline --json Installed native binary 1.01 GiB 88.5ms 11.38 GiB/s 1

Committed fixture performance

Committed small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage.

Fixtures: Claude apps/ccusage/test/fixtures/claude (0.00 MiB, 2 files), Codex apps/ccusage/test/fixtures/codex (0.00 MiB, 1 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published ccusage package from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 2 warmups and 7 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude daily --offline --json 0.00 MiB 25.2ms 25.0ms 1.01x 53.50 MiB 53.75 MiB 1.00x 0.06 MiB/s 0.06 MiB/s
claude session --offline --json 0.00 MiB 26.8ms 23.7ms 1.13x 53.75 MiB 53.50 MiB 1.00x 0.06 MiB/s 0.07 MiB/s
codex daily --offline --json 0.00 MiB 22.8ms 22.9ms 1.00x 53.50 MiB 53.50 MiB 1.00x 0.04 MiB/s 0.04 MiB/s
codex session --offline --json 0.00 MiB 24.3ms 23.6ms 1.03x 53.75 MiB 53.50 MiB 1.00x 0.04 MiB/s 0.04 MiB/s

Large real-world-shaped fixture performance

Generated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2597 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published ccusage package from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude --offline --json 1.01 GiB 269.2ms 255.6ms 1.05x 954.33 MiB 948.33 MiB 0.99x 3.74 GiB/s 3.94 GiB/s
codex --offline --json 1.01 GiB 104.2ms 107.5ms 0.97x 419.04 MiB 427.04 MiB 1.02x 9.66 GiB/s 9.36 GiB/s

Artifact size

Artifact Base PR Delta Ratio
packed ccusage-*.tgz 18.08 KiB 18.08 KiB -0.00 KiB 1.00x
installed native package binary 3851.06 KiB 3851.00 KiB -0.06 KiB 1.00x

Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant