Dependabot triage should avoid agent jobs when no work exists · Issue #14749 · cli/cli · GitHub
Skip to content

Dependabot triage should avoid agent jobs when no work exists #14749

Closed

Description

@williammartin

Describe the feature or problem you’d like to solve

Follow on from #14739

Currently, the hourly Dependabot triage workflow starts its generated agent job even when its deterministic worklist is empty.

The pre-flight step correctly found that #14716 had already been assessed, produced an empty worklist, and caused zero model invocations.

$ tmp=$(mktemp -d); gh aw audit 39137237820 --repo cli/cli --artifacts usage,agent -o "$tmp" >/dev/null; run_dir=$(find "$tmp" -type f -name audit.json -exec dirname {} \; | head -1); grep -E 'PR #[0-9]+: (already assessed|needs assessment)|Work list:' "$run_dir/workflow-logs/3_agent.txt" | sed -E 's/^.*Z //' | tail -2; jq '{working_set:.working_set}' "$run_dir/usage/activity/summary.json"; rm -rf "$tmp"
PR #14716: already assessed at 35c2bf381bb42655fb40ec8ebf2e0a0548ef7543, skipping
Work list: 0 PR(s) -> /tmp/gh-aw/dependabot-worklist.json
{
  "working_set": {
    "cumulative_input_tokens": 0,
    "peak_input_tokens": 0,
    "invocations": 0
  }
}

However, gh-aw records the job as started before its harness observes the noop. The daily guardrail then treats the empty accounting file as unknown and applies the max-ai-credits fallback.

Five consecutive no-work runs were each counted as 1000 AIC, which exhausted the 5000 daily guardrail.

$ gh run view 39141727797 --repo cli/cli --log | grep -E 'Computed run AIC.*(39137237820|39132745345|39128819878|39124342426|39120364428)|Completed AIC inspection window' | sed -E 's/^activation[[:space:]]+UNKNOWN STEP[[:space:]]+//'
[daily-workflow-aic] Computed run AIC: {"runId":39137237820,"aic":1000,"reason":"max_ai_credits_fallback"}
[daily-workflow-aic] Computed run AIC: {"runId":39132745345,"aic":1000,"reason":"scan_cache"}
[daily-workflow-aic] Computed run AIC: {"runId":39128819878,"aic":1000,"reason":"scan_cache"}
[daily-workflow-aic] Computed run AIC: {"runId":39124342426,"aic":1000,"reason":"scan_cache"}
[daily-workflow-aic] Computed run AIC: {"runId":39120364428,"aic":1000,"reason":"scan_cache"}
[daily-workflow-aic] Completed AIC inspection window: {"currentAIC":5000,"threshold":5000,"exceeded":true}

The generated workflow marks execution as started immediately before entering the harness here. The harness can then exit without invoking the model.

Proposed solution

Split scheduling and agent execution into separate workflows.

An ordinary GitHub Actions workflow should run hourly and perform the existing PR listing, CI readiness, and assessed-head checks. It should dispatch dependabot-triage with its existing pr_number input only when a PR needs assessment.

The expected output for this issue is:

  • No agentic workflow run is created when no Dependabot PR needs assessment.
  • Each ready, unassessed Dependabot PR is dispatched for triage.
  • The existing once-per-head-SHA behaviour remains unchanged.
  • The max-daily-ai-credits guardrail remains enabled for genuine agent runs.

Additional context

Lowering max-ai-credits, reducing the schedule frequency, or raising the daily limit would reduce the impact, but none fixes the false accounting path.

The underlying gh-aw behaviour should still be reported upstream. This repository-side split prevents that behaviour from blocking Dependabot triage while preserving the guardrail.

Activity

  1. cli-triage commented on Oct 19, 2026

    @cli-triage

    Classification: Enhancement

    This is a clear, well-evidenced improvement request for the repo's own dependabot-triage agentic workflow (not a gh CLI command, so no command label applies). The reporter shows via gh aw audit logs that a run with an empty worklist (0 model invocations) still gets accounted as the max-ai-credits fallback (1000 AIC) by the daily guardrail, and that five such no-work runs exhausted the 5000 AIC daily cap — consistent with the earlier failure in #14739. The proposed fix (splitting the hourly schedule/pre-flight check into a plain Actions workflow that only workflow_dispatches the agent workflow when a PR actually needs assessment) is concrete and testable.

    Supporting source evidence: the generated workflow writes a "state":"started" execution-evidence marker and begins the Copilot harness invocation before the harness can observe that the worklist is empty — see dependabot-triage.lock.yml lines 940-948 at fa92838be. This supports the reporter's premise that the current single-workflow design can't distinguish "no work" from a genuine (but cheap) agent run for AIC accounting purposes.

    Suggesting enhancement. No duplicates found in open or closed issues; #14739 is a related failure instance (daily AIC guardrail tripped) but not a duplicate of this structural fix request.

    Generated by Issue Triage (skills-driven) for #14749 · copilot · auto · 43.1 AIC · ⌖ 2.06 AIC · ⊞ 9.7K · ◷

  2. added
    tech-debtA chore that addresses technical debt
    and removed on Oct 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    tech-debtA chore that addresses technical debt

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions