You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 7573428
Browse filesBrowse the repository at this point in the historyBrowse files
authored
Gate dry-run scans on added actionable findings (#69193)
* Gate dry-run scans on added actionable findings
Compare scanner findings against pre-compilation lock files, ignore low-severity findings, and apply the shared dry-run zizmor secrets-outside-env filter. Preserve scanner execution failures and normal compilation policy.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
* Fix dry-run scanner review findings
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
* Apply remaining changes
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
* Refresh validation issue schemas after main merge
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
* Verify baseline and current scans in dry-run report test
Assert both ShellCheck passes, their input paths, strict mode, and lock emission ordering after resource baselining added a pre-compilation scan.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy file name to clipboardExpand all lines: .github/aw/debug-agentic-workflow.md
+23-10Lines changed: 23 additions & 10 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -49,7 +49,7 @@ dispatch.
49
49
50
50
3.**Reproduce one boundary.** Choose compilation/prompt/MCP/agent/safe outputs. Run the real component with existing edge test doubles, minimal inputs, output assertions, disposable files and a timeout. Mock missing context/APIs/credentials. Never contact production or disable the firewall; leave OIDC, approvals, hosted tokens and runners unverified.
51
51
52
-
4.**Fix and gate.** Add a minimal fix/regression. Review the revised changes with the [agentic security-review instructions](debug-security-review.md), then compile current source with `--dry-run`: strict, staged, source validation, shellcheck and model checks, warnings as errors. Include emitted locks in the final security review before executing or uploading them. Docker-based checks are optional; Docker unavailability does not block the gate. When Docker is available, recommend `gh aw validate WORKFLOW` for source checks and `gh aw compile WORKFLOW --dry-run --zizmor --actionlint --poutine` for scanner checks. Run these scanners if possible; `validate` itself skips them. Unsupported `--dry-run`, missing required checks or failed compilation blocks live testing, not local diagnosis. Existing/generated locks need a successful gate for current hashes. Never downgrade required checks, suppress findings, substitute `strict: true`, use `--approve`, or commit to erase warnings.
52
+
4. **Fix and gate.** Add a minimal fix/regression. Review the revised changes with the [agentic security-review instructions](debug-security-review.md), then compile current source with `--dry-run`: strict, staged, source validation, shellcheck and model checks, with differential scanner gating described below. Include emitted locks in the final security review before executing or uploading them. Docker-based checks are optional; Docker unavailability does not block the gate. When Docker is available, recommend `gh aw validate WORKFLOW` for source checks and `gh aw compile WORKFLOW --dry-run --zizmor --actionlint --poutine` for scanner checks. Run these scanners if possible; `validate` itself skips them. Unsupported `--dry-run`, missing required checks or failed compilation blocks live testing, not local diagnosis. Existing/generated locks need a successful gate for current hashes. Never downgrade required checks, add ad hoc suppressions beyond the documented dry-run filter, substitute `strict: true`, use `--approve`, or commit to erase warnings.
53
53
54
54
5.**Constrain credentials.** Recommend, not require, a protected test environment and rotated, restricted development credentials. Inspect environments; provision only with authorization and revoke replaced credentials at their issuer. Never copy/rotate production secrets. Authorized repository/organization/enterprise shared secrets remain job-scoped, not automatically OS-exported; test environments do not isolate them. Keep production/dispatch credentials out of harnesses.
compileCmd.Flags().Bool("no-emit", false, "Validate workflow without generating lock files")
811
811
compileCmd.Flags().Bool("purge", false, "Delete .lock.yml files that were not regenerated during compilation (only when no specific files are provided)")
812
812
compileCmd.Flags().Bool("strict", false, "Override frontmatter to enforce strict mode validation for all workflows (enforces action pinning, network config, safe-outputs, disallows write permissions and deprecated fields). Note: Workflows default to strict mode unless frontmatter sets strict: false")
813
-
compileCmd.Flags().Bool("dry-run", false, "Compile with strict validation, shellcheck, and model checks; Docker-based scanners are opt-in; report gate/scanner coverage; disable compiler-managed GitHub mutations, stage safe outputs, and record dry_run in aw_info.json; custom scripts and external MCP effects remain unverified; does not authorize execution, upload, or dispatch workflows")
814
-
compileCmd.Flags().Bool("allow-experimental", false, "Accept experimental-feature notices during dry-run compilation; notices remain visible and all other warnings and scanner failures remain fatal")
813
+
compileCmd.Flags().Bool("dry-run", false, "Compile with strict validation, shellcheck, and model checks; gate added medium-or-higher scanner findings against pre-compilation locks; ignore low severity and zizmor secrets-outside-env; Docker-based scanners are opt-in; report gate/scanner coverage; disable compiler-managed GitHub mutations, stage safe outputs, and record dry_run in aw_info.json; custom scripts and external MCP effects remain unverified; does not authorize execution, upload, or dispatch workflows")
814
+
compileCmd.Flags().Bool("allow-experimental", false, "Record explicit acceptance of nonblocking experimental-feature notices during dry-run compilation; validation errors, added actionable findings, and scanner failures remain fatal")
815
815
compileCmd.Flags().String("environment", "", "Override the environment on every generated job; fails for reusable-workflow caller jobs, which cannot declare an environment")
816
816
compileCmd.Flags().Bool("require-self-hosted-runners", false, "Fail compilation unless every generated workflow job selects a self-hosted runner")
Copy file name to clipboardExpand all lines: docs/src/content/docs/reference/compilation-process.md
+27-11Lines changed: 27 additions & 11 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -229,7 +229,7 @@ Pre-activation runs gating checks sequentially before any AI execution. Any fail
229
229
| `gh aw compile my-workflow` | Compile specific workflow |
230
230
| `gh aw compile --verbose` | Enable verbose output |
231
231
| `gh aw compile --strict` | Enhanced security validation |
232
-
| `gh aw compile my-workflow --dry-run` | Compile for development testing with staging, mandatory validation/shellcheck/model checks, and warnings as errors; Docker scanners are opt-in |
232
+
| `gh aw compile my-workflow --dry-run` | Compile with staging and mandatory validation/shellcheck/model checks; gate added medium-or-higher scanner findings against pre-compilation locks; Docker scanners are opt-in |
233
233
| `gh aw compile my-workflow --environment gh-aw-debug` | Replace the environment on every generated job |
234
234
| `gh aw compile --no-emit` | Validate without generating files |
235
235
| `gh aw compile --actionlint --zizmor --poutine --grant` | Run security scanners |
@@ -260,9 +260,22 @@ pyflakes from `PATH` when those integrations are enabled; it does not install th
**`--dry-run` flag:** Compile with strict source validation, staged safe outputs, shellcheck, model checks, and warnings as errors. Docker-based image validation and scanners are opt-in. Without Docker, required run-step linting uses native shellcheck. Missing required checks or model inventory block compilation. Compiler-managed GitHub mutations are disabled, including push jobs, memory/cache persistence, work-queue operations, reusable safe-output calls, reactions, comments/issues, label removal, and issue locking. Emitted locks remove `OTEL_*` and `GH_AW_OTLP_*` variables from workflow, job, step, container, and service environment mappings and disable automatic OTLP configuration and authentication. The runtime records `dry_run` in `aw_info.json` and warns that custom scripts/jobs, agent shell commands, external MCP servers, and custom credentials remain unverified. Explicitly disabling a required check is rejected; `--no-emit`, `--watch`, `--approve`, and `--allow-action-refs` are incompatible. This flag does not upload or execute workflows and is distinct from `--action-mode dev`. The compile MCP input is `dry_run`; `validate_images: true` opts into required image validation. See [Development Testing Mode](/gh-aw/reference/compilation-process/#development-testing-mode).
392
+
**`--dry-run` flag:** Compile with strict source validation, staged safe outputs, shellcheck, and model checks. Scanner findings are compared against the existing on-disk locks before compilation: only added medium-or-higher or unclassified findings fail. Low-severity findings and zizmor's `secrets-outside-env` rule do not fail the dry-run gate. Docker-based image validation and scanners are opt-in. Without Docker, required run-step linting uses native shellcheck. Missing required checks or model inventory block compilation. Compiler-managed GitHub mutations are disabled, including push jobs, memory/cache persistence, work-queue operations, reusable safe-output calls, reactions, comments/issues, label removal, and issue locking. Emitted locks remove `OTEL_*` and `GH_AW_OTLP_*` variables from workflow, job, step, container, and service environment mappings and disable automatic OTLP configuration and authentication. The runtime records `dry_run` in `aw_info.json` and warns that custom scripts/jobs, agent shell commands, external MCP servers, and custom credentials remain unverified. Explicitly disabling a required check is rejected; `--no-emit`, `--watch`, `--approve`, and `--allow-action-refs` are incompatible. This flag does not upload or execute workflows and is distinct from `--action-mode dev`. The compile MCP input is `dry_run`; `validate_images: true` opts into required image validation. See [Development Testing Mode](/gh-aw/reference/compilation-process/#development-testing-mode).
393
393
394
394
Dry-run locks also omit daily credit accounting, retain existing per-run caps or promote configured/imported daily limits, and add manual dispatch restricted to maintainer/admin actors without bot exemptions. `DEBUG=workflow:compiler_development,cli:compile_development` reports mutations without configuration values.
395
395
@@ -405,7 +405,7 @@ Dry-run locks also omit daily credit accounting, retain existing per-run caps or
405
405
406
406
**JSON Output (`--json`):** Emits an array of `ValidationResult` objects. Each result includes a `labels` field listing all repository labels referenced in safe-outputs (`create-issue.labels`, `create-discussion.labels`, `create-pull-request.labels`, `add-labels.allowed`). Use `--json --no-emit` to collect label references without writing compiled files.
407
407
408
-
With `--dry-run`, workflow warnings invalidate only the affected workflow. Add `--allow-experimental` to accept built-in experimental-feature notices while keeping them visible; all other warnings, validation errors, and scanner failures remain fatal. The JSON dry-run summary records `accepted_experimental_warnings`. Compiler/scanner diagnostics that cannot be attributed to one workflow appear as separate results with `scope: "batch"`; their `workflow` field identifies the diagnostic source rather than a workflow path. Any invalid result fails development compilation, even when individual workflows remain valid.
408
+
With `--dry-run`, actionable workflow warnings invalidate only the affected workflow. Low-severity and built-in experimental-feature notices are nonblocking and remain visible; `--allow-experimental` additionally records explicit acknowledgement in `accepted_experimental_warnings`. Validation errors, unclassified compiler warnings, and scanner execution/parsing failures remain fatal. The JSON summary reports each scanner's baseline, existing, added, and ignored finding counts. Compiler/scanner diagnostics that cannot be attributed to one workflow appear as separate results with `scope: "batch"`; their `workflow` field identifies the diagnostic source rather than a workflow path. Any invalid result fails development compilation, even when individual workflows remain valid.
409
409
410
410
**Dependabot Integration (`--dependabot`):** Generates dependency manifests and `.github/dependabot.yml` by analyzing runtime tools across all workflows. See [Dependabot Support reference](/gh-aw/reference/dependabot/).
0 commit comments