Make Docker checks optional for dry-run compilation (#66591) · github/gh-aw@d94534d · GitHub
Skip to content

Commit d94534d

Browse files
pelikhanCopilot
andauthored
Make Docker checks optional for dry-run compilation (#66591)
Recommend validate and zizmor, actionlint, and poutine when Docker is available; preserve fail-closed checks when explicitly requested. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
1 parent eb79e49 commit d94534d

13 files changed

Lines changed: 193 additions & 40 deletions

‎.github/aw/cli-commands.md‎

Lines changed: 12 additions & 0 deletions

‎.github/aw/debug-agentic-workflow.md‎

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ URLs before replay/report. Never retrieve secrets.
3131

3232
3. **Reproduce one boundary.** Choose compilation/prompt/MCP/agent/safe outputs. Run the real component with existing edge test doubles, minimal inputs, output assertions, disposable files and a timeout. Mock missing context/APIs/credentials. Never contact production or disable the firewall; leave OIDC, approvals, hosted tokens and runners unverified.
3333

34-
4. **Fix and gate.** Add a minimal fix/regression. Compile current source with `--dry-run`: strict, staged, all validation/analysis, warnings as errors. Unsupported `--dry-run`, missing checks or failed compilation blocks live testing, not local diagnosis. Existing/generated locks need a successful gate for current hashes. Never downgrade, suppress findings, substitute `strict: true`, use `--approve`, or commit to erase warnings.
34+
4. **Fix and gate.** Add a minimal fix/regression. Compile current source with `--dry-run`: strict, staged, source validation, shellcheck and model checks, warnings as errors. Docker-based checks are optional; Docker unavailability does not block the gate. When Docker is available, recommend `gh aw validate WORKFLOW` for source checks and `gh aw compile WORKFLOW --dry-run --zizmor --actionlint --poutine` for scanner checks. Run these scanners if possible; `validate` itself skips them. Unsupported `--dry-run`, missing required checks or failed compilation blocks live testing, not local diagnosis. Existing/generated locks need a successful gate for current hashes. Never downgrade required checks, suppress findings, substitute `strict: true`, use `--approve`, or commit to erase warnings.
3535

3636
5. **Constrain credentials.** Recommend, not require, a protected test environment and rotated, restricted development credentials. Inspect environments; provision only with authorization and revoke replaced credentials at their issuer. Never copy/rotate production secrets. Authorized repository/organization/enterprise shared secrets remain job-scoped, not automatically OS-exported; test environments do not isolate them. Keep production/dispatch credentials out of harnesses.
3737

@@ -73,8 +73,22 @@ after explicitly authorized authentication/SSO repair and renewed live validatio
7373
gh aw compile WORKFLOW --dry-run
7474
# Optional reviewed test environment:
7575
gh aw compile WORKFLOW --dry-run --environment gh-aw-debug
76+
# Additional source validation; does not run scanners:
77+
gh aw validate WORKFLOW
78+
# Optional scanners on the emitted dry-run lock file, when Docker is available:
79+
gh aw compile WORKFLOW --dry-run --zizmor --actionlint --poutine
7680
```
7781

82+
Docker-based scanners and `--validate-images` are opt-in, not dry-run gate
83+
requirements. Run zizmor, actionlint and poutine when possible; report unavailable
84+
checks as unverified, not passed. Without Docker, use a native `shellcheck` binary
85+
for the required run-step linting. Findings or failures from requested checks still
86+
fail the dry-run gate.
87+
88+
`validate` uses `--no-emit`, which skips zizmor, actionlint and poutine. Only the
89+
scanner-enabled compile command above runs them; a passing `validate` command
90+
does not establish scanner coverage.
91+
7892
`--environment` replaces every lock-file job's environment, including approval,
7993
custom and framework jobs. Review changed protections before live testing.
8094
Reusable-workflow caller jobs cannot declare environments, so this override fails

‎.github/skills/debugging-workflows/SKILL.md‎

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -487,13 +487,27 @@ gh aw logs --firewall
487487
### Diagnostic and Development Compilation
488488

489489
```bash
490-
# Strict/staged development compilation with checks and warnings as errors
490+
# Strict/staged development compilation; Docker-based checks are optional
491491
gh aw compile <workflow> --dry-run
492492
493493
# Recommended when using a reviewed test environment
494494
gh aw compile <workflow> --dry-run --environment gh-aw-debug
495+
496+
# Additional source validation; does not run scanners
497+
gh aw validate <workflow>
498+
499+
# Run those scanners on the emitted dry-run lock file when possible
500+
gh aw compile <workflow> --dry-run --zizmor --actionlint --poutine
495501
```
496502

503+
Docker unavailability does not block the dry-run gate. Use native `shellcheck`
504+
for required run-step linting without Docker, and report unavailable optional
505+
checks as unverified.
506+
507+
When Docker is available, recommend both commands: `validate` checks source
508+
without emitting files, while the scanner-enabled compile command runs zizmor,
509+
actionlint and poutine. A successful `validate` does not establish scanner coverage.
510+
497511
## Reference Commands
498512

499513
### Log Analysis Commands

‎cmd/gh-aw/compile_flags_test.go‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,42 @@ func TestCompileDevelopmentRejectsExplicitFalseFlags(t *testing.T) {
7777
}
7878
}
7979

80+
func TestCompileDevelopmentAllowsOptionalDockerChecks(t *testing.T) {
81+
t.Parallel()
82+
for _, value := range []string{"false", "true"} {
83+
t.Run(value, func(t *testing.T) {
84+
t.Parallel()
85+
cmd := &cobra.Command{}
86+
cmd.Flags().Bool("dry-run", false, "")
87+
scanners := []string{
88+
"validate-images", "zizmor", "actionlint", "poutine", "runner-guard",
89+
"syft", "grype", "grant", "yamllint",
90+
}
91+
args := []string{"--dry-run"}
92+
for _, name := range scanners {
93+
cmd.Flags().Bool(name, false, "")
94+
args = append(args, "--"+name+"="+value)
95+
}
96+
if err := cmd.ParseFlags(args); err != nil {
97+
t.Fatal(err)
98+
}
99+
opts := getCompileCmdOptions(cmd)
100+
if err := cli.ValidateDevelopmentCompileFlags(opts.dryRun, opts.explicitBoolFlags); err != nil {
101+
t.Fatalf("Docker checks must remain optional in dry-run mode: %v", err)
102+
}
103+
config := opts.toCompileConfig(nil)
104+
for _, actual := range []bool{
105+
config.ValidateImages, config.Zizmor, config.Actionlint, config.Poutine,
106+
config.RunnerGuard, config.Syft, config.Grype, config.Grant, config.Yamllint,
107+
} {
108+
if actual != (value == "true") {
109+
t.Fatalf("optional Docker check value %q was not preserved", value)
110+
}
111+
}
112+
})
113+
}
114+
}
115+
80116
func TestCompileOptionsPropagateForceRefreshContainerPins(t *testing.T) {
81117
t.Parallel()
82118
config := (&compileCmdOptions{forceRefreshContainerPins: true}).toCompileConfig(nil)

‎cmd/gh-aw/main.go‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -809,7 +809,7 @@ func configureCompileBuildFlags() {
809809
compileCmd.Flags().Bool("no-emit", false, "Validate workflow without generating lock files")
810810
compileCmd.Flags().Bool("purge", false, "Delete .lock.yml files that were not regenerated during compilation (only when no specific files are provided)")
811811
compileCmd.Flags().Bool("strict", false, "Override frontmatter to enforce strict mode validation for all workflows (enforces action pinning, network config, safe-outputs, disallows write permissions and deprecated fields). Note: Workflows default to strict mode unless frontmatter sets strict: false")
812-
compileCmd.Flags().Bool("dry-run", false, "Compile with strict validation and all checks; disable compiler-managed GitHub mutations, stage safe outputs, and record dry_run in aw_info.json; custom scripts and external MCP effects remain unverified; does not upload or dispatch workflows")
812+
compileCmd.Flags().Bool("dry-run", false, "Compile with strict validation, shellcheck, and model checks; Docker-based scanners are opt-in; disable compiler-managed GitHub mutations, stage safe outputs, and record dry_run in aw_info.json; custom scripts and external MCP effects remain unverified; does not upload or dispatch workflows")
813813
compileCmd.Flags().String("environment", "", "Override the environment on every generated job; fails for reusable-workflow caller jobs, which cannot declare an environment")
814814
compileCmd.Flags().Bool("require-self-hosted-runners", false, "Fail compilation unless every generated workflow job selects a self-hosted runner")
815815
compileCmd.Flags().Bool("trial", false, "Enable trial mode compilation (modifies workflows for trial execution)")

‎docs/adr/66270-local-first-workflow-debugging-compile-modes.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ Only a hosted run exercises the real lock file, approvals, secrets, and runner b
1212

1313
### Decision
1414

15-
Use local compilation and bounded component reproduction as the first debugging steps. `gh aw compile --dry-run` enables strict validation, staging, image/model/analysis checks, and all scanners, rejecting bypass options and explicitly disabled required flags. Warnings become failures; workflow-local diagnostics remain attached to their workflow, while batch diagnostics are reported separately. Dry-run disables compiler-managed GitHub mutations, including push jobs, memory/cache persistence, reusable safe-output calls, work-queue operations, reactions, comments/issues, label removal, and issue locking. `aw_info.json` records `dry_run`. Diagnostic handlers, summaries, and run artifacts remain enabled. No memory-tool staging fields are added. Custom scripts/jobs, agent shell commands, external MCP servers, and custom credentials remain explicitly unverified rather than being rejected.
15+
Use local compilation and bounded component reproduction as the first debugging steps. `gh aw compile --dry-run` enables strict source validation, staging, shellcheck, and model checks, rejecting bypass options and explicitly disabled required flags. Docker-based scanners and required image validation remain opt-in so Docker unavailability does not block the gate; native shellcheck is required without Docker. When Docker is available, recommend `gh aw validate WORKFLOW` for source checks and `gh aw compile WORKFLOW --dry-run --zizmor --actionlint --poutine` for scanner checks. `validate` uses `--no-emit` and skips these scanners; its success does not establish scanner coverage. Run the scanners when possible. MCP callers opt into required image validation with `validate_images: true`. Unavailable optional checks remain unverified. Warnings and requested scanner failures become failures; workflow-local diagnostics remain attached to their workflow, while batch diagnostics are reported separately. Dry-run disables compiler-managed GitHub mutations, including push jobs, memory/cache persistence, reusable safe-output calls, work-queue operations, reactions, comments/issues, label removal, and issue locking. `aw_info.json` records `dry_run`. Diagnostic handlers, summaries, and run artifacts remain enabled. No memory-tool staging fields are added. Custom scripts/jobs, agent shell commands, external MCP servers, and custom credentials remain explicitly unverified rather than being rejected.
1616

1717
`gh aw compile --environment NAME` replaces the environment on every generated job, including approval jobs, with a validated literal name. Reusable-workflow caller jobs fail explicitly because GitHub Actions forbids an environment on them. Installed and embedded debugging instructions share `.github/aw/debug-agentic-workflow.md`, which unifies local diagnosis/patching, evidence triage, and bounded, human-validated live debugging. Neither flag authorizes workflow execution.
1818

@@ -34,7 +34,7 @@ The override is a debugging-time concern rather than a persistent source setting
3434

3535
#### Positive
3636

37-
- One flag (`--dry-run`) requests the complete development check set without dispatch. Local diagnostics do not replace hosted CI or runtime verification.
37+
- One flag (`--dry-run`) requests the required development check set without dispatch or a Docker dependency. Additional Docker-based checks are opt-in. Local diagnostics do not replace hosted CI or runtime verification.
3838
- Bypass options and explicitly disabled required checks are rejected; JSON diagnostics preserve failures without treating another workflow's warning as a local defect.
3939
- Compiler-managed GitHub mutations are disabled for dry-run testing; diagnostic evidence remains available.
4040
- `--environment` makes live verification explicitly scoped and reviewable, including on approval jobs, instead of relying on authors remembering to hand-edit lock files.

‎docs/src/content/docs/reference/compilation-process.md‎

Lines changed: 15 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -250,9 +250,8 @@ Pre-activation runs gating checks sequentially before any AI execution. Any fail
250250
251251
### Development Testing Mode
252252
253-
`--dry-run` enables `--strict`, `--staged`, `--validate`, `--validate-images`,
254-
`--actionlint`, `--zizmor`, `--poutine`, `--runner-guard`, `--syft`, `--grype`,
255-
`--grant`, `--yamllint`, `--shellcheck`, and `--models`. These settings are
253+
`--dry-run` enables `--strict`, `--staged`, `--validate`, `--shellcheck`,
254+
and `--models`. These settings are
256255
forced regardless of frontmatter; explicitly disabling a required flag is rejected. Compiler warnings,
257256
including safe-update warnings, and scanner failures cause a nonzero exit status
258257
and invalid JSON validation results. Missing required tools block compilation
@@ -261,6 +260,19 @@ model inventory; development compilation fails if that inventory is unavailable
261260
and the workflow declares `models` policies or `engine.models`. It does not prove
262261
live model availability.
263262
263+
Docker-based scanners and `--validate-images` remain opt-in. Docker unavailability
264+
does not block the dry-run gate; install native `shellcheck` for required run-step
265+
linting without Docker. When Docker is available, running `gh aw validate my-workflow`
266+
is recommended for source checks, alongside
267+
`gh aw compile my-workflow --dry-run --zizmor --actionlint --poutine` for scanner
268+
checks. `validate` uses `--no-emit` and skips these scanners; its success does not
269+
establish scanner coverage. Run the scanner-enabled compile command when possible
270+
and report unavailable checks as unverified. Explicitly requested checks
271+
still fail the dry-run gate on findings or scanner failures.
272+
273+
MCP callers can opt into required image validation with `validate_images: true`,
274+
including with `dry_run: true`; Docker unavailability then fails compilation.
275+
264276
```bash
265277
gh aw compile my-workflow --dry-run --environment gh-aw-debug
266278
```

‎docs/src/content/docs/setup/cli.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -345,7 +345,7 @@ Unlike `gh aw upgrade`, `gh aw compile` does not run codemods unless you pass `-
345345

346346
**Options:** `--action-mode`, `--action-tag`, `--actionlint`, `--actions-repo`, `--allow-action-refs`, `--approve`, `--dependabot`, `--dry-run`, `--dir/-d`, `--engine/-e`, `--environment`, `--fail-fast`, `--fix`, `--force/-f`, `--force-refresh-action-pins`, `--force-refresh-container-pins`, `--gh-aw-ref`, `--ghes`, `--grant`, `--grype`, `--json/-j`, `--logical-repo/-l`, `--models`, `--no-check-update`, `--no-emit`, `--poutine`, `--purge`, `--refresh-stop-time`, `--require-self-hosted-runners`, `--runner-guard`, `--schedule-seed`, `--shellcheck`, `--show-all`, `--staged`, `--stats`, `--strict`, `--syft`, `--trial`, `--validate`, `--validate-images`, `--watch/-w`, `--yamllint`, `--zizmor`
347347

348-
**`--dry-run` flag:** Compile with strict validation, staged safe outputs, all analysis/image/model checks, and warnings as errors. Missing checks or required model inventory block compilation. Compiler-managed GitHub mutations are disabled, including push jobs, memory/cache persistence, work-queue operations, reusable safe-output calls, reactions, comments/issues, label removal, and issue locking. The runtime records `dry_run` in `aw_info.json` and warns that custom scripts/jobs, agent shell commands, external MCP servers, and custom credentials remain unverified. Explicitly disabling a required check is rejected; `--no-emit`, `--watch`, `--approve`, and `--allow-action-refs` are incompatible. This flag does not upload or execute workflows and is distinct from `--action-mode dev`. The compile MCP input is `dry_run`. See [Development Testing Mode](/gh-aw/reference/compilation-process/#development-testing-mode).
348+
**`--dry-run` flag:** Compile with strict source validation, staged safe outputs, shellcheck, model checks, and warnings as errors. Docker-based image validation and scanners are opt-in. Without Docker, required run-step linting uses native shellcheck. Missing required checks or model inventory block compilation. Compiler-managed GitHub mutations are disabled, including push jobs, memory/cache persistence, work-queue operations, reusable safe-output calls, reactions, comments/issues, label removal, and issue locking. The runtime records `dry_run` in `aw_info.json` and warns that custom scripts/jobs, agent shell commands, external MCP servers, and custom credentials remain unverified. Explicitly disabling a required check is rejected; `--no-emit`, `--watch`, `--approve`, and `--allow-action-refs` are incompatible. This flag does not upload or execute workflows and is distinct from `--action-mode dev`. The compile MCP input is `dry_run`; `validate_images: true` opts into required image validation. See [Development Testing Mode](/gh-aw/reference/compilation-process/#development-testing-mode).
349349

350350
**`--environment NAME` flag:** Replace the environment on every compiled job, including approval and custom jobs. The name must be literal, non-blank, at most 255 characters, and contain no control characters or GitHub Actions expressions. Reusable-workflow caller jobs cause an error because GitHub Actions forbids an environment on them. Environment overrides do not isolate authorized repository/organization/enterprise secrets or grant execution permission; review replacement approval protections before testing.
351351

‎pkg/cli/compile_development.go‎

Lines changed: 1 addition & 11 deletions

0 commit comments

Comments
 (0)