Repository navigation
Add extensible AST linting and secret-leak taint analysis - #69813
Conversation
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅
|
|
✅ Ponytail Reviewer completed successfully! Ponytail ReviewerOutcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
|
✅ Test Quality Sentinel completed test quality analysis. Test Quality SentinelOutcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
|
✅ Design Decision Gate 🏗️ completed the design decision gate check. See the comment below for the result and any generated ADR draft. Commit pushed:
|
|
✅ PR Code Quality Reviewer completed the code quality review.
|
🏗️ Design Decision Gate — ADR RequiredThis PR adds 2,073 new lines to core business logic under It adds three things that need a recorded decision: a new in-process lint rule registry ( 📄 Draft ADR committed:
📋 What to do next
Once an ADR is linked in the PR body, this gate will re-run and check that the implementation matches the decision. 📋 Michael Nygard ADR Format ReferenceAn ADR must contain these four sections to be considered complete:
ADRs are stored in
|
Comment MemoryPeek at saved memory (pr-code-quality-reviewer)Note This comment is managed by comment memory.Expand the saved memory block to view or edit the persistent context for this thread.
|
There was a problem hiding this comment.
Request changes
The new taint-linting framework still has a few correctness regressions that I would block on: env subcommands are modeled as log dumps, the add-mask shortcut can suppress a real disclosure in the same command, and a malformed companion .md now aborts linting of an otherwise-valid .lock.yml.
Blocking themes
- The shell taint model currently introduces both false positives and false negatives around command handling.
- Lock-file linting is now coupled too tightly to companion source parsing, which regresses normal local iteration on authored workflows.
- Static review only: I did not execute repository code, tests, or builds.
🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt54 · 126.6 AIC · ⌖ 6.14 AIC · ⊞ 15.9K
Comment /review to run again
| } | ||
| } | ||
| case "env", "printenv": | ||
| if len(cmd.Args) == 1 || name == "env" { |
There was a problem hiding this comment.
Treating every env ... invocation as a dump of the full environment is incorrect: env FOO=... cmd executes cmd, it does not print the environment, so this will generate false-positive log leaks and miss the real downstream sink semantics.
💡 Why this matters
The condition len(cmd.Args) == 1 || name == "env" is always true for env, which means any secret passed through env is reported as a log disclosure even when the command is something like env TOKEN=$TOKEN curl --oauth2-bearer .... That is noisy, and worse, it prevents the analyzer from modeling the actual subcommand (for example curl) as a network sink.
Suggested fix:
case "env":
if len(cmd.Args) == 1 {
// dump environment
...
break
}
// skip leading NAME=VALUE assignments / flags, then analyze the subcommand
case "printenv":
...At minimum, split env from printenv and only treat bare env as stdout output.
There was a problem hiding this comment.
env now runs the wrapped command with a temporary environment overlay; it only models environment listing when no command is supplied and reports a coverage gap for unmodeled credential-bearing commands. Regression cases pass (2afe4c3).
| if readErr == nil { | ||
| document, parseErr := aw.Parse(string(source)) | ||
| if parseErr != nil { | ||
| return nil, errors.New("could not parse companion AW document") |
There was a problem hiding this comment.
Hard-failing the lint run when a sibling companion .md does not parse makes gh aw lint unusable for otherwise-valid .lock.yml files during source edits and for lock files that are linted without a trustworthy authoring source.
💡 Why this matters
This command is documented as linting existing lock files from disk. After this change, a transient parse error in foo.md aborts linting of foo.lock.yml entirely, even though the compiled workflow itself may be perfectly valid and actionlint could still provide useful signal. That is a pretty rough regression for local iteration and for consumers who only care about the execution graph.
Suggested fix: treat companion-document parse failures as a warning or coverage gap, then continue linting the lock file. Reserve a hard error for cases where the user explicitly asks to lint authored AW sources rather than compiled lock files.
There was a problem hiding this comment.
A malformed companion AW document is now ignored as unavailable source evidence, while linting continues on the valid Actions AST. The regression test verifies this (9787c5a).
Implement observable CTR-021 trigger selection checks with AW-aware severity, deterministic value-free AST-path diagnostics, cancellation, and parsed/constructed AST tests. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…-secret-leak-analysis
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Inspect direct executable interpolation across shell variants while preserving safe environment expansion. Reuse the Actions parser through a generic value-free text taint API. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Preserve explicit zero job timeouts and lint action pin integrity and explicit job timeout safety with value-free, source-path diagnostics. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Skills-Based Review 🧠
Applied /tdd and /codebase-design (via pr-triage: new_feature). This is a well-structured addition — a clean, symbolic taint-tracking engine (sources/sinks/filters) layered over a new extensible lint-rule registry, with solid test coverage across Bash/sh propagation, Actions expressions, job/env/output propagation, reusable workflow inputs, and 344 real compiled workflows (repository_workflows_test.go). I verified the claimed Gemini API-key regression fix (daily-model-inventory) against the actual committed HEAD and confirmed go build/go test pass cleanly for pkg/workflow/taint, pkg/workflow/lint, and the CLI integration.
📋 Key Themes & Highlights
Key Themes
- One correctness gap found:
artifactContainsinshell.gorelies onfilepath.Match, which doesn't support**as "match any depth" — a commonupload-artifactpattern likedist/**will miss nested files, creating a false negative in the exact kind of leak this PR is designed to catch. Flagged inline with a reproduction and suggested fix. - Everything else reviewed (CLI flag validation in
lint_command.go, Actions-expression taint inexpression.go, rule registry design inlint.go) is clear, well-tested, and consistent with existing codebase conventions.
Positive Highlights
- ✅ Symbolic-only taint tracking (never touches real secret values) is a sound security design choice
- ✅ Extensive, well-named test cases covering branch-union, loop-carried taint, and action credential outputs
- ✅ Documentation updates in
compilation-process.mdandcli.mdclearly explain the newsecret-leakrule semantics
🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · copilot · sonnet50 · 201 AIC · ⌖ 14.8 AIC · ⊞ 10.1K
Comment /matt to run again
Extract branch-scope evaluation below the custom function-size limit and use strings.CutPrefix without changing diagnostics or trigger behavior. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Extract per-step analysis, compare bounds-safe reference prefixes, and use ErrorIs for cancellation assertions. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
🟡 Changes recommended
The default lint path still has blocking correctness and reliability issues.
18 open issues
Reset and correctly propagate tracing state · New Distinguish env listing from command execution · New Guard empty arguments in assignment-only redirections · New Include persisted GITHUB_ENV values in job outputs · New Analyze prompt expressions in the activation job · New Document major changes for mandatory AST checks · New Preserve diagnostics when a rule reports a failure · New Initialize tracing for custom shell invocations · New Track reassigned and unmodeled file targets safely · New Apply prefix assignments to command-local environments · New Preserve taint for add-mask-prefixed output · New Analyze stdin and file uploads in network requests · New Bind aggregate positional parameters in functions · New Parse GITHUB_ENV records from emitted output · New Resolve file targets against the working directory · New Support recursive upload-artifact glob matching · New Resolve inherited and default step shells · New Apply artifact exclusion patterns before tainting · New
What changed in this PR
Adds extensible Actions/AW AST linting to gh-aw, including symbolic secret-flow analysis.
Changes:
- Add rule registration, selection, and structured diagnostics.
- Integrate AST checks into
gh aw lintalongside actionlint. - Add regression tests and documentation, and move Gemini authentication into a header.
| File | Description |
|---|---|
pkg/workflow/taint/workflow.go |
Traverse workflow scopes and propagate taint. |
pkg/workflow/taint/taint.go |
Define taint rules, sources, and findings. |
pkg/workflow/taint/taint_test.go |
Test secret-flow analysis. |
pkg/workflow/taint/shell.go |
Analyze shell commands and file flows. |
pkg/workflow/taint/secrets.go |
Define secret sources and disclosure sinks. |
pkg/workflow/taint/expression.go |
Track Actions expression taint. |
pkg/workflow/lint/secrets.go |
Convert taint results into diagnostics. |
pkg/workflow/lint/repository_workflows_test.go |
Check existing compiled workflows. |
pkg/workflow/lint/lint.go |
Add the extensible rule runner. |
pkg/workflow/lint/lint_test.go |
Test registration and diagnostic handling. |
pkg/cli/lint_workflow.go |
Load workflows and report AST diagnostics. |
pkg/cli/lint_workflow_test.go |
Test CLI integration and reporting. |
pkg/cli/lint_command.go |
Add rule-selection and reporting options. |
go.sum |
Record dependency checksums. |
go.mod |
Add the shell parser dependency. |
docs/src/content/docs/setup/cli.md |
Document lint usage and limitations. |
docs/src/content/docs/reference/glossary.md |
Update the lint command definition. |
docs/src/content/docs/reference/compilation-process.md |
Document rule-extension APIs. |
.github/workflows/daily-model-inventory.md |
Send the Gemini key through a header. |
.github/workflows/daily-model-inventory.lock.yml |
Regenerate the inventory workflow. |
.changeset/minor-workflow-ast-secret-leak-linter.md |
Describe the release changes. |
Review effort: Max
| name, _, found := strings.Cut(payload, "=") | ||
| if !found { | ||
| name, _, _ = strings.Cut(payload, "<<") | ||
| } | ||
| name = strings.TrimSpace(name) |
There was a problem hiding this comment.
Output-record name extraction handles printf assignments and heredoc records; unknown dynamic GITHUB_ENV names produce a coverage gap. The existing propagation tests pass (2afe4c3).
| } | ||
| case "", "/dev/null": | ||
| default: | ||
| s.files[target] = union(s.files[target], sources) |
There was a problem hiding this comment.
Modeled file paths resolve against the effective step/job/workflow working directory and match artifact paths in workspace-relative coordinates; inherited and step-level cases are covered (2afe4c3).
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
@copilot address the following outstanding work in one pass:
Push the necessary fixes, reply to each listed review thread and resolve it when addressed. Ignore feedback already answered or resolved. Use the pr-finisher skill and stop when only human review or CI remains; do not trigger CI. Sous-chef head: b733245
|
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…analysis Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
* Initial plan * Add shared /reflect fixture for model resolution Co-authored-by: SivaKesava1 <11771739+SivaKesava1@users.noreply.github.com> * Resolve configured model aliases with one shared step on every engine Co-authored-by: SivaKesava1 <11771739+SivaKesava1@users.noreply.github.com> * Switch smokes to aliases, recompile locks, document shared model resolution Co-authored-by: SivaKesava1 <11771739+SivaKesava1@users.noreply.github.com> * Apply remaining changes Co-authored-by: SivaKesava1 <11771739+SivaKesava1@users.noreply.github.com> * Remove stray import cache file committed from a test run Co-authored-by: SivaKesava1 <11771739+SivaKesava1@users.noreply.github.com> * Keep pi's audit-classified startup phrase ahead of the shared model_unavailable message Co-authored-by: SivaKesava1 <11771739+SivaKesava1@users.noreply.github.com> * Restore pkg/workflow/.github files from main removed with the stray import cache Co-authored-by: SivaKesava1 <11771739+SivaKesava1@users.noreply.github.com> * Fix typecheck errors in shared model resolution callers Co-authored-by: SivaKesava1 <11771739+SivaKesava1@users.noreply.github.com> * Expect the pi smoke's gpt-5-mini sub-agent alias to resolve to the newest live match Co-authored-by: SivaKesava1 <11771739+SivaKesava1@users.noreply.github.com> * Keep buildInitialWorkflowData at main's length for largefunc lint Co-authored-by: SivaKesava1 <11771739+SivaKesava1@users.noreply.github.com> * Prefer a live exact model id over an alias with the same name; restore pi smoke gpt-5-mini expectation Co-authored-by: SivaKesava1 <11771739+SivaKesava1@users.noreply.github.com> * Merge main; regenerate frontmatter docs for models: aliases Co-authored-by: SivaKesava1 <11771739+SivaKesava1@users.noreply.github.com> * Pass bundled model ids through without AWF, emit GH_AW_MODEL_ALIASES for every harness, validate imported aliases Co-authored-by: SivaKesava1 <11771739+SivaKesava1@users.noreply.github.com> * Ship built-in model aliases with the setup action; emit only workflow aliases Co-authored-by: SivaKesava1 <11771739+SivaKesava1@users.noreply.github.com> * Recompile lock files without built-in alias maps Co-authored-by: SivaKesava1 <11771739+SivaKesava1@users.noreply.github.com> * Fix resolver JSDoc types; document alias env var reduction Co-authored-by: SivaKesava1 <11771739+SivaKesava1@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: SivaKesava1 <11771739+SivaKesava1@users.noreply.github.com> Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Select the workflow's previously successful GPT-5.4 model through the existing Copilot provider. The failing run's complete live catalog advertises this model and its Responses endpoint, but no Gemini models. Preserve Pi, Actions-token authentication, tools, limits and isolation; regenerate the lock. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
…9870) Keep the auto-selected main model and gpt-6-luna delegates while allowing each model to use its supported endpoint. Cover workflow configuration, emitted SDK wiring, model qualification, and prompt preservation. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
* fix: attribute safe-output footers to work queue claims Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * fix: deploy report dependencies for claim footers Include the report renderer and its transitive runtime dependencies in the isolated safe-output bundle. Add a regression that deploys the real bundle and renders a Claim-scoped footer. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
…map rule (#69831) Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Add opt-in per-rule duration markers and execution trace regions. Benchmark every registered rule on safe/unsafe AW and Actions ASTs at 32 and 256 jobs; enforce wall-time and growth budgets in ordinary tests. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Merged latest
The requested feedback is addressed: the remaining malformed-companion failure is fixed in |
Preserve the concurrent remote correction for optional malformed AW lock companions alongside performance instrumentation and main's test fixes. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>



Summary
Add 18 AST-only linters, an extensible rule registry, and reusable source/sink/filter secret-taint analysis. Rules consume parsed AW or Actions ASTs; file discovery and workflow parsing belong to the CLI adapter. No rule recompiles a workflow, fabricates an execution graph, retrieves secret values, or executes analyzed programs.
.md, native Actions.yml/.yaml, and compiled.lock.ymlinputs without compilation. AW-only checks inspect authored execution nodes and prompts and retain unavailable generated-flow coverage. Directory scans avoid duplicate lock/source pairs; Markdown never reaches external actionlint.gh aw lintruns registered checks alongside actionlint on YAML;--rulesselects AST-only operation. Existing extension rules default to Actions-only targets.lint --debugrestores them. Internal AST diagnostics, actual errors, analysis failures, and other rules' warnings remain intact. Shape-preserving JavaScript object rest avoids contaminating public timeout fields with nested credential environment members while retaining real secret flows and shallow aliases.Linters
secret-leak--debugaction-pintemplate-injectionworkflow-run-scopejob-timeoutpermission-scopewrite-alldeclarations and invalid OIDCid-token: read. Scoped writes and OIDC issuance are not automatically violations.version-checkcheck-for-updates.conditional-importif, including false/null values. Runtime prompt conditionals are unaffected.private-to-publicinstall-scriptsallowed-labels*in creation/discussion-update label allowlists. Narrow patterns and literal required-label names remain valid.network-scopedefaults. Domain wildcards remain valid; omission is not missing-firewall evidence.push-targettool-scopesandbox-configimage-argumentcheckout-modefork-associationStrict policy: error for authored strict AW (the default), warning for explicit non-strict AW, and warning for native Actions where applicable. Missing workflow selection and invalid OIDC reads are always errors; wildcard push-target findings remain warnings. AW-only checks accept a standalone AW AST or a
.lock.ymlcompanion and skip native Actions without source evidence. CLI/repository overrides and imported configurations are not reconstructed. These are observable subsets, not assertions of full CTR conformance.Current analysis results
Both completed scans cover all 18 rules across 344 compiled workflows and companion sources on published tree
242e891bfd5c8825b039ff8d1b7e613b67fe06f7:Normal default CLI JSON is
[]. Coverage warnings are normally hidden and restored with--debug; the underlying AST diagnostics remain available. The configured run completed all 344 runners and 6,192 rule checks, with zero runner failures and zero compiler-unavailable/incompatible diagnostics. The other 17 linters report no findings in either scan.Default coverage gaps are 341 unavailable-JavaScript boundaries, seven other external-command boundaries and 124 unsupported/indeterminate shell boundaries. Earlier source-matched shell/file refinements reduced 1,605 warnings to 472; uninspected helpers were not allowlisted.
The new configured total supersedes the earlier child-tree count of 630,179 after main's runtime/source changes. The 14 historical timeout findings were corrected by preserving JavaScript object-rest member provenance, not by suppressing error diagnostics. No interrupted or input-racing partial scan contributes to these final totals. Warnings are incomplete coverage, not vulnerability counts or evidence of safety.
Optional JavaScript inspection uses the actual Go TypeScript compiler through pinned upstream-exported native-preview AST APIs. It confines source reads, preserves cancellation, follows available modules, and retains gaps for unknown calls, cycles, syntax, dynamic reads, and traversal/control-flow limits. Genuine escaped single-record stdout masking commands do not sanitize captured stdout or later uses. Public length/index results require proven types; arbitrary object properties and numeric coercions remain tainted. Authentication inputs are intentional credential consumption; endpoint trust is not inferred.
Remaining compiler obligations require additional trusted evidence: job-purpose authorization, effective sandbox/firewall/output-route semantics, engine enforcement capabilities, package operand semantics, manifests, guarded checkout provenance, repository visibility, successful-job cache gating, runtime bot/App identity, and publisher API provenance. Rules do not infer these through raw YAML, helper basenames, or command substring matching. An empty errors-only report is not proof of whole-workflow safety or complete coverage.
Performance tracking
DEBUG=workflow:lint:perf DEBUG_COLORS=0 gh aw lint --rules= --jsonwrites value-free per-rule start/completion markers, elapsed nanoseconds, diagnostic counts and failure status to stderr. Separate markers measure normalization and total runner time. JSON output and coverage-warning visibility are unchanged. Go execution traces includeworkflow:lint,workflow:lint:rule:<id>, andworkflow:lint:diagnosticsregions.BenchmarkLintRulesexercises safe/unsafe fixtures for every registered rule and supported AST at 32 and 256 jobs, reportingns/op,B/op, andallocs/op. The captured baseline contains 100 cases with three samples each. The slowest median was 2.06 ms for secret-leak on the 256-job Actions unsafe fixture (local Apple M5 Pro; not a portable performance promise).TestLintPerformanceBudgetruns in the ordinary suite. It enforces a 250 ms per-check ceiling and bounds the eight-fold workload increase to 12 times the allocations plus 1,000 and 24 times the runtime plus 5 ms. Safe/unsafe fixture checks prevent skipped rule execution from passing as a fast result. Workflows/frontmatter are parsed before timing; shell/expression parsing inside checks remains measured. The synthetic guard excludes optional external JavaScript compilation and is not a proof of asymptotic complexity.Measured rule-check wall time over the 344-workflow unconfigured corpus (excluding input parsing and output; debug instrumentation enabled):
secret-leaktemplate-injectionaction-pinThe configured native-JavaScript scan identifies the expensive path: aggregate runner wall time was 550.66 s, of which 550.00 s belonged to secret-leak; its slowest individual check took 21.56 s. Template-injection took 0.350 s in total and action-pin 0.046 s. These are measured experimental whole-program-analysis costs, not covered by the small in-process fixture budget and not evidence that every configuration is fast.
CPU, allocation and execution-trace artifacts were captured. The allocation profile exposes internal shell parsing and binding-map cloning as substantial secret-flow costs; no speculative semantic changes or unsafe caching were introduced. Native compiler wall time is included in its calling rule's marker; external compiler CPU work is outside Go CPU profiles.
Dependency license review
mvdan.cc/sh/v3 v3.14.1is BSD-3-Clause, compatible with MIT subject to preserving its copyright, conditions, and disclaimer; endorsement requires permission. Release notice packaging was not audited.Optional
@typescript/native-preview@7.0.0-dev.20260707.2declares Apache-2.0 and ships its license/NOTICE. It is installed separately, not bundled into the Go CLI. Redistribution requires applicable licenses/notices and identification of modified upstream files. Its API is explicitly unstable.Validation
make agent-report-progresspassed after integration, including formatting, custom Go lint, schema freshness, impacted unit tests, and successful recompilation/drift checks for all 344 workflows.TestContributionCheckWorkflowAllowsRequiredShellCommandsandTestSessionParserSourcesIncludeLocalDependencies, pass with CGO enabled. This is a local reproduction result, not a claim that remote CI has finished.