docs: add agent instructions for documentation workflows by pelikhan · Pull Request #70850 · github/gh-aw · GitHub
Skip to content

docs: add agent instructions for documentation workflows - #70850

Merged
pelikhan merged 6 commits into
mainfrom
pelikhan-literate-guide
Oct 21, 2026
Merged

pelikhan merged 6 commits into
mainfrom
pelikhan-literate-guide

Conversation

@pelikhan

@pelikhan pelikhan commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Add agent-facing .github/aw/documentation-workflows.md, lazy-loaded from designer, creator, updater, and pattern guidance. Five instruction files; focused regressions in the existing guidance test module. No public-site changes.

Dense skill style: direct rules, explicit failures, stable terms, one runnable example. Final instruction file: 1,138 words / 9,226 bytes, versus 1,642 / 12,347 before compression (31% fewer words, 25% fewer bytes).

Preserve bounded explicit/merged-PR intent, SHA-bound fail-closed checks, token event suppression, separate inference/mutation credentials, ruleset-compatible post-merge index PRs, and recursion prevention. Separate source-PR input gates from follow-up docs completion.

Close two desk-eval ambiguities:

  • docs:not-needed explicitly stops; only docs:required reaches page edits/new-PR creation.
  • On-demand conversion explicitly rewrites both configuration and prompt: current PR, declared branch-push/comment outputs, no inherited merged-PR reads/new-PR creation/issue escalation unless explicitly configured.

No compiler, runtime, or active workflow changes. #59393 remains a separate compiler follow-up; not every Copilot 429 has the same cause.

Validation

  • Required change-scoped gate passed; latest revision used make agent-report-progress-no-test. Initial impacted Go gate passed and skipped tests because no Go files changed.
  • Both new scenario-focused guidance regressions failed against the old instructions, then passed after correction; all 17 guidance tests pass.
  • Embedded example: current repository compiler + actionlint; 1 succeeded, 0 warnings.
  • Instruction links and all four lazy-loading routes validated.
  • Public docs build/internal links passed after removal of the public guide/gallery link.
  • Clean worktree; committed and pushed; no live execution.

Guidance regressions assert the documented contracts; compilation checks syntax/output wiring. Neither measures agent adherence or live event delivery.

Example credential review

DOCS_APP_PRIVATE_KEY / DOCS_APP_CLIENT_ID are placeholders, not configured credentials. Private key stays in safe-output token minting, never agent tools or generator scripts. Require repository-scoped installation and operation-specific permissions. No active secrets, Actions references, redirects, or deployment settings changed. Consuming repositories must validate scripts, event delivery, and rulesets before deployment.

pelikhan and others added 2 commits October 21, 2026 04:32
Explain explicit and merged-PR invocation, a current-head required docs gate, token event behavior, and ruleset-compatible derived-index PRs.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Move documentation workflow design rules into a lazy-loaded instruction file and route designer, creator, updater, and pattern guidance to it. Remove public documentation changes.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@pelikhan pelikhan changed the title docs: guide intent-driven PR docs and post-merge indexing docs: add agent instructions for documentation workflows Oct 21, 2026
pelikhan and others added 2 commits October 21, 2026 04:52
Replace narrative prose with direct rules, explicit gate failures, and compact event and credential contracts while preserving one runnable example.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Terminate no-docs decisions before writes and require on-demand prompt/output rewrites. Add two focused guidance regressions covering both ambiguities.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@pelikhan
pelikhan marked this pull request as ready for review October 21, 2026 12:06
Copilot AI max review requested due to automatic review settings October 21, 2026 12:06
@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

✅ Ponytail Reviewer completed successfully!

Ponytail Reviewer

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Ponytail Reviewer · gpt56 · 6.01 AIC · ⌖ 5.42 AIC · ⊞ 14.8K · ◷

Generated by Ponytail Reviewer for #70850

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

✅ PR Code Quality Reviewer completed the code quality review.

🔎 Code quality review by PR Code Quality Reviewer

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer

@github-actions

Copy link
Copy Markdown
Contributor

Comment Memory

Peek at saved memory (pr-code-quality-reviewer)
reviewed_at: 2026-10-21T12:09:35Z
review_event: REQUEST_CHANGES
top_themes: [stale SHA-bound docs resolution after branch updates, nondeterministic merged-PR deduplication, ungated index PR contract]
files_reviewed: [.github/aw/create-agentic-workflow.md, .github/aw/designer.md, .github/aw/documentation-workflows.md, .github/aw/patterns.md, .github/aw/update-agentic-workflow.md, .github/skills/agentic-workflows/tests/test_guidance.py]
comment_count: 3

Note

This comment is managed by comment memory.

Expand the saved memory block to view or edit the persistent context for this thread.
Edit only the text inside the backtick fences; workflow metadata and the footer are regenerated automatically.

Learn more about comment memory

🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt50 · 2.74 AIC · ⌖ 0.558 AIC · ⊞ 1.6K · ◷
Comment /review to run again

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request changes

The documentation contract has three correctness gaps that can leave generated PRs permanently blocked or create duplicate follow-up PRs: the on-demand rewrite does not renew the SHA-bound resolution after branch updates, merged-PR deduplication is only a model-time check, and index PRs have no documented way to satisfy the required decision/resolution gate. See the inline comments for surgical fixes.

🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt50 · 2.74 AIC · ⌖ 0.558 AIC · ⊞ 1.6K
Comment /review to run again

Comment thread .github/aw/documentation-workflows.md
Comment thread .github/aw/documentation-workflows.md Outdated
Comment thread .github/aw/documentation-workflows.md Outdated

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skills-Based Review 🧠

Applied /grill-with-docs (per pr-triage) — no blocking issues found; approving.

📋 Verification performed
  • Compiled the embedded aw example (gh aw compile --validate): 1 succeeded, 0 errors. The only warning was the expected "new restricted secret: DOCS_APP_PRIVATE_KEY" safe-update-mode notice, which the PR description already addresses under "Example credential review."
  • Ran .github/skills/agentic-workflows/tests/test_guidance.py: 15/15 pass, including the two new regressions for docs:not-needed stop-before-writes and the on-demand rewrite prompt/output separation.
  • Confirmed terminology consistency: label_command, safe-outputs.md, and the github-app / client-id (alias for app-id) schema all match existing conventions elsewhere in .github/aw/*.md and pkg/workflow/checkout_config_parser.go.
  • Lazy-loading wiring (designer.md, create-agentic-workflow.md, update-agentic-workflow.md, patterns.md) all correctly link to the new file.

Positive highlights

  • ✅ Clear separation of agent (page edits) vs. required check (gate) vs. builder (post-merge index) — matches the project's fail-closed, credential-separation patterns.
  • ✅ Dense, compressed instruction style achieves real token savings (31% fewer words) without losing the explicit failure-mode language.
  • ✅ New regression tests actually fail against the old instructions and pass after the fix, per the PR description — genuine TDD discipline for a docs change.

Notes

  • pr-triage returned a clean documentation classification recommending /grill-with-docs; no fallback heuristic needed.
  • No code, compiler, or runtime paths are touched — scope matches the stated "No compiler, runtime, or active workflow changes."

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · copilot · sonnet50 · 53.8 AIC · ⌖ 14.9 AIC · ⊞ 10.2K
Comment /matt to run again

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The workflow guidance needs correctness fixes, and the new guide is missing from required instruction inventories.

6 open issues
What changed in this PR

Adds internal, lazily loaded guidance for documentation automation in gh-aw, without changing active workflows, compiler behavior, or the public docs site.

Changes:

  • Documents documentation agents, required checks, credentials, and post-merge indexing.
  • Adds four guidance-loading routes.
  • Adds regression tests for no-op decisions and on-demand conversion.
File Description
.github/​skills/​agentic-workflows/​tests/​test_guidance.py Tests stopping and on-demand rewrite contracts.
.github/​aw/​update-agentic-workflow.md Routes documentation-related updates to the guide.
.github/​aw/​patterns.md Adds the documentation workflow pattern route.
.github/​aw/​documentation-workflows.md Defines documentation automation contracts and examples.
.github/​aw/​designer.md Loads the guide during documentation workflow design.
.github/​aw/​create-agentic-workflow.md Loads the guide during documentation workflow creation.

Review effort: Max

Comment thread .github/aw/documentation-workflows.md
Comment thread .github/aw/documentation-workflows.md Outdated
Comment thread .github/aw/documentation-workflows.md Outdated
Comment thread .github/aw/documentation-workflows.md Outdated
Comment thread .github/aw/documentation-workflows.md
Comment thread .github/aw/documentation-workflows.md Outdated
Assign final-head resolution publication to maintainers and trusted index builders, clarify best-effort search versus durable claims, and add focused contract regressions.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@pelikhan

Copy link
Copy Markdown
Collaborator Author

Re: #70850 (comment)

Accepted the evidence limitation; the status notification itself requests no code change. This batch independently passed 21 guidance regressions and example compilation/actionlint, and commit d169e19 was pushed; those checks do not establish live behavior or agent adherence.

@pelikhan

Copy link
Copy Markdown
Collaborator Author

Re: #70850 (comment)

Accepted that reported_no_action is not completion evidence; no implementation change follows from this status alone. The three actionable inline findings were addressed in pushed commit d169e19 and checked independently rather than inferred from the no-op report.

@pelikhan

Copy link
Copy Markdown
Collaborator Author

Re: #70850 (comment)

No separate code change: this is a review-completion notification without a requested change. The associated three inline correctness findings were accepted, addressed in d169e19, replied to individually, and resolved.

@pelikhan

Copy link
Copy Markdown
Collaborator Author

Re: #70850 (comment)

No separate code change: this is a skills-review status notification without actionable feedback. The latest batch preserves agent-facing scope and adds focused completion/idempotency contract coverage; all 21 guidance tests and example compilation/actionlint pass.

@pelikhan

Copy link
Copy Markdown
Collaborator Author

Register the guide in generated inventories, correct merged-mode base and checkout, use a normalization-safe recursion prefix, and narrow on-demand fallback and App permissions.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@pelikhan
pelikhan merged commit f3ac907 into main Oct 21, 2026
89 checks passed
@pelikhan
pelikhan deleted the pelikhan-literate-guide branch October 21, 2026 14:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants