feat: extend agent chat MCP tools for remote UAT evidence loops by ibetitsmike · Pull Request #28233 · coder/coder · GitHub
Skip to content

feat: extend agent chat MCP tools for remote UAT evidence loops - #28233

Merged
ibetitsmike merged 17 commits into
mainfrom
mike/mcp-server-hga6
Aug 18, 2026
Merged

feat: extend agent chat MCP tools for remote UAT evidence loops#28233
ibetitsmike merged 17 commits into
mainfrom
mike/mcp-server-hga6

Conversation

@ibetitsmike

Copy link
Copy Markdown
Collaborator

Extends the Agent-chat MCP tools so an unattended UAT evidence loop can fetch artifacts, monitor long runs, and find prior runs without burning model context.

Backend

  • New chat_files_token crypto key feature (migration 000571) with rotator support and a dedicated signing keycache on coderd.
  • POST /api/experimental/chats/files/{file}/download-url (authenticated) mints a short-lived (5 min) signed URL and returns it with sha256, size_bytes, name, mime_type, and expires_at.
  • GET /api/experimental/chats/files/{file}/download?token= (no session token) redeems the signed URL: verifies the JWS, requires the token's file_id to match the path, and re-checks the minting user's RBAC access live at redemption. Clients can curl -o artifacts with zero credentials in the URL consumer.
  • ChatFileMetadata gains size_bytes (via octet_length, no bytes fetched).

MCP tools (codersdk/toolsdk)

  • coder_download_chat_file: by file_id or chat_id+file_name; returns the signed URL plus checksum and size instead of base64.
  • coder_await_chat: blocks (bounded wait_secs, 1-120) until a chat leaves running/interrupting, using the existing watch stream with subscribe-before-read.
  • coder_list_chats: label, query, and limit filtering; chat projections now include labels.
  • coder_get_chat_messages: after_id forward cursor with next_after_id (exact incremental reads), plus per-message files metadata so artifact-bearing messages are identifiable.
  • coder_get_chat: file listings now include size_bytes and created_at.
  • coder_list_templates: exposes agents_allowed for pre-flight checks.

Testing

  • coderd: mint/redeem happy path with an unauthenticated client, expired/tampered/file-mismatched tokens, auth still required on the plain file endpoint, non-owner mint rejection.
  • toolsdk: harness + integration coverage for all new/changed tools, including signed-URL redemption with checksum verification, forward-cursor exactness, await transition/timeout paths, and label filtering.
  • Remote dogfood UAT (dev.coder.com Coder Agent) passed all six acceptance scenarios end to end over both MCP transports.

Note: go test ./codersdk/toolsdk/ has a pre-existing goleak flake on main (leaked agentssh non-PTY session goroutines from SSH exec tests; reproduced 3/3 on clean b4971bc49f1). It is unrelated to this diff.

Mux acted on Mike's behalf to create this PR.

@github-actions

github-actions Bot commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ce867a0f4f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread codersdk/toolsdk/chats.go Outdated
Comment thread codersdk/toolsdk/chats.go Outdated
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 77699e690c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread codersdk/toolsdk/chats.go Outdated
Comment thread codersdk/toolsdk/chats.go Outdated
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 942a33304a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread codersdk/toolsdk/chats.go Outdated
Comment thread coderd/database/models.go
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4a7a27a62d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread codersdk/toolsdk/chats.go Outdated
Comment thread codersdk/toolsdk/chats.go
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e3b1a8a111

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread codersdk/toolsdk/chats.go Outdated
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

rw.Header().Set("Cache-Control", "private, max-age=31536000, immutable")

P2 Badge Do not cache signed downloads beyond token expiry

When a browser or caching HTTP client fetches a signed download URL, this header allows it to reuse the response from its private cache for a year without contacting downloadChatFile again. The cached URL therefore remains usable after its advertised five-minute expiry and bypasses subsequent user-status and RBAC checks; use no-store or cap freshness at the signed token's remaining lifetime for this public route.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread codersdk/toolsdk/chats.go Outdated
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3ea17e6bfb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/exp_chats.go
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 85e69e1706

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/exp_chats.go
Comment thread codersdk/toolsdk/chats.go Outdated
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Swish!

Reviewed commit: b1f3c3aea9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Review fixes for the signed chat file download flow and the extended
MCP tools:

- Exclude query strings from initial tracing span names so the signed
  download token (and other query credentials) never reaches tracing
  backends via the span name exported at span start.
- Stop echoing internal database error details from the unauthenticated
  signed download endpoint; log them instead.
- Truncate expires_at to whole seconds so the advertised expiry matches
  the JWT exp claim instead of overstating validity by up to a second.
- Make wait_secs a hard upper bound for coder_await_chat: timeout paths
  return the last status observed inside the wait window instead of
  issuing a post-deadline status fetch, and the tool description
  documents the timed_out semantics.
- Surface queued file-only prompts in queued_messages as an attachment
  summary instead of silently dropping them.
- Reject out-of-range wait_secs and limit values instead of silently
  clamping, matching coder_get_chat_messages validation, and fix the
  limit description that claimed newest-to-oldest order for forward
  (after_id) pagination.
- Add regression tests for redemption-time authorization: a revoked
  chat share and a suspended minting user must both invalidate an
  already-minted signed URL. Pin second-precision expires_at in the
  mint test and cover queued file-only prompts in the toolsdk tests.
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

Pushed review fixes in 9691a1f (deep review: 8-domain reviewer pass, cross-checked):

Security

  • Tracing middleware no longer puts the query string in the initial span name, so the signed download token (and other query credentials like coder_session_token) never reaches tracing backends via the span name exported at span start (coderd/tracing/httpmw.go). The mechanism predates this PR, but this PR added a new bearer-credential query param, so it is fixed here.
  • The unauthenticated signed download endpoint no longer echoes internal DB error details in the 500 body; they are logged instead.
  • New regression tests pin the design's central security property (live authorization re-check at redemption): a revoked chat share and a suspended minting user both invalidate an already-minted URL.

Correctness

  • expires_at is truncated to whole seconds so it matches the JWT exp claim instead of overstating validity by up to ~1s (asserted in test).
  • coder_await_chat: wait_secs is now a hard upper bound. Timeout paths return the last status observed inside the wait window instead of issuing a post-deadline fetch (previously a stalled closing fetch could stretch wait_secs: 1 to ~16s). timed_out semantics documented in the tool description.
  • Queued file-only prompts now surface in queued_messages as an attachment summary instead of being silently dropped (the earlier file-only fix covered persisted messages only). Covered by test.
  • wait_secs/limit out-of-range values are rejected with errors instead of silently clamped, matching coder_get_chat_messages validation; fixed the limit description that claimed newest-to-oldest order for forward (after_id) pagination.

Validation: pre-commit (gen/fmt/lint/build) green locally; TestChatFileDownloadURL (8 subtests incl. the 2 new revocation tests) and TestChatTools pass against a real Postgres.

Also verified but intentionally not changed: chat-deletion revocation already works via the link-based dbauthz fallback; the forward cursor cannot skip messages because all inserts run under the chat row lock; and the keycache fetch-error wedge (cryptokeys/cache.go) is pre-existing across all signing features and deserves a separate PR.

Mux acted on Mike's behalf to push these fixes.

@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9691a1f43c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/tracing/httpmw.go
Comment thread codersdk/toolsdk/chats.go Outdated
…y credentials

Codex review fixes for PR #28233:

- coder_await_chat: drop the post-deadline fallback status fetch. When
  the initial status request outlives the wait window, the tool now
  returns the request error instead of running a fresh 15-second fetch
  from the caller context, which could stretch a wait_secs=1 call to
  roughly 16 seconds and report timed_out=false from a post-deadline
  observation. finalStatus now only reports states observed inside the
  window; all of its callers run after lastBusy is set. Regression test
  stalls every request and asserts the tool errors within the bound.

- coderd/tracing: regression test asserting no exported span surface
  (start-time name, final name, or completed-span attributes) carries a
  query-string credential. The vendored otel semconv v1.14 ServerRequest
  deliberately omits http.target, so completed spans do not leak the
  signed download token today; the test pins that invariant against
  future semconv upgrades.
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Delightful!

Reviewed commit: 93d24f7a23

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

…to 000572

Main gained 000571_pool_aware_chat_acquisition_index, so the PR merge
checkout contained two migrations with version 000571 and golang-migrate
failed to init in every CI job that touches the database (gen, lint,
sqlc-vet, offlinedocs, test-go-pg). The schema is unchanged; only the
version number moves past main's.
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fb86d9440d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread codersdk/toolsdk/chats.go Outdated
…alues

searchquery.Chats rejects unquoted values containing colons, so the
advertised diff_url:<url> form fails for any real URL unless quoted.
The coder_list_chats query description now says to quote values with
spaces or colons and shows a quoted diff_url example.
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

@ibetitsmike
ibetitsmike merged commit affeeaf into main Aug 18, 2026
31 checks passed
@ibetitsmike
ibetitsmike deleted the mike/mcp-server-hga6 branch August 18, 2026 17:15
@github-actions github-actions Bot locked and limited conversation to collaborators Aug 18, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants