feat: limit concurrent chat agents with pooled admission by ibetitsmike · Pull Request #27902 · coder/coder · GitHub
Skip to content

feat: limit concurrent chat agents with pooled admission - #27902

Merged
ibetitsmike merged 36 commits into
mainfrom
mike/chatd-pool-admission
Aug 18, 2026
Merged

feat: limit concurrent chat agents with pooled admission#27902
ibetitsmike merged 36 commits into
mainfrom
mike/chatd-pool-admission

Conversation

@ibetitsmike

@ibetitsmike ibetitsmike commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator

Limits concurrent chat generation on capped deployments to 5 root chats and 10 delegated subagent chats. The pools are deployment-wide and independent, so delegated work can continue while root capacity is full.

The default caps live in AGPL code. Enterprise contributes only a licensing unlock, so unlicensed deployments stay capped and cannot fail open. Licensed deployments are uncapped while Agent Hours usage stays below an explicit hard limit. Deployments without a hard limit remain uncapped, and reaching the Agent Hours allocation only triggers warnings.

Admission happens before a worker takes chat ownership. Capped deployments serialize admission across replicas with a transaction-scoped advisory lock and derive active and queued state from current ownership plus fresh runner heartbeats, rather than persisted queue markers or per-replica state. The acquisition query returns a bounded, pool-interleaved candidate set instead of ranking the whole backlog; a migration replaces the acquisition index with a pool-aware one. Refused chats stay running but unowned, and interrupt requests bypass admission so users can stop queued or over-cap chats.

The single-chat API derives queued_for_capacity from live pool state; list endpoints do not report it. The UI polls that value every 5 seconds while a chat is running and shows a callout when the chat is waiting for capacity.

Updates the administrator documentation and deployment-wide Prometheus gauges for active and queued agents. Replica-level values must be aggregated with max, not sum.

Mux updated this PR on Mike's behalf.

@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@coderagents

coderagents Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Documentation Check

New Documentation Needed

  • docs/ai-coder/agents/platform-controls/index.md - Document the new license-gated concurrent-agent capacity limit. Unlicensed/Community deployments are capped at 5 root chats and 10 subagent chats generating at once (two independent deployment-wide pools); deployments with remaining licensed agent_runtime_hours are uncapped. Explain the queued-for-capacity behavior (over-cap chats stay running but queue FIFO and surface a banner until a slot frees) and how to lift the cap (obtain a license / runtime hours). A subsection near Spend management is the natural home, since that section already documents license-gated Coder Agents limits.
    • Addressed in f479526: added a Concurrent agents section covering the 5-agent Community limit, the separate 10-slot subtask pool, the queued banner, and Premium Agent Hours removing the concurrency limit.
  • docs/ai-coder/agents/getting-started.md - Short pointer for Community users so they understand why agents may queue.
    • Addressed in f479526: added a Plan for concurrency limits subsection linking to the new Concurrent agents section.

Note

The auto-generated doc changes in this PR (docs/admin/integrations/prometheus.md, docs/reference/api/chats.md, docs/reference/api/schemas.md) look correct.


Automated review via Coder Agents

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 488948e1e1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/database/queries/chats.sql Outdated

@DanielleMaywood DanielleMaywood left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will perform a more thorough review of the backend tomorrow when I'm more awake but the frontend changes are fine with me

Comment thread coderd/database/dbauthz/dbauthz.go Outdated
Comment thread coderd/database/dbauthz/dbauthz.go Outdated
Comment thread coderd/x/chatd/runner.go Outdated
Comment thread coderd/x/chatd/options.go Outdated
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1eac94c228

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/database/queries/chats.sql Outdated
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8959eba6c9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/x/chatd/worker.go Outdated
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 05375b8e7d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread enterprise/coderd/x/chatd/agentadmission.go Outdated
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2b5a180a2b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread enterprise/coderd/x/chatd/agentadmission.go Outdated
Comment thread site/src/api/queries/chats.ts Outdated
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

Mux requested this review on Mike's behalf.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 21e7706a55

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread site/src/api/queries/chats.ts Outdated
Comment thread enterprise/coderd/x/chatd/agentadmission.go Outdated
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

Mux requested this review on Mike's behalf.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Delightful!

Reviewed commit: 6b109bfc94

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/database/migrations/000563_chat_capacity_queued_at.up.sql Outdated
Comment thread coderd/x/chatd/worker.go Outdated
Comment thread coderd/x/chatd/worker.go Outdated
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Swish!

Reviewed commit: 8d05987999

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/x/chatd/worker.go Outdated
Comment thread coderd/x/chatd/worker.go Outdated
Comment thread coderd/x/chatd/agentadmission.go Outdated
Comment thread coderd/x/chatd/agentadmission.go
Comment thread coderd/database/migrations/000563_chat_capacity_queued_at.up.sql Outdated
@ibetitsmike
ibetitsmike force-pushed the mike/chatd-pool-admission branch from 8d05987 to e7300dc Compare August 6, 2026 12:32
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e7300dcc2f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/x/chatd/runner.go Outdated
Comment thread coderd/x/chatd/worker.go Outdated
Comment thread coderd/database/queries/chats.sql
Comment thread coderd/x/chatd/agentadmission.go Outdated
Comment thread coderd/x/chatd/chatd.go Outdated
@ibetitsmike
ibetitsmike requested a review from johnstcn August 6, 2026 13:53
ibetitsmike and others added 26 commits August 18, 2026 14:20
…rker

Replace the capacity_queued_at column and its mark/clear machinery with
state derived from ownership, status, and pool fullness. AgentAdmission
shrinks to a single Admit method; a separate AgentCapacityPolicy exposes
current limits for read-side derivation. Queue metrics and capacity
events move to the OSS worker, driven by a per-replica in-memory map of
refused chats. FIFO ordering now keys on updated_at alone.
Replica A can refuse a chat and publish the queued capacity event while
replica B later admits it. B has no local refusal entry, so gating the
clear event on the local capacity queue left the banner stale until the
next refetch. On capped deployments, admission of a running chat now
always publishes the clear.

Also assert FIFO admission on the serial Admit call order recorded by
fakeAdmission instead of task-start order, which goroutine scheduling
can reorder, and document the requires_action resume overage bound.
Review feedback from cian: AgentAdmission and AgentCapacityPolicy always
shipped as the same object, so replace them with a single
AgentCapacityLimiter interface (Admit + Limits). Limits returns the caps
plus a capped bool instead of a Capped field. A noop limiter replaces
nil checks; chatd wires it whenever no factory is configured.

Consolidate CountChatCapacityActiveByPool and
CountChatCapacityUnownedByPool into CountChatCapacityByPool so admission
and the metrics loop share one scan, and widen idx_chats_capacity_active
to cover unowned chats so the merged query stays indexed.
…ents

Codex review findings on 546e293:
- An interrupting chat acquired by a replica that never refused it kept
  the queued banner because the clear only covered running acquisitions.
  Interrupting acquisitions now publish the clear too.
- A delayed queued capacity event could arrive after another replica's
  clear and restore the banner. Acquisition bumps updated_at while
  refusals do not, so the frontend merge now applies capacity events
  only when the snapshot is not older than the cache.

Also add the concurrent-agents docs section and point the queued
banner's learn-more links at it.
Codex found the cap can flip to uncapped (license update) between one
replica's refusal, which published queued=true, and another replica's
admission, which then read capped=false and skipped the clear, leaving
a stale banner. The clear condition now depends only on whether a real
limiter is configured: the noop limiter never refuses, so no queued
event can exist, while any real limiter may have refused while capped.
No dynamic state can desync the clear from the queued event anymore.
…pshot

Codex found the skipped-pool path published queued=true from a stale
candidate row: another replica can acquire the chat between the batch
query and the publish, and because the event payload reloaded the chat,
it carried the acquisition's new updated_at and defeated the frontend
freshness guard when delivered after the owner's clear.

enterCapacityQueue now validates and publishes from one snapshot: it
only queues chats that are still running, unarchived, and without a
live owner heartbeat, and the event payload is that same snapshot, so
a stale queued event either never publishes or carries pre-acquisition
updated_at that loses to the clear. Crashed-owner takeovers still queue
because their heartbeat is stale.
…kipped passes

Codex found prune starvation: with a sustained full-pool backlog every
acquisition pass ends on an all-skipped batch, which returned before
pruneCapacityQueue could run, so an entry for a chat another replica
acquired persisted indefinitely, suppressing the next episode's queued
event and measuring wait time from the previous episode.

An all-skipped pass now reconciles the local entries it did not see
with one targeted FilterChatCapacityWaiting query and drops those that
stopped being capacity-wait candidates. One bounded query per pass,
instead of paging the global candidate stream, which under churn had
no termination bound.
An all-skipped acquisition pass ended before reaching chats sorted
behind the known queue heads, so a chat arriving on a saturated pool
never got its queued event and open tabs kept showing plain running
state. Reconcile now lists all waiting chats once per all-skipped
pass, pruning departed entries and queueing arrivals in pools the
pass proved full.
…races

Two races found by review:

Resuming a chat from waiting or error during the runner's abandon gap
preserved worker_id/runner_id, so generation restarted on the retained
runner without capacity admission while the slot release may have
already admitted a queued chat. Idle-state resumes now clear ownership,
routing the resume through worker acquisition and re-admission.

A capacity clear published at acquisition could be rejected by the
frontend freshness guard when a newer status event advanced the cached
updated_at first, leaving the queued banner stuck on a generating chat.
Capacity events now order against a per-chat capacity revision (the
newest capacity event's updated_at) instead of the general guard.
…ity invalidation

The ordering guards can reject a legitimate capacity event when
delivery reorders against message sends that bump updated_at (for
example a queued event loaded before but delivered after a send to the
same still-unowned chat). The worker keeps its local queue entry and
emits no replacement event, so the banner would stay wrong until an
unrelated refetch. After merging a capacity event into the caches,
invalidate the chat entity when the merged result does not reflect the
event; the single-chat GET derives the authoritative queued state.
@ibetitsmike
ibetitsmike force-pushed the mike/chatd-pool-admission branch from 2acf9f1 to f56a4e2 Compare August 18, 2026 14:21
@ibetitsmike
ibetitsmike merged commit 119f2b1 into main Aug 18, 2026
31 checks passed
@ibetitsmike
ibetitsmike deleted the mike/chatd-pool-admission branch August 18, 2026 14:55
@github-actions github-actions Bot locked and limited conversation to collaborators Aug 18, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants