refactor!: consolidate chat model availability by ethanndickson · Pull Request #28440 · coder/coder · GitHub
Skip to content

refactor!: consolidate chat model availability - #28440

Merged
ethanndickson merged 4 commits into
ethan/codagt-718-model-config-auditfrom
ethan/chat-models-availability-consolidation
Aug 24, 2026
Merged

refactor!: consolidate chat model availability#28440
ethanndickson merged 4 commits into
ethan/codagt-718-model-config-auditfrom
ethan/chat-models-availability-consolidation

Conversation

@ethanndickson

@ethanndickson ethanndickson commented Aug 21, 2026

Copy link
Copy Markdown
Member

Summary

Consolidate chat model management and runtime availability into a single organization model collection contract. The organization model response now includes provider availability, unavailable reasons, and unsupported-provider details, so callers no longer need a separate availability request.

Depends on #28439.

Problem

The organization-scoped API exposes two overlapping representations of the same chat models:

GET /api/experimental/organizations/{organization}/chats/models
GET /api/experimental/organizations/{organization}/chats/models/available

The collection route returns organization-owned model configurations and redacted provider descriptors. The availability route returns a second catalog-oriented response with provider availability and synthetic catalog entries. Callers must fetch, reconcile, and cache both responses even though they describe the same effective set of models.

This creates duplicated server logic and separate SDK, OpenAPI, and frontend types. It can also produce inconsistent client state when one request succeeds while the other fails or when the two responses are refreshed at different times.

Fix

Return all model-management and runtime-availability information from the organization model collection route:

GET /api/experimental/organizations/{organization}/chats/models

OrganizationChatModelsResponse continues to return the caller-readable organization model configurations and redacted provider descriptors. Provider descriptors now also include:

  • available
  • unavailable_reason

The response additionally includes unsupported_providers, allowing clients to explain configurations that the Agents harness cannot use.

Frontend model settings, agent creation, existing chat recovery, and model override surfaces now consume this single collection query. This keeps model ownership, caller visibility, provider availability, and unsupported-provider guidance in one cache entry and one authorization path.

The default-organization compatibility route remains available:

GET /api/experimental/chats/models

It now returns the same consolidated OrganizationChatModelsResponse contract instead of the former availability response.

Breaking change

This removes the separate organization model availability endpoint:

GET /api/experimental/organizations/{organization}/chats/models/available

Clients using that endpoint must migrate to:

GET /api/experimental/organizations/{organization}/chats/models

This PR also removes the corresponding public experimental client and schema surface:

  • ExperimentalClient.ChatModelAvailability
  • ChatModelAvailabilityResponse
  • ChatModelProvider
  • ChatModelCatalogEntry
  • The generated OpenAPI schema and frontend API/query types for the availability endpoint

Callers must use ExperimentalClient.ChatModels and OrganizationChatModelsResponse instead. Availability is now reported on each ChatModelProviderDescriptor, and unsupported providers are returned in OrganizationChatModelsResponse.UnsupportedProviders.

The default-organization compatibility route retains its URL, but its response schema changes from ChatModelAvailabilityResponse to OrganizationChatModelsResponse. Direct HTTP clients and generated clients that decode the previous response type must be updated even if they do not use the removed organization-scoped /available route.

@github-actions

Copy link
Copy Markdown
Contributor

@ethanndickson
ethanndickson force-pushed the ethan/chat-models-availability-consolidation branch from 463117f to 580f74b Compare August 21, 2026 18:51
@ethanndickson
ethanndickson force-pushed the ethan/chat-models-availability-consolidation branch from 580f74b to fcb0822 Compare August 21, 2026 19:12
@ethanndickson
ethanndickson force-pushed the ethan/chat-models-availability-consolidation branch from fcb0822 to dddce28 Compare August 21, 2026 19:42
@ethanndickson
ethanndickson force-pushed the ethan/chat-models-availability-consolidation branch from dddce28 to e3bc6db Compare August 21, 2026 19:58
@ethanndickson
ethanndickson force-pushed the ethan/chat-models-availability-consolidation branch from e3bc6db to 24fc8f3 Compare August 21, 2026 20:50
@ethanndickson
ethanndickson marked this pull request as ready for review August 22, 2026 08:27

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 24fc8f3a42

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/exp_chats.go Outdated
Comment thread site/src/pages/AgentsPage/utils/modelOptions.ts
@ethanndickson
ethanndickson force-pushed the ethan/chat-models-availability-consolidation branch from 24fc8f3 to 9536411 Compare August 23, 2026 10:41

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 953641125c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/exp_chats_test.go Outdated
Comment thread site/src/pages/AgentsPage/utils/modelOptions.ts

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 953641125c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread site/src/pages/AgentsPage/AgentSettingsUserAgentsPage.tsx
@ethanndickson
ethanndickson force-pushed the ethan/chat-models-availability-consolidation branch 2 times, most recently from 2206799 to 87d3e99 Compare August 24, 2026 09:30
@ethanndickson
ethanndickson force-pushed the ethan/chat-models-availability-consolidation branch from 87d3e99 to b0939e0 Compare August 24, 2026 09:43
@ethanndickson

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

@ethanndickson
ethanndickson force-pushed the ethan/chat-models-availability-consolidation branch from b0939e0 to 4c58630 Compare August 24, 2026 10:07
@ethanndickson ethanndickson changed the title refactor: consolidate chat model availability refactor!: consolidate chat model availability Aug 24, 2026
@github-actions github-actions Bot added the release/breaking This label is applied to PRs to detect breaking changes as part of the release process label Aug 24, 2026
@ethanndickson
ethanndickson merged commit 3809314 into main Aug 24, 2026
35 of 56 checks passed
@ethanndickson
ethanndickson deleted the ethan/chat-models-availability-consolidation branch August 24, 2026 10:20
ethanndickson added a commit that referenced this pull request Aug 24, 2026
Moves chat model override settings from deployment scope to organization
scope, following the org-scoping of chat model configs in #28440.

Depends on #28440 (based on
`ethan/chat-models-availability-consolidation`).

Remote dogfood UAT passed on this change, including multi-organization
scenarios on a licensed deployment with real models.

## Problem

Chat model configs are now owned by organizations, but the override
settings that reference them stayed deployment-wide: admin
subagent/title/compaction overrides in `site_configs`, personal
overrides in `user_configs`, and the advisor model embedded in the
advisor runtime JSON. A single deployment-level override can only point
at one organization's model config, so overrides effectively worked for
the default organization only, and title generation had to soften its
hard-failure contract to tolerate unusable overrides.

## Fix

Admin overrides (general, explore, title generation, compaction,
advisor) and personal overrides (root, general, explore) become
organization-scoped rows in two new typed tables,
`chat_organization_model_overrides` and `chat_user_model_overrides`.
Composite foreign keys on `(organization_id, model_config_id)` make
cross-organization model references unrepresentable. The migration
deletes the legacy serialized `site_configs`/`user_configs` override
keys instead of migrating them; admins and users re-select models in the
new organization-scoped settings.

New endpoints live under
`/api/experimental/organizations/{organization}/chats/model-overrides`
(admin defaults) and
`/api/experimental/organizations/{organization}/members/{user}/chats/model-overrides`
(personal overrides); the legacy deployment-level routes are removed.
chatd resolves overrides via the chat's organization, restoring the
hard-failure contract for configured-but-unusable title generation
overrides. The deployment-level `AllowUsers` switch and advisor runtime
limits stay deployment-scoped.

The Coder Agents settings UI gains a per-organization "Defaults &
overrides" page under AI Settings > Models, and the personal overrides
page gains an organization selector for users in multiple organizations.
Advisor telemetry now reports per-organization override rows.

**BREAKING CHANGE**: Existing deployment-level overrides are dropped,
not migrated. Every organization starts with no overrides after upgrade,
and model selections must be re-applied in the new settings. The legacy
deployment-level override endpoints are removed.

Closes https://linear.app/coder/issue/CODAGT-872

> [!NOTE]
> Xum (AI agent) authored this PR on behalf of @ibetitsmike.

<!-- xum-attribution: model=claude-opus-4-6 thinking=high -->

---------

Co-authored-by: Ethan Dickson <ethan@coder.com>
aslilac pushed a commit that referenced this pull request Aug 24, 2026
## Summary

Consolidate chat model management and runtime availability into a single
organization model collection contract. The organization model response
now includes provider availability, unavailable reasons, and
unsupported-provider details, so callers no longer need a separate
availability request.

Depends on #28439.

## Problem

The organization-scoped API exposes two overlapping representations of
the same chat models:

```text
GET /api/experimental/organizations/{organization}/chats/models
GET /api/experimental/organizations/{organization}/chats/models/available
```

The collection route returns organization-owned model configurations and
redacted provider descriptors. The availability route returns a second
catalog-oriented response with provider availability and synthetic
catalog entries. Callers must fetch, reconcile, and cache both responses
even though they describe the same effective set of models.

This creates duplicated server logic and separate SDK, OpenAPI, and
frontend types. It can also produce inconsistent client state when one
request succeeds while the other fails or when the two responses are
refreshed at different times.

## Fix

Return all model-management and runtime-availability information from
the organization model collection route:

```text
GET /api/experimental/organizations/{organization}/chats/models
```

`OrganizationChatModelsResponse` continues to return the caller-readable
organization model configurations and redacted provider descriptors.
Provider descriptors now also include:

- `available`
- `unavailable_reason`

The response additionally includes `unsupported_providers`, allowing
clients to explain configurations that the Agents harness cannot use.

Frontend model settings, agent creation, existing chat recovery, and
model override surfaces now consume this single collection query. This
keeps model ownership, caller visibility, provider availability, and
unsupported-provider guidance in one cache entry and one authorization
path.

The default-organization compatibility route remains available:

```text
GET /api/experimental/chats/models
```

It now returns the same consolidated `OrganizationChatModelsResponse`
contract instead of the former availability response.

## Breaking change

This removes the separate organization model availability endpoint:

```text
GET /api/experimental/organizations/{organization}/chats/models/available
```

Clients using that endpoint must migrate to:

```text
GET /api/experimental/organizations/{organization}/chats/models
```

This PR also removes the corresponding public experimental client and
schema surface:

- `ExperimentalClient.ChatModelAvailability`
- `ChatModelAvailabilityResponse`
- `ChatModelProvider`
- `ChatModelCatalogEntry`
- The generated OpenAPI schema and frontend API/query types for the
availability endpoint

Callers must use `ExperimentalClient.ChatModels` and
`OrganizationChatModelsResponse` instead. Availability is now reported
on each `ChatModelProviderDescriptor`, and unsupported providers are
returned in `OrganizationChatModelsResponse.UnsupportedProviders`.

The default-organization compatibility route retains its URL, but its
response schema changes from `ChatModelAvailabilityResponse` to
`OrganizationChatModelsResponse`. Direct HTTP clients and generated
clients that decode the previous response type must be updated even if
they do not use the removed organization-scoped `/available` route.
aslilac pushed a commit that referenced this pull request Aug 24, 2026
Moves chat model override settings from deployment scope to organization
scope, following the org-scoping of chat model configs in #28440.

Depends on #28440 (based on
`ethan/chat-models-availability-consolidation`).

Remote dogfood UAT passed on this change, including multi-organization
scenarios on a licensed deployment with real models.

## Problem

Chat model configs are now owned by organizations, but the override
settings that reference them stayed deployment-wide: admin
subagent/title/compaction overrides in `site_configs`, personal
overrides in `user_configs`, and the advisor model embedded in the
advisor runtime JSON. A single deployment-level override can only point
at one organization's model config, so overrides effectively worked for
the default organization only, and title generation had to soften its
hard-failure contract to tolerate unusable overrides.

## Fix

Admin overrides (general, explore, title generation, compaction,
advisor) and personal overrides (root, general, explore) become
organization-scoped rows in two new typed tables,
`chat_organization_model_overrides` and `chat_user_model_overrides`.
Composite foreign keys on `(organization_id, model_config_id)` make
cross-organization model references unrepresentable. The migration
deletes the legacy serialized `site_configs`/`user_configs` override
keys instead of migrating them; admins and users re-select models in the
new organization-scoped settings.

New endpoints live under
`/api/experimental/organizations/{organization}/chats/model-overrides`
(admin defaults) and
`/api/experimental/organizations/{organization}/members/{user}/chats/model-overrides`
(personal overrides); the legacy deployment-level routes are removed.
chatd resolves overrides via the chat's organization, restoring the
hard-failure contract for configured-but-unusable title generation
overrides. The deployment-level `AllowUsers` switch and advisor runtime
limits stay deployment-scoped.

The Coder Agents settings UI gains a per-organization "Defaults &
overrides" page under AI Settings > Models, and the personal overrides
page gains an organization selector for users in multiple organizations.
Advisor telemetry now reports per-organization override rows.

**BREAKING CHANGE**: Existing deployment-level overrides are dropped,
not migrated. Every organization starts with no overrides after upgrade,
and model selections must be re-applied in the new settings. The legacy
deployment-level override endpoints are removed.

Closes https://linear.app/coder/issue/CODAGT-872

> [!NOTE]
> Xum (AI agent) authored this PR on behalf of @ibetitsmike.

<!-- xum-attribution: model=claude-opus-4-6 thinking=high -->

---------

Co-authored-by: Ethan Dickson <ethan@coder.com>
ethanndickson added a commit that referenced this pull request Aug 27, 2026
> [!IMPORTANT]
> Most of the added lines in this PR are generated API reference content
for publishing the organization-scoped `GET` and `POST
/api/v2/organizations/{organization}/chats/models` replacements. There
is no matching generated-doc deletion because the three removed
default-organization endpoints are experimental and are not present in
the generated public API reference on the current base. The removed
paths remain only as explicit 404 reservations, with no
default-organization shim or functional handler.

## Summary

Remove the unused default-organization chat model collection routes:

```text
GET  /api/experimental/chats/models
GET  /api/experimental/chats/model-configs
POST /api/experimental/chats/model-configs
```

Use the organization-scoped collection instead:

```text
GET  /api/v2/organizations/{organization}/chats/models
POST /api/v2/organizations/{organization}/chats/models
```

## Context

The intention of PR #28440 was to consolidate model availability into
the organization-scoped models collection for CODAGT-898 and remove the
superseded collection routes. That PR removed
`/organizations/{organization}/chats/models/available`, but it missed
these older default-organization routes and explicitly retained one of
them. This PR completes the intended #28440 cutover. The repository has
no SDK, CLI, or frontend consumer for the removed routes.

This change also prevents PR #28496 from promoting the missed
default-organization `/chats/models` route into the stable `/api/v2`
API.

This change is separate from the one-release `/api/experimental`
compatibility window in CODAGT-921. That window preserves experimental
versions of the intended stable API. It does not require compatibility
routes for unused deployment-scoped endpoints.

Refs CODAGT-898.

## Breaking change

Clients that call the removed routes must use the organization-scoped
model collection and provide an organization.

## Changelog

Remove unused default-organization Coder Agents model API routes. API
clients must use the organization-scoped chat models collection.

> [!NOTE]
> Coder Agents generated this pull request on Ethan Dickson's behalf.
ethanndickson added a commit that referenced this pull request Aug 27, 2026
> [!IMPORTANT]
> Most of the added lines in this PR are generated API reference content
for publishing the organization-scoped `GET` and `POST
/api/v2/organizations/{organization}/chats/models` replacements. There
is no matching generated-doc deletion because the three removed
default-organization endpoints are experimental and are not present in
the generated public API reference on the current base. The removed
paths remain only as explicit 404 reservations, with no
default-organization shim or functional handler.

## Summary

Remove the unused default-organization chat model collection routes:

```text
GET  /api/experimental/chats/models
GET  /api/experimental/chats/model-configs
POST /api/experimental/chats/model-configs
```

Use the organization-scoped collection instead:

```text
GET  /api/v2/organizations/{organization}/chats/models
POST /api/v2/organizations/{organization}/chats/models
```

## Context

The intention of PR #28440 was to consolidate model availability into
the organization-scoped models collection for CODAGT-898 and remove the
superseded collection routes. That PR removed
`/organizations/{organization}/chats/models/available`, but it missed
these older default-organization routes and explicitly retained one of
them. This PR completes the intended #28440 cutover. The repository has
no SDK, CLI, or frontend consumer for the removed routes.

This change also prevents PR #28496 from promoting the missed
default-organization `/chats/models` route into the stable `/api/v2`
API.

This change is separate from the one-release `/api/experimental`
compatibility window in CODAGT-921. That window preserves experimental
versions of the intended stable API. It does not require compatibility
routes for unused deployment-scoped endpoints.

Refs CODAGT-898.

## Breaking change

Clients that call the removed routes must use the organization-scoped
model collection and provide an organization.

## Changelog

Remove unused default-organization Coder Agents model API routes. API
clients must use the organization-scoped chat models collection.

> [!NOTE]
> Coder Agents generated this pull request on Ethan Dickson's behalf.

(cherry picked from commit 0f5c7b3)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release/breaking This label is applied to PRs to detect breaking changes as part of the release process

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants