feat(coderd): include chat model organization IDs in telemetry by ethanndickson · Pull Request #27956 · coder/coder · GitHub
Skip to content

feat(coderd): include chat model organization IDs in telemetry - #27956

Merged
ethanndickson merged 1 commit into
mainfrom
ethan/codagt-709-model-config-migration
Aug 24, 2026
Merged

feat(coderd): include chat model organization IDs in telemetry#27956
ethanndickson merged 1 commit into
mainfrom
ethan/codagt-709-model-config-migration

Conversation

@ethanndickson

@ethanndickson ethanndickson commented Aug 10, 2026

Copy link
Copy Markdown
Member

Include the owning organization ID for each chat model configuration in telemetry snapshots.

Depends on #27955

This pull request description was generated by Coder Agents.

@linear-code

linear-code Bot commented Aug 10, 2026

Copy link
Copy Markdown

@ethanndickson

Copy link
Copy Markdown
Member Author

/coder-agents-review

@ethanndickson

Copy link
Copy Markdown
Member Author

@codex review

@coder-agents-review

coder-agents-review Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Chat: Review posted | View chat
Requested: 2026-08-19 18:45 UTC by @ethanndickson

Review history
  • R1 (2026-08-10), 3 Note, 1 P1, 1 P3, COMMENT. Review
  • R2 (2026-08-10), 1 Nit, 5 Note, 1 P0, 1 P1, 2 P3, COMMENT. Review
  • R3 (2026-08-10): 14 reviewers, 1 Nit, 7 Note, 1 P0, 1 P1, 1 P2, 5 P3, COMMENT. Review
  • R4 (2026-08-11): 16 reviewers, 3 Nit, 11 Note, 1 P0, 1 P1, 6 P2, 12 P3, 1 P4, COMMENT. Review
  • R5 (2026-08-18): 7 reviewers, 4 Nit, 12 Note, 1 P0, 1 P1, 7 P2, 12 P3, 1 P4, COMMENT. Review
  • R6 (2026-08-19), 4 Nit, 12 Note, 1 P0, 1 P1, 7 P2, 12 P3, 1 P4, COMMENT. Review

deep-review v0.9.0 | Round 6 | ad400e1..038921b

Last posted: Round 6, 38 findings (1 P0, 1 P1, 7 P2, 12 P3, 1 P4, 4 Nit, 12 Note), COMMENT. Review

Finding inventory

Finding inventory: PR 27956

Findings

# Sev Status Location Summary Round Reviewer Posted
CRF-1 P1 Author fixed (483b499) coderd/database/migrations/migrate_test.go:3177 Rewind re-applies every migration after 566; test fails as soon as any later migration exists (stacked 000567) R1 Netero Yes
CRF-2 P3 Author fixed (483b499, MigrationFS removed entirely) coderd/database/migrations/migrate.go:28 MigrationFS rationale ("commits only when stepper exhausts") is false; each stepper call commits. Same false claim at migrate_test.go:3415 R1 Netero Yes
CRF-3 Note Author fixed (483b499, single INSERT staged via copy map) coderd/database/migrations/000566_chat_model_config_org_explosion.up.sql:92 Soft-deleted-copy INSERT duplicates live-copy INSERT verbatim except the deleted filter R1 Netero Yes
CRF-4 Note Author fixed (483b499, DISTINCT ON removed) coderd/database/migrations/000566_chat_model_config_org_explosion.up.sql:70 DISTINCT ON without ORDER BY is redundant; join is PK-to-PK R1 Netero Yes
CRF-5 Note Author fixed (483b499, down deletes only exact copy_id keys) coderd/database/migrations/000566_chat_model_config_org_explosion.down.sql:104 Down deletes pre-existing dangling threshold keys the up never created; documented and tested, recorded as known decision R1 Netero Yes
CRF-6 P0 Author fixed (2d922f5, enforcement change removed from PR; moved to #27959 scope) coderd/x/chatd/chatd.go:1582 Strict org check breaks 132 top-level chatd tests; seedChatDependenciesWithProvider seeds configs in the default org for a non-default-org chat. This is the CI failure R2 Netero Yes
CRF-7 P3 Author fixed (2d922f5, exp_chats.go changes removed; org-scoped API behavior belongs to #27959) coderd/exp_chats.go:5025-5036 Personal model overrides become inert for non-default-org users post-cutover; setting still appears to save. Needs ticket link or human decision R2 Netero Yes
CRF-8 Nit Author fixed (2d922f5, compaction_override.go changes removed) coderd/x/chatd/compaction_override.go:93 return nil, err where err is provably nil; reads as error propagation, is a soft fallback R2 Netero Yes
CRF-9 Note Author fixed (2d922f5, configcache test changes removed) coderd/x/chatd/configcache_internal_test.go:32 stubChatConfigStore.getDefaultOrganization func field and defaultOrganizationCall counter are dead after test rewrites R2 Netero Yes
CRF-10 Note Author fixed (2d922f5, migration 000566 deleted entirely) coderd/database/migrations/000566_chat_model_config_org_explosion.up.sql:51 Up migration aborts if any live non-default org already owns a live default config (manual rows only); root override key intentionally not remapped R2 Netero Yes
CRF-11 P3 Author fixed (5179677, org2 config seeded and asserted) coderd/telemetry/telemetry_test.go:1938 Both OrganizationID assertions compare default org to default org; test cannot detect uniform misattribution to the default org, the bug class the field exists to catch R3 Bisky P3, Hisoka P3, Mafu-san P3, Mafuuu P3, Pariston P3, Knuckle P3, Chopper P3, Kite P3, Meruem P3, Ryosuke Note Yes
CRF-12 P2 Author fixed (246a83d diff, restatement line removed, invariant kept) coderd/database/queries/chats.sql:2292 First comment line restates the query name and ships twice via sqlc doc-export; only the invariant line earns its place R3 Gon Yes
CRF-13 P3 Author fixed (246a83d diff, IsDefault doc states per-org scope) coderd/telemetry/telemetry.go:2621 IsDefault changed meaning to per-org (000565 per-org unique index); struct doc says nothing, telemetry analysts will misread multiple defaults R3 Leorio Yes
CRF-14 P3 Author fixed (description rewritten, names stack #27955/#27957-#27960) PR description Stacked PR never links the parent PR (000565 schema) or #27959 (enforcement); reviewer cannot navigate the stack R3 Leorio Yes
CRF-15 Note Author accepted R4 (no action needed; stale superseded run, CI re-runs on new head) coderd/database/queries/chats.sql:2294 The 20 failed checks in review context are a cancelled superseded run; live run on same head is green, current required check is success. Verified via public Actions API by two reviewers and orchestrator R3 Pariston, Kite (8 others rated P1 on the stale run; empirically disproved) Yes
CRF-16 Note Author accepted R4; panel re-raised R5; silent R6 (no ticket, no code, no acceptance after re-raise) coderd/telemetry/telemetry.go:2616 Config ownership now visible in snapshot but chat ownership is not; cross-org legacy references remain unobservable, half the join missing R3 Hisoka Yes
CRF-17 Note Dropped by orchestrator (cosmetic per reviewer's own assessment; JSON output unaffected) coderd/telemetry/telemetry.go:2325 ConvertChatModelConfig places OrganizationID second while generated row struct appends it last R3 Mafuuu No
CRF-18 P2 Author fixed (R5, fan-out migration removed from PR; base rebased to 2c7f10b) coderd/database/migrations/migrate_test.go:3290 Seed comment cites 000565/566 which are unrelated oauth2 migrations on this branch (renumber missed strings); ten sibling "566" identifiers plus fixture name/UUID prefix point at the same ghost R4 Netero Yes
CRF-19 P4 Author fixed (R5, fan-out migration removed from PR) coderd/database/migrations/000568_chat_model_config_org_explosion.up.sql:22 Fan-out covers only orgs existing at migration time; orgs created after lose chat creation at the #27959 cutover, no stack PR owns org-creation seeding R4 Netero Yes
CRF-20 Nit Author fixed (d0f5fa0, em-dashes replaced with periods) coderd/telemetry/telemetry_test.go:1663 Two base-side U+2014 em-dashes inside the edited function escape the added-lines-only lint gate R4 Netero Yes
CRF-21 P2 Author fixed (R5, fan-out migration removed; base 000571 backfills default org only) coderd/database/migrations/000568_chat_model_config_org_explosion.up.sql:20 Fan-out merges ahead of org-aware read surfaces: multi-org deployments see N copies per model on settings surfaces, edits diverge per org, and users can pin chats to other orgs' copies during the window (chat picker dedupes, verified) R4 Pariston P2, Hisoka P3 Yes
CRF-22 P3 Author fixed (R5, migration removed) coderd/database/migrations/000568_chat_model_config_org_explosion.up.sql:99 ACL COALESCE fails open: missing everyone entry becomes everyone-read on every copy; user_acl and non-everyone group entries silently dropped; unreachable via product paths today, #27957 sits directly above R4 Meruem P3, Mafuuu P3, Zoro P3, Pariston Note, Chopper Note, Knuckle Note, Kite Note Yes
CRF-23 P3 Author fixed (R5, migration removed) coderd/database/migrations/000568_chat_model_config_org_explosion.down.sql:87 Any post-up edit (copy or original) breaks the 13-field tuple match: copies strand on rollback as deployment-wide duplicates, re-up double-copies; structural alternatives: persist the fan-out map or derive deterministic copy IDs R4 Ryosuke P3, Meruem P3, Knov P3, Chopper P3, Mafuuu Note, Zoro Note, Komugi Note Yes
CRF-24 P3 Author fixed (R5, migration removed) coderd/database/migrations/000568_chat_model_config_org_explosion.up.sql:29 Correlated EXISTS probes cost O(deleted configs x live orgs x message volume) with no index on model_config_id; single-pass referenced-pairs staging is cheaper R4 Knuckle P3, Kite P3, Mafuuu Note Yes
CRF-25 P3 Author fixed (R5, migration removed) coderd/database/migrations/000568_chat_model_config_org_explosion.down.sql:116 Final DELETE fires per-row FK checks against unindexed chat_messages.model_config_id; rollback seq-scans messages once per deleted copy inside the startup-blocking transaction R4 Hisoka Yes
CRF-26 P3 Author fixed (R5, migration test removed) coderd/database/migrations/migrate_test.go:3599 ::uuid cast on key suffix guarded only by sibling regex predicate; Postgres does not guarantee predicate order, a plan change errors the test on the seeded hostile key; same class at 3611 R4 Chopper Yes
CRF-27 P3 Author fixed (R5, migration test removed) coderd/database/migrations/migrate_test.go:3431 Count comment enumerates a config named "late" that does not exist (fourth live config is emptyACLConfigID); auditors hunt a ghost R4 Mafu-san P3, Leorio P3, Mafuuu Nit, Knuckle Nit, Kite Nit, Zoro Nit, Bisky Nit Yes
CRF-28 P2 Author fixed (R5, migration test removed) coderd/database/migrations/migrate_test.go:3429 Test narration comment class: ~18 comments restate the require messages or literals directly below them (3268, 3309, 3328, 3378, 3429, 3477, 3483, 3491, 3518, 3529, 3540, 3566, 3595, 3605, 3621, 3640); delete or trim to the why R4 Gon Yes
CRF-29 P2 Author fixed (d0f5fa0 for in-scope telemetry items; migration items removed) coderd/database/migrations/000568_chat_model_config_org_explosion.up.sql:58 Padding class outside migrate_test.go: up.sql:58 first sentence restates the SELECT, fixture header restates filename, IsDefault doc first sentence subsumed by second (re-examines closed CRF-13: fix padded, not wrong), telemetry_test 1651/1655 restate dbgen calls R4 Gon Yes
CRF-30 P2 Author fixed (R5, migration test removed) coderd/database/migrations/migrate_test.go:3393 Hostile-key comment says the down "must not abort on their uuid cast" but the down never casts key suffixes; describes the deleted CRF-5-era design R4 Gon P2, Bisky Nit, Kite Nit Yes
CRF-31 P3 Author fixed (R5, migration test removed) coderd/database/migrations/migrate_test.go:3236 "zero-member" names the wrong axis: orgC is distinguished by having no chats, not no members; membership is never read R4 Gon Yes
CRF-32 Nit Author fixed (R5, migration removed) coderd/database/migrations/000568_chat_model_config_org_explosion.up.sql:140 ON CONFLICT (user_id, key) DO NOTHING is unreachable (copy_id freshly generated); masks a real invariant break if it ever fires R4 Ryosuke Nit, Meruem Nit, Chopper Note Yes
CRF-33 Note Author fixed (R5, migration removed) coderd/database/migrations/000568_chat_model_config_org_explosion.up.sql:135 Threshold fan-out inserts keys x live-orgs rows regardless of membership; deliberate and reverted by down, but user_configs can grow 200x overnight R4 Hisoka, Knov Yes
CRF-34 Note Author fixed (R5, migration test removed) coderd/database/migrations/migrate_test.go:3306 emptyACLConfigID is the only seed without a purpose comment; its COALESCE-fallback role surfaces 130 lines later R4 Leorio Yes
CRF-35 Note Author fixed (R5, migration test removed) coderd/database/migrations/migrate_test.go:3956 Weakened down-migration field match is detected with ~1/2 probability per run (mutation-verified); deterministic on correct SQL, bounds regression power only R4 Bisky Yes
CRF-36 Note Author fixed (R5, migration test removed) coderd/database/migrations/migrate_test.go:3403 "one config per provider and model" is a seed property, not a migration invariant; no unique index exists and the fan-out copies unconditionally R4 Ryosuke Yes
CRF-37 Note Author fixed (base #27955 CI repaired in 03db647; CI green on R6 head 038921b) coderd/telemetry/telemetry.go:2621 CI failure (test-go-pg ubuntu, pg-17, race-pg) reproduces identically on base commit 2c7f10b (#27955); Linux-only, macOS/Windows green on both; nothing in this diff can fix it. Route to #27955 R5 Netero Yes
CRF-38 P2 Author fixed (R6 head, both comments rewritten to invariant-only) coderd/telemetry/telemetry_test.go:1663 CRF-29 sibling instances missed by the R5 trim: first sentences at 1663 and 1822 restate the dbgen calls below them R5 Gon Yes
CRF-39 Nit Author fixed (R6 head, both comments removed) coderd/telemetry/telemetry_test.go:1651 Replacement comments from the CRF-29 fix regressed differently: "Its" dangles before its antecedent (org2 is declared two lines later), and "Exercises the full dump" names jargon the file never defines R5 Bisky Nit, Leorio Nit, Pariston Note Yes

Law analysis

Round 2, head 483b499, effective +1110 -406 (77.2% test density). Verdict: Don't split. Enforcement: Advisory. Three concerns: (1) migration 000566, (2) telemetry organization_id, (3) strict org-local enforcement (new scope arriving in commit 483b499, not described in the PR title/description). 1 -> 3 dependency chain, single risk story. Advisory conditions: amend the PR description to cover the enforcement change (or split it out as a stacked PR), and the panel must treat the enforcement code as unreviewed-until-now, not as verified fixes.

Contested and acknowledged

CRF-15 (Note, chats.sql:2294) - Stale CI run reported as failure

  • Finding: The 20 failed checks in the R3 review context were a cancelled run superseded by a green run on the same head; recorded so nobody re-investigates.
  • Author accepted (R4): No action needed; the red board on 2d922f5 was a cancelled superseded run and CI re-runs on the new head. The finding requested no change.

CRF-16 (Note, telemetry.go:2616) - Chat ownership not in snapshot

  • Finding: Config ownership became visible in the telemetry snapshot but chat ownership did not, leaving the preserved cross-org legacy config references unobservable.
  • Author accepted (R4): The restored fan-out migration (000568) remaps existing references into each chat's organization, so cross-org references stop being the steady state the snapshot would need to observe. Adding organization_id to telemetry.Chat is a separate product call kept out of scope. The defense engages the consequence: post-migration, the unobservable state is transient, not structural.

CRF-16 (R5 re-raise)

  • New evidence: R5 removed the fan-out migration this closure leaned on. Base feat!: scope chat model configs to organizations #27955's migration 000571 backfills configs to the default org and never touches chats, so every pre-migration non-default-org chat references a default-org config permanently. Cross-org references are steady state, not transient. The technical half of the R4 defense no longer holds; the scope half ("separate product call") stands but is a deferral without a ticket.
  • Panel position (R5): Knuckle and Pariston re-raised; Knov evaluated and would let the closure stand on the scope ground alone. Orchestrator re-raised per the no-agent-accepted-permanence rule: a human must link a ticket for Chat.OrganizationID or explicitly accept the blind spot.

Round log

Round 1

Netero-only (P1 present, panel gated). 1 P1, 1 P3, 3 Note. Reviewed against 425aa7a..9f6d499. Orchestrator independently verified the CRF-1 rewind/re-apply structure in migrate_test.go.

Round 2

Churn guard: PROCEED, all 5 findings classified author-fixed in 483b499 (branch rebased onto ef9a1a7). Netero verified all five fixes against the code (CRF-1..5 confirmed fixed, tests run locally). Law ran (>1000 effective LOC): Don't split, advisory. Netero found 1 P0 (CI root cause), 1 P3, 1 Nit, 2 Notes; P0 gates the panel again (second consecutive Netero-only round, cap reached, panel runs next round regardless). Reviewed against ef9a1a7..483b499.

Round 3

Churn guard: PROCEED, all 5 R2 findings addressed by re-scope in 2d922f5 (branch rewritten; migration 000566 and org enforcement removed, moved to stacked PRs). PR is now telemetry organization_id only: 3 effective files, +17 -13. Netero-only cap reached (R1, R2), panel reviews this round. CI shows 20 failed checks including infra jobs (changes, fmt, lint); panel to assess.

Panel: bisky, hisoka, mafu-san, mafuuu, pariston, ging-go, gon, leorio, knuckle, chopper, komugi, kite, ryosuke + meruem (wildcard, random from distilled set). Cross-check: 8 reviewers rated the red CI board P1 with unverified cause (gh 401 in worktrees); Pariston and Kite reached the public Actions API and proved the 20 failures are a cancelled run superseded by a green run on the same head; orchestrator re-verified (19 cancelled + 1 stale required failure vs 19 success, current required success). P1s empirically disproved, merged into CRF-15 Note. Test blind spot converged 9 reviewers at P3 (CRF-11). New: 1 P2, 3 P3, 2 Notes posted, 1 Note dropped. Reviewed against 46e2aaa..2d922f5.

Round 4

Churn guard: PROCEED. 4 addressed (CRF-11..14), 2 acknowledged (CRF-15, CRF-16). PR restructured again: migration restored as 000568 on new base #27955 (migration 000567), new tests including dedicated regression tests for CRF-10 and the Codex rollback P2. Effective +1092 -17 (74.8% test density). Orchestrator verified the R4 context CI block is again a stale superseded run (19 cancelled + stale required vs 17 success, test-go-race-pg pending on live run). No Law (effective additions shrank vs R2 analysis).

Netero (advisory): CRF-18 P2, CRF-19 P4, CRF-20 Nit. Panel: bisky, hisoka, mafu-san, mafuuu, pariston, ging-go, gon, leorio, knuckle, chopper, komugi, kite, ryosuke, knov + zoro, meruem (wildcards, random). Cross-check: ACL fallback severity conflict resolved upward (P3 over Note, fail-open consequence); mid-stack window P2 over P3 (Pariston's write-path evidence stronger, Hisoka's picker-dedupe verification retained as scope bound); rollback tuple-match fragility converged from 6 reviewers with two structural alternatives preserved; Gon's 22 comment P2s consolidated into two class findings (CRF-28, CRF-29) plus the inaccurate-comment finding (CRF-30). Mutation testing by Bisky and counterfactual old-SQL runs by Mafu-san both reproduced the PR's regression-test claims. New: 4 P2, 7 P3, 1 Nit (+CRF-19 P4, CRF-20 Nit, CRF-18 P2 from Netero), 4 Notes. Reviewed against 38f5109..246a83d.

Round 5

Churn guard: PROCEED. All 19 open findings addressed: the fan-out migration left this PR again (base #27955 now owns migration 000571, a default-org backfill only), and the in-scope telemetry items (CRF-20 em-dashes, CRF-29 comment trims) were fixed in code. PR is telemetry-only: +30 -21, 3 effective files. CI failure is real this round (test-go-pg, pg-17, race-pg failed on the live run; verified via check-runs API, no cancelled runs). Panel to root-cause.

Netero (advisory): CRF-37 Note (CI inherited from base #27955; orchestrator re-verified the base failure signature). Panel: bisky, komugi, knuckle, gon, leorio, pariston + knov (wildcard, random). Komugi and Knov: no findings; Knov independently re-verified the CRF-11 mechanism and type honesty against 000571. Cross-check: CRF-16 re-raised (Knuckle, Pariston vs Knov; orchestrator sided with re-raise, deferral lacks ticket). New: 1 P2 (CRF-38), 1 Nit (CRF-39), 1 Note (CRF-37). Commit-subject wording (Leorio) surfaced in the body. Reviewed against 2c7f10b..d0f5fa0.

Round 6 update

BLOCKED. CRF-37, CRF-38, CRF-39 addressed (verified by churn guard against R6 head 038921b; CI green). CRF-16 silent: the R5 re-raise asked for a ticket, a code addition, or an explicit acceptance on the record, and no author response followed. No Netero, no panel. Review blocked until the author responds to CRF-16 or pushes a fix.

About deep-review

CRF = Coder Review Finding (P0-P4, Nit, Note)

Reviewer Focus
Bisky tests
Chopper ops/errors
Churn-guard change verification
Ging language modernization
Gon naming
Hisoka edge cases
Killua perf
Kite change integrity
Knov contracts
Knuckle SQL
Komugi flake/determinism
Kurapika security
Law decomposition
Leorio docs
Luffy product
Mafu-san process
Mafuuu contracts
Melody dispatch/pairing
Meruem structural
Nami frontend
Netero mechanical checks
Pariston premise testing
Pen-botter product gaps
Razor verification
Robin duplication
Ryosuke Go arch
Takumi concurrency
Zoro shape

🤖 Managed by Coder Agents.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9f6d4996e6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/database/migrations/000566_chat_model_config_org_explosion.up.sql Outdated

@coder-agents-review coder-agents-review Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First-pass review only: these are mechanical findings from Netero. The full review panel has not yet reviewed this PR and will review after these findings are addressed.

The migration itself holds up well under scrutiny: the unique partial default index cannot be violated by the copy insert (pre-566 all configs live in the default org), the 530-line test covers per-org counts, all four reference remaps, ACL re-key, threshold fan-out with hostile keys, and a full down/up round-trip, and the down's accepted fidelity loss is documented in both the file and the PR description. Severity count: 1 P1, 1 P3, 3 Notes.

The P1 is the one that matters given the stack: the test's rewind-and-step-to-latest re-applies every migration after 566, so the stacked RBAC PR adding 000567 will redden this test on its merge ref the moment it rebases onto this commit. Netero verified this by simulation with a fake 000567, and I independently confirmed the rewind/re-apply structure in the test.

Fun quote from the first pass: "The false rationale actively misleads: it hides the fact that stopping the stepper at a target version is safe, which is precisely the fix for the P1 above."

🤖 This review was automatically generated with Coder Agents.

Comment thread coderd/database/migrations/migrate_test.go Outdated
Comment thread coderd/database/migrations/migrate.go Outdated
Comment thread coderd/database/migrations/000566_chat_model_config_org_explosion.up.sql Outdated
Comment thread coderd/database/migrations/000566_chat_model_config_org_explosion.up.sql Outdated
Comment thread coderd/database/migrations/000566_chat_model_config_org_explosion.down.sql Outdated
@ethanndickson
ethanndickson force-pushed the ethan/codagt-709-model-config-migration branch from 9f6d499 to 22afd6a Compare August 10, 2026 06:57
@ethanndickson

Copy link
Copy Markdown
Member Author

@codex review

@ethanndickson

Copy link
Copy Markdown
Member Author

/coder-agents-review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 483b4997e9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/database/migrations/000566_chat_model_config_org_explosion.down.sql Outdated
Comment thread coderd/x/chatd/subagent.go Outdated

@coder-agents-review coder-agents-review Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First-pass review only: these are mechanical findings from Netero, plus a decomposition analysis from Law. The full review panel has not yet reviewed this PR; this is the second consecutive first-pass round, so the panel reviews next round regardless of what remains open.

All five round-1 findings were verified fixed against the code, not taken on claim: the migration test now applies the 000566 up/down SQL directly (no rewind, robust to the stacked 000567), MigrationFS and its false rationale are gone, the up migration stages one INSERT via the copy map, DISTINCT ON is removed, and the down deletes only exact copy_id threshold keys. Good round.

Severity count this round: 1 P0, 1 P3, 1 Nit, 2 Notes. The P0 is the CI failure on test-go-pg/pg-17/race-pg, run-proven and root-caused to a test helper seeding cross-org configs.

Two process observations from Law (verdict: don't split, advisory):

  1. Commit 483b499 (fix: enforce organization-local chat model configs) bundles new feature scope, the deletion of the default-org fallback and org checks across five chatd call paths, with the round-1 finding fixes. The PR title and description ("add organization chat model migration and telemetry") do not mention this access-control behavior change. Please amend the description to cover it, or split it into its own stacked PR.
  2. The panel will treat that enforcement code as unreviewed-until-now code next round, not as verified fixes.

Fun quote from the first pass: "Pre-cutover the default-org fallback masked the mismatch; this PR removes the fallback and adds modelConfig.OrganizationID != organizationID rejection, so every chat created through that helper now fails validation."


coderd/x/chatd/configcache_internal_test.go:32

Note [CRF-9] stubChatConfigStore.getDefaultOrganization and defaultOrganizationCall are now unused. (Netero)

After this PR's test rewrites, no test sets the func field or reads the counter (grep: only the declaration and the GetDefaultOrganization method body reference them). The method itself still serves as a panic guard against a fallback regression, which is worth keeping; the write-only counter and never-set func field are dead.

🤖

🤖 This review was automatically generated with Coder Agents.

Comment thread coderd/x/chatd/chatd.go Outdated
Comment thread coderd/exp_chats.go Outdated
Comment thread coderd/x/chatd/compaction_override.go Outdated
Comment thread coderd/database/migrations/000566_chat_model_config_org_explosion.up.sql Outdated
@ethanndickson
ethanndickson force-pushed the ethan/codagt-709-model-config-migration branch from 483b499 to 2d922f5 Compare August 10, 2026 13:33
@ethanndickson ethanndickson changed the title feat: add organization chat model migration and telemetry feat(coderd): report chat model config organization in telemetry Aug 10, 2026
@ethanndickson

Copy link
Copy Markdown
Member Author

CRF-9 is obsolete after removing the organization-enforcement change. The current PR does not change the Chatd config-cache stub.

@ethanndickson

Copy link
Copy Markdown
Member Author

@codex review

re: organization-local migration and runtime findings

Migration 000566_chat_model_config_org_explosion and the Chatd organization-enforcement changes are removed from this PR. Migration 000565 leaves legacy model configurations in the default organization. The current diff only adds organization_id to chat model configuration telemetry.

Organization-local selection for new chats, including explicit cross-organization rejection, belongs to PR #27959.

Please review the current telemetry-only diff and confirm that deleting migration 566 leaves no stale migration reference or generated-code gap.

@ethanndickson

Copy link
Copy Markdown
Member Author

/coder-agents-review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Hooray!

Reviewed commit: 2d922f561b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@coder-agents-review coder-agents-review Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First panel review of this PR (rounds 1 and 2 were first-pass only). The round-2 re-scope did exactly what was asked: migration 000566 and the org enforcement moved to their own stacked PRs, leaving a single-concern, 4-file telemetry diff. Multiple reviewers independently verified the generated code is byte-identical to fresh sqlc output, the schema contract (NOT NULL, FK, per-org default index) backs every type choice, the field follows the established snapshot convention (workspaces, templates, groups, jobs already report organization_id), and the covering test passes against Postgres including a -race GOMAXPROCS sweep. All 10 prior findings verified closed.

Severity count: 1 P2, 3 P3, 2 Notes.

On the red CI board: eight reviewers flagged it at P1 as an undiagnosed blocker. Two reviewers reached the public Actions API and proved the 20 "failures" are a cancelled run superseded one second later by a duplicate run on the same head; the live run is green and the current required check is success (orchestrator re-verified: 19 cancelled + 1 stale required failure vs 19 success). There is no CI failure. Recorded as a Note so the next reader of this PR does not repeat the investigation.

One process observation (Leorio): two of the three commit subjects describe work the final diff no longer contains, and refactor: on 2d922f5 mislabels a scope removal. Under squash merge this evaporates and the PR title is accurate; if anything other than squash lands this, the history reads as ~1,900 lines of crossed-out surgery.

Fun quote from the panel: "Shall I show you what a 4-file telemetry diff cannot do? It cannot break changes (a path filter), lint-actions (no workflow files touched), offlinedocs, Storybook, and fmt all at once."

🤖 This review was automatically generated with Coder Agents.

Comment thread coderd/telemetry/telemetry_test.go
Comment thread coderd/database/queries/chats.sql Outdated
Comment thread coderd/telemetry/telemetry.go Outdated
Comment thread coderd/telemetry/telemetry.go Outdated
Comment thread coderd/database/queries/chats.sql
Comment thread coderd/telemetry/telemetry.go
@ethanndickson
ethanndickson force-pushed the ethan/codagt-709-model-config-migration branch from 5179677 to 246a83d Compare August 11, 2026 00:14
@ethanndickson ethanndickson changed the title feat(coderd): report chat model config organization in telemetry feat(coderd): fan out chat model configs to every organization Aug 11, 2026
@ethanndickson

Copy link
Copy Markdown
Member Author

@codex review

re: restored fan-out migration and renumbering

This revision restores migration chat_model_config_org_explosion, now numbered 000568, and keeps the telemetry organization_id change. Main added migrations 000565 and 000566, so the chat model config migrations moved to 000567 (schema, #27955), 000568 (fan-out, this PR), and 000569 (scopes, #27957).

Please focus on:

  1. The up migration when a destination organization already owns a live default configuration. The copy must give up is_default, and the migration must not violate idx_chat_model_configs_single_default.
  2. The down migration mapping from copy to original. It matches on every field copied verbatim, excluding is_default, and pairs duplicates with ROW_NUMBER().
  3. The chat_compaction_threshold_pct:<id> copy and delete rules, including malformed, empty-suffix, and unrelated keys.
  4. The reference remaps for chats, chat_messages, chat_queued_messages, and chat_debug_runs.
  5. Migration ordering against main, and any stale migration number reference.

Known gap by design: chat_personal_model_override:root is user-global, so the migration cannot fan it out. The up migration documents this.

Generated by Coder Agents on behalf of @ethanndickson.

@ethanndickson

Copy link
Copy Markdown
Member Author

/coder-agents-review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 246a83d365

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/database/migrations/000568_chat_model_config_org_explosion.down.sql Outdated
Comment thread coderd/database/migrations/migrate_test.go Outdated

@coder-agents-review coder-agents-review Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The restored migration is strong work, and the panel verified that empirically, not on claim: Bisky mutation-tested both regression tests (breaking the is_default CASE and the down's field match made the right tests fail), Mafu-san substituted the old R2-era SQL and reproduced both counterfactual failure claims from the PR description exactly, and multiple reviewers independently confirmed the rank-pairing rollback, the pre-statement-snapshot is_default demotion, and the migration-specific temp-table rationale against the actual pgTxnDriver. The down migration's header was called out as a model for how a data-destroying rollback should be documented. The CI block in the review context is again a stale superseded run (verified: live run green, one job pending at review time); recorded per CRF-15, no action needed.

Severity count: 4 P2, 8 P3, 2 Nits, 4 Notes (includes Netero's first-pass CRF-18 P2, CRF-19 P4, CRF-20 Nit).

The substantive P2 is the mid-stack window (CRF-21): the fan-out merges before any org-aware read surface exists, and during that window multi-org deployments see and can write things this migration exists to eliminate. CRF-19 (P4) is related but distinct: orgs created after the migration get no configs and no stack PR owns seeding them. Both need either an in-PR mitigation or an explicit owner; a human decision, not silence.

Process observation: comment quality is the pattern this round. Gon audited all 50 in-scope comments and 22 are restatements of the code or the assertion messages below them (consolidated into CRF-28/29/30), and three comments describe things that do not exist ("late" config, the uuid-cast hazard, the "zero-member" axis; CRF-27/30/31). Same failure shape as CRF-18's renumber residue: mechanical rewrites landed without a final read of the prose against the code. One sweep fixes the class.

Fun quote from the panel: "The chart names a patient who isn't in the hospital."

🤖 This review was automatically generated with Coder Agents.

Comment thread coderd/database/migrations/migrate_test.go Outdated
Comment thread coderd/database/migrations/000568_chat_model_config_org_explosion.up.sql Outdated
Comment thread coderd/telemetry/telemetry_test.go Outdated
Comment thread coderd/database/migrations/000568_chat_model_config_org_explosion.up.sql Outdated
Comment thread coderd/database/migrations/000568_chat_model_config_org_explosion.up.sql Outdated
Comment thread coderd/database/migrations/000568_chat_model_config_org_explosion.up.sql Outdated
Comment thread coderd/database/migrations/000568_chat_model_config_org_explosion.up.sql Outdated
Comment thread coderd/database/migrations/migrate_test.go Outdated
Comment thread coderd/database/migrations/migrate_test.go Outdated
Comment thread coderd/database/migrations/migrate_test.go Outdated
@ethanndickson
ethanndickson force-pushed the ethan/codagt-709-model-config-migration branch from 246a83d to cc15f61 Compare August 11, 2026 13:18
@coder coder deleted a comment from ccr99bmkkz-coder Aug 16, 2026
@ethanndickson
ethanndickson force-pushed the ethan/codagt-709-model-config-migration branch from 01cc507 to 6ad0c7f Compare August 19, 2026 15:50
@ethanndickson
ethanndickson force-pushed the ethan/codagt-709-model-config-migration branch 2 times, most recently from 0f1416a to 038921b Compare August 19, 2026 17:24

Copy link
Copy Markdown
Member Author

$@codex review\n\nThis review request was generated by Coder Agents.

Copy link
Copy Markdown
Member Author

/coder-agents-review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep it up!

Reviewed commit: 038921bc1a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@coder-agents-review coder-agents-review Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This round is blocked; no panel review ran.

Good progress first: CRF-37 (inherited CI failure) resolved as predicted, the base fix landed on #27955 and CI is green on this head. CRF-38 and CRF-39 (comment padding and wording) are verified fixed in the current head; the rewritten comments state only the invariants.

One finding is silent: CRF-16 (telemetry.go:2616, thread from round 3). The round-5 re-raise laid out the new evidence, with the fan-out migration gone, cross-org chat-to-config references are permanent steady state and the snapshot cannot attribute them because telemetry.Chat carries no OrganizationID, and asked for one of three responses: link a ticket for adding Chat.OrganizationID in a follow-up, add the field in this PR (one field, one projection, one assertion), or explicitly accept the blind spot on the record. No response followed; the thread is still marked resolved with the re-raise as its last message, and no ticket appears in the PR.

A Note-severity finding does not block a merge on its own weight, but an unanswered re-raise blocks further review: the panel cannot re-review code whose one open question the author has not engaged. Reply on the CRF-16 thread with a ticket link, a fix, or an explicit acceptance, and the next round proceeds normally.

🤖 This review was automatically generated with Coder Agents.

@ethanndickson
ethanndickson marked this pull request as ready for review August 20, 2026 07:43
@ethanndickson ethanndickson changed the title feat(coderd): report chat model organizations in telemetry feat(coderd): include chat model organization IDs in telemetry Aug 20, 2026
@ethanndickson
ethanndickson force-pushed the ethan/codagt-709-model-config-migration branch from 038921b to 405b98f Compare August 20, 2026 12:03
@ethanndickson
ethanndickson force-pushed the ethan/codagt-709-model-config-migration branch 3 times, most recently from ca5a73f to 75f63b3 Compare August 21, 2026 19:11
@ethanndickson
ethanndickson force-pushed the ethan/codagt-709-model-config-migration branch from 75f63b3 to d0cca9f Compare August 23, 2026 10:39

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

@ethanndickson
ethanndickson force-pushed the ethan/codagt-709-model-config-migration branch from d0cca9f to ed00958 Compare August 24, 2026 09:04
Base automatically changed from ethan/codagt-709-model-config-org-schema to main August 24, 2026 09:30
@ethanndickson
ethanndickson force-pushed the ethan/codagt-709-model-config-migration branch from ed00958 to 3aa39f6 Compare August 24, 2026 09:30
@ethanndickson
ethanndickson merged commit ac41b5b into main Aug 24, 2026
24 checks passed
@ethanndickson
ethanndickson deleted the ethan/codagt-709-model-config-migration branch August 24, 2026 09:43
@github-actions github-actions Bot locked and limited conversation to collaborators Aug 24, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants