feat(coderd): include chat model organization IDs in telemetry - #27956
Conversation
|
/coder-agents-review |
|
@codex review |
|
Chat: Review posted | View chat Review history
deep-review v0.9.0 | Round 6 | Last posted: Round 6, 38 findings (1 P0, 1 P1, 7 P2, 12 P3, 1 P4, 4 Nit, 12 Note), COMMENT. Review Finding inventoryFinding inventory: PR 27956Findings
Law analysisRound 2, head 483b499, effective +1110 -406 (77.2% test density). Verdict: Don't split. Enforcement: Advisory. Three concerns: (1) migration 000566, (2) telemetry organization_id, (3) strict org-local enforcement (new scope arriving in commit 483b499, not described in the PR title/description). 1 -> 3 dependency chain, single risk story. Advisory conditions: amend the PR description to cover the enforcement change (or split it out as a stacked PR), and the panel must treat the enforcement code as unreviewed-until-now, not as verified fixes. Contested and acknowledgedCRF-15 (Note, chats.sql:2294) - Stale CI run reported as failure
CRF-16 (Note, telemetry.go:2616) - Chat ownership not in snapshot
CRF-16 (R5 re-raise)
Round logRound 1Netero-only (P1 present, panel gated). 1 P1, 1 P3, 3 Note. Reviewed against 425aa7a..9f6d499. Orchestrator independently verified the CRF-1 rewind/re-apply structure in migrate_test.go. Round 2Churn guard: PROCEED, all 5 findings classified author-fixed in 483b499 (branch rebased onto ef9a1a7). Netero verified all five fixes against the code (CRF-1..5 confirmed fixed, tests run locally). Law ran (>1000 effective LOC): Don't split, advisory. Netero found 1 P0 (CI root cause), 1 P3, 1 Nit, 2 Notes; P0 gates the panel again (second consecutive Netero-only round, cap reached, panel runs next round regardless). Reviewed against ef9a1a7..483b499. Round 3Churn guard: PROCEED, all 5 R2 findings addressed by re-scope in 2d922f5 (branch rewritten; migration 000566 and org enforcement removed, moved to stacked PRs). PR is now telemetry organization_id only: 3 effective files, +17 -13. Netero-only cap reached (R1, R2), panel reviews this round. CI shows 20 failed checks including infra jobs (changes, fmt, lint); panel to assess. Panel: bisky, hisoka, mafu-san, mafuuu, pariston, ging-go, gon, leorio, knuckle, chopper, komugi, kite, ryosuke + meruem (wildcard, random from distilled set). Cross-check: 8 reviewers rated the red CI board P1 with unverified cause (gh 401 in worktrees); Pariston and Kite reached the public Actions API and proved the 20 failures are a cancelled run superseded by a green run on the same head; orchestrator re-verified (19 cancelled + 1 stale required failure vs 19 success, current required success). P1s empirically disproved, merged into CRF-15 Note. Test blind spot converged 9 reviewers at P3 (CRF-11). New: 1 P2, 3 P3, 2 Notes posted, 1 Note dropped. Reviewed against 46e2aaa..2d922f5. Round 4Churn guard: PROCEED. 4 addressed (CRF-11..14), 2 acknowledged (CRF-15, CRF-16). PR restructured again: migration restored as 000568 on new base #27955 (migration 000567), new tests including dedicated regression tests for CRF-10 and the Codex rollback P2. Effective +1092 -17 (74.8% test density). Orchestrator verified the R4 context CI block is again a stale superseded run (19 cancelled + stale required vs 17 success, test-go-race-pg pending on live run). No Law (effective additions shrank vs R2 analysis). Netero (advisory): CRF-18 P2, CRF-19 P4, CRF-20 Nit. Panel: bisky, hisoka, mafu-san, mafuuu, pariston, ging-go, gon, leorio, knuckle, chopper, komugi, kite, ryosuke, knov + zoro, meruem (wildcards, random). Cross-check: ACL fallback severity conflict resolved upward (P3 over Note, fail-open consequence); mid-stack window P2 over P3 (Pariston's write-path evidence stronger, Hisoka's picker-dedupe verification retained as scope bound); rollback tuple-match fragility converged from 6 reviewers with two structural alternatives preserved; Gon's 22 comment P2s consolidated into two class findings (CRF-28, CRF-29) plus the inaccurate-comment finding (CRF-30). Mutation testing by Bisky and counterfactual old-SQL runs by Mafu-san both reproduced the PR's regression-test claims. New: 4 P2, 7 P3, 1 Nit (+CRF-19 P4, CRF-20 Nit, CRF-18 P2 from Netero), 4 Notes. Reviewed against 38f5109..246a83d. Round 5Churn guard: PROCEED. All 19 open findings addressed: the fan-out migration left this PR again (base #27955 now owns migration 000571, a default-org backfill only), and the in-scope telemetry items (CRF-20 em-dashes, CRF-29 comment trims) were fixed in code. PR is telemetry-only: +30 -21, 3 effective files. CI failure is real this round (test-go-pg, pg-17, race-pg failed on the live run; verified via check-runs API, no cancelled runs). Panel to root-cause. Netero (advisory): CRF-37 Note (CI inherited from base #27955; orchestrator re-verified the base failure signature). Panel: bisky, komugi, knuckle, gon, leorio, pariston + knov (wildcard, random). Komugi and Knov: no findings; Knov independently re-verified the CRF-11 mechanism and type honesty against 000571. Cross-check: CRF-16 re-raised (Knuckle, Pariston vs Knov; orchestrator sided with re-raise, deferral lacks ticket). New: 1 P2 (CRF-38), 1 Nit (CRF-39), 1 Note (CRF-37). Commit-subject wording (Leorio) surfaced in the body. Reviewed against 2c7f10b..d0f5fa0. Round 6 updateBLOCKED. CRF-37, CRF-38, CRF-39 addressed (verified by churn guard against R6 head 038921b; CI green). CRF-16 silent: the R5 re-raise asked for a ticket, a code addition, or an explicit acceptance on the record, and no author response followed. No Netero, no panel. Review blocked until the author responds to CRF-16 or pushes a fix. About deep-reviewCRF = Coder Review Finding (P0-P4, Nit, Note)
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9f6d4996e6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
First-pass review only: these are mechanical findings from Netero. The full review panel has not yet reviewed this PR and will review after these findings are addressed.
The migration itself holds up well under scrutiny: the unique partial default index cannot be violated by the copy insert (pre-566 all configs live in the default org), the 530-line test covers per-org counts, all four reference remaps, ACL re-key, threshold fan-out with hostile keys, and a full down/up round-trip, and the down's accepted fidelity loss is documented in both the file and the PR description. Severity count: 1 P1, 1 P3, 3 Notes.
The P1 is the one that matters given the stack: the test's rewind-and-step-to-latest re-applies every migration after 566, so the stacked RBAC PR adding 000567 will redden this test on its merge ref the moment it rebases onto this commit. Netero verified this by simulation with a fake 000567, and I independently confirmed the rewind/re-apply structure in the test.
Fun quote from the first pass: "The false rationale actively misleads: it hides the fact that stopping the stepper at a target version is safe, which is precisely the fix for the P1 above."
🤖 This review was automatically generated with Coder Agents.
9f6d499 to
22afd6a
Compare
|
@codex review |
|
/coder-agents-review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 483b4997e9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
First-pass review only: these are mechanical findings from Netero, plus a decomposition analysis from Law. The full review panel has not yet reviewed this PR; this is the second consecutive first-pass round, so the panel reviews next round regardless of what remains open.
All five round-1 findings were verified fixed against the code, not taken on claim: the migration test now applies the 000566 up/down SQL directly (no rewind, robust to the stacked 000567), MigrationFS and its false rationale are gone, the up migration stages one INSERT via the copy map, DISTINCT ON is removed, and the down deletes only exact copy_id threshold keys. Good round.
Severity count this round: 1 P0, 1 P3, 1 Nit, 2 Notes. The P0 is the CI failure on test-go-pg/pg-17/race-pg, run-proven and root-caused to a test helper seeding cross-org configs.
Two process observations from Law (verdict: don't split, advisory):
- Commit 483b499 (
fix: enforce organization-local chat model configs) bundles new feature scope, the deletion of the default-org fallback and org checks across five chatd call paths, with the round-1 finding fixes. The PR title and description ("add organization chat model migration and telemetry") do not mention this access-control behavior change. Please amend the description to cover it, or split it into its own stacked PR. - The panel will treat that enforcement code as unreviewed-until-now code next round, not as verified fixes.
Fun quote from the first pass: "Pre-cutover the default-org fallback masked the mismatch; this PR removes the fallback and adds modelConfig.OrganizationID != organizationID rejection, so every chat created through that helper now fails validation."
coderd/x/chatd/configcache_internal_test.go:32
Note [CRF-9] stubChatConfigStore.getDefaultOrganization and defaultOrganizationCall are now unused. (Netero)
After this PR's test rewrites, no test sets the func field or reads the counter (grep: only the declaration and the
GetDefaultOrganizationmethod body reference them). The method itself still serves as a panic guard against a fallback regression, which is worth keeping; the write-only counter and never-set func field are dead.
🤖
🤖 This review was automatically generated with Coder Agents.
483b499 to
2d922f5
Compare
|
CRF-9 is obsolete after removing the organization-enforcement change. The current PR does not change the Chatd config-cache stub. |
|
@codex review re: organization-local migration and runtime findings Migration Organization-local selection for new chats, including explicit cross-organization rejection, belongs to PR #27959. Please review the current telemetry-only diff and confirm that deleting migration 566 leaves no stale migration reference or generated-code gap. |
|
/coder-agents-review |
|
Codex Review: Didn't find any major issues. Hooray! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
There was a problem hiding this comment.
First panel review of this PR (rounds 1 and 2 were first-pass only). The round-2 re-scope did exactly what was asked: migration 000566 and the org enforcement moved to their own stacked PRs, leaving a single-concern, 4-file telemetry diff. Multiple reviewers independently verified the generated code is byte-identical to fresh sqlc output, the schema contract (NOT NULL, FK, per-org default index) backs every type choice, the field follows the established snapshot convention (workspaces, templates, groups, jobs already report organization_id), and the covering test passes against Postgres including a -race GOMAXPROCS sweep. All 10 prior findings verified closed.
Severity count: 1 P2, 3 P3, 2 Notes.
On the red CI board: eight reviewers flagged it at P1 as an undiagnosed blocker. Two reviewers reached the public Actions API and proved the 20 "failures" are a cancelled run superseded one second later by a duplicate run on the same head; the live run is green and the current required check is success (orchestrator re-verified: 19 cancelled + 1 stale required failure vs 19 success). There is no CI failure. Recorded as a Note so the next reader of this PR does not repeat the investigation.
One process observation (Leorio): two of the three commit subjects describe work the final diff no longer contains, and refactor: on 2d922f5 mislabels a scope removal. Under squash merge this evaporates and the PR title is accurate; if anything other than squash lands this, the history reads as ~1,900 lines of crossed-out surgery.
Fun quote from the panel: "Shall I show you what a 4-file telemetry diff cannot do? It cannot break changes (a path filter), lint-actions (no workflow files touched), offlinedocs, Storybook, and fmt all at once."
🤖 This review was automatically generated with Coder Agents.
5179677 to
246a83d
Compare
|
@codex review re: restored fan-out migration and renumbering This revision restores migration Please focus on:
Known gap by design:
|
|
/coder-agents-review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 246a83d365
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
The restored migration is strong work, and the panel verified that empirically, not on claim: Bisky mutation-tested both regression tests (breaking the is_default CASE and the down's field match made the right tests fail), Mafu-san substituted the old R2-era SQL and reproduced both counterfactual failure claims from the PR description exactly, and multiple reviewers independently confirmed the rank-pairing rollback, the pre-statement-snapshot is_default demotion, and the migration-specific temp-table rationale against the actual pgTxnDriver. The down migration's header was called out as a model for how a data-destroying rollback should be documented. The CI block in the review context is again a stale superseded run (verified: live run green, one job pending at review time); recorded per CRF-15, no action needed.
Severity count: 4 P2, 8 P3, 2 Nits, 4 Notes (includes Netero's first-pass CRF-18 P2, CRF-19 P4, CRF-20 Nit).
The substantive P2 is the mid-stack window (CRF-21): the fan-out merges before any org-aware read surface exists, and during that window multi-org deployments see and can write things this migration exists to eliminate. CRF-19 (P4) is related but distinct: orgs created after the migration get no configs and no stack PR owns seeding them. Both need either an in-PR mitigation or an explicit owner; a human decision, not silence.
Process observation: comment quality is the pattern this round. Gon audited all 50 in-scope comments and 22 are restatements of the code or the assertion messages below them (consolidated into CRF-28/29/30), and three comments describe things that do not exist ("late" config, the uuid-cast hazard, the "zero-member" axis; CRF-27/30/31). Same failure shape as CRF-18's renumber residue: mechanical rewrites landed without a final read of the prose against the code. One sweep fixes the class.
Fun quote from the panel: "The chart names a patient who isn't in the hospital."
🤖 This review was automatically generated with Coder Agents.
246a83d to
cc15f61
Compare
01cc507 to
6ad0c7f
Compare
0f1416a to
038921b
Compare
|
$@codex review\n\nThis review request was generated by Coder Agents. |
|
/coder-agents-review |
|
Codex Review: Didn't find any major issues. Keep it up! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
There was a problem hiding this comment.
This round is blocked; no panel review ran.
Good progress first: CRF-37 (inherited CI failure) resolved as predicted, the base fix landed on #27955 and CI is green on this head. CRF-38 and CRF-39 (comment padding and wording) are verified fixed in the current head; the rewritten comments state only the invariants.
One finding is silent: CRF-16 (telemetry.go:2616, thread from round 3). The round-5 re-raise laid out the new evidence, with the fan-out migration gone, cross-org chat-to-config references are permanent steady state and the snapshot cannot attribute them because telemetry.Chat carries no OrganizationID, and asked for one of three responses: link a ticket for adding Chat.OrganizationID in a follow-up, add the field in this PR (one field, one projection, one assertion), or explicitly accept the blind spot on the record. No response followed; the thread is still marked resolved with the re-raise as its last message, and no ticket appears in the PR.
A Note-severity finding does not block a merge on its own weight, but an unanswered re-raise blocks further review: the panel cannot re-review code whose one open question the author has not engaged. Reply on the CRF-16 thread with a ticket link, a fix, or an explicit acceptance, and the next round proceeds normally.
🤖 This review was automatically generated with Coder Agents.
038921b to
405b98f
Compare
ca5a73f to
75f63b3
Compare
75f63b3 to
d0cca9f
Compare
|
@codex review |
d0cca9f to
ed00958
Compare
ed00958 to
3aa39f6
Compare

Include the owning organization ID for each chat model configuration in telemetry snapshots.
Depends on #27955
This pull request description was generated by Coder Agents.