Repository navigation
refactor(advisor/tidb): migrate 4 STATEMENT_DISALLOW_* rules to omni AST (Phase 1.5 §1.5.3 batch 3) - #20199
Conversation
…AST (Phase 1.5 §1.5.3 batch 3)
First Class IIa batch — advisors that consume both AST nodes AND the
statement's source text for advice content. Four rules migrated:
- advisor_stmt_disallow_commit.go (STATEMENT_DISALLOW_COMMIT)
- advisor_stmt_no_select_all.go (STATEMENT_SELECT_NO_SELECT_ALL)
- advisor_stmt_disallow_order_by.go (STATEMENT_DISALLOW_ORDER_BY)
- advisor_stmt_disallow_limit.go (STATEMENT_DISALLOW_LIMIT)
Three use Recipe A (top-level type switch). One — no_select_all —
uses Recipe B (omniast.Walk) because the wildcard can appear in
nested subqueries; the existing fixture exercises this with
'SELECT a FROM (SELECT * FROM t1 JOIN t2) t' and a top-level switch
silently skips the inner SELECT.
Migration progress: 14 of 51 advisors on omni AST.
New shared helper:
- OmniStmt.TrimmedText() in utils.go — returns Text with surrounding
whitespace removed, for embedding into advice content. Multi-
statement input passes leading newlines through Statement.Text;
pingcap's node.Text() returned the trimmed form.
Notable mappings:
pingcap UpdateStmt/DeleteStmt omni
Order != nil → len(OrderBy) > 0
Limit != nil → Limit != nil
pingcap InsertStmt omni
Select interface (SelectStmt | SetOprStmt) → SelectStmt with SetOp field;
omni unifies both forms, single branch covers them.
pingcap SelectStmt omni
Fields.Fields[].WildCard → *StarExpr in TargetList[]
pingcap CommitStmt omni
*ast.CommitStmt → *ast.CommitStmt (same)
Pre-batch checklist:
1. Coupling scan: clean — useLimit was local to disallow_limit.go;
inlined as insertUsesLimit during migration.
2. Shape audit: pingcap *ast.SetOprStmt collapses to omni's
*SelectStmt with SetOp field (similar to migration_compatibility's
ConstrUnique unification). Fixture exercises the inner-subquery
wildcard case which forced Recipe B for no_select_all.
3. Yaml fixture coverage: TestTiDBRules green for all 4 rules.
4. Pingcap-import quarantine: clean for migrated files.
Validation:
go test -count=1 ./backend/plugin/advisor/tidb/... # green
go test -count=1 ./backend/plugin/advisor/mysql/... # green
go test -count=1 ./backend/plugin/parser/tidb/... # green
go build ... # green
golangci-lint run ... # 0 issues
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…overage Round-1 review feedback on PR #20199. #1 Line-number parity for no_select_all. The pre-migration pingcap advisor set checker.line once per top-level statement, so every wildcard hit (including nested-subquery hits) reported the OUTER statement's line. The omni Recipe B port emitted per-match — using the inner SelectStmt's Loc — which was an arguably-better behavioral change but unacknowledged. Reverted to outer-statement line for migration discipline: capture ostmt.AbsoluteLine(0) once per statement and reuse for all advices. Improving advice line precision is a follow-up decision, not a side-effect of parser migration. #2 UNION fixture coverage. omni's Walk recurses into SelectStmt.Left and .Right (verified in walk_generated.go:756-762), so Recipe B correctly visits UNION arm subqueries. Added three fixture cases to pin the contract: - SELECT * FROM t1 UNION SELECT a FROM t2 → flagged (left arm) - SELECT a FROM t1 UNION SELECT * FROM t2 → flagged (right arm) - SELECT a FROM t1 UNION SELECT b FROM t2 → no advice (control) #2 (cont) insertUsesLimit comment. Added documentation that only the outermost LIMIT is checked — a LIMIT inside a parenthesized UNION arm lives on Left.Limit, not on the outer .Limit, and was not flagged by the pre-migration advisor either. Pingcap parity preserved, behavior intentionally not extended. Validation: go test -count=1 ./backend/plugin/advisor/tidb/... # green go test -count=1 ./backend/plugin/advisor/mysql/... # green golangci-lint run ... # 0 issues Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d4a7ee55c1
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Codex P2: 4 findings of line-number regression for inputs with leading
trivia (whitespace + comments). Empirically verified:
Input: "\n-- cmt\nSTMT;"
pingcap (main): line 2
omni (this PR): line 1 ← regression
Codex's claimed direction was reversed for 3 of the 4 cases (claimed
line 3, actual line 1) but the regression itself is real for all 4.
Root cause: pingcap's lexer strips leading whitespace but keeps
comments in node.Text(), so its OriginTextPosition points at the line
of the first comment OR keyword. Omni's Loc.Start is 0 (start of
segment) for top-level statements; AbsoluteLine(0) gives BaseLine+1.
Fix: new OmniStmt.FirstTokenLine() helper finds the line of the first
non-whitespace character in s.Text. Matches pingcap behavior for the
common cases:
| Input | pingcap | FirstTokenLine |
|----------------------------------|---------|----------------|
| "COMMIT;" | 1 | 1 |
| "\n-- cmt\nCOMMIT;" | 2 | 2 |
| "\n/* block */\nCOMMIT;" | 2 | 2 |
One edge case diverges: " \n COMMIT;" (leading whitespace, no
comments) — pingcap reports 1, FirstTokenLine reports 2. pingcap's
lexer preserves whitespace-only leading text in node.Text() so its
leadingNewlinesStripped is 0 here. Real SQL rarely has this shape;
both reported lines reasonably point at the statement.
All 4 batch-3 advisors (commit, no_select_all, disallow_order_by,
disallow_limit) updated to use FirstTokenLine() instead of
AbsoluteLine(0) or AbsoluteLine(n.Loc.Start).
Validation:
go test -count=1 ./backend/plugin/advisor/tidb/... # green
go test -count=1 ./backend/plugin/advisor/mysql/... # green
golangci-lint run ... # 0 issues
Empirical line-parity test across 6 input shapes: 5/6 match pingcap;
the 1 mismatch is the contrived leading-whitespace-only case
documented above.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 793cc8a1a8
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Codex P2: targetListHasStar only matched *ast.StarExpr, missing the qualified-wildcard form. Empirically verified omni's TargetList shape: SELECT * → *ast.StarExpr SELECT t.* → *ast.ColumnRef with Star: true (NOT a StarExpr) Pre-migration pingcap path used field.WildCard != nil, which was set for both bare and qualified wildcards. omni splits them into two node types; targetListHasStar must check both. Fix: extend the type switch to also match *ast.ColumnRef with Star == true. Sanity-checked by reverting: TestTiDBRules fails on the new "SELECT t.* FROM t" fixture case; restored fix passes. Fixture additions: - SELECT t.* FROM t → flagged (qualified) - SELECT a, t.* FROM t → flagged (mixed with column) - SELECT a, t.b FROM t → no advice (control: qualified non-star) Validation: go test -count=1 ./backend/plugin/advisor/tidb/... # green go test -count=1 ./backend/plugin/advisor/mysql/... # green golangci-lint run ... # 0 issues Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… AST (Phase 1.5 §1.5.3 batch 4) (#20204) * refactor(advisor/tidb): migrate naming index/UK/FK convention to omni AST (Phase 1.5 §1.5.3 batch 4) Migrate 3 advisors to omni AST via Recipe A (top-level type-switch): - advisor_naming_index_convention.go - advisor_naming_unique_key_convention.go - advisor_naming_foreign_key_convention.go Pre-batch coupling scan promoted 2 cross-file helpers to utils.go: - getTemplateRegexp (function) - indexMetaData (struct) Both are AST-agnostic Go (no pingcap types in their signatures), so the move is clean. No Linear cleanup ticket needed (unlike canNull/BYT-9362 or tableNewColumn/BYT-9395, which stayed pingcap-typed for un-migrated consumers). Also added OmniStmt.TrimmedText() and OmniStmt.FirstTokenLine() helpers to utils.go. Same helpers also defined on PR #20199 (batch 3); whichever batch merges second drops the duplicate definition (trivial conflict). NEW shape divergence caught and pinned: omni splits ALTER TABLE ADD INDEX (ATAddIndex) vs ADD CONSTRAINT name INDEX (ATAddConstraint) where pingcap collapsed both into AlterTableAddConstraint. Same split applies to UNIQUE: ADD UNIQUE INDEX goes to ATAddIndex, ADD CONSTRAINT name UNIQUE goes to ATAddConstraint. Switch arms now cover both, mirroring mysql omni rule. Fixture pins for the discovered divergence: - naming_index_idx.yaml: ALTER TABLE ADD CONSTRAINT name INDEX (pass + violation pair). - naming_index_uk.yaml: ALTER TABLE ADD UNIQUE INDEX, ALTER TABLE ADD UNIQUE KEY (4 cases: 2 forms x pass+violation each). These also pin the existing ConstrUnique unification (cumulative table item #2). Behavior preserved (NOT aligned with mysql divergence): CREATE FULLTEXT/SPATIAL INDEX statements continue to be checked by the index naming convention advisor (matching pingcap behavior). The mysql omni rule excludes them; aligning tidb with mysql on this is a separate intentional behavior change deferred from this AST migration. Migration progress: 17 of 51 advisors on omni AST. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(advisor/tidb): extract runNamingConventionRule helper to dedup naming trio Sonar flagged 21.9% duplication across the 3 new naming-convention advisors (33.6% / 32.2% / 11.0% per file). The duplication was real: ~140 LOC of structural overlap across the trio in the Check() preamble (payload validation, template-token check, maxLength default) and the advice-emission loop (regex match + length overflow advices). Extracted runNamingConventionRule(checkCtx, cfg, collect) in utils.go. Each advisor's Check() drops to ~17 lines: a config struct (mismatch code, type noun, internal-error title) plus a closure that maps statement nodes to []*indexMetaData. Collector functions (collect{Index,UK,FK}*) unchanged — those carry the rule-specific logic that genuinely differs. Net LOC: +93 helper / -167 dedup = -102 LOC. Sonar duplication on the trio should drop to ~0%. Mysql analogs left similar duplication inline; that precedent pre-dates Sonar duplication gating on new code. Since the trio is new code, the gate applies; the cumulative cost — naming_table, naming_column, naming_auto_increment_column are queued in the handoff doc and would each replicate the same ~50 LOC scaffold without the helper — justifies the extraction now. Verification: - TestTiDBRules green — advice content is byte-identical to the pre-extraction inline form (helper builds the same "<noun> in table ..." and "<noun> `<name>` in table ..." strings via fmt.Sprintf with cfg.typeNoun substitution). Existing fixtures are the safety net. - go test ./backend/plugin/advisor/tidb/ ./backend/plugin/advisor/mysql/ green (cross-engine non-regression). - golangci-lint 0 issues. - Server build clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(advisor/tidb): guard nil constraints in collectFKCreateTable Codex P2 round-1 catch on PR #20204. The line argument ostmt.AbsoluteLine(constraint.Loc.Start) was evaluated BEFORE buildFKMetaData could nil-check, so a nil entry in n.Constraints would panic with a nil pointer dereference instead of being skipped. The helper's internal `if constraint == nil` was vestigial because of Go's argument-evaluation order. Sibling collectIndexCreateTable / collectUKCreateTable already nil-checked at loop top; collectFKCreateTable was the outlier. collectFKAlterTable was safe by accident — it precomputes stmtLine outside the loop. Fix: nil-check at loop top, mirroring siblings exactly. Added TestCollectFKCreateTableNilConstraintGuard with a synthetic AST containing a nil constraint slot. Omni's parser doesn't produce nils from valid SQL so no YAML fixture can reach this branch — the unit test pins the contract for future refactors. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(advisor/tidb): pin internal-error advice path + clarifying comments Code review pushback on PR #20204 — the runNamingConventionRule extraction's byte-identity claim only stood for the regex-mismatch and length-overflow paths the YAML fixtures exercise. The internal-error advice path (fires when getTemplateRegexp's regexp.Compile fails on the post-substitution template) had no test coverage; an accidental swap of cfg.internalErrorTitle vs cfg.typeNoun, or a regression in Status/Code fields, would go unnoticed. Added TestRunNamingConventionRuleEmitsInternalErrorAdvice — uses an unbalanced-paren format that passes payload-token validation but fails regexp.Compile after no-op substitution. Asserts the advice's Title comes from cfg.internalErrorTitle (not cfg.typeNoun or rule.Type), Code is code.Internal, Status mirrors the rule level, and Content shape matches the pre-extraction inline form. Also added clarifying comments per the same review: - runNamingConventionRule docstring: indexMetaData coupling tradeoff (4-field LCD shared by all callers; future divergence may require widening or replacing the closure shape). - omniIndexColumns docstring: silent skip of expression-based parts (functional indexes contribute no name to column-list substitution; matches pingcap behavior). - Index/UK Check methods: explain the closure-capture pattern for originalMetadata (keeps helper signature uniform across rules that don't need metadata). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…r-batch-2 Conflict on backend/plugin/advisor/tidb/utils.go: - Kept both batch-2's addColumnTargets helper AND main's TrimmedText/ FirstTokenLine helpers (introduced by batch 3 / PR #20199, merged). - Preserved batch-2's intentional concise canNull doc (commit 4e11de1 trimmed it on purpose) over main's verbose form. Same function body. Verified post-merge: - TestTiDBRules + cross-engine mysql tests green. - parser/tidb tests green. - golangci-lint 0 issues. - Server build clean.




Summary
First Class IIa batch — advisors that consume both AST nodes AND the statement's source text for advice content. Four rules migrated:
STATEMENT_DISALLOW_COMMITadvisor_stmt_disallow_commit.go*ast.CommitStmtSTATEMENT_SELECT_NO_SELECT_ALLadvisor_stmt_no_select_all.goomniast.Walk(wildcard can appear in nested subqueries)STATEMENT_DISALLOW_ORDER_BYadvisor_stmt_disallow_order_by.goSTATEMENT_DISALLOW_LIMITadvisor_stmt_disallow_limit.goMigration progress: 14 of 51 advisors on omni AST.
Independent of PRs #20195 (batch 1) and #20198 (batch 2) — different files, parallel-mergeable.
First Recipe B in the migration
no_select_allis the first advisor in this migration that genuinely required Recipe B. The existing yaml fixture exercises this with:The outer SELECT has no wildcard; the inner subquery does. A top-level type switch silently misses it (caught immediately by
TestTiDBRules). Switched toomniast.Walkwith aVisit(node) Visitorimplementation — returningvto recurse,nilto skip subtrees, and ignoring theVisit(nil)post-order signal since this advisor has no post-order work.New shared helper:
OmniStmt.TrimmedText()Added to
utils.go:Mirrors mysql omni's
TrimmedStmtTextpattern. Needed because multi-statement input passes leading newlines throughStatement.Text; pingcap'snode.Text()returned the trimmed form. Without this helper, advice content embeds raw newlines:Commit is not allowed, related statement: "\nCOMMIT;"Commit is not allowed, related statement: "COMMIT;"All 4 advisors in this batch use it.
Notable API mappings
node.Order != nil(UpdateStmt/DeleteStmt)len(node.OrderBy) > 0node.Limit != nilnode.Limit != nil(same shape)node.Select(interface, can be*SelectStmtor*SetOprStmt)node.Select(concrete*SelectStmt; UNION/INTERSECT/EXCEPT collapse to a single SelectStmt with aSetOpfield)node.Fields.Fields[].WildCard != nil*ast.StarExprinTargetList[]*ast.CommitStmt*ast.CommitStmt(same)node.Text()(statement source text, trimmed by parser)ostmt.TrimmedText()Pre-batch checklist
useLimitwas local todisallow_limit.go; inlined asinsertUsesLimitduring migration. No cross-file coupling found.*ast.SetOprStmt(a separate type forSELECT ... UNION SELECT ...) collapses to omni's*SelectStmtwith aSetOpfield — same kind of unification we saw in batch 2's ConstrUnique. The branch logic foruseLimitsimplifies from a type switch to a single field check.TestTiDBRulesgreen for all 4 rules.no_select_all's nested-subquery case is the regression check that forced Recipe B.Validation
References
plans/2026-04-23-omni-tidb-completion-plan.mdPhase 1.5 §1.5.3, §1.5.0 invariants 1, 2, 6, 7.Test plan
TestTiDBRulespasses for all 4 migrated rules (incl. nested-subquery wildcard for no_select_all)TrimmedTexthelper is the right granularity (vs per-callstrings.TrimSpaceat sites)🤖 Generated with Claude Code