Repository navigation
fix(cli): format NOT EXISTS without a nil dereference - #591
Conversation
|
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Pre-merge review: #591
|
| Claim | Result |
|---|---|
The parser builds NOT EXISTS (…) as BinaryExpression{Operator:"NOT", Left: *ExistsExpression, Right: nil} |
Confirmed. pkg/sql/parser/expressions_literal.go is the only place that builds a NOT BinaryExpression, and it always uses this shape. Plain NOT x becomes a UnaryExpression, so the new guard cannot catch any other expression. |
EXISTS is already handled |
Confirmed. case *ast.ExistsExpression exists at sql_formatter.go:850. Point 2 in #587 ("no ExistsExpression case") was wrong. |
| Other binary expressions are unchanged | Confirmed. Corpus output is byte-identical to main except for the two files that used to panic. |
Adversarial cases (CLI formatSQL, compact and readable, then re-parse and re-format to check idempotence)
On main, 26 of these runs panicked. On this branch, none panic.
| Input shape | PR output (compact) | Re-parses | Idempotent |
|---|---|---|---|
WHERE NOT EXISTS (correlated) |
… WHERE NOT EXISTS (SELECT 1 FROM u WHERE u.id = t.id) |
yes | yes |
SELECT NOT EXISTS (…) AS ne (select list) |
unchanged | yes | yes |
CASE WHEN NOT EXISTS (…) THEN … |
unchanged | yes | yes |
HAVING NOT EXISTS (…) |
unchanged | yes | yes |
NOT NOT EXISTS (…) / NOT (NOT EXISTS (…)) |
unchanged | yes | yes |
a = 1 AND NOT EXISTS (…) OR b = 2 |
unchanged | yes | yes |
Nested NOT EXISTS inside a NOT EXISTS subquery |
unchanged | yes | yes |
NOT EXISTS (WITH c AS (…) SELECT …) |
unchanged | yes | yes |
UPDATE … WHERE NOT EXISTS, DELETE … WHERE NOT EXISTS, JOIN … ON … AND NOT EXISTS |
unchanged | yes | yes |
NOT EXISTS (SELECT … UNION SELECT …) |
… WHERE NOT EXISTS () |
no | n/a. Pre-existing bug; see Deferred item 2 |
testdata/mysql/19_not_exists.sql, testdata/corpus/tpch/q21_suppliers_kept_waiting.sql |
panicked on main; now formatted correctly (q21 keeps both the EXISTS and the NOT EXISTS) |
yes | yes |
Corpus totals for testdata/**/*.sql (170 files, compact and readable, 340 runs):
main: 238 ok, 4 panics, 24 re-parse failures, 74 parse errors on input.- PR: 242 ok, 0 panics, 24 re-parse failures (the same ones), 74 parse errors on input.
No regressions.
Should fix before merge (optional, cheap)
-
cmd/gosqlx/cmd/sql_formatter.go:851: the existing comment// Handle EXISTS (subquery) - NOT EXISTS is handled via UnaryExpressionis wrong, and this PR is the right place to fix it. Suggested text:// Handle EXISTS (subquery). NOT EXISTS arrives as BinaryExpression{Operator:"NOT", Right:nil}; see the BinaryExpression case. -
Test coverage (
sql_formatter_test.go,TestSQLFormatter_NotExistsDoesNotPanic): the test covers one shape, readable mode only, and checks a substring. Two changes would make it much stronger:- Make it a table test over a few shapes:
WHERE NOT EXISTS,SELECT NOT EXISTS (…) AS x,CASE WHEN NOT EXISTS,HAVING NOT EXISTS, and nested. - For each shape, run both
Compact: trueandCompact: false, then assert that the output re-parses and that formatting it again gives the same output.
Also, the test never calls
p.Release()and never releases the parsedresultAST; only the separateastObjis released. The newer pattern inTestSQLFormatter_ParenthesizedExpression(defer p.Release()plusdefer ast.ReleaseAST(tree)) is cleaner. Calling(&Formatter{Opts: CLIFormatterOptions{…}}).formatSQL(sql)would go further and test the real CLI path. - Make it a table test over a few shapes:
-
Optional hardening, item 3 of the fix proposed in [BUG] gosqlx format panics (nil pointer) on NOT EXISTS; CLI formatter lacks ExistsExpression case #587: add
if expr == nil { return nil }at the top offormatExpression(sql_formatter.go:619). With it, an unhandled nil sub-expression can no longer crash the CLI throughexpr.TokenLiteral()in thedefault:branch (:894). I found no other nil shape that the parser produces today, so this is defensive only.
Deferred: real bugs, pre-existing on main, out of scope here (I recommend filing issues)
-
ast.SQL()andpkg/formatterrender NOT EXISTS as SQL that does not parse. [BUG] gosqlx format panics (nil pointer) on NOT EXISTS; CLI formatter lacks ExistsExpression case #587 says the library renderers are fine. They are not.- Input:
SELECT a FROM t WHERE NOT EXISTS (SELECT 1 FROM u WHERE u.id = t.id). tree.SQL()returnsSELECT a FROM t WHERE NOT (EXISTS (SELECT 1 FROM u WHERE u.id = t.id) NOT ).formatter.FormatStringgives the same text with;.- Re-parsing either output fails with
E2002 … expected ), got NOT. - Cause: in
pkg/sql/ast/sql.go,BinaryExpression.writeSQLtakes the genericb.Notpath,NOT (+ Left ++ op ++ Right +), which writesop = "NOT"and an empty Right. It needs a branch forOperator == "NOT" && Right == nilthat writesNOTfollowed by Left. - This bug was already there before the perf(ast): make SQL() serialization allocation-light (drop fmt/ToUpper, fix builderPool, writer-based rendering) (#558) #575 rewrite. It affects every
NOT EXISTSgoing throughSQL(),pkg/formatter, and anything else that round-trips through them. I recommend a new high-priority issue.
- Input:
-
The CLI formatter drops set-operation subqueries.
EXISTS (…),IN (…)and the scalar(…)subquery only format*ast.SelectStatement. A*ast.SetOperationbody (UNION/EXCEPT/INTERSECT) is silently dropped:WHERE EXISTS (SELECT 1 FROM u UNION ALL SELECT 2 FROM v)becomesWHERE EXISTS ().a IN (SELECT … UNION SELECT …)becomesa IN ().
Neither output parses.
pkg/formatterhandles both correctly. I recommend a new issue. -
Many expression types are missing from the CLI
formatExpressionswitch. None of them panic. They fall back toTokenLiteral()and corrupt the SQL without any error. These account for the 24 corpus re-parse failures:CastExpression:CAST(a AS INT)anda::intboth becomeCAST.ExtractExpression: becomesEXTRACT.IntervalExpression: becomesINTERVAL.AnyExpression/AllExpression:a = ANY (subq)becomesANY, and the left operand is lost too.TupleExpression:(a, b) IN ((1, 2), (3, 4))becomesTUPLE IN (TUPLE, TUPLE).ArrayConstructorExpression: becomesARRAY.ArraySubscriptExpression/ArraySliceExpression: become[]and[:].RollupExpression/CubeExpression/GroupingSetsExpression: the argument list is dropped.NamedArgument:f(x => 1)becomesf(x).pkg/formatterhas the same bug.
pkg/formatterrenders all of these correctly exceptNamedArgument. I recommend one umbrella issue. The durable fix is to makegosqlx formatdelegate topkg/formatterinstead of keeping a second renderer. Divergence between the two renderers is behind [BUG] GoSQLX does not support parentheses #519, [BUG] ClickHouse parametric aggregates drop params group on render (quantile(0.5)(x) → quantile(x)) #531, [BUG] insert with CTE #537 and [BUG] gosqlx format panics (nil pointer) on NOT EXISTS; CLI formatter lacks ExistsExpression case #587.
Overlap with in-flight PRs #583 and #584
-
sql_formatter.go: no textual overlap.- This PR edits the top of the
*ast.BinaryExpressioncase (around line 665). - fix(cli): render ClickHouse parametric aggregate parameters in gosqlx format (#531) #583 edits the
*ast.FunctionCallcase (around line 699). - fix(parser): accept WITH ... SELECT as the source of INSERT and keep leading CTEs in gosqlx format (#537) #584 edits
formatInsert(around line 259).
Applied in sequence onto
origin/main(fix(cli): format NOT EXISTS without a nil dereference #591, then fix(cli): render ClickHouse parametric aggregate parameters in gosqlx format (#531) #583, then fix(parser): accept WITH ... SELECT as the source of INSERT and keep leading CTEs in gosqlx format (#537) #584), all three apply cleanly to this file. - This PR edits the top of the
-
sql_formatter_test.go: this PR and fix(parser): accept WITH ... SELECT as the source of INSERT and keep leading CTEs in gosqlx format (#537) #584 both append new tests right afterTestSQLFormatter_ParenthesizedExpression, at the end of the file. Whichever merges second gets an append/append conflict. It is trivial to resolve: keep both tests. fix(cli): render ClickHouse parametric aggregate parameters in gosqlx format (#531) #583 adds its own test file, so it does not overlap.
Conventions
- Commit and PR title follow Conventional Commits (
fix(cli): …), and the commit saysFixes #587. - There is no CHANGELOG entry. CONTRIBUTING says "CHANGELOG.md updated (if applicable)", and fix(cli): render ClickHouse parametric aggregate parameters in gosqlx format (#531) #583 and fix(parser): accept WITH ... SELECT as the source of INSERT and keep leading CTEs in gosqlx format (#537) #584 don't add one either. A one-line
### Fixedentry under[Unreleased]would be nice to have: "gosqlx formatno longer panics onNOT EXISTS (subquery)([BUG] gosqlx format panics (nil pointer) on NOT EXISTS; CLI formatter lacks ExistsExpression case #587)". It does not block; a maintainer can add it at release time.
Security and supply chain (fork PR)
No concerns. Safe to run CI: yes.
- Only two files change:
cmd/gosqlx/cmd/sql_formatter.go(+6) andcmd/gosqlx/cmd/sql_formatter_test.go(+30). - Nothing touches
.github/, scripts, the Taskfile,go.modorgo.sum, build tags orinit(). - No network, file or exec calls. The test does no I/O beyond in-memory parsing.
- No non-ASCII or hidden characters in the diff. One commit, authored by the contributor's GitHub noreply address.
- On input handling, the change removes a crash an attacker could trigger. A CLI user or CI job formatting untrusted SQL could abort the process with any
NOT EXISTS. The new code adds one bounded recursion intoLeft, which is no deeper than existing cases and already falls under the parser's recursion limits.
Evidence
go test -race -count=1 ./cmd/gosqlx/...: all ok.go vet ./cmd/gosqlx/...: clean.gofmt -l cmd/gosqlx/cmd: clean.golangci-lint run ./cmd/gosqlx/cmd/...: 0 issues.go test -run TestSQLFormatter_NotExistsDoesNotPanic: PASS on the PR. The issue's reproduction panics onmainwith a nil-pointer dereference atsql_formatter.go:889.- Differential corpus run (CLI
formatSQL, both modes, re-parse and idempotence),mainvs PR: the only differences aremysql/19_not_exists.sqlandcorpus/tpch/q21_suppliers_kept_waiting.sql, which go from PANIC to ok. - Library check on
main:tree.SQL()returnsSELECT a FROM t WHERE NOT (EXISTS (SELECT 1 FROM u WHERE u.id = t.id) NOT ), and re-parsing it fails withE2002 expected ), got NOT.
Independent pre-merge review (adversarial correctness + security/supply-chain), run because the automated claude-review workflow skips fork PRs. Generated with Claude Code.
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
The parser represents NOT EXISTS (subquery) as a NOT with the EXISTS on the left and no right operand. The formatter passed that nil right operand on, and the fallback called TokenLiteral on it, so gosqlx format panicked on any query containing NOT EXISTS. Format the NOT followed by its left operand in that case, and add a test. Fixes #587
9328181 to
396eaf0
Compare

Fixes #587
Summary
Test
🤖 Generated with Claude Code