Fix executable UDF command parameter validation by goutamadwant · Pull Request #107983 · ClickHouse/ClickHouse · GitHub
Skip to content

Fix executable UDF command parameter validation#107983

Merged
alexey-milovidov merged 6 commits into
ClickHouse:masterfrom
goutamadwant:fix-udf-invalid-command-parameter
Jul 2, 2026
Merged

Fix executable UDF command parameter validation#107983
alexey-milovidov merged 6 commits into
ClickHouse:masterfrom
goutamadwant:fix-udf-invalid-command-parameter

Conversation

@goutamadwant

@goutamadwant goutamadwant commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Closes: #52023

Changelog category :

  • Bug Fix (user-visible misbehavior in an official stable release)

Changelog entry :

Fixed executable user-defined function command parameter parsing so placeholders with empty or invalid names are not accepted as parameters.

Version info

  • Merged into: 26.7.1.419

@CLAassistant

CLAassistant commented Jun 19, 2026

Copy link
Copy Markdown

@alexey-milovidov alexey-milovidov self-assigned this Jun 19, 2026
@alexey-milovidov alexey-milovidov added the can be tested Allows running workflows for external contributors label Jun 19, 2026
@clickhouse-gh

clickhouse-gh Bot commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Workflow [PR], commit [42c2808]

Summary:


AI Review

Summary

This PR fixes the dead validation branch in executable UDF command-parameter parsing so empty or otherwise invalid placeholder names are no longer registered as command parameters, and it extends the integration test to cover the empty-name variants that were previously missing. I did not find any remaining correctness, compatibility, or test-coverage gaps in the current diff.

Final Verdict

✅ Approve. No new inline findings.

@clickhouse-gh clickhouse-gh Bot added the pr-bugfix Pull request with bugfix, not backported by default label Jun 19, 2026
Comment thread tests/integration/test_executable_user_defined_function/test.py Outdated
alexey-milovidov and others added 3 commits June 25, 2026 04:53
The previous test only covered a non-empty invalid placeholder name
(`{test parameter:UInt64}`). The fix to `extractParametersFromCommand`
also rejects empty (`{:UInt64}`) and blank (`{ :UInt64}`) placeholder
names, as the changelog promises. Add `executable` UDF configs for both
cases and assert the specific `number of parameters does not match.
Expected 0. Actual 1` error for all three invalid-name variants, so the
test proves the full contract.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@alexey-milovidov

Copy link
Copy Markdown
Member

Merged fresh master into the branch (it was ~4 days / 1091 commits behind and red) to re-trigger CI. The merge was clean — no conflicts, and the net diff is unchanged (the single-character fix in ExternalUserDefinedExecutableFunctionsLoader.cpp plus the two integration-test files).

The only public-CI red was AST fuzzer (amd_debug, targeted, old_compatibility)Logical error: Too large size (A) passed to allocator (STID 1367-1ab3). This is an unrelated, long-standing master-wide fuzzer flake: the same error fires on many unrelated PRs (#100377, #101435, #96630, #106255, …) and directly on master (pull_request_number = 0 on 2026-05-12, 2026-05-11, 2026-04-02). The stack trace is in generic pipeline execution (ExceptionKeepingTransformPipelineExecutor), with nothing touching UDF command-parameter parsing. Tracked by #83577. The fresh CI run should clear it.

Also resolved the addressed (and now outdated) review thread about the dashed parameter name — the test config now uses a space ({test parameter:UInt64}), per the applied suggestion.

The remaining CH Inc sync failure is the internal sync check; the push re-runs it.

@alexey-milovidov

Copy link
Copy Markdown
Member

Re-checked the branch and the fresh CI run (commit 7ed1b6d8). The branch is 0 commits behind master, MERGEABLE, the fix is approved, all review threads are resolved, and AI Review is ✅ Approve with no required actions. The net diff is unchanged: the single-character fix (&&||) in ExternalUserDefinedExecutableFunctionsLoader.cpp plus the two integration-test files.

Both CI reds are unrelated, pre-existing master-wide flakes — neither touches executable-UDF command-parameter parsing:

1. Stateless tests (arm_binary, parallel)04204_global_in_join_max_rows_to_transfer_103333

  • Failure: Code: 102. UNEXPECTED_PACKET_FROM_SERVER: Unexpected packet from server 127.0.0.2:9000 (expected TablesStatusResponse, got ProfileInfo): While executing Remote. — a transient distributed-connection protocol race, not a test-logic failure.
  • The harness's own randomized-settings diagnosis re-ran the test 154 times with the same settings: Runs: 154, Failed: 0"All reruns passed. The failure is not reproducible (likely a transient issue)."
  • The test is a GLOBAL IN / GLOBAL JOIN max_rows_to_transfer regression test — it does not involve UDFs.
  • This expected TablesStatusResponse, got ProfileInfo flake is master-wide: 51 occurrences across 23 distinct tests and 44 distinct PRs in the last 30 days (CIDB), and 04204 itself failed directly on master (pull_request_number = 0) on 2026-06-29 and 2026-06-30.

2. Stress test (arm_tsan)Hung check failed, possible deadlock found

  • Tracked by the open master-wide flake issue Hung check failed, possible deadlock found #107941. The hung-check stack trace is generic query-analyzer / function-execution (QueryAnalyzer::resolveFunction, IExecutableFunction::execute), with nothing specific to UDF command-parameter parsing.

@groeneai, the 04204 red is an instance of the master-wide UNEXPECTED_PACKET_FROM_SERVER (expected TablesStatusResponse, got ProfileInfo) distributed-protocol flake (51 hits / 23 tests / 44 PRs in 30d, including master); it has no tracking issue yet. Could you investigate and provide a fix (or open a tracking issue) in a separate PR? The hung check is already tracked by #107941.

No code change is needed on this PR — it is ready to merge once CI settles.

@groeneai

Copy link
Copy Markdown
Contributor

@alexey-milovidov The 04204 red is that family, and your own #108854 already fixes it - a separate PR or tracking issue would just duplicate it.

I am tracking #108854 to confirm the desync stops on master after it merges. Hung check is #107941. Nothing needed on this PR.

@alexey-milovidov

Copy link
Copy Markdown
Member

Re-checked the completed CI run on commit 7ed1b6d8. The branch is MERGEABLE, the fix is approved, all review threads are resolved, AI Review is ✅ Approve, and the net diff is unchanged: the single-character fix (&&||) in ExternalUserDefinedExecutableFunctionsLoader.cpp plus the two integration-test files. The branch's merge base is from today (only a handful of commits behind master), so no re-merge is needed.

All three CI reds are unrelated, pre-existing master-wide flakes — none touches executable-UDF command-parameter parsing:

1. Stateless tests (arm_binary, parallel)04204_global_in_join_max_rows_to_transfer_103333 — the UNEXPECTED_PACKET_FROM_SERVER (expected TablesStatusResponse, got ProfileInfo) distributed-protocol desync. The harness re-ran it 154 times with the same settings (all passed). Already covered by #108854.

2. Stress test (arm_tsan)Hung check failed, possible deadlock found — tracked by the master-wide flake issue #107941. The hung-check stack is generic query-analyzer/function execution, with nothing specific to UDF parsing.

3. Upgrade check (amd_release)Error message in clickhouse-server.log — this third red surfaced after my earlier triage (the job was still running then), so it was not covered above. It is also an unrelated master-wide flake:

@groeneai, the Upgrade check (amd_release) red here is the FUNCTION_THROW_IF_VALUE_IS_NON_ZERO upgrade-log false-positive coming from the deliberately-failing throwIf mutation in 02597_column_update_tricky_expression_and_replication.sql — 24 distinct PRs in 7d, no tracking issue yet. Could you investigate and provide a fix (or open a tracking issue) in a separate PR? The 04204 desync is covered by #108854 and the hung check by #107941.

No code change is needed on this PR — it is correct, approved, and ready to merge once the flakes clear.

@alexey-milovidov

Copy link
Copy Markdown
Member

Merged fresh master into the branch (head 7ed1b6d842c2808dc47; it was ~1300 commits / ~1.5 days behind and red) to re-trigger CI. The merge was clean — no conflicts, and the net diff is unchanged: the single-character fix (&&||) in ExternalUserDefinedExecutableFunctionsLoader.cpp plus the two integration-test files.

The re-merge is worthwhile now (unlike at the time of the earlier triage) because the fixers for the three flake reds have since landed in master:

So the fresh run should clear reds #1 and #3 deterministically. The remaining possibility is the generic Stress (arm_tsan) hung-check flake tracked by #107941, which is unrelated to executable-UDF parsing and typically does not recur on re-run.

No code change is needed — the fix is correct, approved, and ready to merge once CI settles.

@clickhouse-gh

clickhouse-gh Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

LLVM Coverage Report

Metric Baseline Current Δ
Lines 85.50% 85.50% +0.00%
Functions 92.60% 92.60% +0.00%
Branches 77.60% 77.70% +0.10%

Changed lines: Changed C/C++ lines covered: 0/5 (0.00%) · Uncovered code

Full report · Diff report

@alexey-milovidov alexey-milovidov added this pull request to the merge queue Jul 2, 2026
Merged via the queue into ClickHouse:master with commit 0d8ebcb Jul 2, 2026
176 checks passed
@robot-clickhouse-ci-1 robot-clickhouse-ci-1 added the pr-synced-to-cloud The PR is synced to the cloud repo label Jul 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

can be tested Allows running workflows for external contributors pr-bugfix Pull request with bugfix, not backported by default pr-synced-to-cloud The PR is synced to the cloud repo

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Invalid parameter names in executable UDF commands are silently accepted due to dead validation code

5 participants