{{ message }}
Repository navigation
Step 5: Make Mark to Semi join conversion more stable - #23296
Merged
Merged
Conversation
Member
Member
|
Looks great, thanks! EDIT: I can't add to the merge queue until CI is fixed, maybe @Mytherin can do that. I'll work on fixing CI :) |
lnkuiper
approved these changes
Jun 16, 2026
Member
Mytherin
pushed a commit
to Mytherin/duckdb
that referenced
this pull request
Jun 16, 2026
Saw a flaky failure in duckdb#23296 hope this fixes it
krlmlr
pushed a commit
to krlmlr/duckdb
that referenced
this pull request
Jun 21, 2026
…duckdb/pull/23300) Saw a flaky failure in duckdb#23296 hope this fixes it Upstream-commit: duckdb@68d73b7
krlmlr
pushed a commit
to krlmlr/duckdb
that referenced
this pull request
Jun 21, 2026
…duckdb/pull/23300) Saw a flaky failure in duckdb#23296 hope this fixes it Upstream-commit: duckdb@68d73b7
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
In this PR, I try to make the full decorrelation cases more stable and robust, while allowing more cases. There were a few areas in the optimizer that prevented the system from finding good join types and/or join orders which can regress performance:
convert_mark_to_semiwas statically set to false. However, this was more of a guardrail to prevent the system from establishing semi-optimal join orders, because we were missing some heuristics in thebuild_probe_sideoptimizer for semi joins. This has been added now.... IS NOT DISTINCT ...predicates. Problem is, this can prevent adding dynamic filters.pushdown_mark_joinnow tries to reduce them to regular equality predicates, if the join has been rewritten to a semi join.delim_cte_rewritertechnically did the right thing for full decorrelation cases. However, it is not always desireable to fully decorrelate plans, instead of stopping early. This is a bit hard to decide without cost/cardinality information (maybe we should change that at some point). Therefore, it is a bit conservative when full decorrelation is used. Still, we allow it for more cases now (e. g. TPC-H Q21).For TPC-H Q21, this PR improves performance by about 30-40%, for Q4 by about 25%, while avoiding to regress Q22 and Q17.