{{ message }}
Support spath with dynamic fields#5058
Merged
ykmr1224 merged 28 commits intoJan 26, 2026
Merged
Conversation
Contributor
d1a62c4 to
3defb88
Compare
Signed-off-by: Tomoyuki Morita <moritato@amazon.com>
3defb88 to
2e50f6d
Compare
dai-chen
approved these changes
Jan 26, 2026
dai-chen
left a comment
Collaborator
There was a problem hiding this comment.
Just one concern: I feel it's unclear what future PPL command developers need to understand or implement when adding commands that interact with dynamic fields, since current _MAP implementation seems leaks across multiple layers (field resolution, RelNode visitor, join handling, result post-processing, etc). We can consider improving document or consolidating the logic later if this is valid concern. Thanks!
Contributor
ykmr1224
added a commit
to ykmr1224/sql
that referenced
this pull request
Jan 26, 2026
* Support spath with dynamic fields Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Add explain test and sql conversion test Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Address comments Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix FieldResolutionResultTest Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Address comment Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix explain Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix join logic to adopt dynamic fields Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix join logic Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix spath.md Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Support fillnull and replace Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Update spath.md Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix test failure Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix doc and error for fillnull Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Accept wildcard only at the end of field list Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Minor fix Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Adopt append command to spath Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix join to allow spath in only one input Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix unit test failure Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix join inputs logic Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix test failure Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Move helper methods Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Add _MAP description in the docs Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Extract more from CalciteRelNodeVisitor Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Refactor IT and address comments Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix FieldResolutionResult Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix DynamicFieldsHelper Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Refactor DynamicFieldsHelper Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix DynamicFieldsHelper Signed-off-by: Tomoyuki Morita <moritato@amazon.com> --------- Signed-off-by: Tomoyuki Morita <moritato@amazon.com> (cherry picked from commit 633d760) Signed-off-by: Tomoyuki Morita <moritato@amazon.com>
8 tasks
asifabashar
pushed a commit
to asifabashar/sql
that referenced
this pull request
Jan 26, 2026
* Support spath with dynamic fields Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Add explain test and sql conversion test Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Address comments Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix FieldResolutionResultTest Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Address comment Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix explain Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix join logic to adopt dynamic fields Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix join logic Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix spath.md Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Support fillnull and replace Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Update spath.md Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix test failure Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix doc and error for fillnull Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Accept wildcard only at the end of field list Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Minor fix Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Adopt append command to spath Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix join to allow spath in only one input Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix unit test failure Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix join inputs logic Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix test failure Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Move helper methods Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Add _MAP description in the docs Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Extract more from CalciteRelNodeVisitor Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Refactor IT and address comments Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix FieldResolutionResult Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix DynamicFieldsHelper Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Refactor DynamicFieldsHelper Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix DynamicFieldsHelper Signed-off-by: Tomoyuki Morita <moritato@amazon.com> --------- Signed-off-by: Tomoyuki Morita <moritato@amazon.com>
Merged
LantaoJin
pushed a commit
that referenced
this pull request
Jan 27, 2026
* Support spath with dynamic fields (#5058) * Support spath with dynamic fields Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Add explain test and sql conversion test Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Address comments Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix FieldResolutionResultTest Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Address comment Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix explain Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix join logic to adopt dynamic fields Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix join logic Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix spath.md Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Support fillnull and replace Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Update spath.md Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix test failure Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix doc and error for fillnull Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Accept wildcard only at the end of field list Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Minor fix Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Adopt append command to spath Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix join to allow spath in only one input Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix unit test failure Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix join inputs logic Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix test failure Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Move helper methods Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Add _MAP description in the docs Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Extract more from CalciteRelNodeVisitor Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Refactor IT and address comments Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix FieldResolutionResult Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix DynamicFieldsHelper Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Refactor DynamicFieldsHelper Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix DynamicFieldsHelper Signed-off-by: Tomoyuki Morita <moritato@amazon.com> --------- Signed-off-by: Tomoyuki Morita <moritato@amazon.com> (cherry picked from commit 633d760) Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix for Java 11 Signed-off-by: Tomoyuki Morita <moritato@amazon.com> --------- Signed-off-by: Tomoyuki Morita <moritato@amazon.com>
8 tasks
asifabashar
pushed a commit
to asifabashar/sql
that referenced
this pull request
Jan 27, 2026
* Support spath with dynamic fields Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Add explain test and sql conversion test Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Address comments Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix FieldResolutionResultTest Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Address comment Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix explain Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix join logic to adopt dynamic fields Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix join logic Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix spath.md Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Support fillnull and replace Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Update spath.md Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix test failure Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix doc and error for fillnull Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Accept wildcard only at the end of field list Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Minor fix Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Adopt append command to spath Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix join to allow spath in only one input Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix unit test failure Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix join inputs logic Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix test failure Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Move helper methods Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Add _MAP description in the docs Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Extract more from CalciteRelNodeVisitor Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Refactor IT and address comments Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix FieldResolutionResult Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix DynamicFieldsHelper Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Refactor DynamicFieldsHelper Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix DynamicFieldsHelper Signed-off-by: Tomoyuki Morita <moritato@amazon.com> --------- Signed-off-by: Tomoyuki Morita <moritato@amazon.com>
This was referenced Jan 27, 2026
dai-chen
added a commit
to dai-chen/sql-1
that referenced
this pull request
Feb 11, 2026
This reverts commit 633d760. Signed-off-by: Chen Dai <daichen@amazon.com>
8 tasks
dai-chen
added a commit
to dai-chen/sql-1
that referenced
this pull request
Feb 11, 2026
This reverts commit 633d760. Signed-off-by: Chen Dai <daichen@amazon.com>
dai-chen
added a commit
that referenced
this pull request
Feb 11, 2026
* Revert "Adopt appendcol, appendpipe, multisearch to spath (#5075)" This reverts commit 7630db8. Signed-off-by: Chen Dai <daichen@amazon.com> * Revert "Support spath with dynamic fields (#5058)" This reverts commit 633d760. Signed-off-by: Chen Dai <daichen@amazon.com> * Revert "Implement spath command with field resolution (#5028)" This reverts commit 65baa2a. Signed-off-by: Chen Dai <daichen@amazon.com> * Fix failed IT testSpathWithMvCombine Signed-off-by: Chen Dai <daichen@amazon.com> --------- Signed-off-by: Chen Dai <daichen@amazon.com>
LantaoJin
pushed a commit
to LantaoJin/search-plugins-sql
that referenced
this pull request
Feb 12, 2026
* Revert "Adopt appendcol, appendpipe, multisearch to spath (opensearch-project#5075)" This reverts commit 7630db8. Signed-off-by: Chen Dai <daichen@amazon.com> * Revert "Support spath with dynamic fields (opensearch-project#5058)" This reverts commit 633d760. Signed-off-by: Chen Dai <daichen@amazon.com> * Revert "Implement spath command with field resolution (opensearch-project#5028)" This reverts commit 65baa2a. Signed-off-by: Chen Dai <daichen@amazon.com> * Fix failed IT testSpathWithMvCombine Signed-off-by: Chen Dai <daichen@amazon.com> --------- Signed-off-by: Chen Dai <daichen@amazon.com>
This was referenced Feb 13, 2026
8 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Description
_MAP(dynamic fields), and expanded when result is organized.prefix*,*suffix, and raise error in case those are used.fieldscommand specify*in the middle of field list.appendcol,multisearch,lookupRelated Issues
#4984
Check List
--signoffor-s.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.