Revert dynamic column support by dai-chen · Pull Request #5139 · opensearch-project/sql · GitHub
Skip to content

Revert dynamic column support#5139

Merged
dai-chen merged 4 commits into
opensearch-project:mainfrom
dai-chen:revert-dynamic-column-support
Feb 11, 2026
Merged

Revert dynamic column support#5139
dai-chen merged 4 commits into
opensearch-project:mainfrom
dai-chen:revert-dynamic-column-support

Conversation

@dai-chen

@dai-chen dai-chen commented Feb 11, 2026

Copy link
Copy Markdown
Collaborator

Description

Reverts the dynamic-column feature for spath command introduced in PRs below. This is the first step toward replacing it with a map-based extract-all mode as proposed in #4307 (comment).

Related Issues

Resolves partially #4307

Check List

  • New functionality includes testing.
  • New functionality has been documented.
  • New functionality has javadoc added.
  • New functionality has a user manual doc added.
  • New PPL command checklist all confirmed.
  • API changes companion pull request created.
  • Commits are signed per the DCO using --signoff or -s.
  • Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@dai-chen dai-chen self-assigned this Feb 11, 2026
@coderabbitai

coderabbitai Bot commented Feb 11, 2026

Copy link
Copy Markdown
Contributor

…-project#5075)"

This reverts commit 7630db8.

Signed-off-by: Chen Dai <daichen@amazon.com>
This reverts commit 633d760.

Signed-off-by: Chen Dai <daichen@amazon.com>
…ject#5028)"

This reverts commit 65baa2a.

Signed-off-by: Chen Dai <daichen@amazon.com>
Signed-off-by: Chen Dai <daichen@amazon.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 7

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
core/src/main/java/org/opensearch/sql/expression/function/PPLBuiltinOperators.java (1)

391-401: ⚠️ Potential issue | 🔴 Critical

Change MAP_REMOVE toUDF name from "MAP_REMOVE" to "map_remove" to match the enum definition.

The MAP_REMOVE operator uses uppercase "MAP_REMOVE" in .toUDF(), but BuiltinFunctionName defines it as lowercase "map_remove" (line 76). This mismatch will cause function lookup failures. The correct pattern is shown by MAP_APPEND (line 394), which uses lowercase in both the enum definition and .toUDF("map_append").

core/src/main/java/org/opensearch/sql/expression/function/jsonUDF/JsonExtractAllFunctionImpl.java (1)

75-85: ⚠️ Potential issue | 🟡 Minor

Add JavaDoc with @param and @return for the public eval method.

Public methods must have JavaDoc including @param and @return tags per coding guidelines. The eval method currently has none.

Suggested fix
+  /**
+   * Evaluate JSON input and return extracted key/value map.
+   *
+   * `@param` args function arguments; expects a single JSON string
+   * `@return` map of extracted values or null when input is null/blank/non-object
+   */
   public static Object eval(Object... args) {
🤖 Fix all issues with AI agents
In
`@core/src/main/java/org/opensearch/sql/expression/function/CollectionUDF/MVAppendCore.java`:
- Around line 14-18: Update the JavaDoc for the public method
collectElements(Object... args) in class MVAppendCore: correct the inline tag to
{`@link` MVAppendFunctionImplTest}, add an `@param` tag describing args, add an
`@return` tag that documents the method returns a List of non-null collected
elements or null when all inputs are empty, and add an `@throws` tag (e.g.,
`@throws` NullPointerException) describing that a null args array will cause an
exception; ensure the JavaDoc is placed immediately above the collectElements
method signature.

In
`@core/src/main/java/org/opensearch/sql/expression/function/FunctionResolver.java`:
- Around line 11-12: Replace the invalid Javadoc {`@ref`} tags with {`@link`} in the
interface Javadoc so references to FunctionBuilder and FunctionSignature resolve
correctly; update the doc comment in FunctionResolver (the interface
declaration) to use {`@link` FunctionBuilder} and {`@link` FunctionSignature}
wherever {`@ref` FunctionBuilder} or {`@ref` FunctionSignature} appears.

In
`@core/src/main/java/org/opensearch/sql/expression/function/jsonUDF/JsonExtractAllFunctionImpl.java`:
- Around line 36-40: Update the Javadoc in JsonExtractAllFunctionImpl: replace
the non-standard inline tag {`@ref` JsonExtractAllFunctionImplTest} with the
standard Javadoc tag {`@link` JsonExtractAllFunctionImplTest} so the reference to
the test class is recognized by javadoc; ensure any other occurrences of {`@ref`
...} in the class doc are similarly converted to {`@link` ...}.

In
`@core/src/test/java/org/opensearch/sql/expression/function/CollectionUDF/MapAppendFunctionImplTest.java`:
- Around line 99-107: The test helper assertMapListValues introduces an
unnecessary extracted method; inline its logic back into the tests that call it
and remove the helper. Find the method assertMapListValues in
MapAppendFunctionImplTest, replace each call with the equivalent assertions:
retrieve map.get(key), assert it is a List, cast to List<Object>, assert size
equals expected length and assert each element equals the expectedValues in
order, then delete the assertMapListValues method from the class.

In
`@integ-test/src/test/java/org/opensearch/sql/calcite/remote/CalcitePPLSpathCommandIT.java`:
- Line 27: Update the misleading inline comment that currently reads "Create
test data for string concatenation" to accurately describe the purpose of the
test data for the spath command (e.g., "Create test data for spath command") in
the CalcitePPLSpathCommandIT test so readers know the data is for spath-related
assertions; locate and edit the comment near the test setup in
CalcitePPLSpathCommandIT to make this change.

In
`@integ-test/src/test/java/org/opensearch/sql/calcite/standalone/MapAppendFunctionIT.java`:
- Around line 147-155: The check in assertMapListValue is currently a no-op
(map.containsKey(key)); replace it with an assertion so missing keys fail the
test: in the assertMapListValue method assert that the map contains the provided
key (e.g., using assertTrue or an equivalent assertion helper) before retrieving
value, keeping the rest of the method unchanged (assertMapListValue, map, key,
expectedValues).

In
`@opensearch/src/main/java/org/opensearch/sql/opensearch/request/OpenSearchRequest.java`:
- Around line 57-60: Replace the invalid JavaDoc inline tag {`@ref`
OpenSearchClient.search} in class OpenSearchRequest's method Javadoc with a
proper {`@link`} reference so it renders and passes doclint; change it to {`@link`
OpenSearchClient#search} (or include the full qualified class name if needed) to
correctly link to the search method.
🧹 Nitpick comments (2)
core/src/main/java/org/opensearch/sql/analysis/Analyzer.java (1)

973-981: Consider using PatternUtils constants for consistency.

The aliases "pattern_count" and "sample_logs" are hardcoded here, but CalciteRelNodeVisitor uses PatternUtils.PATTERN_COUNT and PatternUtils.SAMPLE_LOGS constants (lines 820, 823, 844, etc.). Using constants across both files would prevent potential drift if these values need to change.

♻️ Suggested fix using constants
+import org.opensearch.sql.common.patterns.PatternUtils;
+
 ...
 
             new Alias(
-                "pattern_count",
+                PatternUtils.PATTERN_COUNT,
                 new AggregateFunction(BuiltinFunctionName.COUNT.name(), AllFields.of())),
             new Alias(
-                "sample_logs",
+                PatternUtils.SAMPLE_LOGS,
                 new AggregateFunction(
core/src/main/java/org/opensearch/sql/calcite/CalciteRelNodeVisitor.java (1)

1380-1391: Minor: Consider extracting repeated map lookup to a local variable.

The node.getArgumentMap().get("overwrite") is accessed twice in the condition. Extracting it improves readability.

♻️ Optional simplification
-      if (node.getArgumentMap().get("overwrite") == null // 'overwrite' default value is true
-          || (node.getArgumentMap().get("overwrite").equals(Literal.TRUE))) {
+      Literal overwrite = node.getArgumentMap().get("overwrite");
+      if (overwrite == null || overwrite.equals(Literal.TRUE)) { // 'overwrite' default is true
         toBeRemovedFields =
             duplicatedFieldNames.stream()
                 .map(field -> JoinAndLookupUtils.analyzeFieldsForLookUp(field, true, context))

@dai-chen dai-chen merged commit ddb81e9 into opensearch-project:main Feb 11, 2026
39 of 49 checks passed
LantaoJin pushed a commit to LantaoJin/search-plugins-sql that referenced this pull request Feb 12, 2026
* Revert "Adopt appendcol, appendpipe, multisearch to spath (opensearch-project#5075)"

This reverts commit 7630db8.

Signed-off-by: Chen Dai <daichen@amazon.com>

* Revert "Support spath with dynamic fields (opensearch-project#5058)"

This reverts commit 633d760.

Signed-off-by: Chen Dai <daichen@amazon.com>

* Revert "Implement spath command with field resolution (opensearch-project#5028)"

This reverts commit 65baa2a.

Signed-off-by: Chen Dai <daichen@amazon.com>

* Fix failed IT testSpathWithMvCombine

Signed-off-by: Chen Dai <daichen@amazon.com>

---------

Signed-off-by: Chen Dai <daichen@amazon.com>
yuancu added a commit to yuancu/sql-plugin that referenced this pull request Mar 6, 2026
…path PRs

Merge the validation branch with origin/main, resolving conflicts caused
by the revert of dynamic column support (opensearch-project#5139). Spath-related files
(DynamicFieldsHelper, AppendFunctionImpl, spath explain YAMLs) are
deleted to align with the revert. All other conflicts resolved to
preserve the validation round-trip changes from the branch.
yuancu added a commit to yuancu/sql-plugin that referenced this pull request Mar 6, 2026
…path PRs

Merge the validation branch with origin/main, resolving conflicts caused
by the revert of dynamic column support (opensearch-project#5139). Spath-related files
(DynamicFieldsHelper, AppendFunctionImpl, spath explain YAMLs) are
deleted to align with the revert. All other conflicts resolved to
preserve the validation round-trip changes from the branch.

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

maintenance Improves code quality, but not the product

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants