Merge main to 2.x branch by joshuali925 · Pull Request #771 · opensearch-project/sql · GitHub
Skip to content

Merge main to 2.x branch#771

Merged
joshuali925 merged 21 commits into
opensearch-project:2.xfrom
joshuali925:2.x-sync-main
Aug 17, 2022
Merged

Merge main to 2.x branch#771
joshuali925 merged 21 commits into
opensearch-project:2.xfrom
joshuali925:2.x-sync-main

Conversation

@joshuali925

Copy link
Copy Markdown
Member

Description

[Describe what this change achieves]

Issues Resolved

[List any issues this PR will resolve]

Check List

  • New functionality includes testing.
    • All tests pass, including unit test, integration test and doctest
  • New functionality has been documented.
    • New functionality has javadoc added
    • New functionality has user manual doc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

MitchellGale and others added 21 commits August 2, 2022 14:23
… is surrounded by same type of quote (opensearch-project#696)

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Co-authored-by: Andrew Carbonetto <andrewc@AndrewsMBP2022.local>
Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>
… outside of main OS sql project (opensearch-project#704)

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>
* Add support for highlight to parser and AstExpressionBuilder

Signed-off-by: MaxKsyunz <maxk@bitquilltech.com>

* Add unit test for highlight in AstExpressionBuilder

Signed-off-by: MaxKsyunz <maxk@bitquilltech.com>

* Add unit test for highlight in AstBuilderTest

Signed-off-by: MaxKsyunz <maxk@bitquilltech.com>

* Support highlight as an Unresolved expression.

Signed-off-by: MaxKsyunz <maxk@bitquilltech.com>

* Represent highlight as UnresolvedExpression.

Signed-off-by: MaxKsyunz <maxk@bitquilltech.com>

* Support highlight in Analyzer.

Signed-off-by: MaxKsyunz <maxk@bitquilltech.com>

* Treat highlight as a proper function in AST

In particular, highlightField is an expression now.

Signed-off-by: MaxKsyunz <maxk@bitquilltech.com>

* Add support for highlight in Analyzer

HighlightFunction is converted to LogicalHighlight logical plan.

Signed-off-by: MaxKsyunz <maxk@bitquilltech.com>

* Add a simple IT test for highlight.

Signed-off-by: MaxKsyunz <maxk@bitquilltech.com>

* Register highlight function in the BuiltInFunctionRepository

Signed-off-by: MaxKsyunz <maxk@bitquilltech.com>

* Partial support for highlight in physical plan.

Signed-off-by: MaxKsyunz <maxk@bitquilltech.com>

* Add HighlightOperator.

Signed-off-by: MaxKsyunz <maxk@bitquilltech.com>

* Highlight alpha complete.

Signed-off-by: MaxKsyunz <maxk@bitquilltech.com>

* Initial implementation to upporting highlight('*')

Signed-off-by: forestmvey <forestv@bitquilltech.com>

* Add support for multiple highlight calls in select statement.

Signed-off-by: forestmvey <forestv@bitquilltech.com>

* Removed OpenSearchLogicalIndexScan highlightFields and dependencies. Improved test coverage and fixing checkstyle errors.

Signed-off-by: forestmvey <forestv@bitquilltech.com>

* Added HighlightExpressionTest

Signed-off-by: forestmvey <forestv@bitquilltech.com>

* Added javadocs, minor PR revisions, and fixed jacoco errors by improving coverage for OpenSearchIndexScan

Signed-off-by: forestmvey <forestv@bitquilltech.com>

* Code cleanup, adding parsing failure tests, and adding tests for highlight acceptance as a string literal as well as qualified name.

Signed-off-by: forestmvey <forestv@bitquilltech.com>

* Removing HighlightOperator functionality and unnecessary visitHighlight call in PhysicalPlanNodeVisitor..

Signed-off-by: forestmvey <forestv@bitquilltech.com>

* Adding highlight function to functions.rst and removing unecessary function call in OpenSearchIndexScan.

Signed-off-by: forestmvey <forestv@bitquilltech.com>

* Change highlight fields returned format to array list. Changed highlight all and wildcard to unsupported to open up output formatting changes for multiple returned highlight fields. Change tests for updated coverage and disable highlight all and wildcard tests.

Signed-off-by: forestmvey <forestv@bitquilltech.com>

* Fix bug where invalid schema name was being used for returned highlight fields

Signed-off-by: forestmvey <forestv@bitquilltech.com>

* Fix failing integration tests due to schema changes for highlight expression type.

Signed-off-by: forestmvey <forestv@bitquilltech.com>

Co-authored-by: MaxKsyunz <maxk@bitquilltech.com>
Signed-off-by: penghuo <penghuo@gmail.com>
Signed-off-by: Joshua Li <joshuali925@gmail.com>
Signed-off-by: penghuo <penghuo@gmail.com>
…oject#723)

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>
Co-authored-by: Yury Fridlyand <yuryf@bitquilltech.com>
Signed-off-by: Joshua Li <joshuali925@gmail.com>
* Bump wiremock version

Signed-off-by: Chen Dai <daichen@amazon.com>

* Bump guava and json-smart version

Signed-off-by: Chen Dai <daichen@amazon.com>

* Bump httpclient version

Signed-off-by: Chen Dai <daichen@amazon.com>
* Bump jackson version and add version config in root gradle script

Signed-off-by: Chen Dai <daichen@amazon.com>

* Bump jackson version in JDBC driver

Signed-off-by: Chen Dai <daichen@amazon.com>

Signed-off-by: Chen Dai <daichen@amazon.com>
* add maxResultWindow to LogicalRelation

Signed-off-by: Sean Kao <seankao@amazon.com>

* add maxResultWindow to OpenSearchLogicalIndexScan

Signed-off-by: Sean Kao <seankao@amazon.com>

* OpenSearchRequestBuilder init

Signed-off-by: Sean Kao <seankao@amazon.com>

* request builder: push down and build

Signed-off-by: Sean Kao <seankao@amazon.com>

* plan.build() for building request

Signed-off-by: Sean Kao <seankao@amazon.com>

* maxResultWindow for test utils

Signed-off-by: Sean Kao <seankao@amazon.com>

* fix style

Signed-off-by: Sean Kao <seankao@amazon.com>

* remove plan.build()

Signed-off-by: Sean Kao <seankao@amazon.com>

* fetch result in batches

Signed-off-by: Sean Kao <seankao@amazon.com>

* get index.max_result_window settings

Signed-off-by: Sean Kao <seankao@amazon.com>

* use index.max_result_window to decide scroll

Signed-off-by: Sean Kao <seankao@amazon.com>

* maxResultWindow for aggregation

Signed-off-by: Sean Kao <seankao@amazon.com>

* fix fetch size & for aggregation query

Signed-off-by: Sean Kao <seankao@amazon.com>

* fix rest client get max result window

Signed-off-by: Sean Kao <seankao@amazon.com>

* remove maxResultWindow from logical plan

Signed-off-by: Sean Kao <seankao@amazon.com>

* get max result window when building physical plan

Signed-off-by: Sean Kao <seankao@amazon.com>

* move source builder init to request builder

Signed-off-by: Sean Kao <seankao@amazon.com>

* fix max result window for test & rest client

Signed-off-by: Sean Kao <seankao@amazon.com>

* include request builder in equal comparison

Signed-off-by: Sean Kao <seankao@amazon.com>

* rename getIndexMaxResultWindows

Signed-off-by: Sean Kao <seankao@amazon.com>

* open search rest client test

Signed-off-by: Sean Kao <seankao@amazon.com>

* test: request builder, scroll index scan

Signed-off-by: Sean Kao <seankao@amazon.com>

* fix style

Signed-off-by: Sean Kao <seankao@amazon.com>

* remove getMaxResultWindow from base Table

Signed-off-by: Sean Kao <seankao@amazon.com>

* remove unused import from OpenSearchIndexScan

Signed-off-by: Sean Kao <seankao@amazon.com>

* test index scan

Signed-off-by: Sean Kao <seankao@amazon.com>

* integ test for head command

Signed-off-by: Sean Kao <seankao@amazon.com>

* keep request query size for aggregation

Signed-off-by: Sean Kao <seankao@amazon.com>

* fix rest client test coverage

Signed-off-by: Sean Kao <seankao@amazon.com>

* test for node client

Signed-off-by: Sean Kao <seankao@amazon.com>

* test node client default settings

Signed-off-by: Sean Kao <seankao@amazon.com>

* change Elasticsearch to OpenSearch in comment

Signed-off-by: Sean Kao <seankao@amazon.com>

* fix comments

Signed-off-by: Sean Kao <seankao@amazon.com>

* more test for Head IT

Signed-off-by: Sean Kao <seankao@amazon.com>

* ignore some head IT

Signed-off-by: Sean Kao <seankao@amazon.com>

Signed-off-by: Sean Kao <seankao@amazon.com>
Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>
…pensearch-project#744)

* Added new functionality around argument names not case specific for relevancy based functions. Added unit tests to cover updates.

Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>

* Fixed test cases that used uppercase values in arguments.

Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>

Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
…arch-project#728)

Signed-off-by: MaxKsyunz <maxk@bitquilltech.com>

Signed-off-by: MaxKsyunz <maxk@bitquilltech.com>
Signed-off-by: Joshua Li <joshuali925@gmail.com>

Signed-off-by: Joshua Li <joshuali925@gmail.com>
…oject#755)

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>
Signed-off-by: Joshua Li <joshuali925@gmail.com>
Signed-off-by: penghuo <penghuo@gmail.com>
Signed-off-by: Joshua Li <joshuali925@gmail.com>
@joshuali925 joshuali925 requested a review from a team as a code owner August 17, 2022 18:02
@codecov-commenter

Copy link
Copy Markdown

@dai-chen

Copy link
Copy Markdown
Collaborator

@joshuali925 Thanks for syncing main and 2.x. I found some PRs are already merged to 2.x earlier or being reviewed in auto backport PR. Probably in future we need to make sure all PRs merged to main have backport PR and merged to ?.x automatically. For now, should we remove those and cherry-pick the remaining?

@penghuo

penghuo commented Aug 17, 2022

Copy link
Copy Markdown
Collaborator

@joshuali925 joshuali925 merged commit 9819635 into opensearch-project:2.x Aug 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants