JDBC support by allwefantasy · Pull Request #283 · NLPchina/elasticsearch-sql · GitHub
Skip to content

JDBC support #283

Merged
allwefantasy merged 6 commits into
NLPchina:masterfrom
allwefantasy:master
Sep 7, 2016
Merged

JDBC support #283
allwefantasy merged 6 commits into
NLPchina:masterfrom
allwefantasy:master

Conversation

@allwefantasy

Copy link
Copy Markdown
Member

Fix Issue: #28

Example

public void testJDBC() throws Exception {
        Properties properties = new Properties();
        properties.put("url", "jdbc:elasticsearch://127.0.0.1:9300/" + TestsConstants.TEST_INDEX);
        DruidDataSource dds = (DruidDataSource) ElasticSearchDruidDataSourceFactory.createDataSource(properties);
        Connection connection = dds.getConnection();
        PreparedStatement ps = connection.prepareStatement("SELECT  gender,lastname,age from  " + TestsConstants.TEST_INDEX + " where lastname='Heath'");
        ResultSet resultSet = ps.executeQuery();
        List<String> result = new ArrayList<String>();
        while (resultSet.next()) {
              System.out.println(resultSet.getString("lastname") + "," + resultSet.getInt("age") + "," + resultSet.getString("gender"))
        }
        ps.close();
        connection.close();
        dds.close();
    }

Unit test

org.nlpcn.es4sql.JDBCTests.testJDBC

@allwefantasy allwefantasy merged commit 64aa0bb into NLPchina:master Sep 7, 2016
@JichengSong

Copy link
Copy Markdown

1 similar comment
@harbby

harbby commented Sep 28, 2016

Copy link
Copy Markdown

good job !

@weikuo0506

Copy link
Copy Markdown

I run it ,but error occurs below, is there anybody can help me ?
the error:
java.lang.NoClassDefFoundError: org/elasticsearch/common/transport/TransportAddress
at com.alibaba.druid.pool.ElasticSearchDruidDataSource.createPhysicalConnection(ElasticSearchDruidDataSource.java:658)
at com.alibaba.druid.pool.ElasticSearchDruidDataSource.createPhysicalConnection(ElasticSearchDruidDataSource.java:623)
at com.alibaba.druid.pool.ElasticSearchDruidDataSource$CreateConnectionThread.run(ElasticSearchDruidDataSource.java:1802)
Caused by: java.lang.ClassNotFoundException: org.elasticsearch.common.transport.TransportAddress
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)

@allwefantasy

Copy link
Copy Markdown
Member Author

elasticsearch dependency in pom.xml is declared provided,so you should add elasticsearch dependency in your project

@weikuo0506

weikuo0506 commented Oct 18, 2016

Copy link
Copy Markdown

@allwefantasy many thanks, I have add elasticsearch dependency, and that problem seems solved. But another error happends:
Exception in thread "Druid-ConnectionPool-Create-1576809894" java.lang.NoSuchMethodError: com.alibaba.druid.pool.DruidConnectionHolder.(Lcom/alibaba/druid/pool/DruidAbstractDataSource;Ljava/sql/Connection;)V
at com.alibaba.druid.pool.ElasticSearchDruidDataSource.put(ElasticSearchDruidDataSource.java:1635)
at com.alibaba.druid.pool.ElasticSearchDruidDataSource$CreateConnectionThread.run(ElasticSearchDruidDataSource.java:1831)

have any idea?

ps: I have found the solution to the problem: down the druid version to 1.0.15;

thans, I can run it successful now!

@allwefantasy

Copy link
Copy Markdown
Member Author

Good job

@weikuo0506

Copy link
Copy Markdown

@allwefantasy I see you have used TransportClient in ElasticSearchConnection.java; and this is a natural way, cause mysql protocol tcp is TCP based, and so transportClient. But there is a limitation: users can only connect to the transport port (default 9300), other than the http port(9200). The limitation is more obvious when in the product environment where many companies demand that searching must be proxyed by Nginx. Have any idea?

@xiangsiyu

Copy link
Copy Markdown

@allwefantasy If I want to use ‘join‘, how to page?'LIMIT' cannot do that,for example:
image

@eliranmoyal

Copy link
Copy Markdown
Contributor

We don't support paging /scrolling for not native elasticsearch queries now

@xinuo

xinuo commented Jan 4, 2017

Copy link
Copy Markdown

I want to know can the feather support cluster?
For example , i have ES cluster:"192.168.4.91","192.168.4.92","192.168.4.93".
How can i input the url?

@xiangsiyu

Copy link
Copy Markdown

yes, you can do that,for example(elasticsearch.yml):
image

@will0815

will0815 commented Feb 8, 2017

Copy link
Copy Markdown

The es-sql website can run normally
but the code(JDBCTests) throw this exception:
NoNodeAvailableException[None of the configured nodes are available:

@allwefantasy

Copy link
Copy Markdown
Member Author

@will0815 Have you modified the JDBC URL eg.

properties.put("url", "jdbc:elasticsearch://139.196.218.206:9300,139.196.218.205:9300/" + TestsConstants.TEST_INDEX);

@hungrytortoise

Copy link
Copy Markdown

good job

@sansom

sansom commented Feb 28, 2017

Copy link
Copy Markdown

How to driver?

@RaniRaven

Copy link
Copy Markdown

It seems like it misses the java.sql.Driver implementation.
This is a must if I understand in order to use it as a JDBC driver

@stopit

stopit commented Mar 8, 2017

Copy link
Copy Markdown
Contributor

when i want to select from a type,just like: 'select * from index1.type2 where xxx ' , how can i use this jdbc method? But the example only show how to select from index. thx~

@allwefantasy

Copy link
Copy Markdown
Member Author

@RaniRaven com.mysql.jdbc.Driver would be ok

@allwefantasy

Copy link
Copy Markdown
Member Author

@stopit can you try index1/type2 instead of index1.type2?

@coderBaijige

Copy link
Copy Markdown

good job !
support 5.12?

@RaniRaven

Copy link
Copy Markdown

I fail to understand ...
Can I get an example on how to use
Class.forName() & DriverManager.getConnection in java in order to connect with the mysql driver to elasticsearch ?!
I would like to have a connection which is feasible to a standard sql query tool trying to connect.

@coderBaijige

Copy link
Copy Markdown

At first,I think can't support it elasticsearch 5.1.2.
But now I'm done,It can support.

@zhegexiaohuozi

zhegexiaohuozi commented Mar 25, 2017

Copy link
Copy Markdown

@allwefantasy How to support the sql like SELECT * FROM (SELECT user_id,sum(amt) as sum_amt from index1/type2 GROUP BY user_id) as a WHERE a.sum_amt < 500 ORDER BY uid desc limit 100,5?

And I get the Exception:

Exception in thread "main" java.sql.SQLException: Error
	at com.alibaba.druid.pool.ElasticSearchDruidDataSource.handleConnectionException(ElasticSearchDruidDataSource.java:1109)
	at com.alibaba.druid.pool.DruidPooledConnection.handleException(DruidPooledConnection.java:127)
	at com.alibaba.druid.pool.DruidPooledStatement.checkException(DruidPooledStatement.java:68)
	at com.alibaba.druid.pool.ElasticSearchDruidPooledPreparedStatement.executeQuery(ElasticSearchDruidPooledPreparedStatement.java:61)
	at com.xx.test.test.Elastic.jdbc(Elastic.java:79)
	at com.xx.test.test.Elastic.main(Elastic.java:94)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
Caused by: java.lang.ClassCastException: com.alibaba.druid.sql.ast.statement.SQLSubqueryTableSource cannot be cast to com.alibaba.druid.sql.ast.statement.SQLJoinTableSource
	at org.nlpcn.es4sql.parse.SqlParser.findFrom(SqlParser.java:227)
	at org.nlpcn.es4sql.parse.SqlParser.parseSelect(SqlParser.java:48)
	at org.nlpcn.es4sql.parse.SqlParser.parseSelect(SqlParser.java:35)
	at org.nlpcn.es4sql.query.ESActionFactory.create(ESActionFactory.java:61)
	at org.nlpcn.es4sql.SearchDao.explain(SearchDao.java:46)
	at com.alibaba.druid.pool.ElasticSearchDruidPooledPreparedStatement.getObjectResult(ElasticSearchDruidPooledPreparedStatement.java:72)
	at com.alibaba.druid.pool.ElasticSearchDruidPooledPreparedStatement.executeQuery(ElasticSearchDruidPooledPreparedStatement.java:46)
	... 7 more

thx!

detail info #400

@gggyd123

Copy link
Copy Markdown

jdbc对于es7.1.1支持吗?因为我看es7已经关闭了对于transport的连接方式,我个人还是尝试了一下用transport去连接,导入了很多依赖的jar包,但是最后还是没有成功,提示连接关闭或无效,不知道对于es7.x有没有什么特殊配置,还是根本不支持?

@shi-yuan

Copy link
Copy Markdown
Member

PreBuiltXPackTransportClient已经废弃了,要是用的话得申请证书并做下配置:#383

正考虑着切换到highlevelrestclient

@bjzhenglq

Copy link
Copy Markdown

@allwefantasy 你好,请教个问题,我在试用sql查询时,如果sql中试用sum等运算是,无法返回结果,如下图
image
会提示“Fielddata is disabled on text fields by default”,如果是直接sql查询没有问题,String sql = "SELECT name , score FROM score_list_202006";

如果我使用http://host:9200/_xpack/sql?format=json,直接使用图中的sql也是没有问题的,

请问这个问题怎么解决

@ananxuan

ananxuan commented Jun 10, 2020 via email

Copy link
Copy Markdown

@ananxuan

ananxuan commented Jun 10, 2020 via email

Copy link
Copy Markdown

@bjzhenglq

Copy link
Copy Markdown

这个字段类型是text应该不能sum,你要改下mapping吧

---Original--- From: "bjzhenglq"<notifications@github.com> Date: Wed, Jun 10, 2020 17:20 PM To: "NLPchina/elasticsearch-sql"<elasticsearch-sql@noreply.github.com>; Cc: "Comment"<comment@noreply.github.com>;"ananxuan"<493513100@qq.com>; Subject: Re: [NLPchina/elasticsearch-sql] JDBC support (#283) @allwefantasy 你好,请教个问题,我在试用sql查询时,如果sql中试用sum等运算是,无法返回结果,如下图 会提示“Fielddata is disabled on text fields by default”,如果是直接sql查询没有问题,String sql = "SELECT name , score FROM score_list_202006"; 如果我使用http://host:9200/_xpack/sql?format=json,直接使用图中的sql也是没有问题的, 请问这个问题怎么解决 — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

这个字段实际上是个整型的,直接rest接口形式是可以sum的,但是图片中的就是不行

@ananxuan

ananxuan commented Jun 10, 2020 via email

Copy link
Copy Markdown

@ananxuan

ananxuan commented Jun 10, 2020 via email

Copy link
Copy Markdown

@shi-yuan

Copy link
Copy Markdown
Member

@allwefantasy 你好,请教个问题,我在试用sql查询时,如果sql中试用sum等运算是,无法返回结果,如下图
image
会提示“Fielddata is disabled on text fields by default”,如果是直接sql查询没有问题,String sql = "SELECT name , score FROM score_list_202006";

如果我使用http://host:9200/_xpack/sql?format=json,直接使用图中的sql也是没有问题的,

请问这个问题怎么解决

检查下,name字段是不是text类型的,对text类型字段做聚合或者排序是需要开启Fielddata的。
不建议用text类型字段做聚合或者排序,可以考虑keyword类型

@ChiZng

ChiZng commented Sep 27, 2020

Copy link
Copy Markdown

preparedStatement支持set参数吗?

sql = "select * from person where name = ?";
statement = connection.prepareStatement(sql);
statement.setString(1, "小明");
resultSet = statement.executeQuery();

会报错

Caused by: org.nlpcn.es4sql.exception.SqlParseException: Failed to parse SqlExpression of type class com.alibaba.druid.sql.ast.expr.SQLVariantRefExpr. expression value: ?
	at org.nlpcn.es4sql.parse.WhereParser.parseValue(WhereParser.java:616)
	at org.nlpcn.es4sql.parse.WhereParser.explanCond(WhereParser.java:266)
	at org.nlpcn.es4sql.parse.WhereParser.parseWhere(WhereParser.java:93)
	at org.nlpcn.es4sql.parse.WhereParser.findWhere(WhereParser.java:64)
	at org.nlpcn.es4sql.parse.SqlParser.parseSelect(SqlParser.java:91)
	at org.nlpcn.es4sql.parse.SqlParser.parseSelect(SqlParser.java:63)
	at org.nlpcn.es4sql.query.ESActionFactory.create(ESActionFactory.java:65)
	at org.nlpcn.es4sql.SearchDao.explain(SearchDao.java:46)
	at com.alibaba.druid.pool.ElasticSearchDruidPooledPreparedStatement.getObjectResult(ElasticSearchDruidPooledPreparedStatement.java:92)
	at com.alibaba.druid.pool.ElasticSearchDruidPooledPreparedStatement.executeQuery(ElasticSearchDruidPooledPreparedStatement.java:40)
	... 94 common frames omitted

直接查sql="select * from person where name = '小明'"没有问题。
druid版本1.1.16,试了最新的1.1.24也不行,这种情况有什么解决方法吗?

@ananxuan

ananxuan commented Sep 27, 2020 via email

Copy link
Copy Markdown

@shi-yuan

shi-yuan commented Oct 25, 2020

Copy link
Copy Markdown
Member

preparedStatement支持set参数吗?

sql = "select * from person where name = ?";
statement = connection.prepareStatement(sql);
statement.setString(1, "小明");
resultSet = statement.executeQuery();

会报错

Caused by: org.nlpcn.es4sql.exception.SqlParseException: Failed to parse SqlExpression of type class com.alibaba.druid.sql.ast.expr.SQLVariantRefExpr. expression value: ?
	at org.nlpcn.es4sql.parse.WhereParser.parseValue(WhereParser.java:616)
	at org.nlpcn.es4sql.parse.WhereParser.explanCond(WhereParser.java:266)
	at org.nlpcn.es4sql.parse.WhereParser.parseWhere(WhereParser.java:93)
	at org.nlpcn.es4sql.parse.WhereParser.findWhere(WhereParser.java:64)
	at org.nlpcn.es4sql.parse.SqlParser.parseSelect(SqlParser.java:91)
	at org.nlpcn.es4sql.parse.SqlParser.parseSelect(SqlParser.java:63)
	at org.nlpcn.es4sql.query.ESActionFactory.create(ESActionFactory.java:65)
	at org.nlpcn.es4sql.SearchDao.explain(SearchDao.java:46)
	at com.alibaba.druid.pool.ElasticSearchDruidPooledPreparedStatement.getObjectResult(ElasticSearchDruidPooledPreparedStatement.java:92)
	at com.alibaba.druid.pool.ElasticSearchDruidPooledPreparedStatement.executeQuery(ElasticSearchDruidPooledPreparedStatement.java:40)
	... 94 common frames omitted

直接查sql="select * from person where name = '小明'"没有问题。
druid版本1.1.16,试了最新的1.1.24也不行,这种情况有什么解决方法吗?

支持预编译的:
2f581c2

testJDBCWithParameter

@ChiZng

ChiZng commented Nov 4, 2020 via email

Copy link
Copy Markdown

@pengpeng-fan

pengpeng-fan commented Jun 23, 2021

Copy link
Copy Markdown

我的依赖是这样

org.nlpcn
elasticsearch-sql
7.8.0.1


com.alibaba
druid
1.1.16


org.elasticsearch.client
transport
7.10.2


org.springframework.boot
spring-boot-starter-test
test


org.junit.platform
junit-platform-launcher
test


org.elasticsearch.client
x-pack-transport
7.10.2

测试类如下:
@test
public void testJDBC() throws Exception {
Properties properties = new Properties();
properties.put("url", "jdbc:elasticsearch://10.30.23.23:9300/"+"tab_log_info");
properties.put("cluster.name","mycluster");
properties.put("client.transport.sniff",true);
properties.put("xpack.security.transport.ssl.enabled",false);
DruidDataSource dds = (DruidDataSource) ElasticSearchDruidDataSourceFactory.createDataSource(properties);
dds.setInitialSize(1);
dds.setDbType("elastic_search");
Connection connection = dds.getConnection();
PreparedStatement ps = connection.prepareStatement("SELECT * from " +"tab_log_info" + " limit 100");
System.out.println("query ok...");
ResultSet resultSet = ps.executeQuery();
while (resultSet.next()) {
System.out.println(resultSet.getString("address")+"," + resultSet.getString("cityname") + "," + resultSet.getString("mgr") + "," + resultSet.getString("domain"));
}
ps.close();
connection.close();
dds.close();
}

一直报下面的错,请问是哪的原因呢(服务器es版本为7.10.2)
java.lang.NoClassDefFoundError: org/elasticsearch/cluster/metadata/Metadata$Custom
at java.lang.Class.getDeclaredConstructors0(Native Method) ~[na:1.8.0_111]
at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671) ~[na:1.8.0_111]
at java.lang.Class.getConstructors(Class.java:1651) ~[na:1.8.0_111]
at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:588) ~[elasticsearch-7.6.2.jar:7.6.2]
at org.elasticsearch.plugins.PluginsService.(PluginsService.java:119) ~[elasticsearch-7.6.2.jar:7.6.2]
at org.elasticsearch.client.transport.TransportClient.newPluginService(TransportClient.java:111) ~[elasticsearch-7.6.2.jar:7.6.2]
at org.elasticsearch.client.transport.TransportClient.buildTemplate(TransportClient.java:136) ~[elasticsearch-7.6.2.jar:7.6.2]
at org.elasticsearch.client.transport.TransportClient.(TransportClient.java:296) ~[elasticsearch-7.6.2.jar:7.6.2]
at org.elasticsearch.transport.client.PreBuiltTransportClient.(PreBuiltTransportClient.java:130) ~[transport-7.10.2.jar:7.10.2]
at org.elasticsearch.xpack.client.PreBuiltXPackTransportClient.(PreBuiltXPackTransportClient.java:47) ~[x-pack-transport-7.10.2.jar:7.10.2]
at org.elasticsearch.xpack.client.PreBuiltXPackTransportClient.(PreBuiltXPackTransportClient.java:42) ~[x-pack-transport-7.10.2.jar:7.10.2]
at org.elasticsearch.xpack.client.PreBuiltXPackTransportClient.(PreBuiltXPackTransportClient.java:38) ~[x-pack-transport-7.10.2.jar:7.10.2]
at com.alibaba.druid.pool.ElasticSearchDruidDataSource.createPhysicalConnection(ElasticSearchDruidDataSource.java:1034) [elasticsearch-sql-7.8.0.1.jar:na]
at com.alibaba.druid.pool.ElasticSearchDruidDataSource.createPhysicalConnection(ElasticSearchDruidDataSource.java:976) [elasticsearch-sql-7.8.0.1.jar:na]
at com.alibaba.druid.pool.ElasticSearchDruidDataSource.init(ElasticSearchDruidDataSource.java:878) [elasticsearch-sql-7.8.0.1.jar:na]

@shi-yuan

Copy link
Copy Markdown
Member

包版本不一致,x-pack-transport是7.10.2,但elasticsearch是7.6.2

@pengpeng-fan

Copy link
Copy Markdown

@shi-yuan 可以了,感谢

@xuetaoqi

Copy link
Copy Markdown

7.16.2 什么时候出来?

@shi-yuan

Copy link
Copy Markdown
Member

7.16.2 什么时候出来?

#1157

@ITzhangqiang

ITzhangqiang commented Apr 8, 2022

Copy link
Copy Markdown

@shi-yuan 您好,使用jdbc查询,不支持 group by filters,是我使用的版本有问题吗?

<dependency>
            <groupId>org.nlpcn</groupId>
            <artifactId>elasticsearch-sql</artifactId>
            <version>7.8.0.1</version>
</dependency>

@shi-yuan

Copy link
Copy Markdown
Member

这里支持的,7.8.0.1还真不支持,升下级吧
#982

@badBoy-art

Copy link
Copy Markdown

elasticsearch-sql可以将mysql语句转成highlevelrestclient的api吗?

@TimeTravle

Copy link
Copy Markdown

@test 一直报这个错
public void Testjdbc() throws Exception {

    Properties properties = new Properties();
    properties.put("url", "jdbc:es://127.0.0.1:9200");
    DruidDataSource dds = (DruidDataSource) ElasticSearchDruidDataSourceFactory.createDataSource(properties);
    dds.setInitialSize(1);
    Connection connection = dds.getConnection();
    PreparedStatement ps = connection.prepareStatement("SELECT * from datasource ");
    System.out.println("query ok...");
    ResultSet resultSet = ps.executeQuery();
    while (resultSet.next()) {
        System.out.println(resultSet.getString("address")+"," + resultSet.getString("cityname") + "," + resultSet.getString("mgr") + "," + resultSet.getString("domain"));
    }
    ps.close();
    connection.close();
    dds.close();
}

java.sql.SQLException: Error

at com.alibaba.druid.pool.ElasticSearchDruidDataSource.handleConnectionException(ElasticSearchDruidDataSource.java:1109)
at com.alibaba.druid.pool.DruidPooledConnection.handleException(DruidPooledConnection.java:127)
at com.alibaba.druid.pool.DruidPooledStatement.checkException(DruidPooledStatement.java:68)
at com.alibaba.druid.pool.ElasticSearchDruidPooledPreparedStatement.executeQuery(ElasticSearchDruidPooledPreparedStatement.java:61)
at org.nlpcn.es4sql.JDBCTests.Testjdbc(JDBCTests.java:39)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
at com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38)
at com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11)
at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35)
at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:235)
at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:54)

Caused by: NoNodeAvailableException[None of the configured nodes are available: [{#transport#-1}{K9EuakKZQm6PSSU9yLXa8w}{127.0.0.1}{127.0.0.1:9200}]]
at org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(TransportClientNodesService.java:349)
at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:247)
at org.elasticsearch.client.transport.TransportProxyClient.execute(TransportProxyClient.java:60)
at org.elasticsearch.client.transport.TransportClient.doExecute(TransportClient.java:381)
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:407)
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:396)
at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:46)
at org.elasticsearch.action.ActionRequestBuilder.get(ActionRequestBuilder.java:53)
at org.nlpcn.es4sql.query.SqlElasticSearchRequestBuilder.get(SqlElasticSearchRequestBuilder.java:29)
at org.elasticsearch.plugin.nlpcn.QueryActionElasticExecutor.executeSearchAction(QueryActionElasticExecutor.java:25)
at org.elasticsearch.plugin.nlpcn.QueryActionElasticExecutor.executeAnyAction(QueryActionElasticExecutor.java:53)
at com.alibaba.druid.pool.ElasticSearchDruidPooledPreparedStatement.getObjectResult(ElasticSearchDruidPooledPreparedStatement.java:73)
at com.alibaba.druid.pool.ElasticSearchDruidPooledPreparedStatement.executeQuery(ElasticSearchDruidPooledPreparedStatement.java:46)
... 25 more

@shi-yuan

shi-yuan commented Oct 8, 2022

Copy link
Copy Markdown
Member

@test 一直报这个错 public void Testjdbc() throws Exception {

    Properties properties = new Properties();
    properties.put("url", "jdbc:es://127.0.0.1:9200");
    DruidDataSource dds = (DruidDataSource) ElasticSearchDruidDataSourceFactory.createDataSource(properties);
    dds.setInitialSize(1);
    Connection connection = dds.getConnection();
    PreparedStatement ps = connection.prepareStatement("SELECT * from datasource ");
    System.out.println("query ok...");
    ResultSet resultSet = ps.executeQuery();
    while (resultSet.next()) {
        System.out.println(resultSet.getString("address")+"," + resultSet.getString("cityname") + "," + resultSet.getString("mgr") + "," + resultSet.getString("domain"));
    }
    ps.close();
    connection.close();
    dds.close();
}

java.sql.SQLException: Error

at com.alibaba.druid.pool.ElasticSearchDruidDataSource.handleConnectionException(ElasticSearchDruidDataSource.java:1109)
at com.alibaba.druid.pool.DruidPooledConnection.handleException(DruidPooledConnection.java:127)
at com.alibaba.druid.pool.DruidPooledStatement.checkException(DruidPooledStatement.java:68)
at com.alibaba.druid.pool.ElasticSearchDruidPooledPreparedStatement.executeQuery(ElasticSearchDruidPooledPreparedStatement.java:61)
at org.nlpcn.es4sql.JDBCTests.Testjdbc(JDBCTests.java:39)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
at com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38)
at com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11)
at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35)
at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:235)
at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:54)

Caused by: NoNodeAvailableException[None of the configured nodes are available: [{#transport#-1}{K9EuakKZQm6PSSU9yLXa8w}{127.0.0.1}{127.0.0.1:9200}]] at org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(TransportClientNodesService.java:349) at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:247) at org.elasticsearch.client.transport.TransportProxyClient.execute(TransportProxyClient.java:60) at org.elasticsearch.client.transport.TransportClient.doExecute(TransportClient.java:381) at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:407) at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:396) at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:46) at org.elasticsearch.action.ActionRequestBuilder.get(ActionRequestBuilder.java:53) at org.nlpcn.es4sql.query.SqlElasticSearchRequestBuilder.get(SqlElasticSearchRequestBuilder.java:29) at org.elasticsearch.plugin.nlpcn.QueryActionElasticExecutor.executeSearchAction(QueryActionElasticExecutor.java:25) at org.elasticsearch.plugin.nlpcn.QueryActionElasticExecutor.executeAnyAction(QueryActionElasticExecutor.java:53) at com.alibaba.druid.pool.ElasticSearchDruidPooledPreparedStatement.getObjectResult(ElasticSearchDruidPooledPreparedStatement.java:73) at com.alibaba.druid.pool.ElasticSearchDruidPooledPreparedStatement.executeQuery(ElasticSearchDruidPooledPreparedStatement.java:46) ... 25 more

确定是默认的9200,而不是默认的9300?

@TimeTravle

Copy link
Copy Markdown

@shi-yuan
es启动认证方式,会报以下错误
elasticsearch.yml 添加 xpack.security.enabled: true

Properties properties = new Properties();
properties.put("url", "jdbc:elasticsearch://127.0.0.1:9300/datasource");
properties.put("username", "elastic");
properties.put("password", "123456");
DataSource dds = ElasticSearchDruidDataSourceFactory.createDataSource(properties);
Connection connection = dds.getConnection();
PreparedStatement ps = connection.prepareStatement("SELECT * from datasource ");
ResultSet resultSet = ps.executeQuery();
List result = new ArrayList();
while (resultSet.next()) {
System.out.println(resultSet.getString("type"));
}
ps.close();
connection.close();

org.elasticsearch.transport.RemoteTransportException: [ssizsAe][127.0.0.1:9300][cluster:monitor/nodes/liveness]
Caused by: org.elasticsearch.ElasticsearchSecurityException: missing authentication token for action [cluster:monitor/nodes/liveness]
at org.elasticsearch.xpack.core.security.support.Exceptions.authenticationError(Exceptions.java:18)
at org.elasticsearch.xpack.core.security.authc.DefaultAuthenticationFailureHandler.createAuthenticationError(DefaultAuthenticationFailureHandler.java:129)
at org.elasticsearch.xpack.core.security.authc.DefaultAuthenticationFailureHandler.missingToken(DefaultAuthenticationFailureHandler.java:84)

@ChiZng

ChiZng commented Oct 9, 2022

Copy link
Copy Markdown

es开启了密码认证就要开启ssl。jdbc用的是tcp协议。要添加ssl认证

String usernamePassword = pool.getUsername() + ":" + pool.getPassword();
String xPackStr = "xpack.security.user=" + usernamePassword + ";" +
        "xpack.security.transport.ssl.enabled=true;" +
        "xpack.security.transport.ssl.keystore.path=" + keystorePath + ";" +
        "xpack.security.transport.ssl.keystore.password=" + caPassword + ";" +
        "xpack.security.transport.ssl.verification_mode=" + verification_mode;
properties.put(PROP_CONNECTIONPROPERTIES, xPackStr);

参考
#383 (comment)

@TimeTravle

Copy link
Copy Markdown

谢谢,解决了,现在就是想融合mybatis去执行sql,报以下问题
java.lang.IllegalStateException: availableProcessors is already set to [6], rejecting [6]
at io.netty.util.NettyRuntime$AvailableProcessorsHolder.setAvailableProcessors(NettyRuntime.java:51)
at io.netty.util.NettyRuntime.setAvailableProcessors(NettyRuntime.java:87)
at org.elasticsearch.transport.netty4.Netty4Utils.setAvailableProcessors(Netty4Utils.java:83)
at org.elasticsearch.transport.netty4.Netty4Transport.(Netty4Transport.java:112)
at org.elasticsearch.xpack.core.security.transport.netty4.SecurityNetty4Transport.(SecurityNetty4Transport.java:57)
at org.elasticsearch.xpack.core.XPackClientPlugin.lambda$getTransports$1(XPackClientPlugin.java:417)
at org.elasticsearch.client.transport.TransportClient.buildTemplate(TransportClient.java:189)
at org.elasticsearch.client.transport.TransportClient.(TransportClient.java:283)
at org.elasticsearch.transport.client.PreBuiltTransportClient.(PreBuiltTransportClient.java:128)
at org.elasticsearch.xpack.client.PreBuiltXPackTransportClient.(PreBuiltXPackTransportClient.java:47)
at org.elasticsearch.xpack.client.PreBuiltXPackTransportClient.(PreBuiltXPackTransportClient.java:42)
at org.elasticsearch.xpack.client.PreBuiltXPackTransportClient.(PreBuiltXPackTransportClient.java:38)
at com.alibaba.druid.pool.ElasticSearchConnection.(ElasticSearchConnection.java:35)
at com.alibaba.druid.pool.ElasticSearchDruidDataSource.createPhysicalConnection(ElasticSearchDruidDataSource.java:658)
at com.alibaba.druid.pool.ElasticSearchDruidDataSource.createPhysicalConnection(ElasticSearchDruidDataSource.java:623)
at com.alibaba.druid.pool.ElasticSearchDruidDataSource$CreateConnectionThread.run(ElasticSearchDruidDataSource.java:1802)

@TimeTravle

Copy link
Copy Markdown

请问能不能支持es 9200端口的访问呢

@shi-yuan

shi-yuan commented Dec 25, 2022

Copy link
Copy Markdown
Member

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.