[大数据]ElasticSearch的Java API

自己gitee项目地址

https://gitee.com/wang_haoxx/elastic-search-demo.git

Java客户端

1. Elasticsearch Java API有四类client连接方式

TransportClient

RestClient

Jest

Spring Data Elasticsearch

其中TransportClient和RestClient是Elasticsearch原生的api。TransportClient可以支持2.x，5.x版本，TransportClient将会在Elasticsearch 7.0弃用并在8.0中完成删除，替而代之，我们使用Java High Level REST Client，它使用HTTP请求而不是Java序列化请求。
restclient和transprortclient，前者是基于restful的，直接操作各种restful api和query dsl，比较简单，没有orm功能；后者是基于java api封装的orm框架，封装比较死板，不太灵活，兼容性差，不能像调用restful那样直接操作query dsl，所以也不能直接针对query dsl，在head插件或者kibana里面调试和调优query dsl，写出了query dsl还要想方设法转换成对应的java api的调用方式。

Jest是Java社区开发的，是Elasticsearch的Java Http Rest客户端；Spring Data Elasticsearch是spring集成的Elasticsearch开发包。

建议：TransportClient将会在后面的版本中弃用，因此不推荐后续使用；而Jest由于是社区维护，所以更新有一定延迟，目前最新版对接ES6.3.1，近一个月只有四个issue，说明整体活跃度较低，因此也不推荐使用；Spring Data Elasticsearch主要是与Spring生态对接，可以在web系统中整合到Spring中使用。目前比较推荐使用官方的高阶、低阶Rest Client，官方维护，比较值得信赖。本文主要介绍RestClient，其他的只做简单概述。

2. TransportClient

官网：
https://www.elastic.co/guide/en/elasticsearch/client/index.html

RestClient客户端

Elasticsearch提供了2种REST客户端，一种是低级客户端，一种是高级客户端。

Java Low Level REST Client: 官方提供的低级客户端。该客户端通过http来连接Elasticsearch集群。用户在使 用该客户端时需要将请求数据手动拼接成Elasticsearch所需JSON格式进行发送，收到响应时同样也需要将返回 的JSON数据手动封装成对象。虽然麻烦，不过该客户端兼容所有的Elasticsearch版本。

Java High Level REST Client: 官方提供的高级客户端。该客户端基于低级客户端实现，它提供了很多便捷的 API来解决低级客户端需要手动转换数据格式的问题。

application.properties

ElasticSearch.Hosts=ip,ip,ip
ElasticSearch.UserName=账号
ElasticSearch.Password=密码

REST低级客户端(Java Low REST Client)

pom

  <!-- low-level client -->
<dependency>
   <groupId>org.elasticsearch.client</groupId>
   <artifactId>elasticsearch-rest-client</artifactId>
   <version>7.8.0</version>
</dependency>

Configuration

package com.wh.es.config;

import org.apache.http.HttpHost;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.CredentialsProvider;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.impl.nio.client.HttpAsyncClientBuilder;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestClientBuilder;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class ElasticSearchClient {
    @Value("${ElasticSearch.Hosts}")
    private String hosts;
    @Value("${ElasticSearch.UserName}")
    private String userName;
    @Value("${ElasticSearch.Password}")
    private String password;

    @Bean
    public RestClient getClient() {
        String[] hosts = this.hosts.split(",");
        HttpHost[] httpHosts = new HttpHost[hosts.length];
        for (int i = 0; i < hosts.length; i++) {
            httpHosts[i] = new HttpHost(hosts[i], 9200, "http");
        }
        final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
        credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(userName, password));

        RestClientBuilder restClientBuilder = RestClient.builder(httpHosts).setRequestConfigCallback(new RestClientBuilder.RequestConfigCallback() {
            @Override
            public RequestConfig.Builder customizeRequestConfig(RequestConfig.Builder requestConfigBuilder) {
                requestConfigBuilder.setConnectTimeout(-1);
                requestConfigBuilder.setSocketTimeout(-1);
                requestConfigBuilder.setConnectionRequestTimeout(-1);
                return requestConfigBuilder;
            }
        }).setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
            @Override
            public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
                httpClientBuilder.disableAuthCaching();
                return httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
            }
        });
        RestClient client = restClientBuilder.build();
        return client;
    }
}

测试

package com.wh.es;

import org.apache.http.util.EntityUtils;
import org.elasticsearch.client.Request;
import org.elasticsearch.client.Response;
import org.elasticsearch.client.RestClient;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

import java.io.IOException;

@RestController
@RequestMapping("/es")
public class Test {
    @Autowired
    private RestClient restClient;

    /**
     * 根据id查询
     *
     * @throws IOException
     */
    @RequestMapping("/test")
    public void test() throws IOException {
        //ultra_msa_handle_log: index索引
        //NNlUkXwBHnJz4vSzyS9J: id
        Request request = new Request("GET", "/ultra_msa_handle_log/_doc/NNlUkXwBHnJz4vSzyS9J");
        Response response = restClient.performRequest(request);
        System.out.println(response.getStatusLine());
        System.out.println(EntityUtils.toString(response.getEntity()));
    }
}

结果

未添加账号密码启动汇报错

项目里

解决办法

postman

解决办法

Rest高级客户端(Java High Level REST Client)

pom

<!-- high-level client ，默认依赖的elasticsearch存在版本差异，排除后添加统一的es版本-->
<dependency>
    <groupId>org.elasticsearch.client</groupId>
    <artifactId>elasticsearch-rest-high-level-client</artifactId>
    <version>7.8.0</version>
    <exclusions>
        <exclusion>
            <groupId>org.elasticsearch</groupId>
            <artifactId>elasticsearch</artifactId>
        </exclusion>
    </exclusions>
</dependency>
  <!-- 使用low-level client不需要，但是high-level client需要依赖-->
<dependency>
    <groupId>org.elasticsearch</groupId>
    <artifactId>elasticsearch</artifactId>
    <version>7.8.0</version>
</dependency>

Configuration

/**
 * es 高级客户端配置
 */
@Configuration
public class ElasticSearchHeignClient {
    @Value("${ElasticSearch.Hosts}")
    private String hosts;
    @Value("${ElasticSearch.UserName}")
    private String userName;
    @Value("${ElasticSearch.Password}")
    private String password;

    @Bean
    public RestHighLevelClient restHighLevelClient() {
        String[] hosts = this.hosts.split(",");
        HttpHost[] httpHosts = new HttpHost[hosts.length];
        for (int i = 0; i < hosts.length; i++) {
            httpHosts[i] = new HttpHost(hosts[i], 9200, "http");
        }
        final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
        credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(userName, password));

        RestClientBuilder restClientBuilder = RestClient.builder(httpHosts).setRequestConfigCallback(new RestClientBuilder.RequestConfigCallback() {
            @Override
            public RequestConfig.Builder customizeRequestConfig(RequestConfig.Builder requestConfigBuilder) {
                requestConfigBuilder.setConnectTimeout(-1);
                requestConfigBuilder.setSocketTimeout(-1);
                requestConfigBuilder.setConnectionRequestTimeout(-1);
                return requestConfigBuilder;
            }
        }).setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
            @Override
            public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
                httpClientBuilder.disableAuthCaching();
                return httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
            }
        });
        RestHighLevelClient restHighLevelClient = new RestHighLevelClient(restClientBuilder);
        return restHighLevelClient;
    }