SpringBoot整合ElasticSearch
初识ElasticSearch
自从学到后面的整合我直接心态爆炸，所以重新又学了一遍，现重新整理了一些笔记，以供参考。不多逼逼，开始逼着自己学！
1.什么是ElasticSearch？
ElasticSearch是一个分布式的开源搜索与分析引擎，ElasticSearch着重于数据的检索与分析。但是众所周知MySQL等数据库也可以实现对于数据的分析与检索，但是闻道有先后，术业有专攻。MySQL主要用于数据的持久化管理以及存储，但用MySQL对海量数据进行检索与分析，ElasticSearch无疑更加在行。
Elastic 的底层是开源库Lucene。但是，你没法直接用Lucene，必须自己写代码去调用它的
接口。Elastic 是Lucene 的封装，提供了REST API 的操作接口，开箱即用。
REST API：天然的跨平台。
官方文档：https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html
官方中文：https://www.elastic.co/guide/cn/elasticsearch/guide/current/foreword_id.html
2.基本概念
- Index（索引）
动词，相当于insert，索引了一条数据就是插入了一条数据
名词，相当于Database，就是数据库 - Type（类型）
对应的是数据库中的表，相比于存储到哪个数据库的某个表中，ElasticSearch就是存储到某个索引的某个类型中。 - Document（文档）
在类型中存储的数据称之为文档，所有的存储数据都是Json格式。
- 倒排索引
elasticSearch在存储记录的时候会有一张倒排索引表，将所有的话拆分为一个个的单词。
3.安装elasticSearch以及可视化工具Kibana
- 下载镜像文件(注意版本的一致性)
docker pull elasticsearch:7.4.2 //存储和检索数据
docker pull kibana:7.4.2 //可视化检索数据
- 创建实例
mkdir -p /mydata/elasticsearch/config
mkdir -p /mydata/elasticsearch/data
echo "http.host: 0.0.0.0" >> /mydata/elasticsearch/config/elasticsearch.yml
chmod -R 777 /mydata/elasticsearch/ 保证权限
docker run --name elasticsearch -p 9200:9200 -p 9300:9300 \
-e "discovery.type=single-node" \
-e ES_JAVA_OPTS="-Xms64m -Xmx512m" \
-v /mydata/elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml \
-v /mydata/elasticsearch/data:/usr/share/elasticsearch/data \
-v /mydata/elasticsearch/plugins:/usr/share/elasticsearch/plugins \
-d elasticsearch:7.4.2
这样就说明es安装成功，尤其要注意文件夹的权限。
- 安装kibana
docker run --name kibana -e ELASTICSEARCH_HOSTS=http://192.168.56.10:9200 -p 5601:5601 \
-d kibana:7.4.2
http://192.168.56.10:9200 一定改为自己虚拟机的地址
入门ES
1._cat(查看es的信息)
GET /_cat/nodes：查看所有节点
GET /_cat/health：查看es 健康状况
GET /_cat/master：查看主节点
GET /_cat/indices：查看所有索引 相当于show databases；
通过PostMan进行测试这里就不一一列举了
2.索引一个文档
保存一个数据，保存在哪个索引的哪个类型下，指定用哪个唯一标识
PUT customer/external/1
{
"name": "John Doe"
}
PUT 和POST 都可以，
POST 新增。如果不指定id，会自动生成id。指定id 就会修改这个数据，并新增版本号
PUT 可以新增可以修改。PUT 必须指定id；由于PUT 需要指定id，我们一般都用来做修改
操作，不指定id 会报错。
3.查询文档
GET customer/external/1
结果：
{
"_index": "customer", //在哪个索引
"_type": "external", //在哪个类型
"_id": "1", //记录id
"_version": 2, //版本号
"_seq_no": 1, //并发控制字段，每次更新就会+1，用来做乐观锁
"_primary_term": 1, //同上，主分片重新分配，如重启，就会变化
"found": true,
"_source": { //真正的内容
"name": "John Doe"
}
}
更新携带?if_seq_no=0&if_primary_term=1
4.更新文档
POST customer/external/1/_update
{
"doc":{
"name": "John Doew"
}
}
POST customer/external/1
{
"name": "John Doe2"
}
PUT customer/external/1
{
"name": "John Doe"
}
不同：POST 操作会对比源文档数据，如果相同不会有什么操作，文档version 不增加
PUT 操作总会将数据重新保存并增加version 版本；
带_update 对比元数据如果一样就不进行任何操作。
看场景；
对于大并发更新，不带update，不会对比元数据；
对于大并发查询偶尔更新，带update，带doc{…}；对比元数据，一致则不更新，不一致则更新。
5.删除文档与索引
DELETE customer/external/1
DELETE customer
掌握了基本语法，导入一些数据进行实际学习
这里才是重点，之前一轮很随意，疏忽了，到后面发现就看不懂了，后面全是套娃，DSL的语法是真的离谱，这里一定得跟着来一遍，切记！跟着来一遍！
1.bulk 批量API
POST customer/external/_bulk
{"index":{"_id":"1"}}
{"name": "John Doe" }
{"index":{"_id":"2"}}
{"name": "Jane Doe" }
{
"took" : 4, //花费的
"errors" : false, //没有发生错误
"items" : [
{
"index" : {
"_index" : "customer",
"_type" : "external",
"_id" : "1",
"_version" : 2,
"result" : "updated",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 2,
"_primary_term" : 1,
"status" : 200
}
},
{
"index" : {
"_index" : "customer",
"_type" : "external",
"_id" : "2",
"_version" : 2,
"result" : "updated",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 3,
"_primary_term" : 1,
"status" : 200
}
}
]
}
2.导入整体数据
地址：https://gitee.com/xlh_blog/common_content/blob/master/es%E6%B5%8B%E8%AF%95%E6%95%B0%E6%8D%AE.json
- 进阶学习（这里的语法真的很重要，给我接着干！）
Elasticsearch 提供了一个可以执行查询的Json 风格的DSL（domain-specific language 领域特
定语言）。这个被称为Query DSL。该查询语言非常全面，并且刚开始的时候感觉有点复杂，
真正学好它的方法是从一些基础的示例开始的。
3.语法介绍
GET bank/_search //查询哪张里的信息
{
"query": { //按照条件查询，括号里的就是具体的条件
"match_all": {}
},
"sort": [ //按照什么进行排序
{
"balance": {
"order": "desc"
}
}
],
"from": 5, //从第几条记录开始
"size": 5, //每页展示多少条记录
"_source": ["balance","firstname"] //类似于select balance，firstname from bank order by balance desc
}
- match（全文检索）
可以用来精确查询，也可以用来模糊匹配。最终会按照评分进行排序，会对检索条件进行分词匹配。
GET bank/_search
{
"query": {
"match": {
"address": "Kings"
}
}
}
GET bank/_search //精确匹配
{
"query": {
"match": {
"address.keyword": "789 Madison"
}
}
}
- match_phrase
对短语进行匹配，不会被分割
GET bank/_search
{
"query": {
"match_phrase": {
"address": "Mill Lane"
}
}
}
- multi_match
多字段匹配，会进行分词
GET bank/_search
{
"query": {
"multi_match": {
"query": "mill Lopezo",
"fields": ["address","city"]
}
}
}
- bool复合查询（极其重要）
合并多个查询条件，这些条件都必须被满足
GET bank/_search
{
"query": {
"bool": {
"must": [
{
"match": {
"gender": "M"
}
},
{
"match": {
"address": "mill"
}
}
],
"must_not": [
{
"match": {
"age": "28"
}
}
],
"should": [
{
"match": {
"lastname": "Wallace"
}
}
]
}
}
}
must以及should被满足会获得相关性得分，must_not会被当成过滤器，这就引出了过滤器，过滤器不会提供相关性得分。
- filter结果过滤
不会计算相关性得分
GET bank/_search
{
"query": {
"bool": {
"must": [
{
"range": {
"age": {
"gte": 18,
"lte": 30
}
}
}
]
}
}
}
GET bank/_search
{
"query": {
"bool": {
"filter": {
"range": {
"age": {
"gte": 18,
"lte": 30
}
}
}
}
}
}
-
term（非常重要！）
和match 一样。匹配某个属性的值。全文检索字段用match，其他非text 字段匹配用term。
数字——>term
文本——>match -
aggregations（执行聚合）
查数据的时候就可以获取到聚合信息。
##搜索address 中包含mill 的所有人的年龄分布以及平均年龄，但不显示这些人的详情
GET bank/_search
{
"query": {
"match": {
"address": "mill"
}
},
"aggs": {
"age_agg": {
"terms": {
"field": "age",
"size": 10
}
},
"age_avg": {
"avg": {
"field": "age"
}
},
"balance_avg": {
"avg": {
"field": "balance"
}
}
},
"size": 0
}
##按照年龄聚合，并且请求这些年龄段的这些人的平均薪资
GET bank/_search
{
"query": {
"match_all": {}
},
"aggs": {
"age_agg": {
"terms": {
"field": "age",
"size": 100
},
"aggs": {
"balanceAvg": {
"avg": {
"field": "balance"
}
}
}
}
}
}
##查出所有年龄分布，并且这些年龄段中M 的平均薪资和F的平均薪资以及这个年龄段的总体平均薪资
GET bank/_search
{
"query": {
"match_all": {}
},
"aggs": {
"age_agg": {
"terms": {
"field": "age",
"size": 100
},
"aggs": {
"genderAgg": {
"terms": {
"field": "gender.keyword",
"size": 10
},
"aggs": {
"balanceAgg": {
"avg": {
"field": "balance"
}
}
}
},
"ageBalanceAvg": {
"avg": {
"field": "balance"
}
}
}
}
}
}
4.映射
es在6之后移除了类型，直接将数据存储到索引中去了。第一次存数据的时候，es会自动猜测存储类型。而在保存数据前，我们可以指定映射，从而确保文档类型是我们想要的。注意：映射是不允许更新的，需要改的话，需要使用数据迁移（下节），创建新索引，重新保存。
PUT /my_index //创建映射
{
"mappings": {
"properties": {
"age": {"type": "integer"},
"email": {"type": "keyword"},
"name": {"type": "text"}
}
}
}
PUT /my_index/_mapping //在映射中添加新的字段
{
"properties": {
"employee_ID": {
"type": "keyword",
"index": false
}
}
}
5.数据迁移
- 创建新索引，并将老索引中的复制过来，然后修改成自己想要的类型
PUT /newbank
{
"mappings": {
"properties": {
"account_number": {
"type": "long"
},
"address": {
"type": "text"
},
"age": {
"type": "integer"
},
"balance": {
"type": "long"
},
"city": {
"type": "keyword"
},
"email": {
"type": "keyword"
},
"employer": {
"type": "keyword"
},
"firstname": {
"type": "text"
},
"gender": {
"type": "keyword"
},
"lastname": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"state": {
"type": "keyword"
}
}
}
}
- 数据迁移命令
POST _reindex [固定写法]
{
"source": {
"index": "twitter"
},
"dest": {
"index": "new_twitter"
}
}
SpringBoot整合high-level-client
之前有点飘了以为ES不重要，一直没仔细看基础。现在仔细看看也没有那么难了，现在开始整合SpringBoot，兄弟们，开干！
1.导入依赖（注意ESclient的版本需要和elasticSearch的版本一致）
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-high-level-client</artifactId>
<version>7.4.2</version>
</dependency>
2.编写配置类，给容器中注入RestHighLevelClient（虚拟机ip改成自己的）
@Configuration
public class ElasticSearchConfig {
public static final RequestOptions COMMON_OPTIONS;
static {
RequestOptions.Builder builder = RequestOptions.DEFAULT.toBuilder();
COMMON_OPTIONS = builder.build();
}
@Bean
public RestHighLevelClient esRestClient(){
RestHighLevelClient client = new RestHighLevelClient(
RestClient.builder(new HttpHost("192.168.75.128", 9200, "http")));
return client;
}
}
3.编写测试类
GET bank/_search
{
"query": {
"match": {
"address": "mill"
}
},
"aggs": {
"age_agg": {
"terms": {
"field": "age",
"size": 10
}
},
"age_avg": {
"avg": {
"field": "age"
}
},
"balance_avg": {
"avg": {
"field": "balance"
}
}
},
"size": 0
}
@Test
public void searchData() throws IOException {
//1. 创建检索请求
SearchRequest searchRequest = new SearchRequest();
//1.1）指定索引
searchRequest.indices("bank");
//1.2）构造检索条件
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
sourceBuilder.query(QueryBuilders.matchQuery("address", "Mill"));
//1.2.1)按照年龄分布进行聚合
TermsAggregationBuilder ageAgg = AggregationBuilders.terms("ageAgg").field("age").size(10);
sourceBuilder.aggregation(ageAgg);
//1.2.2)计算平均年龄
AvgAggregationBuilder ageAvg = AggregationBuilders.avg("ageAvg").field("age");
sourceBuilder.aggregation(ageAvg);
//1.2.3)计算平均薪资
AvgAggregationBuilder balanceAvg = AggregationBuilders.avg("balanceAvg").field("balance");
sourceBuilder.aggregation(balanceAvg);
System.out.println("检索条件：" + sourceBuilder);
searchRequest.source(sourceBuilder);
//2. 执行检索
SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
System.out.println("检索结果：" + searchResponse);
//3. 将检索结果封装为Bean
SearchHits hits = searchResponse.getHits();
SearchHit[] searchHits = hits.getHits();
for (SearchHit searchHit : searchHits) {
String sourceAsString = searchHit.getSourceAsString();
Account account = JSON.parseObject(sourceAsString, Account.class);
System.out.println(account);
}
//4. 获取聚合信息
Aggregations aggregations = searchResponse.getAggregations();
Terms ageAgg1 = aggregations.get("ageAgg");
for (Terms.Bucket bucket : ageAgg1.getBuckets()) {
String keyAsString = bucket.getKeyAsString();
System.out.println("年龄：" + keyAsString + " ==> " + bucket.getDocCount());
}
Avg ageAvg1 = aggregations.get("ageAvg");
System.out.println("平均年龄：" + ageAvg1.getValue());
Avg balanceAvg1 = aggregations.get("balanceAvg");
System.out.println("平均薪资：" + balanceAvg1.getValue());
}
SearchRequest的构建-检索
1.首先对传入参数实体类进行封装(检索条件)
/**
* 封装页面所有可能传递过来的参数
*/
@Data
public class SearchParam {
/**
* 页面传递过来的全文匹配关键字
* &keyword=小米
*/
private String keyword;
/**
* 品牌id,可以多选
*
*/
private List<Long> brandId;
/**
* 三级分类id
* &catalog3Id=255
*/
private Long catalog3Id;
/**
* 排序条件：sort=price/salecount/hotscore_desc/asc
*/
private String sort;
/**
* 是否显示有货
* &hasStock=0/1
*/
private Integer hasStock;
/**
* 价格区间查询
* &skuPrice=1_500/_500/500_
*/
private String skuPrice;
/**
* 按照属性进行筛选
* attrs=1_其他：安卓&attrs=2_5寸：6寸
*/
private List<String> attrs;
/**
* 页码
*/
private Integer pageNum = 1;
/**
* 原生的所有查询条件
*/
private String _queryString;
}
2.封装结果返回实体类
@Data
public class SearchResult {
/**
* 查询到的所有商品信息
*/
private List<SkuEsModel> product;
/**
* 当前页码
*/
private Integer pageNum;
/**
* 总记录数
*/
private Long total;
/**
* 总页码
*/
private Integer totalPages;
private List<Integer> pageNavs;
/**
* 当前查询到的结果，所有涉及到的品牌
*/
private List<BrandVo> brands;
/**
* 当前查询到的结果，所有涉及到的所有属性
*/
private List<AttrVo> attrs;
/**
* 当前查询到的结果，所有涉及到的所有分类
*/
private List<CatalogVo> catalogs;
//===========================以上是返回给页面的所有信息============================//
/* 面包屑导航数据 */
private List<NavVo> navs;
@Data
public static class NavVo {
private String navName;
private String navValue;
private String link;
}
@Data
public static class BrandVo {
private Long brandId;
private String brandName;
private String brandImg;
}
@Data
public static class AttrVo {
private Long attrId;
private String attrName;
private List<String> attrValue;
}
@Data
public static class CatalogVo {
private Long catalogId;
private String catalogName;
}
}
3.编写DSL语句
查询过滤
GET gulimall_product/_search
{
"query": {
"bool": {
"must": [
{
"match": {
"skuTitle": "华为"
}
}
],
"filter": [
{
"term": {
"catalogId": "225"
}
},
{
"terms": {
"brandId": [
"1",
"2"
]
}
},
{
"nested": {
"path": "attrs",
"query": {
"bool": {
"must": [
{
"term": {
"attrs.attrId": {
"value": "8"
}
}
},
{
"terms": {
"attrs.attrValue": [
"4G",
"5G"
]
}
}
]
}
}
}
},
{
"term": {
"hasStock": {
"value": "true"
}
}
},
{
"range": {
"skuPrice": {
"gte": 2000,
"lte": 2500
}
}
}
]
}
},
"sort": [
{
"skuPrice": {
"order": "desc"
}
}
],
"from": 0,
"size": 2,
"highlight": {
"fields": {"skuTitle": {}},
"pre_tags":"<b style='color:red'>",
"post_tags": "</b>"
}
}
聚合分析
GET gulimall_product/_search
{
"query": {
"match_all": {}
},
"aggs": {
"brandAgg": {
"terms": {
"field": "brandId",
"size": 2
},
"aggs": {
"barndNameAgg": {
"terms": {
"field": "brandName",
"size": 10
}
},
"brandImgAgg": {
"terms": {
"field": "brandImg",
"size": 10
}
}
}
},
"catalogAgg": {
"terms": {
"field": "catalogId",
"size": 10
},
"aggs": {
"catalogNameAgg": {
"terms": {
"field": "catalogName",
"size": 10
}
}
}
},
"attrsAgg": {
"nested": {
"path": "attrs"
},
"aggs": {
"attrIdAgg": {
"terms": {
"field": "attrs.attrId",
"size": 10
},
"aggs": {
"attrNameAgg": {
"terms": {
"field": "attrs.attrName",
"size": 10
}
},
"attrValueAgg": {
"terms": {
"field": "attrs.attrValue",
"size": 10
}
}
}
}
}
}
}
}
4.根据DSL语句编写检索逻辑的业务逻辑代码
- 由于这里前端页面使用的是thymeleaf模板。前后端分离的话可以做出相应修改。
@GetMapping("/list.html")
public String listPage(SearchParam param, Model model) {
SearchResult search = mallSearchService.search(param);
model.addAttribute("result",search);
return "list";
}
- 具体serviceImpl实现
- 准备检索请求
- 执行检索请求
- 分析响应数据，封装成我们需要的格式
public SearchResult search(SearchParam param) {
//1、动态构建出查询需要的DSL语句
SearchResult result = null;
//1、准备检索请求
SearchRequest searchRequest = buildSearchRequest(param);
try {
//2、执行检索请求
SearchResponse response = esRestClient.search(searchRequest, GulimallElasticSearchConfig.COMMON_OPTIONS);
//3、分析响应数据，封装成我们需要的格式
result = buildSearchResult(response,param);
} catch (IOException e) {
e.printStackTrace();
}
return result;
}
由于业务逻辑复杂，所以就不在该方法中实现，接下来我们单独对准备检索请求，以及构建响应数据进行实现。
- 首先实现对检索请求进行封装，这部分需要完成三部分，首先是第一部分模糊匹配，过滤（按照属性，分类，品牌，价格区间，库存）
//构建DSL语句
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
/**
* 模糊匹配，过滤（按照属性，分类，品牌，价格区间，库存）
*/
//1. 构建bool-query
BoolQueryBuilder boolQueryBuilder=new BoolQueryBuilder();
//1.1 bool-must
if(!StringUtils.isEmpty(param.getKeyword())){
boolQueryBuilder.must(QueryBuilders.matchQuery("skuTitle",param.getKeyword()));
}
//1.2 bool-fiter
//1.2.1 catelogId
if(null != param.getCatalog3Id()){
boolQueryBuilder.filter(QueryBuilders.termQuery("catalogId",param.getCatalog3Id()));
}
//1.2.2 brandId
if(null != param.getBrandId() && param.getBrandId().size() >0){
boolQueryBuilder.filter(QueryBuilders.termsQuery("brandId",param.getBrandId()));
}
//1.2.3 attrs
if(param.getAttrs() != null && param.getAttrs().size() > 0){
param.getAttrs().forEach(item -> {
//attrs=1_5寸:8寸&2_16G:8G
BoolQueryBuilder boolQuery = QueryBuilders.boolQuery();
//attrs=1_5寸:8寸
String[] s = item.split("_");
String attrId=s[0];
String[] attrValues = s[1].split(":");//这个属性检索用的值
boolQuery.must(QueryBuilders.termQuery("attrs.attrId",attrId));
boolQuery.must(QueryBuilders.termsQuery("attrs.attrValue",attrValues));
//每一个必须都得生成一个嵌入式查询
NestedQueryBuilder nestedQueryBuilder = QueryBuilders.nestedQuery("attrs",boolQuery, ScoreMode.None);
boolQueryBuilder.filter(nestedQueryBuilder);
});
}
//1.2.4 hasStock 按照有无库存进行检索
if(null != param.getHasStock()){
boolQueryBuilder.filter(QueryBuilders.termQuery("hasStock",param.getHasStock() == 1));
}
//1.2.5 skuPrice 按照价格区间进行检索
if(!StringUtils.isEmpty(param.getSkuPrice())){
//skuPrice形式为：1_500或_500或500_
RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("skuPrice");
String[] price = param.getSkuPrice().split("_");
if(price.length==2){
rangeQueryBuilder.gte(price[0]).lte(price[1]);
}else if(price.length == 1){
if(param.getSkuPrice().startsWith("_")){
rangeQueryBuilder.lte(price[1]);
}
if(param.getSkuPrice().endsWith("_")){
rangeQueryBuilder.gte(price[0]);
}
}
boolQueryBuilder.filter(rangeQueryBuilder);
}
//封装所有的查询条件
searchSourceBuilder.query(boolQueryBuilder);
- 第二部分就是实现排序，分页，高亮
/**
* 排序，分页，高亮
*/
//排序
//形式为sort=hotScore_asc/desc
if(!StringUtils.isEmpty(param.getSort())){
String sort = param.getSort();
String[] sortFileds = sort.split("_");
SortOrder sortOrder="asc".equalsIgnoreCase(sortFileds[1])?SortOrder.ASC:SortOrder.DESC;
searchSourceBuilder.sort(sortFileds[0],sortOrder);
}
//分页
searchSourceBuilder.from((param.getPageNum()-1)*EsConstant.PRODUCT_PAGESIZE);
searchSourceBuilder.size(EsConstant.PRODUCT_PAGESIZE);
//高亮
if(!StringUtils.isEmpty(param.getKeyword())){
HighlightBuilder highlightBuilder = new HighlightBuilder();
highlightBuilder.field("skuTitle");
highlightBuilder.preTags("<b style='color:red'>");
highlightBuilder.postTags("</b>");
searchSourceBuilder.highlighter(highlightBuilder);
}
- 聚合分析
/**
* 聚合分析
*/
//1. 按照品牌进行聚合
TermsAggregationBuilder brand_agg = AggregationBuilders.terms("brand_agg");
brand_agg.field("brandId").size(50);
//1.1 品牌的子聚合-品牌名聚合
brand_agg.subAggregation(AggregationBuilders.terms("brand_name_agg")
.field("brandName").size(1));
//1.2 品牌的子聚合-品牌图片聚合
brand_agg.subAggregation(AggregationBuilders.terms("brand_img_agg")
.field("brandImg").size(1));
searchSourceBuilder.aggregation(brand_agg);
//2. 按照分类信息进行聚合
TermsAggregationBuilder catalog_agg = AggregationBuilders.terms("catalog_agg");
catalog_agg.field("catalogId").size(20);
catalog_agg.subAggregation(AggregationBuilders.terms("catalog_name_agg").field("catelogName").size(1));
searchSourceBuilder.aggregation(catalog_agg);
//2. 按照属性信息进行聚合
NestedAggregationBuilder attr_agg = AggregationBuilders.nested("attr_agg", "attrs");
//2.1 按照属性ID进行聚合
TermsAggregationBuilder attr_id_agg = AggregationBuilders.terms("attr_id_agg").field("attrs.attrId");
attr_agg.subAggregation(attr_id_agg);
//2.1.1 在每个属性ID下，按照属性名进行聚合
attr_id_agg.subAggregation(AggregationBuilders.terms("attr_name_agg").field("attrs.attrName").size(1));
//2.1.1 在每个属性ID下，按照属性值进行聚合
attr_id_agg.subAggregation(AggregationBuilders.terms("attr_value_agg").field("attrs.attrValue").size(50));
searchSourceBuilder.aggregation(attr_agg);
System.out.println("构建的DSL语句:"+searchSourceBuilder.toString());
SearchRequest searchRequest = new SearchRequest(new String[]{EsConstant.PRODUCT_INDEX},searchSourceBuilder);
return searchRequest;
此时通过postman测试构建的DSL语句是可以输出的
SearchResult的分析与封装
我们需要返回一下数据：
- 返回所有查询到的商品
SearchResult result = new SearchResult();
//1、返回的所有查询到的商品
SearchHits hits = response.getHits();
List<SkuEsModel> esModels = new ArrayList<>();
//遍历所有商品信息
if (hits.getHits() != null && hits.getHits().length > 0) {
for (SearchHit hit : hits.getHits()) {
String sourceAsString = hit.getSourceAsString();
SkuEsModel esModel = JSON.parseObject(sourceAsString, SkuEsModel.class);
//判断是否按关键字检索，若是就显示高亮，否则不显示
if (!StringUtils.isEmpty(param.getKeyword())) {
//拿到高亮信息显示标题
HighlightField skuTitle = hit.getHighlightFields().get("skuTitle");
String skuTitleValue = skuTitle.getFragments()[0].string();
esModel.setSkuTitle(skuTitleValue);
}
esModels.add(esModel);
}
}
result.setProduct(esModels);
- 当前商品所涉及到的属性信息
//2、当前商品涉及到的所有属性信息
List<SearchResult.AttrVo> attrVos = new ArrayList<>();
//获取属性信息的聚合
ParsedNested attrsAgg = response.getAggregations().get("attrAgg");
ParsedLongTerms attrIdAgg = attrsAgg.getAggregations().get("attrIdAgg");
for (Terms.Bucket bucket : attrIdAgg.getBuckets()) {
SearchResult.AttrVo attrVo = new SearchResult.AttrVo();
//1、得到属性的id
long attrId = bucket.getKeyAsNumber().longValue();
attrVo.setAttrId(attrId);
//2、得到属性的名字
ParsedStringTerms attrNameAgg = bucket.getAggregations().get("attrNameAgg");
String attrName = attrNameAgg.getBuckets().get(0).getKeyAsString();
attrVo.setAttrName(attrName);
//3、得到属性的所有值
ParsedStringTerms attrValueAgg = bucket.getAggregations().get("attrValueAgg");
List<String> attrValues = attrValueAgg.getBuckets().stream().map(item -> item.getKeyAsString()).collect(Collectors.toList());
attrVo.setAttrValue(attrValues);
attrVos.add(attrVo);
}
result.setAttrs(attrVos);
- 当前商品所涉及到的品牌信息
//3、当前商品涉及到的所有品牌信息
List<SearchResult.BrandVo> brandVos = new ArrayList<>();
//获取到品牌的聚合
ParsedLongTerms brandAgg = response.getAggregations().get("brandAgg");
for (Terms.Bucket bucket : brandAgg.getBuckets()) {
SearchResult.BrandVo brandVo = new SearchResult.BrandVo();
//1、得到品牌的id
long brandId = bucket.getKeyAsNumber().longValue();
brandVo.setBrandId(brandId);
//2、得到品牌的名字
ParsedStringTerms brandNameAgg = bucket.getAggregations().get("brandNameAgg");
String brandName = brandNameAgg.getBuckets().get(0).getKeyAsString();
brandVo.setBrandName(brandName);
//3、得到品牌的图片
ParsedStringTerms brandImgAgg = bucket.getAggregations().get("brandImgAgg");
String brandImg = brandImgAgg.getBuckets().get(0).getKeyAsString();
brandVo.setBrandImg(brandImg);
brandVos.add(brandVo);
}
result.setBrands(brandVos);
- 当前商品所涉及到的分类信息
//4、当前商品涉及到的所有分类信息
//获取到分类的聚合
List<SearchResult.CatalogVo> catalogVos = new ArrayList<>();
ParsedLongTerms catalogAgg = response.getAggregations().get("catalogAgg");
for (Terms.Bucket bucket : catalogAgg.getBuckets()) {
SearchResult.CatalogVo catalogVo = new SearchResult.CatalogVo();
//得到分类id
String keyAsString = bucket.getKeyAsString();
catalogVo.setCatalogId(Long.parseLong(keyAsString));
//得到分类名
ParsedStringTerms catalogNameAgg = bucket.getAggregations().get("catalogNameAgg");
String catalogName = catalogNameAgg.getBuckets().get(0).getKeyAsString();
catalogVo.setCatalogName(catalogName);
catalogVos.add(catalogVo);
}
result.setCatalogs(catalogVos);
- 分页信息-页码，总记录数
//5、分页信息-页码
result.setPageNum(param.getPageNum());
//5、1分页信息、总记录数
long total = hits.getTotalHits().value;
result.setTotal(total);
//5、2分页信息-总页码-计算
int totalPages = (int)total % EsConstant.PRODUCT_PAGESIZE == 0 ?
(int)total / EsConstant.PRODUCT_PAGESIZE : ((int)total / EsConstant.PRODUCT_PAGESIZE + 1);
result.setTotalPages(totalPages);
到此为止，ES的后端就结束了，通过postman测试，构建的DSL语句如下所示：
GET gulimall_product/_search
{
"from": 0,
"size": 2,
"query": {
"bool": {
"adjust_pure_negative": true,
"boost": 1
}
},
"aggregations": {
"brandAgg": {
"terms": {
"field": "brandId",
"size": 50,
"min_doc_count": 1,
"shard_min_doc_count": 0,
"show_term_doc_count_error": false,
"order": [
{
"_count": "desc"
},
{
"_key": "asc"
}
]
},
"aggregations": {
"brandNameAgg": {
"terms": {
"field": "brandName",
"size": 1,
"min_doc_count": 1,
"shard_min_doc_count": 0,
"show_term_doc_count_error": false,
"order": [
{
"_count": "desc"
},
{
"_key": "asc"
}
]
}
},
"brandImgAgg": {
"terms": {
"field": "brandImg",
"size": 1,
"min_doc_count": 1,
"shard_min_doc_count": 0,
"show_term_doc_count_error": false,
"order": [
{
"_count": "desc"
},
{
"_key": "asc"
}
]
}
}
}
},
"catalogAgg": {
"terms": {
"field": "catalogId",
"size": 20,
"min_doc_count": 1,
"shard_min_doc_count": 0,
"show_term_doc_count_error": false,
"order": [
{
"_count": "desc"
},
{
"_key": "asc"
}
]
},
"aggregations": {
"catalogNameAgg": {
"terms": {
"field": "catalogName",
"size": 1,
"min_doc_count": 1,
"shard_min_doc_count": 0,
"show_term_doc_count_error": false,
"order": [
{
"_count": "desc"
},
{
"_key": "asc"
}
]
}
}
}
},
"attrAgg": {
"nested": {
"path": "attrs"
},
"aggregations": {
"attrIdAgg": {
"terms": {
"field": "attrs.attrId",
"size": 10,
"min_doc_count": 1,
"shard_min_doc_count": 0,
"show_term_doc_count_error": false,
"order": [
{
"_count": "desc"
},
{
"_key": "asc"
}
]
},
"aggregations": {
"attrNameAgg": {
"terms": {
"field": "attrs.attrName",
"size": 1,
"min_doc_count": 1,
"shard_min_doc_count": 0,
"show_term_doc_count_error": false,
"order": [
{
"_count": "desc"
},
{
"_key": "asc"
}
]
}
},
"attrValueAgg": {
"terms": {
"field": "attrs.attrValue",
"size": 50,
"min_doc_count": 1,
"shard_min_doc_count": 0,
"show_term_doc_count_error": false,
"order": [
{
"_count": "desc"
},
{
"_key": "asc"
}
]
}
}
}
}
}
}
}
}
总结：初学ES还是挺煎熬的，看着他的语法头是一阵一阵的大，但是看熟悉了ES的DSL语法也就没那么离谱了，至此，感谢雷神的细心讲解。