ElasticSearch 及 Kibana 下载地址 : https://www.elastic.co/
IK分词器 : https://github.com/medcl/elasticsearch-analysis-ik/releases(根据ES版本选择对应的IK sersion)
head插件 : https://github.com/mobz/elasticsearch-head
外网下载慢的话,可以联系我帮你下载任意版本!
学习内容参考up主:狂神说java
1 安装ElasticSearch
1.1 解压ElasticSearch
bin 启动文件
config 配置文件
log4j2 日志配置文件
jvm.options java虚拟机相关配置,默认最小启动内存为1g
elasticsearch.yml elasticsearch相关配置,默认9200端口
lib 相关jar包
modules 功能模块
plugins 插件
logs 日志
1.2 启动 elasticsearch.bat
1.3 返回结果
{
"name": "YUCHEN",
"cluster_name": "elasticsearch",
"cluster_uuid": "omczgHLHR-6NKPhe6gcp7A",
"version": {
"number": "7.14.0",
"build_flavor": "default",
"build_type": "zip",
"build_hash": "dd5a0a2acaa2045ff9624f3729fc8a6f40835aa1",
"build_date": "2021-07-29T20:49:32.864135063Z",
"build_snapshot": false,
"lucene_version": "8.9.0",
"minimum_wire_compatibility_version": "6.8.0",
"minimum_index_compatibility_version": "6.0.0-beta1"
},
"tagline": "You Know, for Search"
}
2 安装可视化界面
安装head插件前,需有前端环境 nodejs
2.1 解压 head 插件
2.2 在 head 插件目录中执行命令
npm install
npm run start
2.3 启动成功后访问:127.0.0.1:9100
? 由于可视化插件的端口为 9100 ,elasticsearch 的端口为 9200 ,产生了跨域问题,所以需在 elasticsearch 的配置文件中添加开启跨域支持
# 开启跨域支持
http.cors.enabled: true
# 支持所有端口访问
http.cors.allow-origin: "*"
3 安装Kibana
只需要解压 kibana 即可,但是启动 kibana 需要使用前端环境
3.1 解压 kibana 文件
3.2 启动 kibana.bat
3.3 汉化
? 目录 \x-pack\plugins\translations\translations 下有中文翻译 zh-CN.json ,默认 kibana.yml 设置为英文环境,只需修改为中文即可
i18n.locale: "zh-CN"
4 ES概念
ES 是面向文档,关系型数据库和ES对比;
MySQL | ES |
---|
数据库(database) | 索引(indices) | 表(tables) | types(7.0之后被弃用) | 行(rows) | documents | 字段(columns) | fields |
4.1 分片
? 一个elasticsearch索引由多个lucene索引聚合而成,使用的是倒排索引
4.2 倒排索引
? 正排索引可以理解为:拿着文档去找关键词
? 倒排索引可以理解为:拿着关键词找文档
5 IK分词器
5.1 概念
把一段文字信息划分成一个个的关键字。
IK提供了两个分词算法:
- ik_smart:会做最粗粒度的拆分,比如会将“中华人民共和国人民大会堂”拆分为中华人民共和国、人民大会堂。
- ik_max_word:会将文本做最细粒度的拆分,比如会将“中华人民共和国人民大会堂”拆分为“中华人民共和国、中华人民、中华、华人、人民共和国、人民、共和国、大会堂、大会、会堂等词语。
5.2 使用场景
索引时使用 ik_max_word ,搜索时使用 ik_smart
5.3 安装
根据对应的ES版本下载不同的 IK version ,下载后解压成一个目录并放到 elasticsearch 的 plugins 目录下即可
5.4 重启 elasticsearch
-
重启 elasticsearch 后发现,加载了 ik分词器 插件 -
可以使用命令查看被加载的插件
elasticsearch-plugin list
- 使用
kibana 测试
? 使用 ik_smart 切分
# 使用 ik_smart 切分
GET _analyze
{
"analyzer": "ik_smart",
"text": ["世界和平日"]
}
# 响应
{
"tokens" : [
{
"token" : "世界和平",
"start_offset" : 0,
"end_offset" : 4,
"type" : "CN_WORD",
"position" : 0
},
{
"token" : "日",
"start_offset" : 4,
"end_offset" : 5,
"type" : "CN_CHAR",
"position" : 1
}
]
}
? 使用 ik_max_word 切分
# 使用 ik_max_word 切分
GET _analyze
{
"analyzer": "ik_max_word",
"text": ["世界和平日"]
}
# 响应
{
"tokens" : [
{
"token" : "世界和平",
"start_offset" : 0,
"end_offset" : 4,
"type" : "CN_WORD",
"position" : 0
},
{
"token" : "世界",
"start_offset" : 0,
"end_offset" : 2,
"type" : "CN_WORD",
"position" : 1
},
{
"token" : "和平",
"start_offset" : 2,
"end_offset" : 4,
"type" : "CN_WORD",
"position" : 2
},
{
"token" : "平日",
"start_offset" : 3,
"end_offset" : 5,
"type" : "CN_WORD",
"position" : 3
}
]
}
- 自定义分词
? 例如像 于晨 这个词,不管用 ik_smart 还是 ik_max_word 都会被默认拆分为两个单独的词,因为没有词典里有 于晨 这个词,所以需要自定义分词。配置文件 IKAnalyzer.cfg.xml 就是帮我们引入外部自定义的分词内容。
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE properties SYSTEM "http://java.sun.com/dtd/properties.dtd">
<properties>
<comment>IK Analyzer 扩展配置</comment>
<entry key="ext_dict">my.dic</entry>
<entry key="ext_stopwords"></entry>
</properties>
6 索引基本操作
method | url | desc |
---|
PUT | 127.0.0.1:9200/索引名称/类型名称/文档id | 创建文档(指定文档id) | POST | 127.0.0.1:9200/索引名称/类型名称 | 创建文档(随机文档id) | POST | 127.0.0.1:9200/索引名称/类型名称/文档id/_update | 修改文档 | DELETE | 127.0.0.1:9200/索引名称/类型名称/文档id | 删除文档 | GET | 127.0.0.1:9200/索引名称/类型名称/文档id | 通过文档id查询文档 | POST | 127.0.0.1:9200/索引名称/类型名称/_search | 查询所有数据 |
6.1 创建索引
**注意:**如果创建索引时没有指定文档字段,es会帮我们默认配置字段类型!
# /索引名称/类型/文档id
PUT /test1/type1/1
{
"name": "于晨",
"age": 24
}
# 响应
{
"_index" : "test1",
"_type" : "type1",
"_id" : "1",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 0,
"_primary_term" : 1
}
6.2 字段类型
详情点击:https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-types.html
6.3 创建索引规则
# 创建索引规则,不创建任何字段和添加数据
PUT /test2
{
"mappings": {
"properties": {
"name": {
"type": "text"
},
"age": {
"type": "long"
},
"birthday": {
"type": "date"
}
}
}
}
# 响应
{
"acknowledged" : true,
"shards_acknowledged" : true,
"index" : "test2"
}
6.4 查看索引信息
# 查看索引信息
GET test2
# 响应
{
"test2" : {
"aliases" : { },
"mappings" : {
"properties" : {
"age" : {
"type" : "long"
},
"birthday" : {
"type" : "date"
},
"name" : {
"type" : "text"
}
}
},
"settings" : {
"index" : {
"routing" : {
"allocation" : {
"include" : {
"_tier_preference" : "data_content"
}
}
},
"number_of_shards" : "1",
"provided_name" : "test2",
"creation_date" : "1630220026847",
"number_of_replicas" : "1",
"uuid" : "oj3fflPzSCOocpHFtBjKHQ",
"version" : {
"created" : "7140099"
}
}
}
}
}
6.5 修改
-
直接覆盖原有字段值 # 直接覆盖已有的数据更新
PUT /test1/type1/1
{
"name": "于晨1",
"age": 23
}
# 响应
{
"_index" : "test1",
"_type" : "type1",
"_id" : "1",
"_version" : 2,
"result" : "updated",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 1,
"_primary_term" : 1
}
-
使用POST请求修改 # 请求
POST /test1/type1/1/_update
{
"doc": {
"name": "测试修改",
"age": 22
}
}
# 响应
{
"_index" : "test1",
"_type" : "type1",
"_id" : "1",
"_version" : 3,
"result" : "updated",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 4,
"_primary_term" : 1
}
6.6 删除
# 删除索引
DELETE test1
# 响应
{
"acknowledged" : true
}
7 复杂操作
7.1 加数据
# 创建文档
PUT /yuchen/user/1
{
"name": "于晨",
"age": 24,
"desc": "羞答答的玫瑰,火红红的开",
"tags": ["宅男","强迫症","秀儿"]
}
# 响应
{
"_index" : "yuchen",
"_type" : "user",
"_id" : "1",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 0,
"_primary_term" : 1
}
7.2 简单查询一条数据
hits:包含索引和文档的信息、查询的结果总数、source
_source:包含文档中字段及其值,可以使用此字段,过滤需要查询的内容
# 查询name包含于晨的
GET /yuchen/user/_search
{
"query": {
"match": {
"name": "于晨"
}
}
}
# 响应
{
"took" : 707,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 2,
"relation" : "eq"
},
"max_score" : 1.4523083,
"hits" : [
{
"_index" : "yuchen",
"_type" : "user",
"_id" : "1",
"_score" : 1.4523083,
"_source" : {
"name" : "于晨",
"age" : 24,
"desc" : "羞答答的玫瑰,火红红的开",
"tags" : [
"宅男",
"强迫症",
"秀儿"
]
}
},
{
"_index" : "yuchen",
"_type" : "user",
"_id" : "6",
"_score" : 1.2199391,
"_source" : {
"name" : "于晨12",
"age" : 3,
"desc" : "羞答答的玫瑰,火红红的开",
"tags" : [
"宅男",
"强迫症",
"秀儿"
]
}
}
]
}
}
# 多个条件用空格隔开
GET /yuchen/user/_search
{
"query": {
"match": {
"tags": "友 渣"
}
}
}
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 2,
"relation" : "eq"
},
"max_score" : 1.8392417,
"hits" : [
{
"_index" : "yuchen",
"_type" : "user",
"_id" : "2",
"_score" : 1.8392417,
"_source" : {
"name" : "张三",
"age" : 23,
"desc" : "法外狂徒",
"tags" : [
"渣男",
"强迫症",
"交友"
]
}
},
{
"_index" : "yuchen",
"_type" : "user",
"_id" : "3",
"_score" : 0.7654058,
"_source" : {
"name" : "李四",
"age" : 22,
"desc" : "憨里憨气的",
"tags" : [
"憨批",
"mmp",
"渣女"
]
}
}
]
}
}
7.3 过滤字段查询
# 只查询某些字段
GET /yuchen/user/_search
{
"query": {
"match": {
"name": "于晨"
}
},
"_source": ["name", "desc"]
}
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 2,
"relation" : "eq"
},
"max_score" : 1.4523083,
"hits" : [
{
"_index" : "yuchen",
"_type" : "user",
"_id" : "1",
"_score" : 1.4523083,
"_source" : {
"name" : "于晨",
"desc" : "羞答答的玫瑰,火红红的开"
}
},
{
"_index" : "yuchen",
"_type" : "user",
"_id" : "6",
"_score" : 1.2199391,
"_source" : {
"name" : "于晨12",
"desc" : "羞答答的玫瑰,火红红的开"
}
}
]
}
}
7.4 按照年龄排序查询
# 根据年龄排序
GET /yuchen/user/_search
{
"query": {
"match": {
"name": "于晨"
}
},
"sort": {
"age": {
"order": "asc"
}
}
}
# 响应
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 2,
"relation" : "eq"
},
"max_score" : null,
"hits" : [
{
"_index" : "yuchen",
"_type" : "user",
"_id" : "6",
"_score" : null,
"_source" : {
"name" : "于晨12",
"age" : 3,
"desc" : "羞答答的玫瑰,火红红的开",
"tags" : [
"宅男",
"强迫症",
"秀儿"
]
},
"sort" : [
3
]
},
{
"_index" : "yuchen",
"_type" : "user",
"_id" : "1",
"_score" : null,
"_source" : {
"name" : "于晨",
"age" : 24,
"desc" : "羞答答的玫瑰,火红红的开",
"tags" : [
"宅男",
"强迫症",
"秀儿"
]
},
"sort" : [
24
]
}
]
}
}
7.5 分页查询
# 根据年龄排序 from 从第几个数据开始, size 每页显示数量
GET /yuchen/user/_search
{
"query": {
"match": {
"name": "于晨"
}
},
"from": 0,
"size": 1
}
# 响应
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 2,
"relation" : "eq"
},
"max_score" : 1.4523083,
"hits" : [
{
"_index" : "yuchen",
"_type" : "user",
"_id" : "1",
"_score" : 1.4523083,
"_source" : {
"name" : "于晨",
"age" : 24,
"desc" : "羞答答的玫瑰,火红红的开",
"tags" : [
"宅男",
"强迫症",
"秀儿"
]
}
}
]
}
}
7.6 bool 多条件查询
7.6.1 must (and)
# bool多条件匹配查询,name = "于晨" and age = 3
GET /yuchen/user/_search
{
"query": {
"bool": {
"must": [
{
"match": {
"name" : "于晨"
}
},
{
"match": {
"age": "3"
}
}
]
}
}
}
# 响应
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 1,
"relation" : "eq"
},
"max_score" : 2.2199392,
"hits" : [
{
"_index" : "yuchen",
"_type" : "user",
"_id" : "6",
"_score" : 2.2199392,
"_source" : {
"name" : "于晨12",
"age" : 3,
"desc" : "羞答答的玫瑰,火红红的开",
"tags" : [
"宅男",
"强迫症",
"秀儿"
]
}
}
]
}
}
7.6.2 should (or)
# bool多条件匹配查询,name = "于晨" or age = 3
GET /yuchen/user/_search
{
"query": {
"bool": {
"should": [
{
"match": {
"name" : "于晨"
}
},
{
"match": {
"age": "3"
}
}
]
}
}
}
# 响应
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 2,
"relation" : "eq"
},
"max_score" : 2.2199392,
"hits" : [
{
"_index" : "yuchen",
"_type" : "user",
"_id" : "6",
"_score" : 2.2199392,
"_source" : {
"name" : "于晨12",
"age" : 3,
"desc" : "羞答答的玫瑰,火红红的开",
"tags" : [
"宅男",
"强迫症",
"秀儿"
]
}
},
{
"_index" : "yuchen",
"_type" : "user",
"_id" : "1",
"_score" : 1.4523083,
"_source" : {
"name" : "于晨",
"age" : 24,
"desc" : "羞答答的玫瑰,火红红的开",
"tags" : [
"宅男",
"强迫症",
"秀儿"
]
}
}
]
}
}
7.6.3 must not (not)
# bool多条件匹配查询,age not 3 and age not 22 and age not 23
GET /yuchen/user/_search
{
"query": {
"bool": {
"must_not": [
{
"match": {
"age": 3
}
},
{
"match": {
"age": 22
}
},
{
"match": {
"age": 23
}
}
]
}
}
}
# 响应
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 1,
"relation" : "eq"
},
"max_score" : 0.0,
"hits" : [
{
"_index" : "yuchen",
"_type" : "user",
"_id" : "1",
"_score" : 0.0,
"_source" : {
"name" : "于晨",
"age" : 24,
"desc" : "羞答答的玫瑰,火红红的开",
"tags" : [
"宅男",
"强迫症",
"秀儿"
]
}
}
]
}
}
7.7 filter 过滤
# filter 过滤年龄在3-20之间的
GET /yuchen/user/_search
{
"query": {
"bool": {
"must": [
{
"match": {
"name" : "于晨"
}
}
],
"filter": [
{
"range": {
"age": {
"gte": 3,
"lte": 20
}
}
}
]
}
}
}
# 响应
{
"took" : 2,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 1,
"relation" : "eq"
},
"max_score" : 1.2199391,
"hits" : [
{
"_index" : "yuchen",
"_type" : "user",
"_id" : "6",
"_score" : 1.2199391,
"_source" : {
"name" : "于晨12",
"age" : 3,
"desc" : "羞答答的玫瑰,火红红的开",
"tags" : [
"宅男",
"强迫症",
"秀儿"
]
}
}
]
}
}
7.8 term 精确查询
7.8.1 text 与 keyword 区别
- text:会被进行分析后查询
- keyword:不会被分词器分析
-
创建索引 PUT /test3
{
"mappings": {
"properties": {
"name":{
"type": "text"
},
"desc":{
"type":"keyword"
}
}
}
}
PUT /test3/_doc/1
{
"name": "于晨",
"desc": "于晨"
}
PUT /test3/_doc/2
{
"name": "于晨2",
"desc": "于晨2"
}
-
获取name GET test3/_search
{
"query":{
"term" : {
"name" : "于"
}
}
}
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 2,
"relation" : "eq"
},
"max_score" : 0.14874382,
"hits" : [
{
"_index" : "test3",
"_type" : "_doc",
"_id" : "1",
"_score" : 0.14874382,
"_source" : {
"name" : "于晨",
"desc" : "于晨"
}
},
{
"_index" : "test3",
"_type" : "_doc",
"_id" : "2",
"_score" : 0.12703526,
"_source" : {
"name" : "于晨2",
"desc" : "于晨2"
}
}
]
}
}
-
获取desc GET test3/_search
{
"query":{
"term" : {
"desc" : "于晨"
}
}
}
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 1,
"relation" : "eq"
},
"max_score" : 0.9808291,
"hits" : [
{
"_index" : "test3",
"_type" : "_doc",
"_id" : "1",
"_score" : 0.9808291,
"_source" : {
"name" : "于晨",
"desc" : "于晨"
}
}
]
}
}
7.9 高亮显示
默认高亮为 <em> 标签,可以使用 pre_tags、post_tags 来自定义标签高亮显示
# 高亮显示name字段
GET yuchen/user/_search
{
"query":{
"match": {
"name" : "于晨"
}
},
"highlight": {
"pre_tags": "<p class='key' style='color:red'",
"post_tags": "/>",
"fields": {
"name": {}
}
}
}
{
"took" : 35,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 2,
"relation" : "eq"
},
"max_score" : 1.4523083,
"hits" : [
{
"_index" : "yuchen",
"_type" : "user",
"_id" : "1",
"_score" : 1.4523083,
"_source" : {
"name" : "于晨",
"age" : 24,
"desc" : "羞答答的玫瑰,火红红的开",
"tags" : [
"宅男",
"强迫症",
"秀儿"
]
},
"highlight" : {
"name" : [
"<p class='key' style='color:red'于/><p class='key' style='color:red'晨/>"
]
}
},
{
"_index" : "yuchen",
"_type" : "user",
"_id" : "6",
"_score" : 1.2199391,
"_source" : {
"name" : "于晨12",
"age" : 3,
"desc" : "羞答答的玫瑰,火红红的开",
"tags" : [
"宅男",
"强迫症",
"秀儿"
]
},
"highlight" : {
"name" : [
"<p class='key' style='color:red'于/><p class='key' style='color:red'晨/>12"
]
}
}
]
}
}
|