[大数据]elasticsearch笔记

目录复制

安装

1. docker pull elasticsearch:7.4.2
2. docker pull kibana:7.4.2
3. free -m 查看虚拟机可用内存
4. mkdir -p /mydata/elasticsearch/config
5. mkdir -p /mydata/elasticsearch/data
6. echo “http.host: 0.0.0.0”>>/mydata/elasticsearch/config/elasticsearch.yml
7. chmod -R 777 /mydata/elasticsearch

docker run --name elasticsearch \
-p 9200:9200 -p 9300:9300 \
-e "discovery.type=single-node" \
-e "ES_JAVA_OPTS=-Xms64m -Xmx128m" \
-v /mydata/elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearche.yml \
-v /mydata/elasticsearch/data:/usr/share/elasticsearch/data \
-v /mydata/elasticsearch/plugins:/usr/share/elasticsearch/plugins \
-d elasticsearch:7.4.2

docker run --name kibaba -e ELASTICSEARCH_HOSTS=http://192.168.224.128:9200 -p 5601:5601 -d kibana:7.4.2

9. 查看节点信息_cat

http://192.168.224.128:9200/_cat/health 查看健康状况信息
http://192.168.224.128:9200/_cat/master 查看主节点信息
http://192.168.224.128:9200/_cat/indices 查看所有索引

保存数据
(PUT方式,必须带ID)
http://192.168.224.128:9200/custom/external/1
{
	"name":"Tom"
}
(POST方式， 带了Id ， 新增或修改，不带id，永远是新增)
http://192.168.224.128:9200/custom/external/1
{
	"name":"Tom"
}
(GET)	查询数据
http://192.168.224.128:9200/custom/external/1

乐观锁修改(带上if_seq_no)
http://192.168.224.128:9200/custom/external/1?if_seq_no=4&if_primary_term=1

(_update)(POST) 会对比原来数据，重复不更新，不同再更新
http://192.168.224.128:9200/custom/external/1/_update

(DELETE)删除数据
http://192.168.224.128:9200/custom/external/1

(DELETE)删除整个索引
http://192.168.224.128:9200/custom

批量操作
POST  /custom/external/_bulk
{"index":{"_id":1}}
{"name": "name1"}
{"index":{"_id":2}}
{"name": "name2"}

POST  /_bulk
{"delete": {"_index":"website","_type":"blog","_id":"123"}}
{"create": {"_index":"website","_type":"blog","_id":"123"}}
{"title":"my first blog post"}
{"index": {"_index":"website","_type":"blog"}}
{"title":"my secode blog post"}
{"update": {"_index":"website","_type":"blog","_id":"123"}}
{"doc":{"title":"my secode blog post"}}

10.ElasticSearch官方测试数据
github地址：https://github.com/elastic/elasticsearch/blob/master/docs/src/test/resources/accounts.json
调用地址：
POST /bank/account/_bulk

设置虚拟机docker 容器开机自动启动
docker update abc --restart=always

第一种查询方式:查询所有数据，按account_number升续排序
GET bank/_search/?q=*&sort=account_number:asc
第二种查询方式
GET bank/_search
{
“query”: {
“match_all”: {}
},
“sort”: [
{
“account_number”: “asc”
}
]
}

GET bank/_search
{
“query”: {
“match_all”: {}
},
“sort”: [
{
“account_number”: “asc”
}
],
“from”: 3,
“size”: 5,
“_source”: [“age”,“firstname”]
}
GET bank/_search
{
“query”: {“match”: {
“age”: 22
}}
}

GET bank/_search
{
“query”: {“match”: {
“address”: “Street”
}}
}

包含Madison 或者 Street的都能查询出来
GET bank/_search
{
“query”: {“match”: {
“address”: “Madison Street”
}}
}

match_phrase 完全匹配包含
GET bank/_search
{
“query”: {“match_phrase”: {
“address”: “Place”
}}
}
多字段匹配,任意一个字段包含任意一个单词
GET bank/_search
{
“query”: {“multi_match”: {
“query”: “Place Limestone Nogal”,
“fields”: [“address”, “city”]
}}
}

多条件查询
GET bank/_search
{
“query”: {
“bool”: {
“must”: [
{“match”: {
“address”:“Street”
}}
],
“must_not”: [

    {"match": {
      "age":28
    }}
  ],
  "should": [
    {"match": {
      "city" : "Sunriver"
    }}]
}

}
}

filter 不会贡献得分
GET bank/_search
{
“query”: {
“bool”: {
“filter”: {“range”: {
“age”: {
“gte”: 10,
“lte”: 20
}
}}
}
}
}
全文检索用match, 其他非text字段匹配用term

精确匹配keyword
GET bank/_search
{
“query”: {
“match”: {
“address.keyword”: “776 Little Street”
}
}
}

聚合

GET bank/_search
{
  "query": {"match_all": {}},
  "size": 0, 
  "aggs": {
    "ageAgg": {
      "terms": {
        "field": "age",
        "size": 100
      },
      "aggs": {
        "totalAvgBlance":{
          "avg": {
            "field": "balance"
          }
        },
        "genderAgg": {
          "terms": {
            "field": "gender.keyword",
            "size": 100
          },
          "aggs": {
            "avgBlance": {
              "avg": {
                "field": "balance"
              }
            }
          }
        }
        
      }
    }
  }
}

获取类型信息
Get /bank/_mapping

映射:

PUT /my_index
{
  "mappings": {
    "properties": {
      "age":{"type":"integer"},
      "email":{"type":"keyword"},
      "adress":{"type":"text"}
    }
  }
}

添加属性 (index 为 false 表示不作为索引进行查询)

PUT /my_index/_mapping
{
  "properties":{
    "abc":{
      "type":"keyword",
      "index":false 
    }
  }
}

迁移索引

POST _reindex
{
  "source": {
    "index": "bank",
    "type": "account"
  },
  "dest": {
    "index": "newbank"
  }
}

分词

POST _analyze
{
  "analyzer": "standard",
  "text":"abc def g hello aa aa"
}

下载中文分词器
https://github.com/medcl/elasticsearch-analysis-ik/releases/tag/v7.4.2
进入到/mydata/elasticsearch/plugins
上传该文件

查看docker 容器IP

docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' 容器ID或容器名称

进入容器kibana:7.4.2 修改vi kibana.yml中的地址

docker exec -it 602 /bin/bash
cd config/
vi kibana.yml

2.yum 安装

yum install -y unzip zip

解压
unzip 文件

智能分词

POST _analyze
{
  "analyzer": "ik_smart",
  "text":"你好我是"
}

POST _analyze
{
  "analyzer": "ik_max_word",
  "text":"你好我是中国人"
}

安装nginx

docker run -p 80:80 --name nginx -d nginx:1.10

复制docker中nginx中的配置文件

docker container cp nginx:/etc/nginx .

修改文件名nginx为config
mv nginx config
创建nginx
mkdir nginx

docker run --privileged -p 80:80 --name nginx \
-v /mydata/nginx/html:/usr/share/nginx/html \
-v /mydata/nginx/logs:/var/log/nginx \
-v /mydata/nginx/conf:/etc/nginx \
-d nginx:latest

docker run -p 80:80 --name nginx \
 -v /mydata/nginx/conf:/etc/nginx \
 -d nginx:1.10