[大数据]Docker Swarm部署Elasticsearch集群

1、部署环境：

2、给相关Swarm节点添加标签，使elasticsearch实例部署到指定服务器节点上

#  在任意一个Manager节点上执行
docker node update --label-add elasticsearch.cluster=node1 swarm01
docker node update --label-add elasticsearch.cluster=node2 swarm02
docker node update --label-add elasticsearch.cluster=node3 swarm03

执行以上命令后,各节点Node labels如下：

如：compose编排脚本中，用如下的方式，将elasticsearch1服务部署到docker04节点上

deploy:
      mode: global
      placement:
        constraints:
          - node.labels.elasticsearch.cluster == node1

1.3、创建数据卷映射目录

# 在三个服务器节点上，分别执行以下命令
mkdir -p /usr/local/elasticsearch/data

1.4、修改系统参数

# 分别在三个服务器节点上(宿主机)，修改/etc/sysctl.conf，添加参数vm.max_map_count（一个进程可以拥有的VMA(虚拟内存区域)的数量）
vi /etc/sysctl.conf
vm.max_map_count=262144

然后执行sysctl -p , 使用修改生效

4、部署文件说明

涉及到的部署文件有两个：

1. 编排脚本：elasticsearch-compose.yml

定义了构成应用程序的所有服务

2. 环境变量：elasticsearch.env

将compose脚本中重复使用的参数提取成变量，部署时只修改环境变量中定义的值即可，避免直接修改庞大的compose脚本导致出错

4.1、编排脚本：elasticsearch-compose.yml

version: "3.8"
services:
  elasticsearch1:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.13.2
    environment:
      network.publish_host: elasticsearch1
      network.host: 0.0.0.0
      node.name: es01
      cluster.name: es-docker-cluster
      discovery.seed_hosts: elasticsearch2,elasticsearch3
      cluster.initial_master_nodes: es01,es02,es03
      ES_JAVA_OPTS: ${ES_JAVA_OPTS}
      indices.fielddata.cache.size: 20%
      indices.breaker.fielddata.limit: 40%
      indices.breaker.request.limit: 40%
      indices.breaker.total.limit: 95%
      TAKE_FILE_OWNERSHIP: "true"
    volumes:
      - "/elasticsearch/data:/usr/share/elasticsearch/data"
    networks:
      - middle
    deploy:
      mode: global
      placement:
        constraints:
          - "node.labels.elasticsearch.cluster==node1"
  elasticsearch2:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.13.2
    environment:
      network.publish_host: elasticsearch2
      network.host: 0.0.0.0
      node.name: es02
      cluster.name: es-docker-cluster
      discovery.seed_hosts: elasticsearch1,elasticsearch3
      cluster.initial_master_nodes: es01,es02,es03
      ES_JAVA_OPTS: ${ES_JAVA_OPTS}
      indices.fielddata.cache.size: 20%
      indices.breaker.fielddata.limit: 40%
      indices.breaker.request.limit: 40%
      indices.breaker.total.limit: 95%
      TAKE_FILE_OWNERSHIP: "true"
    volumes:
      - "/elasticsearch/data:/usr/share/elasticsearch/data"
    networks:
      - middle
    deploy:
      mode: global
      placement:
        constraints:
          - "node.labels.elasticsearch.cluster==node2"
  elasticsearch3:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.13.2
    environment:
      network.publish_host: elasticsearch3
      network.host: 0.0.0.0
      node.name: es03
      cluster.name: es-docker-cluster
      discovery.seed_hosts: elasticsearch1,elasticsearch2
      cluster.initial_master_nodes: es01,es02,es03
      ES_JAVA_OPTS: ${ES_JAVA_OPTS}
      indices.fielddata.cache.size: 20%
      indices.breaker.fielddata.limit: 40%
      indices.breaker.request.limit: 40%
      indices.breaker.total.limit: 95%
      TAKE_FILE_OWNERSHIP: "true"
    volumes:
      - "/elasticsearch/data:/usr/share/elasticsearch/data"
    networks:
      - middle
    deploy:
      mode: global
      placement:
        constraints:
          - "node.labels.elasticsearch.cluster==node3"
networks:
  middle:
    name: middleware-network
    external: true

4.2、环境变量：elk.env

# ES内存设置
ES_JAVA_OPTS=-Xms512m -Xmx512m

5、部署

部署compose脚本的方式有两种，分别是：命令行、portainer工具

5.1、通过命令行部署

登录到任意一个Manager节点，将以下4个文件：
elasticsearch-compose.yml
elasticsearch.env
放到同一目录下，比如放到/usr/local/elasticsearch/目录

cd /usr/local/elasticsearch/
 ls ./
elasticsearch-compose.yml  elasticsearch.env

然后执行docker stack deploy 命令

cd /usr/local/elasticsearch/
set -a && . ./elasticsearch.env && set +a && docker stack deploy -c elasticsearch-compose.yml elasticsearch

5.2、通过Portainer部署

1. 点击菜单Stacks,点击按钮Add Stack，在Name输入框输入本Stack的名称,如elasticsearch，在Build method选项中选择Web editor,并在Web editor文本域中输入文件elasticsearch-compose.yml 的内容；
2. 点击Environment variables中的 Advanced mode，并在文本域中填入文件elasticsearch.env的内容；
3. 点击页面底部的Deploy the stack按钮；