docker三剑客swarm及docker集群
一.Docker Swarm 简介
Docker Swarm 优点
- 任何规模都有高性能表现
- 灵活的容器调度
- 服务的持续可用性
- 和 Docker API 及整合支持的兼容性
- Docker Swarm 为 Docker 化应用的核心功能（诸如多主机网络和存储卷管理）提供原生支持。
docker swarm 相关概念 - 节点分为管理 (manager) 节点和工作 (worker) 节点
- 任务 （Task）是 Swarm 中的最小的调度单位，目前来说就是一个单一的容器。
- 服务 （Services） 是指一组任务的集合，服务定义了任务的属性。
二.Docker Swarm建立集群
初始化集群
[root@server3 ~]# docker swarm init
Swarm initialized: current node (c2n90m20je6e7olg3715o69bj) is now a manager.
To add a worker to this swarm, run the following command:
docker swarm join --token SWMTKN-1-5ypf8jbkg3gd9flyz0wdt8zadjfblvv6gouhubio9gw3f50pyc-crjz805czwr4s0i02e1zlpgns 172.25.3.3:2377
To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.
在servr2/3/4执行，加入集群
docker swarm join --token SWMTKN-1-5ypf8jbkg3gd9flyz0wdt8zadjfblvv6gouhubio9gw3f50pyc-crjz805czwr4s0i02e1zlpgns 172.25.3.3:2377
设置server4为备用主机
[root@server3 docker]# docker node promote server4
Node server4 promoted to a manager in the swarm.
[root@server3 docker]# docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION
l5haztljobe5b6qp1ky8bscrt server1 Ready Active 19.03.15
zcl17mdf9p2z8s52e8a8wjf1e server2 Ready Active 19.03.15
c2n90m20je6e7olg3715o69bj * server3 Ready Active Leader 19.03.15
jkbdykkyc0aid2ilq9kmb5mt3 server4 Ready Active Reachable 19.03.15
删除4主机身份
[root@server3 docker]# docker node demote server2
Manager server4 demoted in the swarm.
[root@server3 docker]# docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION
l5haztljobe5b6qp1ky8bscrt server1 Ready Active 19.03.15
zcl17mdf9p2z8s52e8a8wjf1e server2 Ready Active 19.03.15
c2n90m20je6e7olg3715o69bj * server3 Ready Active Leader 19.03.15
jkbdykkyc0aid2ilq9kmb5mt3 server4 Ready Active 19.03.15
传递文件 repo文件、harbor仓库认证文件目录cert.d、本地解析、daemon.json（解决网络访问问题）
scp daemon.json server1:/etc/docker/
scp daemon.json server2:/etc/docker/
scp daemon.json server3:/etc/docker/
scp daemon.json server4:/etc/docker/
ls
scp -r certs.d/ server1:/etc/docker/
scp -r certs.d/ server2:/etc/docker/
scp -r certs.d/ server4:/etc/docker/
scp /etc/hosts server1:/etc/hosts
scp /etc/hosts server2:/etc/hosts
scp /etc/hosts server4:/etc/hosts
scp /etc/yum.repos.d/dvd.repo server1:/etc/yum.repos.d/dvd.repo
scp /etc/yum.repos.d/dvd.repo server2:/etc/yum.repos.d/dvd.repo
scp /etc/yum.repos.d/dvd.repo server4:/etc/yum.repos.d/dvd.repo
配置仓库，开启harbor仓库，为集群提供镜像支持
docker-compose start
[root@server3 harbor]# docker-compose start
Starting log ... done
Starting registry ... done
Starting registryctl ... done
Starting postgresql ... done
Starting portal ... done
Starting redis ... done
Starting core ... done
Starting jobservice ... done
Starting proxy ... done
测试
myapp部署
载入镜像，内有v1 v2两个版本
[root@server3 ~]# docker load -i myapp.tar
d39d92664027: Loading layer [==================================================>] 4.232MB/4.232MB
8460a579ab63: Loading layer [==================================================>] 11.61MB/11.61MB
c1dc81a64903: Loading layer [==================================================>] 3.584kB/3.584kB
68695a6cfd7d: Loading layer [==================================================>] 4.608kB/4.608kB
05a9e65e2d53: Loading layer [==================================================>] 16.38kB/16.38kB
a0d2c4392b06: Loading layer [==================================================>] 7.68kB/7.68kB
Loaded image: ikubernetes/myapp:v1
Loaded image: ikubernetes/myapp:v2
[root@server3 ~]# docker tag ikubernetes/myapp:v1 reg.westos.org/library/myapp:v1
[root@server3 ~]# docker tag ikubernetes/myapp:v2 reg.westos.org/library/myapp:v2
#
镜像传入仓库
[root@server3 ~]# docker push reg.westos.org/library/myapp:v2
The push refers to repository [reg.westos.org/library/myapp]
05a9e65e2d53: Pushed
68695a6cfd7d: Pushed
c1dc81a64903: Pushed
8460a579ab63: Pushed
d39d92664027: Pushed
v2: digest: sha256:5f4afc8302ade316fc47c99ee1d41f8ba94dbe7e3e7747dd87215a15429b9102 size: 1362
[root@server3 ~]# docker push reg.westos.org/library/myapp:v1
The push refers to repository [reg.westos.org/library/myapp]
a0d2c4392b06: Pushed
05a9e65e2d53: Layer already exists
68695a6cfd7d: Layer already exists
c1dc81a64903: Layer already exists
8460a579ab63: Layer already exists
d39d92664027: Layer already exists
v1: digest: sha256:9eeca44ba2d410e54fccc54cbe9c021802aa8b9836a0bcf3d3229354e4c8870e size: 1569
部署webservice
docker service create --name webservice --publish 80:80 --replicas 3 myapp:v1
[root@server3 ~]# docker service create --name webservice --publish 80:80 --replicas 3 myapp:v1
ik2nbwlygupyqgz0fsy7yqfns
overall progress: 3 out of 3 tasks
1/3: running [==================================================>]
2/3: running [==================================================>]
3/3: running [==================================================>]
verify: Service converged
#
查看部署情况
[root@server3 ~]# docker service ps webservice
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
772hhsiuopqn webservice.1 myapp:v1 server4 Running Running 26 seconds ago
s84m4h8nluo2 webservice.2 myapp:v1 server3 Running Running 27 seconds ago
x16v1vc6cgz1 webservice.3 myapp:v1 server1 Running Running 25 seconds ago
测试访问与负载均衡
[root@server3 ~]# curl 172.25.3.1
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
[root@server3 ~]# curl 172.25.3.1/hostname.html
f8f1f6d00f83
[root@server3 ~]# curl 172.25.3.1/hostname.html
45cabc6b659a
[root@server3 ~]# curl 172.25.3.1/hostname.html
a75ad712c849
[root@server3 ~]# curl 172.25.3.1/hostname.html
f8f1f6d00f83
[root@server3 ~]# curl 172.25.3.1/hostname.html
45cabc6b659a
#
三. visualizer监控
载入镜像
[root@server4 ~]# docker load -i visualizer.tar
5bef08742407: Loading layer [==================================================>] 4.221MB/4.221MB
5f70bf18a086: Loading layer [==================================================>] 1.024kB/1.024kB
0a19bde117a5: Loading layer [==================================================>] 60.01MB/60.01MB
f7e883283ebc: Loading layer [==================================================>] 3.942MB/3.942MB
dfd8ee95c7e7: Loading layer [==================================================>] 1.536kB/1.536kB
300a6cad969a: Loading layer [==================================================>] 8.704kB/8.704kB
d1627040da6d: Loading layer [==================================================>] 489kB/489kB
00ed018016c5: Loading layer [==================================================>] 2.56kB/2.56kB
d5aa1ab1b431: Loading layer [==================================================>] 4.096kB/4.096kB
2d6a463420f7: Loading layer [==================================================>] 4.608kB/4.608kB
53888d7f4cca: Loading layer [==================================================>] 2.56kB/2.56kB
ea93ed99abca: Loading layer [==================================================>] 2.598MB/2.598MB
fa467b43abc0: Loading layer [==================================================>] 4.096kB/4.096kB
94cd25765710: Loading layer [==================================================>] 96.48MB/96.48MB
Loaded image: dockersamples/visualizer:latest
修改tag，上传harbor
[root@server4 ~]# docker tag dockersamples/visualizer:latest reg.westos.org/library/visualizer:latest
[root@server4 ~]# docker push reg.westos.org/library/visualizer:latest
创建监控
[root@server4 ~]# docker service create --name=viz --publish=8080:8080/tcp \
> --constraint=node.role==manager \
> --mount=type=bind,src=/var/run/docker.sock,dst=/var/run/docker.sock visualizer
> zvacnii44vd8y5gyqwjoxbnsa
> overall progress: 1 out of 1 tasks
> 1/1: running [==================================================>]
> verify: Service converged
> [root@server4 ~]# docker se
#
查看
[root@server4 ~]# docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
zvacnii44vd8 viz replicated 1/1 visualizer:latest *:8080->8080/tcp
创建webservice
[root@server4 ~]# docker service create --name webservice --publish 80:80 --replicas 3 myapp:v1
glzsj1ivs2w4nncodr1vxaau7
overall progress: 3 out of 3 tasks
1/3: running [==================================================>]
2/3: running [==================================================>]
3/3: running [==================================================>]
verify: Service converged
[root@server3 ~]# docker service ps webservice
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
772hhsiuopqn webservice.1 myapp:v1 server4 Running Running 3 minutes ago
s84m4h8nluo2 webservice.2 myapp:v1 server3 Running Running 3 minutes ago owpapp6u0fri webservice.3 myapp:v1 server2 Running Running 12 seconds ago
[root@server3 ~]# logout
查看监控界面
拉伸
[root@server4 ~]# docker service scale webservice=30
webservice scaled to 30
overall progress: 30 out of 30 tasks
verify: Service converged
回滚
rollback: manually requested rollback
overall progress: rolling back update: 30 out of 30 tasks
verify: Service converged
逐渐更改
[root@server3 ~]# curl 172.25.3.1
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
[root@server3 ~]# curl 172.25.3.1
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
[root@server3 ~]# curl 172.25.3.1
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
[root@server3 ~]# curl 172.25.3.1
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
[root@server3 ~]# curl 172.25.3.1
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
[root@server3 ~]# curl 172.25.3.1
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
四.compose+swarm 创建集群
vim docker-compose.yml
[root@server4 ~]# cat docker-compose.yml
version: "3.8"
services:
web:
image: myapp:v1
ports:
- "80:80"
networks:
- webnet
deploy:
replicas: 60
update_config:
parallelism: 20
delay: 1s
restart_policy:
condition: on-failure
visualizer:
image: visualizer:latest
ports:
- "8080:8080"
stop_grace_period: 1m30s
volumes:
- "/var/run/docker.sock:/var/run/docker.sock"
deploy:
placement:
constraints:
- "node.role==manager"
networks:
webnet:
启动
[root@server4 ~]# docker stack deploy -c docker-compose.yml mycluster
Creating network mycluster_webnet
Creating network mycluster_default
Creating service mycluster_web
Creating service mycluster_visualizer
查看集群内容器
[root@server4 ~]# docker stack ps mycluster
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
4gjshcghdvlv mycluster_visualizer.1 visualizer:latest server4 Running Starting 23 seconds ago
alk5ccscnffw mycluster_web.1 myapp:v2 server4 Running Running 19 seconds ago
gox0p14er63i mycluster_web.2 myapp:v2 server1 Running Running 20 seconds ago
nkg7oa6280xc mycluster_web.3 myapp:v2 server4 Running Running 16 seconds ago
9wiho7q6uong mycluster_web.4 myapp:v2 server2 Running Running 16 seconds ago
52tsq0wmetee mycluster_web.5 myapp:v2 server2 Running Running 16 seconds ago
nera3duna6cz mycluster_web.6 myapp:v2 server1 Running Running 17 seconds ago
测试负载均衡访问
执行更新
docker stack deploy -c docker-compose.yml mycluster
[root@server4 ~]# docker stack deploy -c docker-compose.yml mycluster
Updating service mycluster_web (id: kp0bwngqp5c4zgi8w83lnd4bu)
Updating service mycluster_visualizer (id: b4vxwj6fqpiyyu01e3akrzttq)
访问版本逐渐切换
五.portainer图形化集群管理工具
加载镜像
[root@server4 ~]# cd portainer/
[root@server4 portainer]# ls
portainer-agent-stack.yml portainer-agent.tar portainer.tar
[root@server4 portainer]# docker load -i portainer-agent.tar
dd4969f97241: Loading layer [==================================================>] 278kB/278kB
1f0ac9aec537: Loading layer [==================================================>] 2.048kB/2.048kB
0cf0d1d03535: Loading layer [==================================================>] 12.15MB/12.15MB
Loaded image: portainer/agent:latest
[root@server4 portainer]# docker load -i portainer.tar
e7260fd2a5f2: Loading layer [==================================================>] 73.85MB/73.85MB
Loaded image: portainer/portainer:latest
[root@server4 portainer]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
nginx latest 4bb46517cac3 11 months ago 133MB
portainer/portainer latest 19d07168491a 2 years ago 74.1MB
portainer/agent latest 9335796fedf9 2 years ago 12.4MB
myapp <none> d4a5e0eaa84f 3 years ago 15.5MB
myapp <none> 54202d3f0f35 3 years ago 15.5MB
reg.westos.org/library/visualizer latest 17e55a9b2354 3 years ago 148MB
dockersamples/visualizer latest 17e55a9b2354 3 years ago 148MB
visualizer <none> 17e55a9b2354 3 years ago 148MB
更改tag
[root@server4 portainer]# docker tag portainer/portainer:latest reg.westos.org/portainer/portainer:latest
[root@server4 portainer]# docker tag portainer/agent:latest reg.westos.org/portainer/agent:latest
push 进入portainer仓库
[root@server4 portainer]# docker push reg.westos.org/portainer/portainer:latest
The push refers to repository [reg.westos.org/portainer/portainer]
e7260fd2a5f2: Pushed
dd4969f97241: Pushed
latest: digest: sha256:5702df0ee639d9e003eee626d905e7d4d7ee1a2c7d40a7ec961e3996924f0285 size: 739
[root@server4 portainer]# docker push reg.westos.org/portainer/agent:latest
The push refers to repository [reg.westos.org/portainer/agent]
0cf0d1d03535: Pushed
1f0ac9aec537: Pushed
dd4969f97241: Mounted from portainer/portainer
latest: digest: sha256:d5e9b7b6e0ed94f868c64127fe8d4a2c3dc1c2fa771c6695f1f9c8924c646bac size: 945
创建集群
[root@server4 portainer]# ls
portainer-agent-stack.yml portainer-agent.tar portainer.tar
[root@server4 portainer]# docker stack deploy -c portainer-agent-stack.yml portainer
Creating network portainer_agent_network
Creating service portainer_portainer
Creating service portainer_agent
查看部署情况
[root@server4 portainer]# docker stack ls
NAME SERVICES ORCHESTRATOR
mycluster 2 Swarm
portainer 2 Swarm
[root@server4 portainer]# docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
8den1ppqapbu portainer_agent global 4/4 portainer/agent:latest
hwgc9adq1b8d portainer_portainer replicated 1/1 portainer/portainer:latest *:9000->9000/tcp
firefox访问 172.25.3.4:9000
