在kubeadm扩容k8s集群node节点的时候,出现以下错误:
[ERROR FileAvailable--etc-kubernetes-pki-ca.crt]: /etc/kubernetes/pki/ca.crt already exists
意思:/etc/kubernetes/pki/ca.crt已存在;
报错原因:在执行命令前,我把相关证书文件复制到这个扩容节点,而扩容命令会自动复制相关的证书文件到扩容节点。
解决:手动删除证书文件,然后重新执行扩容命令
[root@adm-master02 ~]# ls /etc/kubernetes/
admin.conf manifests pki
[root@adm-master02 ~]# rm -rf /etc/kubernetes/pki/*
[root@adm-master02 ~]# kubeadm join 192.168.2.41:6443 --token x5hhhs.lbw9rpqqk38ptdv4 --discovery-token-ca-cert-hash sha256:832ce791483907d42eb46def29bba10852172560844c48a8682c683ff418cd21
[preflight] Running pre-flight checks
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.
Run 'kubectl get nodes' on the control-plane to see this node join the cluster.
?验证是否扩容成功:
[root@ADM-master01-41 ~]# kubectl get nodes -owide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
adm-master01-41 Ready control-plane,master 49d v1.20.11 192.168.2.41 <none> CentOS Linux 7 (Core) 4.19.12-1.el7.elrepo.x86_64 docker://19.3.8
adm-master02 NotReady <none> 6m42s v1.20.11 192.168.2.44 <none> CentOS Linux 7 (Core) 4.19.12-1.el7.elrepo.x86_64 docker://19.3.8
adm-node01-2.42 Ready <none> 49d v1.20.11 192.168.2.42 <none> CentOS Linux 7 (Core) 4.19.12-1.el7.elrepo.x86_64 docker://19.3.8
adm-node02 Ready <none> 7d12h v1.20.11 192.168.2.43 <none> CentOS Linux 7 (Core) 4.19.12-1.el7.elrepo.x86_64 docker://19.3.8
[root@ADM-master01-41 ~]#
?扩容节点状态是NotReady,说明该节点虽然加入成功,但还无法正常使用,继续排查:
[root@ADM-master01-41 ~]# kubectl -n kube-system get pods -owide | grep 0/1
calico-node-k6n6d 0/1 Init:0/3 0 13m 192.168.2.44 adm-master02 <none> <none>
kube-proxy-b6fxs 0/1 ContainerCreating 0 13m 192.168.2.44 adm-master02 <none> <none>
这是扩容节点没有calico-node和kube-proxy镜像,将master上的这两个镜像打包,然后上传到该节点即可。
[root@ADM-master01-41 ~]# docker images | grep calico
calico/node v3.21.2 f1bca4d4ced2 7 weeks ago 214MB
calico/pod2daemon-flexvol v3.21.2 7778dd57e506 7 weeks ago 21.3MB
calico/cni v3.21.2 4c5c32530391 7 weeks ago 239MB
calico/kube-controllers v3.21.2 b20652406028 7 weeks ago 132MB
[root@ADM-master01-41 ~]# docker images | grep kube-proxy
k8s.gcr.io/kube-proxy v1.20.11 f4a6053ca28d 4 months ago 99.7MB
[root@ADM-master01-41 ~]# docker images | grep pause
k8s.gcr.io/pause 3.2 80d28bedfe5d 23 months ago 683kB
--------------------------------------------------------------------------
[root@ADM-master01-41 ~]# docker save calico/node:v3.21.2 -o calico-node-v3.21.2.tar
[root@ADM-master01-41 ~]# docker save k8s.gcr.io/kube-proxy:v1.20.11 -o kube-proxy-v1.20.11.tar
[root@ADM-master01-41 ~]# docker save k8s.gcr.io/pause:3.2 -o pause.tar
----------------------------------------------------------------------------------------
[root@ADM-master01-41 ~]# scp calico-node-v3.21.2.tar kube-proxy-v1.20.11.tar pause.tar 192.168.2.44:/root/
root@192.168.2.44's password:
calico-node-v3.21.2.tar 100% 208MB 63.0MB/s 00:03
kube-proxy-v1.20.11.tar 100% 97MB 67.7MB/s 00:01
pause.tar 100% 677KB 28.2MB/s 00:00
扩容节点导入镜像:
[root@adm-master02 ~]# ls
anaconda-ks.cfg calico-node-v3.21.2.tar kube-proxy-v1.20.11.tar pause.tar
[root@adm-master02 ~]# docker load -i calico-node-v3.21.2.tar
d149a79af148: Loading layer [==================================================>] 218.3MB/218.3MB
f2c81f625b1c: Loading layer [==================================================>] 13.82kB/13.82kB
Loaded image: calico/node:v3.21.2
[root@adm-master02 ~]# docker load -i kube-proxy-v1.20.11.tar
48b90c7688a2: Loading layer [==================================================>] 61.99MB/61.99MB
dfec24feb8ab: Loading layer [==================================================>] 39.49MB/39.49MB
Loaded image: k8s.gcr.io/kube-proxy:v1.20.11
[root@adm-master02 ~]# docker load -i pause.tar
ba0dae6243cc: Loading layer [==================================================>] 684.5kB/684.5kB
Loaded image: k8s.gcr.io/pause:3.2
[root@adm-master02 ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
calico/node v3.21.2 f1bca4d4ced2 7 weeks ago 214MB
k8s.gcr.io/kube-proxy v1.20.11 f4a6053ca28d 4 months ago 99.7MB
k8s.gcr.io/pause 3.2 80d28bedfe5d 23 months ago 683kB
验证
[root@ADM-master01-41 ~]# kubectl -n kube-system get pods -owide | grep 192.168.2.44
calico-node-k6n6d 1/1 Running 0 36m 192.168.2.44 adm-master02 <none> <none>
kube-proxy-b6fxs 1/1 Running 0 36m 192.168.2.44 adm-master02 <none> <none>
[root@ADM-master01-41 ~]# kubectl get node -owide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
adm-master01-41 Ready control-plane,master 49d v1.20.11 192.168.2.41 <none> CentOS Linux 7 (Core) 4.19.12-1.el7.elrepo.x86_64 docker://19.3.8
adm-master02 Ready <none> 37m v1.20.11 192.168.2.44 <none> CentOS Linux 7 (Core) 4.19.12-1.el7.elrepo.x86_64 docker://19.3.8
adm-node01-2.42 Ready <none> 49d v1.20.11 192.168.2.42 <none> CentOS Linux 7 (Core) 4.19.12-1.el7.elrepo.x86_64 docker://19.3.8
adm-node02 Ready <none> 7d12h v1.20.11 192.168.2.43 <none> CentOS Linux 7 (Core) 4.19.12-1.el7.elrepo.x86_64 docker://19.3.8
可以看到,扩容节点可正常使用了
|