[系统运维]Kubernetes安装手册（Ubuntu非高可用版-CNI-flannel)

CKA考试环境 --Kubernetes 安装手册（Ubuntu非高可用版）

安装前准备工作

1. 设置hosts解析

操作节点：所有节点（k8s-master）均需执行

• 修改hostname
  hostname必须只能包含小写字母、数字、","、"-"，且开头结尾必须是小写字母或数字

# 在master节点
$ hostnamectl set-hostname k8s-master #设置master节点的hostname

2. 调整系统配置

操作节点： 所有的master和slave节点（k8s-master,k8s-slave）需要执行

本章下述操作均以k8s-master为例，其他节点均是相同的操作（ip和hostname的值换成对应机器的真实值）

设置iptables

$ iptables -P FORWARD ACCEPT
$ /etc/init.d/ufw stop

• 关闭swap

swapoff -a
# 防止开机自动挂载 swap 分区
sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab

• 修改内核参数

cat <<EOF >  /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward=1
vm.max_map_count=262144
EOF
modprobe br_netfilter
sysctl -p /etc/sysctl.d/k8s.conf

• 设置apt源

#替换ａｐｔ源
$ vi /etc/apt/sources.list
deb http://mirrors.aliyun.com/ubuntu/ bionic main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-security main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-security main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-updates main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-updates main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-proposed main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-proposed main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-backports main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-backports main restricted universe multiverse

$ apt-get update && apt-get install -y apt-transport-https ca-certificates software-properties-common 
$ curl https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | apt-key add - 
$ curl -fsSL https://mirrors.ustc.edu.cn/docker-ce/linux/ubuntu/gpg | sudo apt-key add 
$ add-apt-repository "deb [arch=amd64] https://mirrors.ustc.edu.cn/docker-ce/linux/ubuntu $(lsb_release -cs) stable"
$ tee /etc/apt/sources.list.d/kubernetes.list <<-'EOF'
deb https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial main
EOF

$ apt-get update   
#若上步出现NO_PUBLICKEY问题，参考https://www.cnblogs.com/jiangzuo/p/13667011.html


-----------------------------------------------------------------------------------------------------
注：
# 若出现：资源锁被别的进程占用
$ apt-get update && apt-get install -y apt-transport-https ca-certificates software-properties-common 
Reading package lists... Done
E: Could not get lock /var/lib/apt/lists/lock - open (11: Resource temporarily unavailable)
E: Unable to lock directory /var/lib/apt/lists/

$ apt-get update   
Reading package lists... Done
E: Could not get lock /var/lib/apt/lists/lock - open (11: Resource temporarily unavailable)
E: Unable to lock directory /var/lib/apt/lists/

# 则查看是哪些进程占用， kill掉这三个进程即可
$  ps -e | grep apt
  3895 ?        00:00:00 apt.systemd.dai
  3911 ?        00:00:00 apt.systemd.dai
  4005 ?        00:00:00 apt-get

$  kill -9 3895 3911 4005

# 若出现这种情况，解决如下
$ add-apt-repository "deb [arch=amd64] https://mirrors.ustc.edu.cn/docker-ce/linux/ubuntu $(lsb_release -cs) stable"
...            
Reading package lists... Done
W: Failed to fetch https://mirrors.ustc.edu.cn/docker-ce/linux/ubuntu/dists/bionic/InRelease  Reading from proxy failed - read (115: Operation now in progress) [IP: 157.148.73.220 443]
W: Some index files failed to download. They have been ignored, or old ones used instead.

$ vim /etc/resolv.conf 
nameserver 127.0.0.53
options edns0
# search local 注释掉搜索域

# 升级
sudo apt upgrade

总结：这里的报错应该是装机的时候没注意把阿里云的源填到代理地址上了

3. 安装docker

操作节点： 所有节点

$ apt-get install docker-ce=5:19.03.9~3-0~ubuntu-bionic
## 启动docker
$ systemctl enable docker && systemctl start docker
$ ps aux | grep docker

部署kubernetes

1. 安装 kubeadm, kubelet 和 kubectl

操作节点： 所有的master和slave节点(k8s-master,k8s-slave) 需要执行

$ apt-get install kubelet=1.18.8-00 kubectl=1.18.8-00 kubeadm=1.18.8-00
## 查看kubeadm 版本
$ kubeadm version
## 设置kubelet开机启动
$ systemctl enable kubelet 

2. 初始化配置文件

操作节点： 只在master节点（k8s-master）执行

$ kubeadm config print init-defaults > kubeadm.yaml
$ cat kubeadm.yaml
apiVersion: kubeadm.k8s.io/v1beta2
bootstrapTokens:
- groups:
  - system:bootstrappers:kubeadm:default-node-token
  token: abcdef.0123456789abcdef
  ttl: 24h0m0s
  usages:
  - signing
  - authentication
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: 192.168.136.135   # 修改成master主机ip
  bindPort: 6443
nodeRegistration:
  criSocket: /var/run/dockershim.sock
  name: k8s-master
  taints:
  - effect: NoSchedule
    key: node-role.kubernetes.io/master
---
apiServer:
  timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta2
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
dns:
  type: CoreDNS
etcd:
  local:
    dataDir: /var/lib/etcd
imageRepository: registry.aliyuncs.com/google_containers   # 修改为国内源
kind: ClusterConfiguration
kubernetesVersion: v1.18.8    # 修改为v1.18.8
networking:
  dnsDomain: cluster.local
  podSubnet: 10.244.0.0/16			# 添加pod网段
  serviceSubnet: 10.96.0.0/12
scheduler: {}

3. 提前下载镜像

操作节点：只在master节点（k8s-master）执行

  # 提前下载镜像到本地
$ kubeadm config images pull --config kubeadm.yaml
[config/images] Pulled registry.aliyuncs.com/google_containers/kube-apiserver:v1.18.8
[config/images] Pulled registry.aliyuncs.com/google_containers/kube-controller-manager:v1.18.8
[config/images] Pulled registry.aliyuncs.com/google_containers/kube-scheduler:v1.18.8
[config/images] Pulled registry.aliyuncs.com/google_containers/kube-proxy:v1.18.8
[config/images] Pulled registry.aliyuncs.com/google_containers/pause:3.2
[config/images] Pulled registry.aliyuncs.com/google_containers/etcd:3.4.3-0
[config/images] Pulled registry.aliyuncs.com/google_containers/coredns:1.6.7

4. 初始化master节点

操作节点：只在master节点（k8s-master）执行

$ kubeadm init --config kubeadm.yaml

若初始化成功后，最后会提示如下信息：

...
Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 172.21.51.5:6443 --token abcdef.0123456789abcdef \
    --discovery-token-ca-cert-hash sha256:0535e8ac702893ac7fb3c9ceeaf91f12d2c3266995af0a6a3d53ae09465b4cf6 

接下来按照上述提示信息操作，配置kubectl客户端的认证

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

**??注意：**此时使用 kubectl get nodes查看节点应该处于notReady状态，因为还未配置网络插件

若执行初始化过程中出错，根据错误信息调整后，执行kubeadm reset后再次执行init操作即可

5. 添加slave节点到集群中

操作节点：所有的slave节点（k8s-slave）需要执行
在每台slave节点，执行如下命令，该命令是在kubeadm init成功后提示信息中打印出来的，需要替换成实际init后打印出的命令。

kubeadm join 172.21.51.5:6443 --token abcdef.0123456789abcdef \
    --discovery-token-ca-cert-hash sha256:0535e8ac702893ac7fb3c9ceeaf91f12d2c3266995af0a6a3d53ae09465b4cf6 

6. 安装flannel插件

操作节点：只在master节点（k8s-master）执行

• 下载flannel的yaml文件

wget https://raw.githubusercontent.com/coreos/flannel/2140ac876ef134e0ed5af15c65e414cf26827915/Documentation/kube-flannel.yml

• 修改配置，指定网卡名称，大概在文件的190行，添加一行配置：

$ vi kube-flannel.yml
...      
      containers:
      - name: kube-flannel
        image: quay.io/coreos/flannel:v0.11.0-amd64
        command:
        - /opt/bin/flanneld
        args:
        - --ip-masq
        - --kube-subnet-mgr
        - --iface=eth0  # 如果机器存在多网卡的话，指定内网网卡的名称，默认不指定的话会找第一块网
        resources:
          requests:
            cpu: "100m"
...

• 执行安装flannel网络插件

# 先拉取镜像,此过程国内速度比较慢
$ docker pull quay.io/coreos/flannel:v0.11.0-amd64
# 执行flannel安装
$ kubectl create -f kube-flannel.yml

7. 设置master节点为可调度

操作节点：k8s-master

默认部署成功后，master节点无法调度业务pod，如需设置master节点也可以参与pod的调度，需执行：

$ kubectl taint node k8s-master node-role.kubernetes.io/master:NoSchedule-

8. 验证集群

操作节点： 在master节点（k8s-master）执行

$ kubectl get nodes  #观察集群节点是否全部Ready

创建测试nginx服务

$ kubectl run  test-nginx --image=nginx:alpine

查看pod是否创建成功，并访问pod ip测试是否可用

$ kubectl get po -o wide
NAME                          READY   STATUS    RESTARTS   AGE   IP           NODE         NOMINATED NODE   READINESS GATES
test-nginx-5bd8859b98-5nnnw   1/1     Running   0          9s    10.244.1.2   k8s-slave1   <none>           <none>
$ curl 10.244.1.2
...
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

10. 清理环境

如果你的集群安装过程中遇到了其他问题，我们可以使用下面的命令来进行重置：

$ kubeadm reset
$ ifconfig cni0 down && ip link delete cni0
$ ifconfig flannel.1 down && ip link delete flannel.1
$ rm -rf /var/lib/cni/