[系统运维]Kubernetes安装过程全攻略|Minikube版

参考资料|网址

K8S和Docker介绍

安装方案参考

k8s拉取镜像解决方案

官方文档

dockerHub

虚拟化

Minikube测试教程

安装攻略

前言

安装环境是云服务器，Ubuntu18的系统，无法采用虚拟机那种形式。
安装目的主要是为了学习，所以选择了Minikube。

生成环境版，之后也许也会出一个教程，但暂时还不想牺牲自己服务器的swap分区，所以见谅啦。

Minikube

MiniKube是K8S官方为了开放者能在个人电脑上运行K8S提供的一套工具。
由Go语言实现，通过调用虚拟化管理程序，创建出一个运行在虚拟机内的单点集群。

安装kubectl

K8S集群是典型的C/S架构，有一个官方提供的名为kubectl的CLI工具。
尽量保证kubectl版本和K8S集群版本保持一致，或版本相差在一个小版本内。

可以选择指定版本，也可以直接选择安装最新的。

这个是指定版本：

curl -LO https://dl.k8s.io/release/v1.10.7/bin/linux/amd64/kubectl

curl -LO https://dl.k8s.io/v1.10.7/bin/linux/amd64/kubectl.sha256

echo “$(<kubectl.sha256) kubectl” | sha256sum --check -c

可能会遇到如上错误，按如下解决即可：
install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl

安装最新版本

官方文档

curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"

curl -LO "https://dl.k8s.io/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl.sha256"

echo "$(<kubectl.sha256) kubectl" | sha256sum --check

安装Minikube

先看，不要着急操作！

• wget -O minikube https://github.com/kubernetes/minikube/releases/download/v0.28.2/minikube-linux-amd64

• chmod +x minikube 赋予执行权限

• sudo mv minikube /usr/local/bin/minikube

• /usr/local/bin/minikube version
  minikube version: v0.28.2

# 安装最新的:
curl -Lo minikube https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64

chmod +x minikube

sudo cp ./minikube /usr/local/bin/minikube

使用 Minikube 创建集群，只要简单的执行 minikube start 即可

root@VM-0-11-ubuntu:/dbroot/NNroot# minikube start
Starting local Kubernetes v1.10.0 cluster...
Starting VM...
E0709 16:38:40.267790    7566 start.go:174] Error starting host: Error creating host: Error executing step: Running precreate checks.
: VBoxManage not found. Make sure VirtualBox is installed and VBoxManage is in the path.

 Retrying.
E0709 16:38:40.268011    7566 start.go:180] Error starting host:  Error creating host: Error executing step: Running precreate checks.
: VBoxManage not found. Make sure VirtualBox is installed and VBoxManage is in the path
================================================================================
An error has occurred. Would you like to opt in to sending anonymized crash
information to minikube to help prevent future errors?
To opt out of these messages, run the command:
        minikube config set WantReportErrorPrompt false
================================================================================
Please enter your response [Y/n]:
Y
E0709 16:38:48.449769    7566 util.go:151] Error uploading error message: : Post https://clouderrorreporting.googleapis.com/v1beta1/projects/k8s-minikube/events:report?key=AIzaSyACUwzG0dEPcl-eOgpDKnyKoUFgHdfoFuA: read tcp 172.17.0.11:41168->172.217.160.74:443: read: connection reset by peer

Error starting host: Error creating host: Error executing step: Running precreate checks.: VBoxManage not found.

因为没有使用虚拟机，应该将 Minikube 的 --vm-driver 参数设置为 none ，并且在本机已经正确安装 Docker。 这种方式是无需虚拟化支持的。minikube start --vm-driver=none

root@VM-0-11-ubuntu:/dbroot/NNroot# minikube start --vm-driver=none
Starting local Kubernetes v1.10.0 cluster...
Starting VM...
Getting VM IP address...
Moving files into cluster...
Downloading kubeadm v1.10.0
Downloading kubelet v1.10.0
Finished Downloading kubeadm v1.10.0
Finished Downloading kubelet v1.10.0
Setting up certs...
Connecting to cluster...
Setting up kubeconfig...
Starting cluster components...
E0709 17:15:00.768230   21541 start.go:300] Error starting cluster:  kubeadm init error
sudo /usr/bin/kubeadm init --config /var/lib/kubeadm.yaml --ignore-preflight-errors=                       DirAvailable--etc-kubernetes-manifests --ignore-preflight-errors=DirAvailable--data-                       minikube --ignore-preflight-errors=Port-10250 --ignore-preflight-errors=FileAvailabl                       e--etc-kubernetes-manifests-kube-scheduler.yaml --ignore-preflight-errors=FileAvaila                       ble--etc-kubernetes-manifests-kube-apiserver.yaml --ignore-preflight-errors=FileAvai                       lable--etc-kubernetes-manifests-kube-controller-manager.yaml --ignore-preflight-erro                       rs=FileAvailable--etc-kubernetes-manifests-etcd.yaml --ignore-preflight-errors=Swap                        --ignore-preflight-errors=CRI  &&
sudo /usr/bin/kubeadm alpha phase addon kube-dns
 running command: : running command:
sudo /usr/bin/kubeadm init --config /var/lib/kubeadm.yaml --ignore-preflight-errors=                       DirAvailable--etc-kubernetes-manifests --ignore-preflight-errors=DirAvailable--data-                       minikube --ignore-preflight-errors=Port-10250 --ignore-preflight-errors=FileAvailabl                       e--etc-kubernetes-manifests-kube-scheduler.yaml --ignore-preflight-errors=FileAvaila                       ble--etc-kubernetes-manifests-kube-apiserver.yaml --ignore-preflight-errors=FileAvai                       lable--etc-kubernetes-manifests-kube-controller-manager.yaml --ignore-preflight-erro                       rs=FileAvailable--etc-kubernetes-manifests-etcd.yaml --ignore-preflight-errors=Swap                        --ignore-preflight-errors=CRI  &&
sudo /usr/bin/kubeadm alpha phase addon kube-dns

 output: [init] Using Kubernetes version: v1.10.0
[init] Using Authorization modes: [Node RBAC]
[preflight] Running pre-flight checks.
[preflight] The system verification failed. Printing the output from the verificatio                       n:
KERNEL_VERSION: 4.15.0-118-generic
CONFIG_NAMESPACES: enabled
CONFIG_NET_NS: enabled
CONFIG_PID_NS: enabled
CONFIG_IPC_NS: enabled
CONFIG_UTS_NS: enabled
CONFIG_CGROUPS: enabled
CONFIG_CGROUP_CPUACCT: enabled
CONFIG_CGROUP_DEVICE: enabled
CONFIG_CGROUP_FREEZER: enabled
CONFIG_CGROUP_SCHED: enabled
CONFIG_CPUSETS: enabled
CONFIG_MEMCG: enabled
CONFIG_INET: enabled
CONFIG_EXT4_FS: enabled
CONFIG_PROC_FS: enabled
CONFIG_NETFILTER_XT_TARGET_REDIRECT: enabled (as module)
CONFIG_NETFILTER_XT_MATCH_COMMENT: enabled (as module)
CONFIG_OVERLAY_FS: enabled (as module)
CONFIG_AUFS_FS: enabled (as module)
CONFIG_BLK_DEV_DM: enabled
DOCKER_VERSION: 20.10.0
OS: Linux
CGROUPS_CPU: enabled
CGROUPS_CPUACCT: enabled
CGROUPS_CPUSET: enabled
CGROUPS_DEVICES: enabled
CGROUPS_FREEZER: enabled
CGROUPS_MEMORY: enabled
        [WARNING Hostname]: hostname "minikube" could not be reached
        [WARNING Hostname]: hostname "minikube" lookup minikube on 127.0.0.53:53: se                       rver misbehaving
        [WARNING FileExisting-crictl]: crictl not found in system path
Suggestion: go get github.com/kubernetes-incubator/cri-tools/cmd/crictl
Flag --admission-control has been deprecated, Use --enable-admission-plugins or --di                       sable-admission-plugins instead. Will be removed in a future version.
[preflight] Some fatal errors occurred:
        [ERROR SystemVerification]: unsupported docker version: 20.10.0
[preflight] If you know what you are doing, you can make a check non-fatal with `--i                       gnore-preflight-errors=...`
: running command:
sudo /usr/bin/kubeadm init --config /var/lib/kubeadm.yaml --ignore-preflight-errors=                       DirAvailable--etc-kubernetes-manifests --ignore-preflight-errors=DirAvailable--data-                       minikube --ignore-preflight-errors=Port-10250 --ignore-preflight-errors=FileAvailabl                       e--etc-kubernetes-manifests-kube-scheduler.yaml --ignore-preflight-errors=FileAvaila                       ble--etc-kubernetes-manifests-kube-apiserver.yaml --ignore-preflight-errors=FileAvai                       lable--etc-kubernetes-manifests-kube-controller-manager.yaml --ignore-preflight-erro                       rs=FileAvailable--etc-kubernetes-manifests-etcd.yaml --ignore-preflight-errors=Swap                        --ignore-preflight-errors=CRI  &&
sudo /usr/bin/kubeadm alpha phase addon kube-dns

.: exit status 2
E0709 17:15:30.768922   21541 util.go:151] Error uploading error message: : Post https://clouderrorreporting.googleapis.com/v1beta1/projects/k8s-minikube/events:report?key=AIzaSyACUwzG0dEPcl-eOgpDKnyKoUFgHdfoFuA: dial tcp 172.217.160.74:443: i/o timeout

好像Docker的版本太高了，是20的。
尝试降低版本

sudo systemctl stop docker
sudo apt-get remove -y docker-ce docker-ce-cli
sudo apt-get update
sudo apt-get install -y docker-ce=5:18.09.4~3-0~ubuntu-bionic docker-ce-cli=5:18.09.4~3-0~ubuntu-bionic
docker version

sudo systemctl stop docker
sudo apt-get remove -y docker-ce docker-ce-cli
sudo apt-get update

sudo apt-get install -y apt-transport-https ca-certificates curl software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -


sudo apt-get install -y docker-ce=17.12.0~ce-0~ubuntu docker-ce-cli=17.12.0~ce-0~ubuntu
docker version

apt-get install docker-ce=17.03.0~ce-0~ubuntu docker-ce-cli=17.03.0~ce-0~ubuntu

发现折腾来折腾去反而导致我Docker装不上了。
人生苦短，我选择快速安装，我要装回我原本高版本的Docker。

curl -fsSL https://get.docker.com | bash -s docker --mirror Aliyun

这里我想特别的说一下，应该保存一个服务器快照的。这样万一有什么事情发生，也方便恢复。

如果遇到apt_pkg的错误

1、进入包文件目录

cd /usr/lib/python3/dist-packages/
Select CodeCopy

2、执行软连接

ln -s apt_pkg.cpython-36m-x86_64-linux-gnu.so apt_pkg.so
Select CodeCopy
如果apt_pkg.so已存在，则使用：

ln -fs apt_pkg.cpython-36m-x86_64-linux-gnu.so apt_pkg.so
Select CodeCopy
或者可以使用：

apt install python3-apt
cd /usr/lib/python3/dist-packages
cp apt_pkg.cpython-35m-x86_64-linux-gnu.so apt_pkg.cpython-36m-x86_64-linux-gnu.so

add-apt-repository “deb [arch=amd64] https://download.docker.com/linux/ubuntu xenial stable”

如果遇到cgroupfs的错误

参考了这位大佬的博客进行了处理：

在/etc/docker下创建daemon.json并编辑：

mkdir /etc/docker/daemon.json
加入以下内容：

{
“exec-opts”:[“native.cgroupdriver=systemd”]
}

我要用最新的！

事实证明用旧的版本不如用新的，重来，从Docker开始全部都升到新的！

这是重新来过的结果：
我的服务器是双核的，如果你是单核可能还要再设置一下。

单核 cpu配置请设置 1
sudo minikube start --driver=none
–extra-config=kubeadm.ignore-preflight-errors=NumCPU
–force --cpus 1
–image-repository=registry.cn-hangzhou.aliyuncs.com/google_containers

还是遇到了镜像的问题😭

试了如下方式统统不行，太痛苦了😣

• minikube start --driver=none --image-repository=registry.cn-hangzhou.aliyuncs.com/google_containers

• minikube start --driver=none --registry-mirror=https://dockerhub.azk8s.cn --image-repository=registry.cn-hangzhou.aliyuncs.com/google_containers

设置了image-repository完全不起作用！
以下是详细错误：

        [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver.                                                                   The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/doc                                                                  s/setup/cri/
error execution phase preflight: [preflight] Some fatal errors occurred:
        [ERROR ImagePull]: failed to pull image k8s.gcr.io/kube-apiserver:v1.21.2: output                                                                  : Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled whil                                                                  e waiting for connection (Client.Timeout exceeded while awaiting headers)
, error: exit status 1
        [ERROR ImagePull]: failed to pull image k8s.gcr.io/kube-controller-manager:v1.21.                                                                  2: output: Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canc                                                                  eled while waiting for connection (Client.Timeout exceeded while awaiting headers)
, error: exit status 1
        [ERROR ImagePull]: failed to pull image k8s.gcr.io/kube-scheduler:v1.21.2: output                                                                  : Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled whil                                                                  e waiting for connection (Client.Timeout exceeded while awaiting headers)
, error: exit status 1
        [ERROR ImagePull]: failed to pull image k8s.gcr.io/kube-proxy:v1.21.2: output: Er                                                                  ror response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while wa                                                                  iting for connection (Client.Timeout exceeded while awaiting headers)
, error: exit status 1
        [ERROR ImagePull]: failed to pull image k8s.gcr.io/pause:3.4.1: output: Error res                                                                  ponse from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting f                                                                  or connection (Client.Timeout exceeded while awaiting headers)
, error: exit status 1
        [ERROR ImagePull]: failed to pull image k8s.gcr.io/etcd:3.4.13-0: output: Error r                                                                  esponse from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting                                                                   for connection (Client.Timeout exceeded while awaiting headers)
, error: exit status 1
        [ERROR ImagePull]: failed to pull image k8s.gcr.io/coredns/coredns:v1.8.0: output                                                                  : Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled whil                                                                  e waiting for connection (Client.Timeout exceeded while awaiting headers)
, error: exit status 1
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore                                                                  -preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher

脚本解决镜像问题！

由于国内镜像问题，导致了上面一系列的ERROR。

写脚本从DockerHub的其他仓库里拉取镜像。
去DockerHub搜索kube_proxy、kube_kube-apiserver等等。

就是上图报错中我框出来的这些东西。

向下面这样搜索：

vim pull_k8s_images.sh

内容：

set -o errexit
set -o nounset
set -o pipefail

KUBE_VERSION=v1.21.2
KUBE_PAUSE_VERSION=3.4.1
ETCD_VERSION=3.4.13-0
DNS_VERSION=v1.8.0

GCR_URL=k8s.gcr.io
## k8smx这个你可以换
DOCKERHUB_URL=k8smx

images=(
kube-proxy:${KUBE_VERSION}
kube-scheduler:${KUBE_VERSION}
kube-controller-manager:${KUBE_VERSION}
kube-apiserver:${KUBE_VERSION}
pause:${KUBE_PAUSE_VERSION}
etcd:${ETCD_VERSION}
coredns:${DNS_VERSION}
)

for imageName in ${images[@]} ; do
  docker pull $DOCKERHUB_URL/$imageName
  docker tag $DOCKERHUB_URL/$imageName $GCR_URL/$imageName
  docker rmi $DOCKERHUB_URL/$imageName
done

授予执行权

chmod +x ./pull_k8s_images.sh

执行

./pull_k8s_images.sh

你可以查看自己已经有了哪些镜像：

docker images

比如说我框出来的这些，就和ERROR的内容是匹配的

如果你发现还有镜像没有拉下来的，可以再写一个脚本，单独处理。
比如说coredns=v1.8.0没拉下来 :

set -o errexit
set -o nounset
set -o pipefail

DNS_VERSION=v1.8.0

GCR_URL=k8s.gcr.io/coredns
DOCKERHUB_URL=gotok8s

images=(
coredns:${DNS_VERSION}
)

for imageName in ${images[@]} ; do
  docker pull $DOCKERHUB_URL/$imageName
  docker tag $DOCKERHUB_URL/$imageName $GCR_URL/$imageName
  docker rmi $DOCKERHUB_URL/$imageName
done

使用方式和上面是一样的。

k8smx、gotok8s都是自己可以选的，这个仓库不行就换一个。

拿到所有镜像后，minikube start --driver=none

已经OK了。