我選擇 Container Runtime 是 CRI-O，是個專為 K8s 而設計，並且完全支持符合 OCI 規範的任何容器。
像是 Redhat 商用 K8s 平台：OpenShift 即使以 CRI-O 作為預設的 Container Runtime。
開源的 CRI 專案網址：
官網網址：
# 即刻啟用 module 方式
sudo modprobe overlay
sudo modprobe br_netfilter
# 開機啟用 module 方式
echo br_netfilter > /etc/modules-load.d/br_netfilter.conf
# 查看啟用的 module
lsmod |grep br_netfilter
# Setup required sysctl params, these persist across reboots.
[vagrant@kk8s-1 ~]$ sudo vi /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
[vagrant@kk8s-1 ~]$ sudo sysctl --system
* Applying /etc/sysctl.d/k8s.conf ...
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
vm.swappiness = 0
* Applying /etc/sysctl.conf ...
# Install prerequisites
[vagrant@kk8s-1 ~]$ sudo yum-config-manager --add-repo=https://cbs.centos.org/repos/paas7-crio-311-candidate/x86_64/os/
# Install CRI-O
[vagrant@kk8s-1 ~]$ sudo yum install --nogpgcheck cri-o
#
Installed:
cri-o.x86_64 0:1.11.8-2.rhaos3.11.git71cc465.el7
Dependency Installed:
audit-libs-python.x86_64 0:2.8.1-3.el7_5.1 checkpolicy.x86_64 0:2.5-6.el7 container-selinux.noarch 2:2.68-1.el7
containernetworking-plugins.x86_64 0:0.7.1-1.el7 criu.x86_64 0:3.5-4.el7 libcgroup.x86_64 0:0.41-15.el7
libnet.x86_64 0:1.1.6-7.el7 libsemanage-python.x86_64 0:2.5-11.el7 policycoreutils-python.x86_64 0:2.5-22.el7
protobuf-c.x86_64 0:1.0.2-3.el7 python-IPy.noarch 0:0.75-6.el7 runc.x86_64 0:1.0.0-52.dev.git70ca035.el7_5
setools-libs.x86_64 0:3.3.8-2.el7 skopeo-containers.x86_64 1:0.1.31-1.dev.gitae64ff7.el7.centos
Dependency Updated:
audit.x86_64 0:2.8.1-3.el7_5.1 audit-libs.x86_64 0:2.8.1-3.el7_5.1
[vagrant@kk8s-1 ~]$ sudo systemctl start crio
[vagrant@kk8s-1 ~]$ sudo systemctl enable crio
[vagrant@kk8s-1 ~]$ sudo systemctl status crio
● crio.service - Open Container Initiative Daemon
Loaded: loaded (/usr/lib/systemd/system/crio.service; disabled; vendor preset: disabled)
Active: active (running) since Sat 2018-11-10 11:52:18 UTC; 3s ago
Docs: https://github.com/kubernetes-sigs/cri-o
Main PID: 16765 (crio)
CGroup: /system.slice/crio.service
└─16765 /usr/bin/crio
Nov 10 11:52:18 kk8s-1 systemd[1]: Starting Open Container Initiative Daemon...
Nov 10 11:52:18 kk8s-1 crio[16765]: time="2018-11-10 11:52:18.905802494Z" level=error msg="watcher.Add("/usr/share/containers/oci/hooks.d") failed: no such file or directory"
Nov 10 11:52:18 kk8s-1 systemd[1]: Started Open Container Initiative Daemon.
# 上述出現一個 error，透過 Google 搜尋後發現需要 hook 目錄，重啟 crio 即可。
# https://github.com/containers/libpod/blob/master/pkg/hooks/docs/oci-hooks.5.md
[vagrant@kk8s-1 ~]$ sudo mkdir /usr/share/containers/oci/
[vagrant@kk8s-1 ~]$ sudo mkdir /usr/share/containers/oci/hooks.d
[vagrant@kk8s-1 ~]$ sudo systemctl restart crio
[vagrant@kk8s-1 ~]$ crio --version
crio version 1.11.8
# 因 Lab 環境有兩個網路介面，為了確定 kubeadm 抓取正確的 IP 參數，透過 hosts 確認主機名稱解析的 IP。
sudo vi /etc/hosts
# 初始化叢集 Master node
sudo kubeadm init --apiserver-advertise-address=192.168.42.191
# 因預設 kubeadm init 會偵測環境中 docker 要素但我沒安裝 docker！
# [preflight] WARNING: Couldn't create the interface used for talking to the container runtime: docker is required for container runtime: exec: "docker": executable file not found in $PATH
# 我採用 CRI-O 作為 Container Runtime，故 init arg 需增加 --cri-socket，如下：
sudo kubeadm init --cri-socket="/var/run/crio/crio.sock" --apiserver-advertise-address=192.168.42.191
### 結果失敗，主因出在 kubelet 服務上～只好繼續排查問題。
# 下篇繼續～
This error is likely caused by:
The kubelet is not running
The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
