使用 kubeadm 快速创建一个集群。

主机初始化

这里用的是 Debian 12

关闭 swap

关闭防火墙

配置 hosts

cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
br_netfilter
overlay

sudo modprobe overlay
sudo modprobe br_netfilter

cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
EOF

sudo sysctl --system

确认是否加载模块
lsmod | grep br_netfilter
lsmod | grep overlay

安装 kubeadm,kubelet 和 kubectl

首先配置 apt 代理：

vim /etc/apt/apt.conf
Acquire::http::Proxy "http://10.0.10.7:8118";

sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates curl

curl -x 'http://10.0.10.7:8118' -fsSL https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-archive-keyring.gpg

echo "deb [signed-by=/etc/apt/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list

sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl

安装 containerd

containerd 发行版的包由 docker 维护：

sudo apt-get update
sudo apt-get install ca-certificates curl gnupg

sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/debian/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg

echo \
  "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/debian \
  "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

sudo apt-get install containerd.io

生成默认配置并修改:

containerd config default | sudo tee /etc/containerd/config.toml

# 需要修改的配置

[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
  SystemdCgroup = true

[plugins."io.containerd.grpc.v1.cri"]
  sandbox_image = "registry.aliyuncs.com/google_containers/pause:3.9"

[plugins."io.containerd.grpc.v1.cri".registry]
  [plugins."io.containerd.grpc.v1.cri".registry.mirrors]
  # 添加下面两个配置
    [plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
      endpoint = ["https://ekxinbbh.mirror.aliyuncs.com"]
    [plugins."io.containerd.grpc.v1.cri".registry.mirrors."k8s.gcr.io"]
      endpoint = ["https://gcr.k8s.li"]

sudo systemctl restart containerd

ctr version

注意：需要确保 kubelet 和 containerd 使用 systemd 作为 cgroup 驱动。

生成 kubeadm.yaml：

kubeadm config print init-defaults --kubeconfig ClusterConfiguration > kubeadm.yml

注意几个地方：

apiVersion: kubeadm.k8s.io/v1beta3
bootstrapTokens:
- groups:
  - system:bootstrappers:kubeadm:default-node-token
  token: abcdef.0123456789abcdef
  ttl: 24h0m0s
  usages:
  - signing
  - authentication
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: 192.168.122.154
  bindPort: 6443
nodeRegistration:
  criSocket: unix:///var/run/containerd/containerd.sock
  imagePullPolicy: IfNotPresent
  name: k8s-master
  taints: null
---
apiServer:
  timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta3
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
dns: {}
etcd:
  local:
    dataDir: /var/lib/etcd
imageRepository: registry.aliyuncs.com/google_containers
kind: ClusterConfiguration
kubernetesVersion: 1.27.0
networking:
  podSubnet: 10.0.0.0/16    # 注意这个 cidr 不要和主机的网络重叠
  dnsDomain: cluster.local
  serviceSubnet: 10.96.0.0/12
scheduler: {}
---
kind: KubeletConfiguration
apiVersion: kubelet.config.k8s.io/v1beta1
cgroupDriver: systemd
---
kind: KubeProxyConfiguration
apiVersion: kubeproxy.config.k8s.io/v1alpha1
mode: ipvs

拉取镜像:

kubeadm config images list --config kubeadm.yml

安装：

kubeadm init --config=kubeadm.yml --upload-certs | tee kubeadm-init.log

join 工作节点:

kubeadm join 192.168.122.154:6443 --token abcdef.0123456789abcdef \
	--discovery-token-ca-cert-hash sha256:d53020265c2bae4f691258966b3d35f99a9cc2dc530514888d85e916b2844525

安装 flannel

kubectl apply -f https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml

注意修改配置文件中的网段和 podSubnet 一致。

使 master 节点也能运行 pod

kubectl taint nodes --all node-role.kubernetes.io/control-plane-

创建测试容器

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  # 创建2个nginx容器
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.18.0
        ports:
        - containerPort: 80
---
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-svc
  labels:
    name: nginx-svc
spec:
  type: ClusterIP
  ports:
  - port: 80         
    targetPort: 80  
    protocol: TCP
  selector:
    app: nginx

查看 pod 和 service 创建是否正常。

reset 步骤(master 和 node 节点都需要)

如果发现有集群有问题的地方，需要重置，可以按照下面的步骤来，特别是网络有问题的话，一定要先删除网卡：

# kubeadm reset
# iptables -F
# iptables -X
# ipvsadm -C
# systemctl stop kubelet
# systemctl stop containerd
# rm -rf /var/lib/cni/
# rm -rf /var/lib/kubelet/*
# rm -rf /etc/cni/
# ifconfig cni0 down
# ifconfig flannel.1 down
# ip link delete cni0
# ip link delete flannel.1
# systemctl start containerd