Memo: Bootstrapping Kubernets using Kubeadm

Details: https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/

Pre-requisite

A compatible Linux host.

Verify the MAC address and product_uuid are unique for every node. Kubernetes uses these values to uniquely identify the nodes in the cluster.

  # Verify the mac-address
  ip link

  # The product_uuid can verfied using following commands. 
  # If these values are not unique to each node, the installation process may fail.
  sudo cat /sys/class/dmi/id/product_uuid

Check network adapters
Check required ports
These required ports need to be open in order for Kubernetes components to communicate with each other. The pod network plugin you use may also require certain ports to be open. Since this differs with each pod network plugin

nc 127.0.0.1 6443 -v

2 GB or more of RAM per machine (any less will leave little room for your apps).
2 CPUs or more.
Swap configuration. The default behavior of a kubelet was to fail to start if swap memory was detected on a node. See Swap memory management for more details.
- You MUST disable swap if the kubelet is not properly configured to use swap. For example, sudo swapoff -a will disable swapping temporarily. To make this change persistent across reboots, make sure swap is disabled in config files like /etc/fstab, systemd.swap, depending how it was configured on your system.

    sudo swapoff -a

Network Configuration

Enable IPv4 packet forwarding:

# sysctl params required by setup, params persist across reboots
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.ipv4.ip_forward = 1
EOF

# Apply sysctl params without reboot
sudo sysctl --system

#verify
sysctl net.ipv4.ip_forward

Configure cgroup drivers

NOTE: Starting with v1.22 and later, when creating a cluster with kubeadm, if the user does not set the cgroupDriver field under KubeletConfiguration, kubeadm defaults it to systemd.

Install Containerd

## Containerd
curl -LO https://github.com/containerd/containerd/releases/download/v1.7.18/containerd-1.7.18-linux-amd64.tar.gz
sudo tar Cxzvf /usr/local containerd-1.7.18-linux-amd64.tar.gz

## Systemd
curl -O https://raw.githubusercontent.com/containerd/containerd/main/containerd.service
sudo mv containerd.service /lib/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now containerd

Runc

curl -LO  https://github.com/opencontainers/runc/releases/download/v1.1.12/runc.amd64
sudo install -m 755 runc.amd64 /usr/local/sbin/runc

Install CNI plugins

~> curl -LO https://github.com/containernetworking/plugins/releases/download/v1.5.0/cni-plugins-linux-amd64-v1.5.0.tgz
~> mkdir -p /opt/cni/bin
~> sudo tar Cxzvf /opt/cni/bin cni-plugins-linux-amd64-v1.5.0.tgz

~> ls /opt/cni/bin/
LICENSE    bandwidth  dhcp   firewall     host-local  loopback  portmap  sbr     tap     vlan
README.md  bridge     dummy  host-device  ipvlan      macvlan   ptp      static  tuning  vrf

Configure containerd

sudo mkdir -p /etc/containerd/
containerd config default | sudo tee  /etc/containerd/config.toml
# Update the SystemdCgroup = false line to SystemdCgroup = true

# You need CRI support enabled to use containerd with Kubernetes. 
#Make sure that cri is not included in thedisabled_plugins list
# within /etc/containerd/config.toml; if you made changes to that 
# file, also restart containerd.
sudo vim /etc/containerd/config.toml
sudo systemctl restart containerd

Installing kubeadm, kubelet and kubectl

kubeadm will not install or manage kubelet or kubectl for you, so you will need to ensure they match the version of the Kubernetes control plane you want kubeadm to install for you.

Network setup

ip route show

Optional: Preparing the required container images
This step is optional and only applies in case you wish kubeadm init and kubeadm join to not download the default container images which are hosted at registry.k8s.io.

Initializing your control-plane node

(Recommended) If you have plans to upgrade this single control-plane kubeadm cluster to high availability you should specify the --control-plane-endpoint to set the shared endpoint for all control-plane nodes. Such an endpoint can be either a DNS name or an IP address of a load-balancer.
Choose a Pod network add-on, and verify whether it requires any arguments to be passed to kubeadm init. Depending on which third-party provider you choose, you might need to set the --pod-network-cidr to a provider-specific value. See Installing a Pod network add-on.
(Optional) kubeadm tries to detect the container runtime by using a list of well known endpoints. To use different container runtime or if there are more than one installed on the provisioned node, specify the --cri-socket argument to kubeadm. See Installing a runtime.

Run init

~> sudo kubeadm init --pod-network-cidr 10.33.0.0/16
Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 10.0.8.15:6443 --token vajf1q.ahw188f33xklo8um \
        --discovery-token-ca-cert-hash sha256:58e837184a19c648286f21ccac308b2894fc0592ceee7c24f1ce0e616885bc52 

### Sanity Check
kubect get nodes
kubect get pods -n kube-system

Deploy CNI Plugins

#for single node cluster
kubectl taint nodes vagrant node-role.kubernetes.io/control-plane-
curl https://raw.githubusercontent.com/projectcalico/calico/v3.28.0/manifests/calico.yaml -O
kubectl apply -f calico.yaml

Untaint Nodes

kubectl taint nodes vagrant node.cilium.io/agent-not-ready-

Balman's Blog