# Memo: Bootstrapping Kubernets using Kubeadm

Details: [https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/](https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/)

## Pre-requisite

* A compatible Linux host.
    
* **Verify the MAC address and product\_uuid** are unique for every node. Kubernetes uses these values to uniquely identify the nodes in the cluster.
    
    ```bash
    # Verify the mac-address
    ip link
    
    # The product_uuid can verfied using following commands. 
    # If these values are not unique to each node, the installation process may fail.
    sudo cat /sys/class/dmi/id/product_uuid
    ```
    
* **Check network adapters**
    
* **Check required ports**  
    These [required ports need to be op](https://kubernetes.io/docs/reference/networking/ports-and-protocols/)en in order for Kubernetes components to communicate with each other. The pod network plugin you use may also require certain ports to be open. Since this differs with each pod network plugin
    

```bash
nc 127.0.0.1 6443 -v
```

* 2 GB or more of RAM per machine (any less will leave little room for your apps).
    
* 2 CPUs or more.
    
* Swap configuration. The default behavior of a kubelet was to fail to start if swap memory was detected on a node. See [Swap memory management for more details.](https://kubernetes.io/docs/concepts/architecture/nodes/#swap-memory)
    
    * You **MUST** disable swap if the kubelet is not properly configured to use swap. For example, `sudo swapoff -a` will disable swapping temporarily. [To make this change p](https://kubernetes.io/docs/concepts/architecture/nodes/#swap-memory)ersistent across reboots, make sure swap is disabled in config files like `/etc/fstab`, `systemd.swap`, depending how it was configured on your system.
        
    
    ```bash
    sudo swapoff -a
    ```
    

### Network Configuration

Enable IPv4 packet forwarding:

```bash
# sysctl params required by setup, params persist across reboots
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.ipv4.ip_forward = 1
EOF

# Apply sysctl params without reboot
sudo sysctl --system

#verify
sysctl net.ipv4.ip_forward
```

### Configure cgroup drivers

**NOTE**: `Starting with v1.22 and later, when creating a cluster with kubeadm, if the user does not set the cgroupDriver field under KubeletConfiguration, kubeadm defaults it to systemd.`

### **Install** [**Containerd**](https://github.com/containerd/containerd/blob/main/docs/getting-started.md)

```bash
## Containerd
curl -LO https://github.com/containerd/containerd/releases/download/v1.7.18/containerd-1.7.18-linux-amd64.tar.gz
sudo tar Cxzvf /usr/local containerd-1.7.18-linux-amd64.tar.gz

## Systemd
curl -O https://raw.githubusercontent.com/containerd/containerd/main/containerd.service
sudo mv containerd.service /lib/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now containerd
```

### Runc

```bash
curl -LO  https://github.com/opencontainers/runc/releases/download/v1.1.12/runc.amd64
sudo install -m 755 runc.amd64 /usr/local/sbin/runc
```

### Install CNI plugins

```bash
~> curl -LO https://github.com/containernetworking/plugins/releases/download/v1.5.0/cni-plugins-linux-amd64-v1.5.0.tgz
~> mkdir -p /opt/cni/bin
~> sudo tar Cxzvf /opt/cni/bin cni-plugins-linux-amd64-v1.5.0.tgz

~> ls /opt/cni/bin/
LICENSE    bandwidth  dhcp   firewall     host-local  loopback  portmap  sbr     tap     vlan
README.md  bridge     dummy  host-device  ipvlan      macvlan   ptp      static  tuning  vrf
```

### Configure containerd

```bash
sudo mkdir -p /etc/containerd/
containerd config default | sudo tee  /etc/containerd/config.toml
# Update the SystemdCgroup = false line to SystemdCgroup = true

# You need CRI support enabled to use containerd with Kubernetes. 
#Make sure that cri is not included in thedisabled_plugins list
# within /etc/containerd/config.toml; if you made changes to that 
# file, also restart containerd.
sudo vim /etc/containerd/config.toml
sudo systemctl restart containerd
```

* [**Installing kubeadm, kubelet and kubectl**](https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#installing-kubeadm-kubelet-and-kubectl)
    

kubeadm will not install or manage `kubelet` or `kubectl` for you, so you will need to ensure they match the version of the Kubernetes control plane you want kubeadm to install for you.

* #### [**Network setup**](https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/#network-setup)
    

```bash
ip route show
```

**Optional**: **Preparing the required container images**  
This step is optional and only applies in case you wish `kubeadm init` and `kubeadm join` to not download the default container images which are hosted at [`registry.k8s.io`](http://registry.k8s.io).

### **Initializing your control-plane node**

* (Recommended) If you have plans to upgrade this single control-plane `kubeadm` cluster to high availability you should specify the `--control-plane-endpoint` to set the shared endpoint for all control-plane nodes. Such an endpoint can be either a DNS name or an IP address of a load-balancer.
    
* Choose a Pod network add-on, and verify whether it requires any arguments to be passed to `kubeadm init`. Depending on which third-party provider you choose, you might need to set the `--pod-network-cidr` to a provider-specific value. See [Installing a Pod network add-on](https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/#pod-network).
    
* (Optional) `kubeadm` tries to detect the container runtime by using a list of well known endpoints. To use different container runtime or if there are more than one installed on the provisioned node, specify the `--cri-socket` argument to `kubeadm`. See [Installing a runtime](https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#installing-runtime).
    

## Run init

```bash
~> sudo kubeadm init --pod-network-cidr 10.33.0.0/16
Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 10.0.8.15:6443 --token vajf1q.ahw188f33xklo8um \
        --discovery-token-ca-cert-hash sha256:58e837184a19c648286f21ccac308b2894fc0592ceee7c24f1ce0e616885bc52 

### Sanity Check
kubect get nodes
kubect get pods -n kube-system
```

### Deploy CNI Plugins

```bash
#for single node cluster
kubectl taint nodes vagrant node-role.kubernetes.io/control-plane-
curl https://raw.githubusercontent.com/projectcalico/calico/v3.28.0/manifests/calico.yaml -O
kubectl apply -f calico.yaml
```

### Untaint Nodes

```bash
kubectl taint nodes vagrant node.cilium.io/agent-not-ready-
```