CKAD field manual

Kubernetes kubeadm upgrade example

On each node, upgrade kubeadm, run apply or upgrade node, drain, upgrade the kubelet, restart it, and uncordon. One node at a time. Do not skip a minor.

Two crew carry a glowing core toward an open fighter. Two crew carry a glowing core toward an open fighter.
First control plane 
apt-mark unhold kubeadm
apt-get update
apt-get install -y kubeadm='1.35.0-*'
apt-mark hold kubeadm
kubeadm version
kubeadm upgrade plan
kubeadm upgrade apply v1.35.0
Every other node 
apt-mark unhold kubeadm kubectl
apt-get install -y kubeadm='1.35.0-*' kubectl='1.35.0-*'
apt-mark hold kubeadm kubectl
kubeadm upgrade node
Kubelet on that same node 
kubectl drain node-1 --ignore-daemonsets --delete-emptydir-data
apt-mark unhold kubelet
apt-get install -y kubelet='1.35.0-*'
apt-mark hold kubelet
systemctl daemon-reload
systemctl restart kubelet
kubectl uncordon node-1
Join a node 
kubeadm init --pod-network-cidr=10.244.0.0/16
kubeadm token create --print-join-command
kubeadm join 10.0.0.1:6443 --token <token> --discovery-token-ca-cert-hash sha256:<hash>

Fields

apt-mark
Packages are held. unhold before install, hold again after. Drop sudo when the shell is already root.
apt-get
Install the version the task names. 1.35.0-* is the package. There is no v on the apt pin.
kubeadm version
Confirms the kubeadm binary before plan. It must be the target minor.
kubeadm upgrade plan
First control plane only. Prints the versions you can apply. Every node must be Ready.
kubeadm upgrade apply
First control plane only, the node with /etc/kubernetes/admin.conf. The argument is v1.35.0, with the v.
kubeadm upgrade node
Every other control plane, and every worker. Do not run upgrade apply there.
kubectl drain
Before the kubelet package on that node. --ignore-daemonsets --delete-emptydir-data. Run it from a node that still has kubectl.
systemctl
daemon-reload, then restart kubelet. systemctl status kubelet shows the new process.
kubectl uncordon
After the kubelet is back. The node can take Pods again. Then start the next node.
kubeadm init
Creates the first control plane. A CNI still has to be installed before nodes go Ready.
kubeadm token
Prints a join command. The token and the ca-cert hash are what kubeadm join consumes.
kubeadm join
Run on the new node. API server host:6443, the token, and --discovery-token-ca-cert-hash sha256:…

Watch

  • Do one node at a time. kubeadm upgrade apply is the first control plane, the one with /etc/kubernetes/admin.conf. Every later control plane and every worker is kubeadm upgrade node. Do not skip a minor version.
  • upgrade apply wants every node Ready, and it does not upgrade the kubelet binary. On that same node, drain, install the kubelet package, daemon-reload, restart kubelet, uncordon. Then the next node.
  • The packages are held. unhold, install, hold. kubeadm upgrade apply takes v1.35.0. The apt pin is 1.35.0-* with no v. Replace 1.35.0 with the version the task names. On a worker, upgrade kubectl in the same step as kubeadm.
  • When the node cannot reach a package mirror, the debs are already on disk. Install the version the task names. kubeadm init does not install a CNI. Nodes stay NotReady until the network add-on is in place.

Official docs Upgrading kubeadm clusters Upgrading Linux nodes

Practice these objects on a live cluster →