CKAD field manual
Kubernetes etcd snapshot example
etcd stores every API object. snapshot save copies that store. snapshot restore writes a new directory on the node — the static Pod's hostPath has to mount it, or the cluster comes back empty.
ETCDCTL_API=3 etcdctl endpoint health --endpoints=https://127.0.0.1:2379 --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/server.crt --key=/etc/kubernetes/pki/etcd/server.key
ETCDCTL_API=3 etcdctl member list --write-out=table --endpoints=https://127.0.0.1:2379 --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/server.crt --key=/etc/kubernetes/pki/etcd/server.key
ETCDCTL_API=3 etcdctl snapshot save /var/lib/etcd-backup.db --endpoints=https://127.0.0.1:2379 --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/server.crt --key=/etc/kubernetes/pki/etcd/server.key
etcdutl snapshot status /var/lib/etcd-backup.db --write-out=table
mv /etc/kubernetes/manifests/kube-apiserver.yaml /tmp/kube-apiserver.yaml
mv /etc/kubernetes/manifests/etcd.yaml /tmp/etcd.yaml
etcdutl snapshot restore /var/lib/etcd-backup.db --data-dir=/var/lib/etcd-from-backup --name=controlplane --initial-cluster=controlplane=https://10.0.0.1:2380 --initial-advertise-peer-urls=https://10.0.0.1:2380 --initial-cluster-token=etcd-restore-1
spec:
containers:
- name: etcd
command:
- etcd
- --data-dir=/var/lib/etcd
- --name=controlplane
- --initial-cluster=controlplane=https://10.0.0.1:2380
- --initial-advertise-peer-urls=https://10.0.0.1:2380
- --initial-cluster-token=etcd-restore-1
volumeMounts:
- name: etcd-data
mountPath: /var/lib/etcd
volumes:
- name: etcd-data
hostPath:
path: /var/lib/etcd-from-backup
type: DirectoryOrCreate
mv /tmp/etcd.yaml /etc/kubernetes/manifests/etcd.yaml
mv /tmp/kube-apiserver.yaml /etc/kubernetes/manifests/kube-apiserver.yaml
Fields
- etcdctl endpoint
- Live member. https://127.0.0.1:2379 and the certs under /etc/kubernetes/pki/etcd/ on kubeadm — unless the task prints others.
- etcdctl member
- Name and peer URL (port 2380). Those two values are what --name and --initial-cluster must repeat on restore.
- etcdctl snapshot
- Writes one file from a live member. ETCDCTL_API=3. Does not stop etcd. If etcdctl is missing, it is often only inside the etcd static Pod.
- etcdutl snapshot
- Runs on the file, not the cluster. No certs. status prints hash, revision, keys, size. restore creates --data-dir and refuses if that directory already exists.
- --data-dir
- Path inside the container. Leave it at /var/lib/etcd. The restored files arrive through the volume.
- --initial-cluster
- name=https://peer:2380, copied from the manifest. One member on a single control plane.
- --initial-cluster-token
- A new token on restore, then the same token here. The old token still names the old cluster.
- hostPath.path
- Path on the node — the directory snapshot restore created. Mounted onto mountPath, which stays /var/lib/etcd.
- mv
- Out of /etc/kubernetes/manifests stops the static Pod. Back in starts it. Edit the file while it sits in /tmp.
Watch
- snapshot restore does not overwrite /var/lib/etcd. It creates a new directory. In the static Pod, change only hostPath.path to that directory. Leave --data-dir and mountPath at /var/lib/etcd so the volume lands where etcd opens it.
- Copy the member name and the peer URL from etcdctl member list into --name, --initial-cluster and --initial-advertise-peer-urls. Peer port is 2380. Client port 2379 is only for --endpoints.
- Pass a new --initial-cluster-token to snapshot restore and set that same token in the static Pod. The old token still names the old cluster.
- snapshot save is etcdctl, against a live member, with the certs. snapshot status and snapshot restore are etcdutl, on the file, with no certs. If etcdutl is missing, ETCDCTL_API=3 etcdctl snapshot restore takes the same flags — deprecated, removed in etcd 3.6.
- Not a CKAD object. On the CKA, use the endpoint and cert paths the task prints. These are the kubeadm defaults.
Official docs Operating etcd clusters etcd disaster recovery