Fix Kubernetes Certificate Expired Error Easily
When I started my Kubernetes Cluster and look to check the pods status. I got below errors:
root@kb-master:~# kubectl get pods
E0730 08:15:05.250099 5547 memcache.go:265] couldn't get current server API group list: Get "https://kb-master:6443/api?timeout=32s": dial tcp 192.168.64.230:6443: connect: connection refused
E0730 08:15:05.255138 5547 memcache.go:265] couldn't get current server API group list: Get "https://kb-master:6443/api?timeout=32s": dial tcp 192.168.64.230:6443: connect: connection refused
E0730 08:15:05.260390 5547 memcache.go:265] couldn't get current server API group list: Get "https://kb-master:6443/api?timeout=32s": dial tcp 192.168.64.230:6443: connect: connection refused
E0730 08:15:05.264253 5547 memcache.go:265] couldn't get current server API group list: Get "https://kb-master:6443/api?timeout=32s": dial tcp 192.168.64.230:6443: connect: connection refused
E0730 08:15:05.268127 5547 memcache.go:265] couldn't get current server API group list: Get "https://kb-master:6443/api?timeout=32s": dial tcp 192.168.64.230:6443: connect: connection refusedSo not sure what was the issue, Ultimately after some time, I found the issue was “Expired Certificate”.
When managing a Kubernetes cluster with kubeadm, certificate expiration is inevitable. If you encounter <invalid> messages during kubeadm certs check-expiration, don’t panic. In this blog, we will walk you through how to fix the Kubernetes certificate expired error smoothly and securely.
When running:
# kubeadm certs check-expirationYou may receive output like:
admin.conf Dec 30, 2024 11:15 UTC <invalid>
apiserver Dec 30, 2024 11:15 UTC <invalid>
...This indicates that several key Kubernetes certificates have expired, even though the certificate authority (ca, etcd-ca, front-proxy-ca) remains valid. As a result, services like kube-apiserver, controller-manager, and etcd may fail to start properly, leading to cluster-wide issues.
Why This Happens
Kubeadm-generated certificates are valid by default for 1 year. If not renewed in time, these certificates will cause services to crash silently or generate cryptic TLS errors such as:
x509: certificate has expired or is not yet validHow to Fix Expired Kubernetes Certificates
kubeadm offers a straightforward way to renew certificates. you can resolve the issue step-by-step:
Step1: Backup Your Existing Configuration
Before making changes, always back up critical directories:
root@kb-master:~/.kube# cp -r /etc/kubernetes /etc/kubernetes.bak-$(date +%F)
root@kb-master:~/.kube# cp -r ~/.kube ~/.kube.bak-$(date +%F)Step2: Renew All Certificates Using kubeadm
To regenerate all expired Kubernetes certificates, run:
root@kb-master:~/.kube# kubeadm certs renew all
[renew] Reading configuration from the cluster...
[renew] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[renew] Error reading configuration from the Cluster. Falling back to default configuration
certificate embedded in the kubeconfig file for the admin to use and for kubeadm itself renewed
certificate for serving the Kubernetes API renewed
certificate the apiserver uses to access etcd renewed
certificate for the API server to connect to kubelet renewed
certificate embedded in the kubeconfig file for the controller manager to use renewed
certificate for liveness probes to healthcheck etcd renewed
certificate for etcd nodes to communicate with each other renewed
certificate for serving etcd renewed
certificate for the front proxy client renewed
certificate embedded in the kubeconfig file for the scheduler manager to use renewed
Done renewing certificates. You must restart the kube-apiserver, kube-controller-manager, kube-scheduler and etcd, so that they can use the new certificates.This command renews internal certs like:
apiserveradmin.confcontroller-manager.confscheduler.confetcdclient/server certs
Step3: Regenerate Kubeconfig Files
If certificates inside admin.conf, controller-manager.conf, or scheduler.conf were expired, regenerate them:
root@kb-master:~/.kube# kubeadm init phase kubeconfig admin
root@kb-master:~/.kube# kubeadm init phase kubeconfig controller-manager
root@kb-master:~/.kube# kubeadm init phase kubeconfig schedulerThen, reconfigure kubectl
root@kb-master:~/.kube# mkdir -p $HOME/.kube
root@kb-master:~/.kube# cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
cp: overwrite '/root/.kube/config'? y
root@kb-master:~/.kube# chown $(id -u):$(id -g) $HOME/.kube/configStep4: Restart the Kubelet Service
root@kb-master:~/.kube# systemctl restart kubeletNow check the nodes:
root@kb-master:~/.kube# kubectl get nodes
NAME STATUS ROLES AGE VERSION
kb-master Ready control-plane 576d v1.27.9
worker1 Ready <none> 576d v1.27.9
worker2 Ready <none> 576d v1.27.9
worker3 Ready <none> 576d v1.27.9Final Thoughts
Even though Kubernetes certificate expiration can temporarily break the cluster, the fix is straightforward with the help of kubeadm. By regularly checking expiration dates and renewing certs proactively, you can keep your cluster healthy and avoid downtime.
So, the next time you see <invalid> in kubeadm certs check-expiration, remember: you’re just a few commands away from full recovery.
Visit My GitHub VARELITE
I enjoyed reading this article. Thanks for sharing your insights.