SRE文档

深浅模式

文档,https://kubernetes.io/zh-cn/docs/tasks/administer-cluster/kubeadm/kubeadm-certs/

1. 查看证书时间 ​

而默认情况下ca证书是十年，而其他证书都只有一年

1.1 第一种方式 ​

bash
#根据自己的路径进行修改
cd /etc/kubernetes/pki

for i in $(ls *.crt); do echo "====================== $i ========"; openssl x509 -in $i -text -noout | grep -A 3 'Validity' ; done

====================== apiserver.crt ========
        Validity
            Not Before: Apr 10 08:31:49 2024 GMT
            Not After : Apr 10 08:31:49 2025 GMT
        Subject: CN=kube-apiserver
.....
#根据自己的路径进行修改
cd /etc/kubernetes/pki

for i in $(ls *.crt); do echo "====================== $i ========"; openssl x509 -in $i -text -noout | grep -A 3 'Validity' ; done

====================== apiserver.crt ========
        Validity
            Not Before: Apr 10 08:31:49 2024 GMT
            Not After : Apr 10 08:31:49 2025 GMT
        Subject: CN=kube-apiserver
.....

1.2 第二种方式 ​

bash
[root@kube-master pki]# kubeadm certs check-expiration
[check-expiration] Reading configuration from the cluster...
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'

CERTIFICATE                EXPIRES                  RESIDUAL TIME   CERTIFICATE AUTHORITY   EXTERNALLY MANAGED
admin.conf                 Apr 10, 2025 08:31 UTC   273d            ca                      no
apiserver                  Apr 10, 2025 08:31 UTC   273d            ca                      no
apiserver-etcd-client      Apr 10, 2025 08:31 UTC   273d            etcd-ca                 no
apiserver-kubelet-client   Apr 10, 2025 08:31 UTC   273d            ca                      no
controller-manager.conf    Apr 10, 2025 08:31 UTC   273d            ca                      no
etcd-healthcheck-client    Apr 10, 2025 08:31 UTC   273d            etcd-ca                 no
etcd-peer                  Apr 10, 2025 08:31 UTC   273d            etcd-ca                 no
etcd-server                Apr 10, 2025 08:31 UTC   273d            etcd-ca                 no
front-proxy-client         Apr 10, 2025 08:31 UTC   273d            front-proxy-ca          no
scheduler.conf             Apr 10, 2025 08:31 UTC   273d            ca                      no

CERTIFICATE AUTHORITY   EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
ca                      Apr 08, 2034 08:31 UTC   9y              no
etcd-ca                 Apr 08, 2034 08:31 UTC   9y              no
front-proxy-ca          Apr 08, 2034 08:31 UTC   9y              no
[root@kube-master pki]# kubeadm certs check-expiration
[check-expiration] Reading configuration from the cluster...
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'

CERTIFICATE                EXPIRES                  RESIDUAL TIME   CERTIFICATE AUTHORITY   EXTERNALLY MANAGED
admin.conf                 Apr 10, 2025 08:31 UTC   273d            ca                      no
apiserver                  Apr 10, 2025 08:31 UTC   273d            ca                      no
apiserver-etcd-client      Apr 10, 2025 08:31 UTC   273d            etcd-ca                 no
apiserver-kubelet-client   Apr 10, 2025 08:31 UTC   273d            ca                      no
controller-manager.conf    Apr 10, 2025 08:31 UTC   273d            ca                      no
etcd-healthcheck-client    Apr 10, 2025 08:31 UTC   273d            etcd-ca                 no
etcd-peer                  Apr 10, 2025 08:31 UTC   273d            etcd-ca                 no
etcd-server                Apr 10, 2025 08:31 UTC   273d            etcd-ca                 no
front-proxy-client         Apr 10, 2025 08:31 UTC   273d            front-proxy-ca          no
scheduler.conf             Apr 10, 2025 08:31 UTC   273d            ca                      no

CERTIFICATE AUTHORITY   EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
ca                      Apr 08, 2034 08:31 UTC   9y              no
etcd-ca                 Apr 08, 2034 08:31 UTC   9y              no
front-proxy-ca          Apr 08, 2034 08:31 UTC   9y              no

2. 证书类型 ​

2.1 集群根证书 ​

bash
[root@kube-master kubeadm]# ll /etc/kubernetes/pki/ca*
-rw-r--r-- 1 root root 1099 4月  10 16:31 /etc/kubernetes/pki/ca.crt
-rw------- 1 root root 1679 4月  10 16:31 /etc/kubernetes/pki/ca.key
[root@kube-master kubeadm]# ll /etc/kubernetes/pki/ca*
-rw-r--r-- 1 root root 1099 4月  10 16:31 /etc/kubernetes/pki/ca.crt
-rw------- 1 root root 1679 4月  10 16:31 /etc/kubernetes/pki/ca.key

由此集群根证书签发的证书有：

  1. kube-apiserver 组件持有的服务端证书
bash
[root@kube-master kubeadm]# ll /etc/kubernetes/pki/apiserver.*
-rw-r--r-- 1 root root 1289 4月  10 16:31 /etc/kubernetes/pki/apiserver.crt
-rw------- 1 root root 1679 4月  10 16:31 /etc/kubernetes/pki/apiserver.key
[root@kube-master kubeadm]# ll /etc/kubernetes/pki/apiserver.*
-rw-r--r-- 1 root root 1289 4月  10 16:31 /etc/kubernetes/pki/apiserver.crt
-rw------- 1 root root 1679 4月  10 16:31 /etc/kubernetes/pki/apiserver.key
  1. kubelet 组件持有的客户端证书
bash
[root@kube-master kubeadm]# ll /etc/kubernetes/pki/apiserver-kubelet-client.*
-rw-r--r-- 1 root root 1164 4月  10 16:31 /etc/kubernetes/pki/apiserver-kubelet-client.crt
-rw------- 1 root root 1675 4月  10 16:31 /etc/kubernetes/pki/apiserver-kubelet-client.key
[root@kube-master kubeadm]# ll /etc/kubernetes/pki/apiserver-kubelet-client.*
-rw-r--r-- 1 root root 1164 4月  10 16:31 /etc/kubernetes/pki/apiserver-kubelet-client.crt
-rw------- 1 root root 1675 4月  10 16:31 /etc/kubernetes/pki/apiserver-kubelet-client.key

❌ 注意

kubelet的/var/lib/kubelet/config.yaml 配置文件中一般不会明确指定服务端证书，而是只指定ca根证书，让kubelet根据本地主机信息自动生成服务端证书并保存到配置的cert-dir文件夹中

2.2 汇聚层根证书 ​

bash
[root@kube-master kubeadm]# ll /etc/kubernetes/pki/front-proxy-ca.*
-rw-r--r-- 1 root root 1115 4月  10 16:31 /etc/kubernetes/pki/front-proxy-ca.crt
-rw------- 1 root root 1675 4月  10 16:31 /etc/kubernetes/pki/front-proxy-ca.key
[root@kube-master kubeadm]# ll /etc/kubernetes/pki/front-proxy-ca.*
-rw-r--r-- 1 root root 1115 4月  10 16:31 /etc/kubernetes/pki/front-proxy-ca.crt
-rw------- 1 root root 1675 4月  10 16:31 /etc/kubernetes/pki/front-proxy-ca.key

由此汇聚层根证书签发的证书有

  1. 代理端使用的客户端证书，用作代用户与 kube-apiserver 认证
bash
[root@kube-master kubeadm]# ll /etc/kubernetes/pki/front-proxy-client.*
-rw-r--r-- 1 root root 1119 4月  10 16:31 /etc/kubernetes/pki/front-proxy-client.crt
-rw------- 1 root root 1679 4月  10 16:31 /etc/kubernetes/pki/front-proxy-client.key
[root@kube-master kubeadm]# ll /etc/kubernetes/pki/front-proxy-client.*
-rw-r--r-- 1 root root 1119 4月  10 16:31 /etc/kubernetes/pki/front-proxy-client.crt
-rw------- 1 root root 1679 4月  10 16:31 /etc/kubernetes/pki/front-proxy-client.key

2.3 etcd 集群根证书 ​

bash
[root@kube-master kubeadm]# ll /etc/kubernetes/pki/etcd/ca.*
-rw-r--r-- 1 root root 1086 4月  10 16:31 /etc/kubernetes/pki/etcd/ca.crt
-rw------- 1 root root 1679 4月  10 16:31 /etc/kubernetes/pki/etcd/ca.key
[root@kube-master kubeadm]# ll /etc/kubernetes/pki/etcd/ca.*
-rw-r--r-- 1 root root 1086 4月  10 16:31 /etc/kubernetes/pki/etcd/ca.crt
-rw------- 1 root root 1679 4月  10 16:31 /etc/kubernetes/pki/etcd/ca.key

由此 etcd 根证书签发的证书有：

  1. etcd server 服务端证书
bash
[root@kube-master kubeadm]#  ll /etc/kubernetes/pki/etcd/server.*
-rw-r--r-- 1 root root 1208 4月  10 16:31 /etc/kubernetes/pki/etcd/server.crt
-rw------- 1 root root 1679 4月  10 16:31 /etc/kubernetes/pki/etcd/server.key
[root@kube-master kubeadm]#  ll /etc/kubernetes/pki/etcd/server.*
-rw-r--r-- 1 root root 1208 4月  10 16:31 /etc/kubernetes/pki/etcd/server.crt
-rw------- 1 root root 1679 4月  10 16:31 /etc/kubernetes/pki/etcd/server.key
  1. etcd 集群中 peer 节点互相通信使用的客户端证书
bash
[root@kube-master kubeadm]# ll /etc/kubernetes/pki/etcd/peer.*
-rw-r--r-- 1 root root 1208 4月  10 16:31 /etc/kubernetes/pki/etcd/peer.crt
-rw------- 1 root root 1675 4月  10 16:31 /etc/kubernetes/pki/etcd/peer.key
[root@kube-master kubeadm]# ll /etc/kubernetes/pki/etcd/peer.*
-rw-r--r-- 1 root root 1208 4月  10 16:31 /etc/kubernetes/pki/etcd/peer.crt
-rw------- 1 root root 1675 4月  10 16:31 /etc/kubernetes/pki/etcd/peer.key
  1. Pod 中定义 Liveness 探针使用的客户端证书
bash
[root@kube-master kubeadm]# ll /etc/kubernetes/pki/etcd/healthcheck-client.*
-rw-r--r-- 1 root root 1159 4月  10 16:31 /etc/kubernetes/pki/etcd/healthcheck-client.crt
-rw------- 1 root root 1675 4月  10 16:31 /etc/kubernetes/pki/etcd/healthcheck-client.key
[root@kube-master kubeadm]# ll /etc/kubernetes/pki/etcd/healthcheck-client.*
-rw-r--r-- 1 root root 1159 4月  10 16:31 /etc/kubernetes/pki/etcd/healthcheck-client.crt
-rw------- 1 root root 1675 4月  10 16:31 /etc/kubernetes/pki/etcd/healthcheck-client.key
  1. 配置在 kube-apiserver 中用来与 etcd server 做双向认证的客户端证书
bash
[root@kube-master kubeadm]#  ll /etc/kubernetes/pki/apiserver-etcd-client.*
-rw-r--r-- 1 root root 1155 4月  10 16:31 /etc/kubernetes/pki/apiserver-etcd-client.crt
-rw------- 1 root root 1679 4月  10 16:31 /etc/kubernetes/pki/apiserver-etcd-client.key
[root@kube-master kubeadm]#  ll /etc/kubernetes/pki/apiserver-etcd-client.*
-rw-r--r-- 1 root root 1155 4月  10 16:31 /etc/kubernetes/pki/apiserver-etcd-client.crt
-rw------- 1 root root 1679 4月  10 16:31 /etc/kubernetes/pki/apiserver-etcd-client.key

2.4 Serveice Account 密钥 ​

bash
[root@kube-master kubeadm]# ll /etc/kubernetes/pki/sa.*
-rw------- 1 root root 1675 4月  10 16:31 /etc/kubernetes/pki/sa.key
-rw------- 1 root root  451 4月  10 16:31 /etc/kubernetes/pki/sa.pub
[root@kube-master kubeadm]# ll /etc/kubernetes/pki/sa.*
-rw------- 1 root root 1675 4月  10 16:31 /etc/kubernetes/pki/sa.key
-rw------- 1 root root  451 4月  10 16:31 /etc/kubernetes/pki/sa.pub

❌ 注意

Serveice Account密钥对仅提供给kube-controller-manager使用，kube-controller-manager通过sa.key对token进行签名，Master节点通过公钥sa.pub进行签名的验证

2.5 API Server身份验证过程 ​

API Server 的 Authentication 环节支持多种身份校验方式：Client Cert、Bearer Token、Static Password Auth 等，这些方式中只要有一种方式通过 Authentication（Kubernetes API Server 会逐个方式尝试），那么身份校验就会通过

一旦 API Server 发现 Client 发起的 Request 使用的是 Service Account Token 的方式，API Server 就会自动采用 Signed Bearer Token 方式进行身份校验。而 Request 就会使用携带的 Service Account Token 参与验证。该 Token 是 API Server 在创建 Service Account 时用 kube-controller-manager 启动参数 --service-account-private-key-file 指定的私钥签署 (sign) 的，同时必须指定 kube-apiserver 参数 --service-account-key-file（如果没指定的话，会使用 --tls-private-key-file 替代）为该私钥对应的公钥，用来在认证阶段验证 Token，也就是说该证书对通过 CN 和 O 指定了 ServiceAccount 的授权权限

通过 Authentication 后，API Server 将根据 Pod 所属 ServiceAccount 的用户名（以 system:serviceaccount: 为前缀）和组（以 system:serviceaccounts: 前缀）的权限对其进行 Authorization 和 Admission Control 两个环节的处理

不管是自动生成的 Token 还是手动创建的 Token 的值都是一样的，因为进行签署 Token 的 -–service-account-key-file 是同一个

ServiceAccount 中的 Token 是 API server 私钥签署的，Pod 在对 API Server 发起请求的时候会带上该 Token，以确保能够通过 API Server 的认证。对 ServiceAccount 的授权通过对 ServiceAccount 对应的用户或组进行 RBAC 控制即可

3. 更新证书 ​

3.1 根据源码方式 ​

https://www.cnblogs.com/guangdelw/p/17575730.html

https://www.youqiqi.cn/archives/kubernetesxiu-gai-kubeadmzheng-shu

3.2 kubeadm方式 ​

官当

0.查看kubeamd版本 ​

bash
kubeadm version

#或者
[root@kube-master kubernetes]# kubectl version --short
Client Version: v1.22.17
Server Version: v1.22.17
kubeadm version

#或者
[root@kube-master kubernetes]# kubectl version --short
Client Version: v1.22.17
Server Version: v1.22.17

1. 检查证书是否过期 ​

kubeadm certs check-expiration  //新版本,1.22之后

kubeadm alpha certs check-expiration //老版本，在 1.20 之前使用
kubeadm certs check-expiration  //新版本,1.22之后

kubeadm alpha certs check-expiration //老版本，在 1.20 之前使用

2. 备份集群证书、配置信息 ​

bash
## 将k8s和tecd相关文件做备份
cp -r /etc/kubernetes /tmp/kubernetes.bak
cp -r /var/lib/etcd /tmp/etcd.bak
## 将k8s和tecd相关文件做备份
cp -r /etc/kubernetes /tmp/kubernetes.bak
cp -r /var/lib/etcd /tmp/etcd.bak

3. 执行更新 ​

#续订所有证书
kubeadm certs renew all
#续订所有证书
kubeadm certs renew all

4. 重启相关应用 ​

（所有master都要执行）

bash
docker ps |grep -E 'k8s_kube-apiserver|k8s_kube-controller-manager|k8s_kube-scheduler|k8s_etcd_etcd' | awk -F ' ' '{print $1}' |xargs docker restart
docker ps |grep -E 'k8s_kube-apiserver|k8s_kube-controller-manager|k8s_kube-scheduler|k8s_etcd_etcd' | awk -F ' ' '{print $1}' |xargs docker restart
bash
crictl ps |grep -E 'kube-apiserver|kube-controller-manager|kube-scheduler|etcd' | awk -F ' ' '{print $1}' |xargs crictl stop
crictl ps |grep -E 'kube-apiserver|kube-controller-manager|kube-scheduler|etcd' | awk -F ' ' '{print $1}' |xargs crictl stop

5.kubeconfig ​

bash
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config

#更新kubeconfig
kubeadm init phase kubeconfig all
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config

#更新kubeconfig
kubeadm init phase kubeconfig all

6.验证 ​

bash
[root@kube-master ~]# echo | openssl s_client -showcerts -connect 127.0.0.1:6443 -servername api 2>/dev/null | openssl x509 -noout -enddate
notAfter=Oct 22 06:15:11 2025 GMT

#或者
kubeadm certs check-expiration

#或者
 for item in `find /etc/kubernetes/pki -maxdepth 2 -name "*.crt"`;do openssl x509 -in $item -text -noout| grep Not;echo ======================$item===============;done
 
#执行get pod
kubectl get pod 如果能显示出来，说明更新证书成功
[root@kube-master ~]# echo | openssl s_client -showcerts -connect 127.0.0.1:6443 -servername api 2>/dev/null | openssl x509 -noout -enddate
notAfter=Oct 22 06:15:11 2025 GMT

#或者
kubeadm certs check-expiration

#或者
 for item in `find /etc/kubernetes/pki -maxdepth 2 -name "*.crt"`;do openssl x509 -in $item -text -noout| grep Not;echo ======================$item===============;done
 
#执行get pod
kubectl get pod 如果能显示出来，说明更新证书成功

https://cloud.google.com/kubernetes-engine/distributed-cloud/bare-metal/docs/troubleshooting/expired-certs?hl=zh-cn

4.二进制证书 ​

涉及到证书

img

直接重新生成新的证书,替换重启服务即可

#查看证书到期
openssl x509 -in /etc/kubernetes/pki/kubelet.crt -noout -text |grep 'Not'
#查看证书到期
openssl x509 -in /etc/kubernetes/pki/kubelet.crt -noout -text |grep 'Not'

5.etcd证书更换 ​

etcd部署在外部

5.1查看证书路径 ​

bash
systemctl cat   etcd
systemctl cat   etcd

5.2备份 ​

bash
#停止Etcd操作
systemctl stop etcd

#备份Etcd数据
cd /var/lib
tar -zvcf etcd.tar.gz etcd/

#备份原有ssl
cp -r /etc/etcd/ssl /tmp/etcd_ssl_backup
#停止Etcd操作
systemctl stop etcd

#备份Etcd数据
cd /var/lib
tar -zvcf etcd.tar.gz etcd/

#备份原有ssl
cp -r /etc/etcd/ssl /tmp/etcd_ssl_backup

5.3 创建配置文件 ​

配置文件，在证书目录创建openssl.conf，最下面一行，如果是三台机器构建的Etcd集群，则三台ip都需要加上

bash
[root@kube-master-01 ssl]# vi openssl.conf
[req]
req_extensions = v3_req
distinguished_name = req_distinguished_name

[req_distinguished_name]

[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
subjectAltName = @alt_names

[ ssl_client ]
extendedKeyUsage = clientAuth, serverAuth
basicConstraints = CA:FALSE
subjectKeyIdentifier=hash
authorityKeyIdentifier=keyid,issuer
subjectAltName = @alt_names

[ v3_ca ]
basicConstraints = CA:TRUE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
subjectAltName = @alt_names
authorityKeyIdentifier=keyid:always,issuer

[alt_names]
DNS.1 = localhost
DNS.2 = etcd.kube-system.svc.cluster.local
DNS.3 = etcd.kube-system.svc
DNS.4 = etcd.kube-system
DNS.5 = etcd
DNS.6 = lb.kubesphere.local
DNS.7 = kube-master-01
DNS.8 = kube-node-01
DNS.9 = kube-node-02
IP.1 = 127.0.0.1
IP.2 = 10.103.236.150
IP.3 = 10.103.236.151
IP.4 = 10.103.236.152
[root@kube-master-01 ssl]# vi openssl.conf
[req]
req_extensions = v3_req
distinguished_name = req_distinguished_name

[req_distinguished_name]

[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
subjectAltName = @alt_names

[ ssl_client ]
extendedKeyUsage = clientAuth, serverAuth
basicConstraints = CA:FALSE
subjectKeyIdentifier=hash
authorityKeyIdentifier=keyid,issuer
subjectAltName = @alt_names

[ v3_ca ]
basicConstraints = CA:TRUE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
subjectAltName = @alt_names
authorityKeyIdentifier=keyid:always,issuer

[alt_names]
DNS.1 = localhost
DNS.2 = etcd.kube-system.svc.cluster.local
DNS.3 = etcd.kube-system.svc
DNS.4 = etcd.kube-system
DNS.5 = etcd
DNS.6 = lb.kubesphere.local
DNS.7 = kube-master-01
DNS.8 = kube-node-01
DNS.9 = kube-node-02
IP.1 = 127.0.0.1
IP.2 = 10.103.236.150
IP.3 = 10.103.236.151
IP.4 = 10.103.236.152

5.4生成ca ​

如果没有过期，不需要执行这步

bash
openssl genrsa -out ca-key.pem 2048 > /	dev/null 2>&1
openssl req -x509 -new -nodes -key ca-key.pem -days 3650 -out ca.pem -subj "/CN=etcd-ca" > /dev/null 2>&1
openssl genrsa -out ca-key.pem 2048 > /	dev/null 2>&1
openssl req -x509 -new -nodes -key ca-key.pem -days 3650 -out ca.pem -subj "/CN=etcd-ca" > /dev/null 2>&1

生成etcd ​

bash
for host in kube-node-01 kube-node-02 ; do
  # Member key
  # 用于 etcd 节点之间的通信
  openssl genrsa -out member-${host}-key.pem 2048 > /dev/null 2>&1
  openssl req -new -key member-${host}-key.pem -out member-${host}.csr -subj "/CN=etcd-member-${host}" -config openssl.conf > /dev/null 2>&1
  openssl x509 -req -in member-${host}.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out member-${host}.pem -days 3650 -extensions ssl_client -extfile openssl.conf > /dev/null 2>&1

  # Admin key
  # 用于 etcd 集群的管理
  openssl genrsa -out admin-${host}-key.pem 2048 > /dev/null 2>&1
  openssl req -new -key admin-${host}-key.pem -out admin-${host}.csr -subj "/CN=etcd-admin-${host}" > /dev/null 2>&1
  openssl x509 -req -in admin-${host}.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out admin-${host}.pem -days 3650 -extensions ssl_client -extfile openssl.conf > /dev/null 2>&1

  # Node keys
  # 用于 etcd 节点与客户端之间的通信
  openssl genrsa -out node-${host}-key.pem 2048 > /dev/null 2>&1
  openssl req -new -key node-${host}-key.pem -out node-${host}.csr -subj "/CN=etcd-node-${host}" > /dev/null 2>&1
  openssl x509 -req -in node-${host}.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out node-${host}.pem -days 3650 -extensions ssl_client -extfile openssl.conf > /dev/null 2>&1

  # 将证书复制到每个节点
  scp -r /etc/etcd/ssl/* root@${host}:/etc/etcd/ssl/
done
for host in kube-node-01 kube-node-02 ; do
  # Member key
  # 用于 etcd 节点之间的通信
  openssl genrsa -out member-${host}-key.pem 2048 > /dev/null 2>&1
  openssl req -new -key member-${host}-key.pem -out member-${host}.csr -subj "/CN=etcd-member-${host}" -config openssl.conf > /dev/null 2>&1
  openssl x509 -req -in member-${host}.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out member-${host}.pem -days 3650 -extensions ssl_client -extfile openssl.conf > /dev/null 2>&1

  # Admin key
  # 用于 etcd 集群的管理
  openssl genrsa -out admin-${host}-key.pem 2048 > /dev/null 2>&1
  openssl req -new -key admin-${host}-key.pem -out admin-${host}.csr -subj "/CN=etcd-admin-${host}" > /dev/null 2>&1
  openssl x509 -req -in admin-${host}.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out admin-${host}.pem -days 3650 -extensions ssl_client -extfile openssl.conf > /dev/null 2>&1

  # Node keys
  # 用于 etcd 节点与客户端之间的通信
  openssl genrsa -out node-${host}-key.pem 2048 > /dev/null 2>&1
  openssl req -new -key node-${host}-key.pem -out node-${host}.csr -subj "/CN=etcd-node-${host}" > /dev/null 2>&1
  openssl x509 -req -in node-${host}.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out node-${host}.pem -days 3650 -extensions ssl_client -extfile openssl.conf > /dev/null 2>&1

  # 将证书复制到每个节点
  scp -r /etc/etcd/ssl/* root@${host}:/etc/etcd/ssl/
done

查看 ​

bash
openssl x509 -in ca.pem -noout -text | grep 'Not'
openssl x509 -in member-k8s-master1.pem -noout -text | grep 'Not'
openssl x509 -in admin-k8s-master1.pem -noout -text | grep 'Not'
openssl x509 -in node-k8s-master1.pem -noout -text | grep 'Not'
openssl x509 -in ca.pem -noout -text | grep 'Not'
openssl x509 -in member-k8s-master1.pem -noout -text | grep 'Not'
openssl x509 -in admin-k8s-master1.pem -noout -text | grep 'Not'
openssl x509 -in node-k8s-master1.pem -noout -text | grep 'Not'

5.5修改配置文件 ​

修改所有etcd节点

bash
Type=notify
WorkingDirectory=/var/lib/etcd/
ExecStart=/usr/local/bin/etcd \
  --name=k8s-master1 \
  --cert-file=/etc/etcd/ssl/member-k8s-master1.pem \
  --key-file=/etc/etcd/ssl/member-k8s-master1-key.pem \
  --peer-cert-file=/etc/etcd/ssl/member-k8s-master1.pem \
  --peer-key-file=/etc/etcd/ssl/member-k8s-master1-key.pem \
  --trusted-ca-file=/etc/etcd/ssl/ca.pem \
  --peer-trusted-ca-file=/etc/etcd/ssl/ca.pem \
Type=notify
WorkingDirectory=/var/lib/etcd/
ExecStart=/usr/local/bin/etcd \
  --name=k8s-master1 \
  --cert-file=/etc/etcd/ssl/member-k8s-master1.pem \
  --key-file=/etc/etcd/ssl/member-k8s-master1-key.pem \
  --peer-cert-file=/etc/etcd/ssl/member-k8s-master1.pem \
  --peer-key-file=/etc/etcd/ssl/member-k8s-master1-key.pem \
  --trusted-ca-file=/etc/etcd/ssl/ca.pem \
  --peer-trusted-ca-file=/etc/etcd/ssl/ca.pem \
  • 验证服务
bash
#重启etcd服务(记住，要3个节点一起重启，不然会hang住)
systemctl restart etcd

etcdctl --cacert=/etc/etcd/ssl/ca.pem --cert=/etc/etcd/ssl/node-node1.pem --key=/etc/etcd/ssl/node-node1-key.pem --endpoints=https://192.168.0.3:2379 endpoint health
#重启etcd服务(记住，要3个节点一起重启，不然会hang住)
systemctl restart etcd

etcdctl --cacert=/etc/etcd/ssl/ca.pem --cert=/etc/etcd/ssl/node-node1.pem --key=/etc/etcd/ssl/node-node1-key.pem --endpoints=https://192.168.0.3:2379 endpoint health
  • key和crt 转 pem
bash
.key 转换成 .pem：

openssl rsa -in temp.key -out temp.pem
.crt 转换成 .pem：

openssl x509 -in tmp.crt -out tmp.pem
.key 转换成 .pem：

openssl rsa -in temp.key -out temp.pem
.crt 转换成 .pem：

openssl x509 -in tmp.crt -out tmp.pem