SRE文档

深浅模式

1. Ingress-Nginx 进行灰度(金丝雀)发布 ​

1.1 发布方式 ​

滚动更新；

蓝绿发布；

灰度发布（金丝雀发布）；

1.2 Ingress-Nginx Canary介绍 ​

Nginx Ingress Controller 作为项目对外的流量入口和项目中各个服务的反向代理.

官方文档概述：Annotations - Ingress-Nginx Controller (kubernetes.github.io)Nginx Annotations 的几种 Canary 规则：

Annotations说明
nginx.ingress.kubernetes.io/canary必须设置该Annotation值为 true ，否则其它规
则将不会生效。取值：
true ：启用 canary 功能。
false ：不启用 canary 功能
nginx.ingress.kubernetes.io/canary-
by-header		表示基于请求头的名称进行灰度发布。
请求头名称的特殊取值：
always ：无论什么情况下，流量均会进入灰度
服务。
never ：无论什么情况下，流量均不会进入灰度
服务。
若没有指定请求头名称的值，则只要该头存在，
都会进行流量转发.
nginx.ingress.kubernetes.io/canary-
by-header-value		表示基于请求头的值进行灰度发布。
需要与 canary-by-header 头配合使用
nginx.ingress.kubernetes.io/canary-
by-header-pattern		表示基于请求头的值进行灰度发布，并对请求头
的值进行正则匹配。
需要与 canary-by-header 头配合使用。
取值为用于匹配请求头的值的正则表达式。
nginx.ingress.kubernetes.io/canary-
by-cookie		表示基于Cookie进行灰度发布。例如，
nginx.ingress.kubernetes.io/canary-by-
cookie: foo 。
Cookie内容的取值：
always ：当 foo=always ，流量会进入灰度服
务。
never ：当 foo=never ，流量不会进入灰度服
务。
只有当Cookie存在，且值为 always 时，才会进
行流量转发。
nginx.ingress.kubernetes.io/canary-
weight		表示基于权重进行灰度发布。
取值范围：0~权重总值。
若未设定总值，默认总值为100。
nginx.ingress.kubernetes.io/canary-
weight-total		表示设定的权重总值。
若未设定总值，默认总值为100。

❌ 注意

不同灰度方式的优先级 由高到低 为：

canary-by-header --> canary-by-cookie --> canary-weight

1.2 基于客户端请求的流量切分场景 ​

希望将请求头中包含 foo=bar 或者Cookie中包含 foo=bar 的客户端请求转发到ServiceV2服务中.

待运行一段时间稳定后，可将所有的流量从Service V1切换到Service V2服务中，再平滑地将Service V1服务下线.

思路:

1.创两个新的系统,一个old,一个new(cannary)

2.定义两个service,一个正常提供服务,一个增加cannary的annotions

3.待cannary版本没有问题之后,切换到新的版本上面

创建old-yaml ​

包含了deployment、service、ingress

yaml
[root@kube-master ingress]# cat old-demo-ingress.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: old-demo
spec:
  replicas: 2
  selector:
    matchLabels:
      run: old-demo
  template:
    metadata:
      labels:
        run: old-demo
    spec:
      containers:
      - name: old-demo
        image: registry.cn-zhangjiakou.aliyuncs.com/hsuing/demoapp:v1
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 80
          protocol: TCP
      restartPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
  name: old-demo
spec:
  type: ClusterIP
  ports:
  - port: 80
    targetPort: 80
  selector:
    run: old-demo
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: old-demo
spec:
  ingressClassName: nginx
  rules:
  - host: java.host.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: old-demo
            port:
              number: 80
[root@kube-master ingress]# cat old-demo-ingress.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: old-demo
spec:
  replicas: 2
  selector:
    matchLabels:
      run: old-demo
  template:
    metadata:
      labels:
        run: old-demo
    spec:
      containers:
      - name: old-demo
        image: registry.cn-zhangjiakou.aliyuncs.com/hsuing/demoapp:v1
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 80
          protocol: TCP
      restartPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
  name: old-demo
spec:
  type: ClusterIP
  ports:
  - port: 80
    targetPort: 80
  selector:
    run: old-demo
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: old-demo
spec:
  ingressClassName: nginx
  rules:
  - host: java.host.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: old-demo
            port:
              number: 80
  • 执行
bash
kubectl apply -f old-demo-ingress.yaml
kubectl apply -f old-demo-ingress.yaml
  • 测试效果
bash
#curl -H "Host: java.host.com" http://java.host.com
hsuing demoapp v1.1 !! ClientIP: 172.30.0.128, PodName: old-demo-687c6ccd99-mf7ks, PodIP: 172.23.127.96!
#curl -H "Host: java.host.com" http://java.host.com
hsuing demoapp v1.1 !! ClientIP: 172.30.0.128, PodName: old-demo-687c6ccd99-mf7ks, PodIP: 172.23.127.96!

创建灰度发布new-yaml ​

yaml
[root@kube-master ingress]# cat new-demo-ingress.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: new-demo
spec:
  replicas: 1
  selector:
    matchLabels:
      run: new-demo
  template:
    metadata:
      labels:
        run: new-demo
    spec:
      containers:
      - name: new-demo
        image: registry.cn-zhangjiakou.aliyuncs.com/hsuing/demoapp:v2
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 80
          protocol: TCP
      restartPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
  name: new-demo
spec:
  type: ClusterIP
  ports:
  - port: 80
    targetPort: 80
  selector:
    run: new-demo
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: new-demo-ingress
  annotations:
    # 开启Canary
    nginx.ingress.kubernetes.io/canary: "true"
    # 请求头为foo且值为bar的流量进入canary
    nginx.ingress.kubernetes.io/canary-by-header: "foo"
    nginx.ingress.kubernetes.io/canary-by-header-value: "bar"
spec:
  ingressClassName: nginx
  rules:
  - host: java.host.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: new-demo #选择新pod的svc
            port:
              number: 80
[root@kube-master ingress]# cat new-demo-ingress.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: new-demo
spec:
  replicas: 1
  selector:
    matchLabels:
      run: new-demo
  template:
    metadata:
      labels:
        run: new-demo
    spec:
      containers:
      - name: new-demo
        image: registry.cn-zhangjiakou.aliyuncs.com/hsuing/demoapp:v2
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 80
          protocol: TCP
      restartPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
  name: new-demo
spec:
  type: ClusterIP
  ports:
  - port: 80
    targetPort: 80
  selector:
    run: new-demo
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: new-demo-ingress
  annotations:
    # 开启Canary
    nginx.ingress.kubernetes.io/canary: "true"
    # 请求头为foo且值为bar的流量进入canary
    nginx.ingress.kubernetes.io/canary-by-header: "foo"
    nginx.ingress.kubernetes.io/canary-by-header-value: "bar"
spec:
  ingressClassName: nginx
  rules:
  - host: java.host.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: new-demo #选择新pod的svc
            port:
              number: 80
  • 测试效果
bash
#curl -H "Host: java.host.com" http://java.host.com
hsuing demoapp v1.1 !! ClientIP: 172.30.0.128, PodName: old-demo-687c6ccd99-mf7ks, PodIP: 172.23.127.96!

#仅请求头中满足 foo=bar 的客户端请求才能路由到新版本服务
#curl -H "Host: java.host.com" -H "foo:bar" http://java.host.com
hsuing demoapp v1.2 !! ClientIP: 10.103.236.202, PodName: new-demo-5d96dfb47d-s4v8j, PodIP: 172.30.0.180!
#curl -H "Host: java.host.com" http://java.host.com
hsuing demoapp v1.1 !! ClientIP: 172.30.0.128, PodName: old-demo-687c6ccd99-mf7ks, PodIP: 172.23.127.96!

#仅请求头中满足 foo=bar 的客户端请求才能路由到新版本服务
#curl -H "Host: java.host.com" -H "foo:bar" http://java.host.com
hsuing demoapp v1.2 !! ClientIP: 10.103.236.202, PodName: new-demo-5d96dfb47d-s4v8j, PodIP: 172.30.0.180!

1.3 按权重发布 ​

yaml
nginx.ingress.kubernetes.io/canary-weight: "50" #新添加这项
nginx.ingress.kubernetes.io/canary-weight: "50" #新添加这项

系统运行一段时间后，当新版本服务已经稳定并且符合预期后，需要下线老版本的服务，仅保留新版本服务在线上运行

为了达到该目标，需要将旧版本的Service指向新版本服务的Deployment（根据标签），并且删除旧版本的Deployment和新版本的Service

2. Ingress高级用法 ​

通过修改 nginx.ingress.kubernetes.io/configuration-snippet 配置，并且配置 正则实现：

  • nginx.ingress.kubernetes.io/configuration-snippet （用于插入 location块代码段）;
  • nginx.ingress.kubernetes.io/server-snippet （用于插入 server 块中的代码段）;