istio: 取消注入sidecar

发布时间 2023-04-19 18:15:16作者: 若-飞

1. 问题

Consul注入istio以后,功能异常了,后台管理页面都打不开,无法选择leader

2023-04-19T09:33:27.886Z [ERROR] agent: Coordinate update error: error="No cluster leader"

2023-04-19T09:33:46.598Z [ERROR] agent.anti_entropy: failed to sync remote state: error="No cluster leader"

2023-04-19T09:33:57.469Z [ERROR] agent: Coordinate update error: error="No cluster leader"

2023-04-19T09:34:16.298Z [ERROR] agent.anti_entropy: failed to sync remote state: error="No cluster leader"

2023-04-19T09:34:27.653Z [ERROR] agent: Coordinate update error: error="No cluster leader"

2023-04-19T09:34:39.173Z [ERROR] agent.anti_entropy: failed to sync remote state: error="No cluster leader"

2023-04-19T09:34:59.006Z [ERROR] agent: Coordinate update error: error="No cluster leader"

2023-04-19T09:35:03.442Z [ERROR] agent.anti_entropy: failed to sync remote state: error="No cluster leader”

先不研究如何让consul兼容istio的问题,先考虑如何让consul不注入istio,

2. 取消注入sidecar

看了下官网,取消sidecar注入的配置很简单,只要在对应的deployment/daemonset/statefulset配置sidecar.istio.io/inject:"false"即可

apiVersion: apps/v1
spec:
  template:
    metadata:
      annotations:
        sidecar.istio.io/inject: “false"

 

3. consul取消注入sidecar实例

3.1. 注入以后的部署情况:

qiteck@server:~/program/docker_service/consul/cicd/deploy/k8s/nodeport$ sudo kubectl get pods --all-namespaces -o wide |grep consul
default         consul-client-hrlxb                          2/2     Running            0                84s     10.244.1.42    server               <none>           <none>
default         consul-server-0                              2/2     Running            0                85s     10.244.1.41    server               <none>           <none>
default         consul-server-1                              2/2     Running            0                66s     10.244.1.43    server               <none>           <none>
default         consul-server-2                              2/2     Running            0                52s     10.244.1.44    server               <none>           <none>

可以看到每个pod部署2个container

3.2. 取消注入配置

consul-client
 apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: consul-client
  labels:
    name: consul-client
spec:
  selector:
    matchLabels:
      name: consul-client
  template:
    metadata:
      labels:
        name: consul-client
      annotations:
        sidecar.istio.io/inject: "false"
    spec:
      containers:
        - name: consul
          image: consul:1.10.3
          imagePullPolicy: IfNotPresent
          args:
            - "agent"
            - "-ui"
            - "-data-dir=/consul/data"
            - "-bind=0.0.0.0"
            - "-client=0.0.0.0"
            - "-advertise=$(POD_IP)"
            - "-retry-join=consul-server-0.consul-server.$(NAMESPACE).svc.cluster.local"
            - "-retry-join=consul-server-1.consul-server.$(NAMESPACE).svc.cluster.local"
            - "-retry-join=consul-server-2.consul-server.$(NAMESPACE).svc.cluster.local"
            - "-domain=consul"          ## 默认情况下,Consul响应“consul”中的DNS查询。域。该标志可用于更改该域。该域中的所有查询都假定由Consul处理,不会递归解决。
            - "-disable-host-node-id"
          env:
            - name: POD_IP
              valueFrom:
                fieldRef:
                  fieldPath: status.podIP
            - name: NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          lifecycle:
            postStart:
              exec:
                command:
                  - /bin/sh
                  - -c
                  - consul reload
            preStop:
              exec:
                command:
                  - /bin/sh
                  - -c
                  - consul leave
          volumeMounts:
            - name: consul-data-dir
              mountPath: /consul/data
          ports:
            - containerPort: 8500
              name: http
            - containerPort: 8400
              name: rpc
            - containerPort: 8443
              name: https
            - containerPort: 8301
              name: serf-lan
            - containerPort: 8301
              name: serf-lan-8301
              protocol: UDP
            - containerPort: 8302
              name: serf-wan
            - containerPort: 8302
              name: serf-wan-udp
              protocol: UDP
            - name: dns-tcp
              containerPort: 8600
            - name: dns-udp
              containerPort: 8600
              protocol: UDP
            - containerPort: 8300
              name: server
      volumes:
        - name: consul-data-dir
          hostPath:
            path: /Users/xuan/desktop/consul/data
            type: DirectoryOrCreate
consul-server
 apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: consul-server
  labels:
    name: consul-server
spec:
  serviceName: consul-server
  selector:
    matchLabels:
      name: consul-server
  replicas: 3
  template:
    metadata:
      labels:
        name: consul-server
      annotations:
        sidecar.istio.io/inject: "false"
    spec:
      terminationGracePeriodSeconds: 10
      containers:
        - name: consul
          image: consul:1.10.3
          imagePullPolicy: IfNotPresent
          args:
            - "agent"
            - "-server"
            - "-bootstrap-expect=3"
            - "-ui"
            - "-data-dir=/consul/data"
            - "-bind=0.0.0.0"
            - "-client=0.0.0.0"
            - "-advertise=$(POD_IP)"
            - "-retry-join=consul-server-0.consul-server.$(NAMESPACE).svc.cluster.local"
            - "-retry-join=consul-server-1.consul-server.$(NAMESPACE).svc.cluster.local"
            - "-retry-join=consul-server-2.consul-server.$(NAMESPACE).svc.cluster.local"
            - "-domain=consul"    ## 默认情况下,Consul响应“consul”中的DNS查询。域。该标志可用于更改该域。该域中的所有查询都假定由Consul处理,不会递归解决。
            - "-disable-host-node-id"
          env:
            - name: POD_IP
              valueFrom:
                fieldRef:
                  fieldPath: status.podIP
            - name: NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          ports:
            - containerPort: 8500
              name: http
            - containerPort: 8400
              name: rpc
            - containerPort: 8443
              name: https-port
            - containerPort: 8301
              name: serf-lan
            - containerPort: 8301
              name: serf-lan-8301
              protocol: UDP
            - containerPort: 8302
              name: serf-wan
            - containerPort: 8302
              name: serf-wan-udp
              protocol: UDP
            - name: dns-tcp
              containerPort: 8600
              protocol: TCP
            - name: dns-udp
              containerPort: 8600
              protocol: UDP
            - containerPort: 8300
              name: server

3.3.取消注入以后的部署情况:

qiteck@server:~/program/docker_service/consul/cicd/deploy/k8s/nodeport$ sudo kubectl get pods --all-namespaces -o wide |grep consul
default         consul-client-44ksc                          1/1     Running            2 (60s ago)       2m     10.244.1.45    server               <none>           <none>
default         consul-server-0                              1/1     Running            0                 114s   10.244.1.46    server               <none>           <none>
default         consul-server-1                              1/1     Running            0                 110s   10.244.1.47    server               <none>           <none>
default         consul-server-2                              1/1     Running            0                 107s   10.244.1.48    server               <none>           <none>

可以看到每个pod部署1个container了。

然后功能也正常了。