14.5-3 - 14.5-7 流量治理实践

发布时间 2023-11-10 21:33:36作者: zhan0

一、超时策略(针对VirtualService)

如果访问某个后端的服务非常卡顿,会严重影响到用户体验,所以可以在前端设置一个超时时间,如果超过该时间则直接返回一个错误码(504)而不是一直等待。
在前面章节的学习中,我们了解到,天气预报应用中frontend会调用forecast,而forecast会调用recommendation
依据此调用关系,来设置一个超时策略,比如将超时策略设置在frontend和forecast中间,然后在forecast和recommendation中间设置一个延迟注入的策略,来模拟超时。

1.1 设置超时策略

[root@master-1-230 11.6]# pwd
/root/k8s/14.4/cloud-native-istio/11_traffic-management/11.6
[root@master-1-230 11.6]# kubectl  apply -f vs-forecast-timeout.yaml 
virtualservice.networking.istio.io/forecast-route created
查看策略
kubectl get vs forecast-route -n weather -o yaml
[root@master-1-230 11.6]# kubectl get vs forecast-route -n weather -o yaml
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"networking.istio.io/v1alpha3","kind":"VirtualService","metadata":{"annotations":{},"name":"forecast-route","namespace":"weather"},"spec":{"hosts":["forecast"],"http":[{"match":[{"headers":{"User-Agent":{"regex":".*(Chrome/([\\d.]+)).*"}}}],"route":[{"destination":{"host":"forecast","subset":"v2"}}]},{"route":[{"destination":{"host":"forecast","subset":"v1"}}]}]}}
  creationTimestamp: "2023-11-07T15:01:36Z"
  generation: 4
  name: forecast-route
  namespace: weather
  resourceVersion: "704691"
  uid: a728357f-78b9-42d7-9341-679750758ddd
spec:
  hosts:
  - forecast
  http:
  - match:
    - headers:
        User-Agent:
          regex: .*(Chrome/([\d.]+)).*
    route:
    - destination:
        host: forecast
        subset: v2
  - route:
    - destination:
        host: forecast
        subset: v1
测试,进入frontend的pod里,curl访问forecast
[root@master-1-230 11.6]# kubectl -n weather exec -it `kubectl get po -n weather|grep frontend-v1|awk '{print $1}'` -- bash -c 'curl -I http://forecast:3002/weather?locate=hangzhou'
HTTP/1.1 200 OK
content-type: application/json; charset=utf-8
content-length: 25
date: Thu, 09 Nov 2023 13:13:21 GMT
x-envoy-upstream-service-time: 855
server: envoy

1.2 注册延迟策略

[root@master-1-230 11.6]# kubectl apply -f vs-recommendation-fault-delay.yaml -n weather
virtualservice.networking.istio.io/recommendation-route configured

查看策略

[root@master-1-230 11.6]# kubectl get vs recommendation-route -n weather -o yaml
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"networking.istio.io/v1alpha3","kind":"VirtualService","metadata":{"annotations":{},"name":"recommendation-route","namespace":"weather"},"spec":{"hosts":["recommendation"],"http":[{"fault":{"delay":{"fixedDelay":"4s","percentage":{"value":100}}},"route":[{"destination":{"host":"recommendation","subset":"v1"}}]}]}}
  creationTimestamp: "2023-11-07T15:06:14Z"
  generation: 2
  name: recommendation-route
  namespace: weather
  resourceVersion: "721026"
  uid: 8e0f9d36-dc20-4884-84ab-1f1d71ff9cb9
spec:
  hosts:
  - recommendation
  http:
  - fault:
      delay:
        fixedDelay: 4s
        percentage:
          value: 100
    route:
    - destination:
        host: recommendation
        subset: v1

测试:frontend的pod里,curl访问forecast

[root@master-1-230 11.6]# kubectl -n weather exec -it `kubectl get po -n weather|grep frontend-v1|awk '{print $1}'` -- bash -c 'curl -I http://forecast:3002/weather?locate=hangzhou'
HTTP/1.1 200 OK
content-type: application/json; charset=utf-8
content-length: 25
date: Thu, 09 Nov 2023 13:52:05 GMT
x-envoy-upstream-service-time: 75
server: envoy

二、重试策略(针对VirtualService)

为了让系统更加健壮,是有必要设置重试策略的,比如当访问遇到问题(比如超时),我们应该让它再次尝试访问,而不是直接放弃。
比如,我们可以在frontend访问forecast时,让它遇到问题时重试5次。
[root@master-1-230 11.7]# pwd
/root/k8s/14.4/cloud-native-istio/11_traffic-management/11.7
[root@master-1-230 11.7]# ll
总用量 8
-rw-r--r-- 1 root root 279 11月  7 23:00 vs-forecast-retry.yaml
-rw-r--r-- 1 root root 310 11月  7 23:00 vs-recommendation-fault-abort.yaml
[root@master-1-230 11.7]# kubectl apply -f vs-forecast-retry.yaml -n weather
virtualservice.networking.istio.io/forecast-route configured

查看策略

[root@master-1-230 11.7]# kubectl get vs forecast-route -n weather -o yaml
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"networking.istio.io/v1alpha3","kind":"VirtualService","metadata":{"annotations":{},"name":"forecast-route","namespace":"weather"},"spec":{"hosts":["forecast"],"http":[{"retries":{"attempts":5,"perTryTimeout":"1s","retryOn":"5xx"},"route":[{"destination":{"host":"forecast","subset":"v2"}}]}]}}
  creationTimestamp: "2023-11-07T15:01:36Z"
  generation: 6
  name: forecast-route
  namespace: weather
  resourceVersion: "727741"
  uid: a728357f-78b9-42d7-9341-679750758ddd
spec:
  hosts:
  - forecast
  http:
  - retries:
      attempts: 5
      perTryTimeout: 1s
      retryOn: 5xx
    route:
    - destination:
        host: forecast
        subset: v2

模拟故障

[root@master-1-230 11.7]# kubectl apply -f vs-recommendation-fault-abort.yaml -n weather
virtualservice.networking.istio.io/recommendation-route configured
[root@master-1-230 11.7]# 

查看策略

[root@master-1-230 11.7]# kubectl get vs recommendation-route -n weather -o yaml
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"networking.istio.io/v1alpha3","kind":"VirtualService","metadata":{"annotations":{},"name":"recommendation-route","namespace":"weather"},"spec":{"hosts":["recommendation"],"http":[{"fault":{"abort":{"httpStatus":500,"percentage":{"value":100}}},"route":[{"destination":{"host":"recommendation","subset":"v1"}}]}]}}
  creationTimestamp: "2023-11-07T15:06:14Z"
  generation: 5
  name: recommendation-route
  namespace: weather
  resourceVersion: "727888"
  uid: 8e0f9d36-dc20-4884-84ab-1f1d71ff9cb9
spec:
  hosts:
  - recommendation
  http:
  - fault:
      abort:
        httpStatus: 500
        percentage:
          value: 100
    route:
    - destination:
        host: recommendation
        subset: v1

测试:

[root@master-1-230 11.7]# kubectl -n weather exec -it `kubectl get po -n weather|grep frontend-v1|awk '{print $1}'` -- bash -c 'curl -I http://forecast:3002/weather?locate=hangzhou'
HTTP/1.1 503 Service Unavailable
content-length: 114
content-type: text/plain
date: Thu, 09 Nov 2023 13:56:00 GMT
server: envoy

查看grep frontend-v1 日志

[root@master-1-230 ~]# kubectl logs -n weather `kubectl get po -n weather |grep frontend-v1 |awk '{print $1}'` -c istio-proxy |grep " 503 "
[2023-11-09T13:55:09.317Z] "HEAD /weather?locate=hangzhou HTTP/1.1" 503 UF,URX upstream_reset_before_response_started{connection_failure} - "-" 0 0 51666 - "-" "curl/7.52.1" "a16ad6ef-21f3-90e6-bab3-7fd553721a2a" "forecast:3002" "10.244.154.24:3002" outbound|3002|v2|forecast.weather.svc.cluster.local - 10.111.255.74:3002 10.244.167.139:42884 - -
[2023-11-09T13:57:20.871Z] "HEAD /weather?locate=hangzhou HTTP/1.1" 503 URX via_upstream - "-" 0 0 490 490 "-" "curl/7.52.1" "a5572535-4cd5-9c97-a744-f078dab485d3" "forecast:3002" "10.244.154.24:3002" outbound|3002|v2|forecast.weather.svc.cluster.local 10.244.167.139:40276 10.111.255.74:3002 10.244.167.139:44140 - -
[2023-11-09T13:57:23.610Z] "HEAD /weather?locate=hangzhou HTTP/1.1" 503 URX via_upstream - "-" 0 0 387 386 "-" "curl/7.52.1" "eb72f5e8-3e7c-9a92-9fd2-db915f226958" "forecast:3002" "10.244.154.24:3002" outbound|3002|v2|forecast.weather.svc.cluster.local 10.244.167.139:40308 10.111.255.74:3002 10.244.167.139:44172 - -
[2023-11-09T13:57:24.896Z] "HEAD /weather?locate=hangzhou HTTP/1.1" 503 URX via_upstream - "-" 0 0 325 325 "-" "curl/7.52.1" "d6c9e62b-9962-91fc-b91b-fe0ea7681de4" "forecast:3002" "10.244.154.24:3002" outbound|3002|v2|forecast.weather.svc.cluster.local 10.244.167.139:40308 10.111.255.74:3002 10.244.167.139:44186 - -
[2023-11-09T13:57:26.142Z] "HEAD /weather?locate=hangzhou HTTP/1.1" 503 URX via_upstream - "-" 0 0 305 305 "-" "curl/7.52.1" "bad2f7d1-cf4a-913c-b725-265c2b7999a1" "forecast:3002" "10.244.154.24:3002" outbound|3002|v2|forecast.weather.svc.cluster.local 10.244.167.139:40308 10.111.255.74:3002 10.244.167.139:44208 - -
[2023-11-09T13:57:28.313Z] "HEAD /weather?locate=hangzhou HTTP/1.1" 503 URX via_upstream - "-" 0 0 235 235 "-" "curl/7.52.1" "bbb9c74c-4c09-9c30-a36f-f13f5006f8ea" "forecast:3002" "10.244.154.24:3002" outbound|3002|v2|forecast.weather.svc.cluster.local 10.244.167.139:40308 10.111.255.74:3002 10.244.167.139:44222 - -
[root@master-1-230 ~]# 

三、HTTP重定向和重写(针对VirtualService)

在Nginx中经常配置重定向或重写规则,而Istio也可以轻松实现。
下面,我们针对frontend到advertisement的请求,来实现具体的规则

1)redirect(重定向)

配置规则:
[root@master-1-230 11.8]# pwd
/root/k8s/14.4/cloud-native-istio/11_traffic-management/11.8
[root@master-1-230 11.8]# kubectl apply -f redirect.yaml  -n weather
virtualservice.networking.istio.io/advertisement-route configured
[root@master-1-230 11.8]# 

查看规则

[root@master-1-230 11.8]# kubectl get vs advertisement-route -n weather -o yaml
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"networking.istio.io/v1alpha3","kind":"VirtualService","metadata":{"annotations":{},"name":"advertisement-route","namespace":"weather"},"spec":{"hosts":["advertisement"],"http":[{"match":[{"uri":{"prefix":"/ad"}}],"redirect":{"authority":"advertisement.weather.svc.cluster.local","uri":"/maintenanced"}}]}}
  creationTimestamp: "2023-11-07T15:01:36Z"
  generation: 4
  name: advertisement-route
  namespace: weather
  resourceVersion: "728939"
  uid: a22c0500-c1a9-480a-a721-8974f2f5ad81
spec:
  hosts:
  - advertisement
  http:
  - match:
    - uri:
        prefix: /ad
    redirect:
      authority: advertisement.weather.svc.cluster.local
      uri: /maintenanced

测试:

[root@master-1-230 11.8]# kubectl -n weather exec -it `kubectl get po -n weather|grep frontend-v1|awk '{print $1}'` -- bash -c 'curl -I http://advertisement:3003/ad'
HTTP/1.1 301 Moved Permanently
location: http://advertisement.weather.svc.cluster.local/maintenanced
date: Thu, 09 Nov 2023 14:00:05 GMT
server: envoy
transfer-encoding: chunked

2)rewrite(重写)

配置规则
[root@master-1-230 11.9]# pwd
/root/k8s/14.4/cloud-native-istio/11_traffic-management/11.9
[root@master-1-230 11.9]# kubectl apply -f rewrite.yaml -n weather
virtualservice.networking.istio.io/advertisement-route configured
[root@master-1-230 11.9]# 

查看规则

[root@master-1-230 11.9]# kubectl get vs advertisement-route -n weather -o yaml
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"networking.istio.io/v1alpha3","kind":"VirtualService","metadata":{"annotations":{},"name":"advertisement-route","namespace":"weather"},"spec":{"hosts":["advertisement"],"http":[{"match":[{"uri":{"prefix":"/demo/"}}],"rewrite":{"uri":"/"},"route":[{"destination":{"host":"advertisement","subset":"v1"}}]}]}}
  creationTimestamp: "2023-11-07T15:01:36Z"
  generation: 5
  name: advertisement-route
  namespace: weather
  resourceVersion: "729193"
  uid: a22c0500-c1a9-480a-a721-8974f2f5ad81
spec:
  hosts:
  - advertisement
  http:
  - match:
    - uri:
        prefix: /demo/
    rewrite:
      uri: /
    route:
    - destination:
        host: advertisement
        subset: v1

测试:

[root@master-1-230 11.9]# kubectl -n weather exec -it `kubectl get po -n weather|grep frontend-v1|awk '{print $1}'` -- bash -c 'curl  http://advertisement:3003/ad --silent -w "Status: %{http_code}\n"'
Status: 404
[root@master-1-230 11.9]# kubectl -n weather exec -it `kubectl get po -n weather|grep frontend-v1|awk '{print $1}'` -- bash -c 'curl  http://advertisement:3003/demo/ad --silent -w "Status: %{http_code}\n"'
{"adImgName":"tshirtImg"}
Status: 200

把advertisement-route这条规则删除,再来测试:

[root@master-1-230 11.9]# kubectl delete vs advertisement-route -n weather
virtualservice.networking.istio.io "advertisement-route" deleted
[root@master-1-230 11.9]# kubectl -n weather exec -it `kubectl get po -n weather|grep frontend-v1|awk '{print $1}'` -- bash -c 'curl  http://advertisement:3003/demo/ad --silent -w "Status: %{http_code}\n"'
404 page not foundStatus: 404
[root@master-1-230 11.9]# kubectl -n weather exec -it `kubectl get po -n weather|grep frontend-v1|awk '{print $1}'` -- bash -c 'curl  http://advertisement:3003/ad --silent -w "Status: %{http_code}\n"'
{"adImgName":"mountainClimbingImg"}
Status: 200

四、熔断(针对DestinationRule)

在日常生活中,部署电源总闸时,必须要上熔断策略(当电流过高,就会自动掉闸,或者自动将保险丝烧毁),否则就会有很大的风险。
而在Istio中,这个熔断指的是,当业务流量非常大的时候,为了保护系统整体可用性,Istio会根据连接池的熔断配置对一部分请求直接拒绝访问(即,返回503状态码)
 

1)连接池配置

下面,我们来模拟一下这个场景,将连接forecast的连接池配置为最多接收3个并发连接,如果超过3个,那么多余的就会触发熔断机制,即返回503错误码。
首先部署一个用来测试的客户端pod
[root@master-1-230 11.10]# pwd
/root/k8s/14.4/cloud-native-istio/11_traffic-management/11.10
[root@master-1-230 11.10]# kubectl apply -f fortio-deploy.yaml -n weather
service/fortio created
deployment.apps/fortio-deploy created

查看pod

[root@master-1-230 11.10]# kubectl get po -n weather |grep fortio
fortio-deploy-689bd5969b-z2xq7       0/2     PodInitializing   0             25s
[root@master-1-230 11.10]# 

将forecast-v2扩展到5个pod

# kubectl -n weather edit deploy forecast-v2 
deployment.apps/forecast-v2 edited

查看pod

[root@master-1-230 11.10]# kubectl get po -n weather |grep forecast-v2
forecast-v2-655c4f98d7-86bh2         2/2     Running   0             28s
forecast-v2-655c4f98d7-9dzjd         2/2     Running   0             28s
forecast-v2-655c4f98d7-d4m6n         2/2     Running   0             28s
forecast-v2-655c4f98d7-t562b         2/2     Running   0             28s
forecast-v2-655c4f98d7-zmfq6         2/2     Running   6 (58m ago)   46h
配置熔断之前, 我们先做一些准备工作:
需要将前面设置过的recommendation-dr和recommendation-route删除掉,否则会影响到实验效果。
[root@master-1-230 11.10]# kubectl delete vs recommendation-route -n weather
virtualservice.networking.istio.io "recommendation-route" deleted
[root@master-1-230 11.10]# kubectl delete dr recommendation-dr -n weather
destinationrule.networking.istio.io "recommendation-dr" deleted

测试没有熔断策略时的效果,发起10个请求,其中-c指定并发数,-qps定义每秒查询数,如果为0表示不限制,-n指定请求总数:

# kubectl -n weather exec -it `kubectl get po -n weather |grep fortio|awk '{print $1}'` -- /usr/bin/fortio load -c 10 -qps 0 -n 10 http://forecast:3002/weather?locate=hangzhou
 [root@master-1-230 11.10]# kubectl -n weather exec -it `kubectl get po -n weather |grep fortio|awk '{print $1}'` -- /usr/bin/fortio load -c 10 -qps 0 -n 10  http://forecast:3002/weather?locate=hangzhou
14:06:28.013 r1 [INF] scli.go:123> Starting, command="Φορτίο", version="1.60.3 h1:adR0uf/69M5xxKaMLAautVf9FIVkEpMwuEWyMaaSnI0= go1.20.10 amd64 linux"
Fortio 1.60.3 running at 0 queries per second, 2->2 procs, for 10 calls: http://forecast:3002/weather?locate=hangzhou
14:06:28.013 r1 [INF] httprunner.go:121> Starting http test, run=0, url="http://forecast:3002/weather?locate=hangzhou", threads=10, qps="-1.0", warmup="parallel", conn-reuse=""
Starting at max qps with 10 thread(s) [gomax 2] for exactly 10 calls (1 per thread + 0)

14:06:31.022 r35 [ERR] http_client.go:1084> Read error, err="read tcp 10.244.29.34:59130->10.111.255.74:3002: i/o timeout", size=0, dest="10.111.255.74:3002", url="http://forecast:3002/weather?locate=hangzhou", thread=6, run=0
14:06:31.022 r33 [ERR] http_client.go:1084> Read error, err="read tcp 10.244.29.34:59136->10.111.255.74:3002: i/o timeout", size=0, dest="10.111.255.74:3002", url="http://forecast:3002/weather?locate=hangzhou", thread=4, run=0
14:06:31.022 r35 [INF] periodic.go:850> T006 ended after 3.000835719s : 1 calls. qps=0.33324050152710144
14:06:31.022 r33 [INF] periodic.go:850> T004 ended after 3.000834982s : 1 calls. qps=0.3332405833704054
14:06:31.022 r36 [ERR] http_client.go:1084> Read error, err="read tcp 10.244.29.34:59134->10.111.255.74:3002: i/o timeout", size=0, dest="10.111.255.74:3002", url="http://forecast:3002/weather?locate=hangzhou", thread=7, run=0
14:06:31.022 r38 [ERR] http_client.go:1084> Read error, err="read tcp 10.244.29.34:59122->10.111.255.74:3002: i/o timeout", size=0, dest="10.111.255.74:3002", url="http://forecast:3002/weather?locate=hangzhou", thread=9, run=0
14:06:31.022 r36 [INF] periodic.go:850> T007 ended after 3.000897308s : 1 calls. qps=0.3332336622563294
14:06:31.022 r38 [INF] periodic.go:850> T009 ended after 3.000909714s : 1 calls. qps=0.3332322846418031
14:06:31.022 r16 [ERR] http_client.go:1084> Read error, err="read tcp 10.244.29.34:59132->10.111.255.74:3002: i/o timeout", size=0, dest="10.111.255.74:3002", url="http://forecast:3002/weather?locate=hangzhou", thread=3, run=0
14:06:31.022 r37 [ERR] http_client.go:1084> Read error, err="read tcp 10.244.29.34:59138->10.111.255.74:3002: i/o timeout", size=0, dest="10.111.255.74:3002", url="http://forecast:3002/weather?locate=hangzhou", thread=8, run=0
14:06:31.022 r16 [INF] periodic.go:850> T003 ended after 3.000936919s : 1 calls. qps=0.3332292637238204
14:06:31.022 r37 [INF] periodic.go:850> T008 ended after 3.000950169s : 1 calls. qps=0.33322779242723244
14:06:31.022 r13 [ERR] http_client.go:1084> Read error, err="read tcp 10.244.29.34:59124->10.111.255.74:3002: i/o timeout", size=0, dest="10.111.255.74:3002", url="http://forecast:3002/weather?locate=hangzhou", thread=0, run=0
14:06:31.022 r15 [ERR] http_client.go:1084> Read error, err="read tcp 10.244.29.34:59126->10.111.255.74:3002: i/o timeout", size=0, dest="10.111.255.74:3002", url="http://forecast:3002/weather?locate=hangzhou", thread=2, run=0
14:06:31.022 r13 [INF] periodic.go:850> T000 ended after 3.001011452s : 1 calls. qps=0.33322098765519803
14:06:31.022 r15 [INF] periodic.go:850> T002 ended after 3.001025729s : 1 calls. qps=0.33321940239853237
14:06:31.022 r14 [ERR] http_client.go:1084> Read error, err="read tcp 10.244.29.34:59120->10.111.255.74:3002: i/o timeout", size=0, dest="10.111.255.74:3002", url="http://forecast:3002/weather?locate=hangzhou", thread=1, run=0
14:06:31.022 r34 [ERR] http_client.go:1084> Read error, err="read tcp 10.244.29.34:59128->10.111.255.74:3002: i/o timeout", size=0, dest="10.111.255.74:3002", url="http://forecast:3002/weather?locate=hangzhou", thread=5, run=0
14:06:31.022 r14 [INF] periodic.go:850> T001 ended after 3.001057569s : 1 calls. qps=0.333215867076224
14:06:31.022 r34 [INF] periodic.go:850> T005 ended after 3.001076852s : 1 calls. qps=0.3332137260442273
Ended after 3.001099298s : 10 calls. qps=3.3321
14:06:31.022 r1 [INF] periodic.go:581> Run ended, run=0, elapsed="3.001099298s", calls=10, qps=3.332112338523495
Aggregated Function Time : count 10 avg 3.0006142 +/- 0.0001508 min 3.000402709 max 3.000931934 sum 30.0061423
# range, mid point, percentile, count
>= 3.0004 <= 3.00093 , 3.00067 , 100.00, 10
# target 50% 3.00064
# target 75% 3.00078
# target 90% 3.00087
# target 99% 3.00093
# target 99.9% 3.00093
Error cases : count 10 avg 3.0006142 +/- 0.0001508 min 3.000402709 max 3.000931934 sum 30.0061423
# range, mid point, percentile, count
>= 3.0004 <= 3.00093 , 3.00067 , 100.00, 10
# target 50% 3.00064
# target 75% 3.00078
# target 90% 3.00087
# target 99% 3.00093
# target 99.9% 3.00093
# Socket and IP used for each connection:
[0]   1 socket used, resolved to 10.111.255.74:3002, connection timing : count 1 avg 0.000477594 +/- 0 min 0.000477594 max 0.000477594 sum 0.000477594
[1]   1 socket used, resolved to 10.111.255.74:3002, connection timing : count 1 avg 0.000865135 +/- 0 min 0.000865135 max 0.000865135 sum 0.000865135
[2]   1 socket used, resolved to 10.111.255.74:3002, connection timing : count 1 avg 0.000434009 +/- 0 min 0.000434009 max 0.000434009 sum 0.000434009
[3]   1 socket used, resolved to 10.111.255.74:3002, connection timing : count 1 avg 0.000219892 +/- 0 min 0.000219892 max 0.000219892 sum 0.000219892
[4]   1 socket used, resolved to 10.111.255.74:3002, connection timing : count 1 avg 0.000123581 +/- 0 min 0.000123581 max 0.000123581 sum 0.000123581
[5]   1 socket used, resolved to 10.111.255.74:3002, connection timing : count 1 avg 0.000362421 +/- 0 min 0.000362421 max 0.000362421 sum 0.000362421
[6]   1 socket used, resolved to 10.111.255.74:3002, connection timing : count 1 avg 0.000277783 +/- 0 min 0.000277783 max 0.000277783 sum 0.000277783
[7]   1 socket used, resolved to 10.111.255.74:3002, connection timing : count 1 avg 0.000183097 +/- 0 min 0.000183097 max 0.000183097 sum 0.000183097
[8]   1 socket used, resolved to 10.111.255.74:3002, connection timing : count 1 avg 4.6949e-05 +/- 0 min 4.6949e-05 max 4.6949e-05 sum 4.6949e-05
[9]   1 socket used, resolved to 10.111.255.74:3002, connection timing : count 1 avg 0.000558737 +/- 0 min 0.000558737 max 0.000558737 sum 0.000558737
Connection time histogram (s) : count 10 avg 0.0003549198 +/- 0.0002292 min 4.6949e-05 max 0.000865135 sum 0.003549198
# range, mid point, percentile, count
>= 4.6949e-05 <= 0.000865135 , 0.000456042 , 100.00, 10
# target 50% 0.000410587
# target 75% 0.000637861
# target 90% 0.000774225
# target 99% 0.000856044
# target 99.9% 0.000864226
Sockets used: 10 (for perfect keepalive, would be 10)
Uniform: false, Jitter: false, Catchup allowed: true
IP addresses distribution:
10.111.255.74:3002: 10
Code  -1 : 10 (100.0 %)
Response Header Sizes : count 10 avg 0 +/- 0 min 0 max 0 sum 0
Response Body/Total Sizes : count 10 avg 0 +/- 0 min 0 max 0 sum 0
All done 10 calls (plus 0 warmup) 3000.614 ms avg, 3.3 qps
配置熔断策略:
[root@master-1-230 11.10]# kubectl apply -f circuit-breaking.yaml -n weather
destinationrule.networking.istio.io/forecast-dr configured

查看策略

[root@master-1-230 11.10]# kubectl get dr forecast-dr -n weather -o yaml
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"networking.istio.io/v1alpha3","kind":"DestinationRule","metadata":{"annotations":{},"name":"forecast-dr","namespace":"weather"},"spec":{"host":"forecast","subsets":[{"labels":{"version":"v1"},"name":"v1"},{"labels":{"version":"v2"},"name":"v2"}],"trafficPolicy":{"connectionPool":{"http":{"http1MaxPendingRequests":5,"maxRequestsPerConnection":1},"tcp":{"maxConnections":3}},"outlierDetection":{"baseEjectionTime":"2m","consecutive5xxErrors":2,"interval":"10s","maxEjectionPercent":40}}}}
  creationTimestamp: "2023-11-07T15:01:37Z"
  generation: 3
  name: forecast-dr
  namespace: weather
  resourceVersion: "730921"
  uid: 4dd284d2-8c68-46bc-8e61-9698d6873b0a
spec:
  host: forecast
  subsets:
  - labels:
      version: v1
    name: v1
  - labels:
      version: v2
    name: v2
  trafficPolicy:
    connectionPool:
      http:
        http1MaxPendingRequests: 5
        maxRequestsPerConnection: 1
      tcp:
        maxConnections: 3
    outlierDetection:
      baseEjectionTime: 2m
      consecutive5xxErrors: 2
      interval: 10s
      maxEjectionPercent: 40
说明:
maxConnections: 3   同一时间最多只能有3个tcp连接,超过就要拒绝。
http1MaxPendingRequests: 5  定义http请求pending状态的最大数值为5个。服务器的承载能力有限,如果并发量很大,那么就会排队,这个参数用来设置当队列已经满了,然后最多有几个请求被挂起,这个被挂起其实就是在队列里排队了,超过这个数值,直接拒绝。
maxRequestsPerConnection: 设置每个连接最大的http请求数量为1个,就是说一个tcp连接最多只能有一个http请求,第二个就不让连了。
 
测试(并发数为10,总共进行100个请求):
# kubectl -n weather exec -it `kubectl get po -n weather |grep fortio|awk '{print $1}'` -- /usr/bin/fortio load -c 10 -qps 0 -n 100 http://forecast:3002/weather?locate=hangzhou
 [root@master-1-230 11.10]# kubectl -n weather exec -it `kubectl get po -n weather |grep fortio|awk '{print $1}'` -- /usr/bin/fortio load -c 10 -qps 0 -n 100  http://forecast:3002/weather?locate=hangzhou
14:09:29.320 r1 [INF] scli.go:123> Starting, command="Φορτίο", version="1.60.3 h1:adR0uf/69M5xxKaMLAautVf9FIVkEpMwuEWyMaaSnI0= go1.20.10 amd64 linux"
Fortio 1.60.3 running at 0 queries per second, 2->2 procs, for 100 calls: http://forecast:3002/weather?locate=hangzhou
14:09:29.321 r1 [INF] httprunner.go:121> Starting http test, run=0, url="http://forecast:3002/weather?locate=hangzhou", threads=10, qps="-1.0", warmup="parallel", conn-reuse=""
Starting at max qps with 10 thread(s) [gomax 2] for exactly 100 calls (10 per thread + 0)
14:09:29.335 r38 [WRN] http_client.go:1104> Non ok http code, code=503, status="HTTP/1.1 503", thread=1, run=0
14:09:29.335 r41 [WRN] http_client.go:1104> Non ok http code, code=503, status="HTTP/1.1 503", thread=4, run=0
14:09:29.335 r45 [WRN] http_client.go:1104> Non ok http code, code=503, status="HTTP/1.1 503", thread=8, run=0
14:09:29.335 r42 [WRN] http_client.go:1104> Non ok http code, code=503, status="HTTP/1.1 503", thread=5, run=0
14:09:29.342 r43 [WRN] http_client.go:1104> Non ok http code, code=503, status="HTTP/1.1 503", thread=6, run=0
14:09:29.346 r45 [WRN] http_client.go:1104> Non ok http code, code=503, status="HTTP/1.1 503", thread=8, run=0
14:09:30.238 r40 [WRN] http_client.go:1104> Non ok http code, code=503, status="HTTP/1.1 503", thread=3, run=0
14:09:30.238 r46 [WRN] http_client.go:1104> Non ok http code, code=503, status="HTTP/1.1 503", thread=9, run=0
14:09:30.314 r41 [WRN] http_client.go:1104> Non ok http code, code=503, status="HTTP/1.1 503", thread=4, run=0
14:09:30.315 r46 [WRN] http_client.go:1104> Non ok http code, code=503, status="HTTP/1.1 503", thread=9, run=0
14:09:30.317 r46 [WRN] http_client.go:1104> Non ok http code, code=503, status="HTTP/1.1 503", thread=9, run=0
14:09:30.429 r40 [WRN] http_client.go:1104> Non ok http code, code=503, status="HTTP/1.1 503", thread=3, run=0
14:09:31.146 r41 [WRN] http_client.go:1104> Non ok http code, code=503, status="HTTP/1.1 503", thread=4, run=0
14:09:31.158 r45 [WRN] http_client.go:1104> Non ok http code, code=503, status="HTTP/1.1 503", thread=8, run=0
14:09:31.183 r44 [WRN] http_client.go:1104> Non ok http code, code=503, status="HTTP/1.1 503", thread=7, run=0
14:09:31.185 r40 [WRN] http_client.go:1104> Non ok http code, code=503, status="HTTP/1.1 503", thread=3, run=0
14:09:31.188 r40 [WRN] http_client.go:1104> Non ok http code, code=503, status="HTTP/1.1 503", thread=3, run=0
14:09:31.263 r39 [WRN] http_client.go:1104> Non ok http code, code=503, status="HTTP/1.1 503", thread=2, run=0
14:09:31.264 r40 [INF] periodic.go:850> T003 ended after 1.935531802s : 10 calls. qps=5.1665387206073925
14:09:31.339 r39 [WRN] http_client.go:1104> Non ok http code, code=503, status="HTTP/1.1 503", thread=2, run=0
14:09:31.342 r39 [WRN] http_client.go:1104> Non ok http code, code=503, status="HTTP/1.1 503", thread=2, run=0
14:09:31.342 r39 [INF] periodic.go:850> T002 ended after 2.012844977s : 10 calls. qps=4.968092483159969
14:09:31.418 r46 [INF] periodic.go:850> T009 ended after 2.08939857s : 10 calls. qps=4.7860662602061605
14:09:31.470 r41 [INF] periodic.go:850> T004 ended after 2.140883451s : 10 calls. qps=4.670968891524212
14:09:31.544 r44 [INF] periodic.go:850> T007 ended after 2.215320319s : 10 calls. qps=4.514019897815057
14:09:31.599 r37 [INF] periodic.go:850> T000 ended after 2.269671775s : 10 calls. qps=4.405923407141105
14:09:31.621 r45 [INF] periodic.go:850> T008 ended after 2.292342549s : 10 calls. qps=4.362349773755389
14:09:31.689 r38 [INF] periodic.go:850> T001 ended after 2.359638731s : 10 calls. qps=4.2379368793298555
14:09:31.741 r42 [INF] periodic.go:850> T005 ended after 2.411801376s : 10 calls. qps=4.146278420565923
14:09:31.809 r43 [INF] periodic.go:850> T006 ended after 2.480401389s : 10 calls. qps=4.031605547532614
Ended after 2.480452985s : 100 calls. qps=40.315
14:09:31.809 r1 [INF] periodic.go:581> Run ended, run=0, elapsed="2.480452985s", calls=100, qps=40.315216859472145
Aggregated Function Time : count 100 avg 0.22203631 +/- 0.3728 min 0.000482827 max 1.703815091 sum 22.2036307
# range, mid point, percentile, count
>= 0.000482827 <= 0.001 , 0.000741413 , 4.00, 4
> 0.001 <= 0.002 , 0.0015 , 9.00, 5
> 0.002 <= 0.003 , 0.0025 , 13.00, 4
> 0.003 <= 0.004 , 0.0035 , 14.00, 1
> 0.005 <= 0.006 , 0.0055 , 17.00, 3
> 0.006 <= 0.007 , 0.0065 , 18.00, 1
> 0.011 <= 0.012 , 0.0115 , 19.00, 1
> 0.012 <= 0.014 , 0.013 , 21.00, 2
> 0.016 <= 0.018 , 0.017 , 22.00, 1
> 0.025 <= 0.03 , 0.0275 , 23.00, 1
> 0.05 <= 0.06 , 0.055 , 34.00, 11
> 0.06 <= 0.07 , 0.065 , 44.00, 10
> 0.07 <= 0.08 , 0.075 , 50.00, 6
> 0.08 <= 0.09 , 0.085 , 53.00, 3
> 0.09 <= 0.1 , 0.095 , 57.00, 4
> 0.1 <= 0.12 , 0.11 , 63.00, 6
> 0.12 <= 0.14 , 0.13 , 71.00, 8
> 0.14 <= 0.16 , 0.15 , 78.00, 7
> 0.16 <= 0.18 , 0.17 , 79.00, 1
> 0.18 <= 0.2 , 0.19 , 80.00, 1
> 0.2 <= 0.25 , 0.225 , 83.00, 3
> 0.25 <= 0.3 , 0.275 , 84.00, 1
> 0.45 <= 0.5 , 0.475 , 85.00, 1
> 0.5 <= 0.6 , 0.55 , 87.00, 2
> 0.6 <= 0.7 , 0.65 , 88.00, 1
> 0.7 <= 0.8 , 0.75 , 90.00, 2
> 0.8 <= 0.9 , 0.85 , 91.00, 1
> 0.9 <= 1 , 0.95 , 96.00, 5
> 1 <= 1.70382 , 1.35191 , 100.00, 4
# target 50% 0.08
# target 75% 0.151429
# target 90% 0.8
# target 99% 1.52786
# target 99.9% 1.68622
Error cases : count 20 avg 0.0034419284 +/- 0.003366 min 0.000482827 max 0.012673404 sum 0.068838568
# range, mid point, percentile, count
>= 0.000482827 <= 0.001 , 0.000741413 , 20.00, 4
> 0.001 <= 0.002 , 0.0015 , 45.00, 5
> 0.002 <= 0.003 , 0.0025 , 65.00, 4
> 0.003 <= 0.004 , 0.0035 , 70.00, 1
> 0.005 <= 0.006 , 0.0055 , 85.00, 3
> 0.006 <= 0.007 , 0.0065 , 90.00, 1
> 0.011 <= 0.012 , 0.0115 , 95.00, 1
> 0.012 <= 0.0126734 , 0.0123367 , 100.00, 1
# target 50% 0.00225
# target 75% 0.00533333
# target 90% 0.007
# target 99% 0.0125387
# target 99.9% 0.0126599
# Socket and IP used for each connection:
[0]   1 socket used, resolved to 10.111.255.74:3002, connection timing : count 1 avg 0.000357145 +/- 0 min 0.000357145 max 0.000357145 sum 0.000357145
[1]   2 socket used, resolved to 10.111.255.74:3002, connection timing : count 2 avg 0.000315006 +/- 6.966e-05 min 0.00024535 max 0.000384662 sum 0.000630012
[2]   3 socket used, resolved to 10.111.255.74:3002, connection timing : count 3 avg 0.00019915367 +/- 0.0001532 min 8.936e-05 max 0.000415862 sum 0.000597461
[3]   5 socket used, resolved to 10.111.255.74:3002, connection timing : count 5 avg 0.0001812922 +/- 0.0001354 min 4.6072e-05 max 0.00037281 sum 0.000906461
[4]   4 socket used, resolved to 10.111.255.74:3002, connection timing : count 4 avg 0.00019919625 +/- 0.0001088 min 7.3066e-05 max 0.000331688 sum 0.000796785
[5]   2 socket used, resolved to 10.111.255.74:3002, connection timing : count 2 avg 0.0001133065 +/- 7.015e-05 min 4.3157e-05 max 0.000183456 sum 0.000226613
[6]   2 socket used, resolved to 10.111.255.74:3002, connection timing : count 2 avg 0.0002655545 +/- 0.0001994 min 6.6189e-05 max 0.00046492 sum 0.000531109
[7]   2 socket used, resolved to 10.111.255.74:3002, connection timing : count 2 avg 0.000110885 +/- 2.451e-06 min 0.000108434 max 0.000113336 sum 0.00022177
[8]   4 socket used, resolved to 10.111.255.74:3002, connection timing : count 4 avg 0.0001507745 +/- 9.826e-05 min 4.0342e-05 max 0.000294316 sum 0.000603098
[9]   4 socket used, resolved to 10.111.255.74:3002, connection timing : count 4 avg 0.0002043145 +/- 0.0001627 min 7.8604e-05 max 0.000483016 sum 0.000817258
Connection time histogram (s) : count 29 avg 0.000196128 +/- 0.0001383 min 4.0342e-05 max 0.000483016 sum 0.005687712
# range, mid point, percentile, count
>= 4.0342e-05 <= 0.000483016 , 0.000261679 , 100.00, 29
# target 50% 0.000253774
# target 75% 0.000368395
# target 90% 0.000437168
# target 99% 0.000478431
# target 99.9% 0.000482558
Sockets used: 29 (for perfect keepalive, would be 10)
Uniform: false, Jitter: false, Catchup allowed: true
IP addresses distribution:
10.111.255.74:3002: 29
Code 200 : 80 (80.0 %)
Code 503 : 20 (20.0 %)
Response Header Sizes : count 100 avg 139.66 +/- 69.83 min 0 max 176 sum 13966
Response Body/Total Sizes : count 100 avg 719.86 +/- 239.4 min 241 max 841 sum 71986
All done 100 calls (plus 0 warmup) 222.036 ms avg, 40.3 qps
 

2)异常点检查

如果请求的目标服务一旦返回了50x的错误码,那么Istio就会将该服务隔离,即不再将请求转发到该服务上。
下面针对从forecast-v2到recommendation的请求来做此策略。
[root@master-1-230 11.11]# pwd
/root/k8s/14.4/cloud-native-istio/11_traffic-management/11.11
[root@master-1-230 11.11]# sed -i 's/replicas: 1/replicas: 6/' recommendation-all.yaml
[root@master-1-230 11.11]# kubectl apply -f forecast-v2-deployment.yaml -n weather
deployment.apps/forecast-v2 configured
[root@master-1-230 11.11]# kubectl apply -f recommendation-all.yaml -n weather
service/recommendation unchanged
deployment.apps/recommendation-v1 configured
destinationrule.networking.istio.io/recommendation-dr created
virtualservice.networking.istio.io/recommendation-route created
查看策略
[root@master-1-230 11.11]# kubectl get dr recommendation-dr -n weather -o yaml
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"networking.istio.io/v1alpha3","kind":"DestinationRule","metadata":{"annotations":{},"name":"recommendation-dr","namespace":"weather"},"spec":{"host":"recommendation","subsets":[{"labels":{"version":"v1"},"name":"v1"}]}}
  creationTimestamp: "2023-11-09T14:10:58Z"
  generation: 1
  name: recommendation-dr
  namespace: weather
  resourceVersion: "731379"
  uid: 18813486-e7cc-40ab-84a6-046ee31f488e
spec:
  host: recommendation
  subsets:
  - labels:
      version: v1
    name: v1
outlierDetection这部分配置就是异常点检测的配置。
这段配置的含义是:每10秒扫描一次recommendation服务查看访问其状态码,如果连续两次都是5xx的状态码,则要剔除50%的异常实例(比如一共有6个pod,出现异常的实例有2个,那么将会剔除1个pod)

五、流量镜像

将流量发给服务A的同时,再复制一份给服务B。
流量镜像的使用场景:
  • 监控和故障排除:通过将流量复制到专门的监控工具或服务,您可以实时监视服务的性能指标、请求和响应数据,并进行故障排除。这使您能够快速检测问题、分析错误和调查潜在的性能瓶颈。
  • 安全分析和审计:流量镜像使您能够将流量复制到安全分析工具或服务,以进行入侵检测、恶意行为分析和安全审计。通过分析复制的流量,您可以识别潜在的安全威胁、异常行为和漏洞。
  • A/B 测试和灰度发布:通过将流量复制到用于测试和评估新功能或代码的目标服务,您可以进行 A/B 测试和灰度发布。这使您能够在真实流量环境中评估新功能的性能和用户体验,而不会对主要流量产生影响。
实验目标:
设置流量镜像策略,达到如下效果:
让访问forecast-v2的请求镜像到访问forecast-v1。
 
准备工作:

1)将流量全部发往forecast-v2,而不发往forecast-v1

[root@master-1-230 10.2]# pwd
/root/k8s/14.4/cloud-native-istio/10_canary-release/10.2
[root@master-1-230 10.2]# kubectl  apply -f vs-forecast-weight-based-v2.yaml 
virtualservice.networking.istio.io/forecast-route configured
查看vs策略
[root@master-1-230 10.2]# kubectl -n weather get vs forecast-route -o yaml
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"networking.istio.io/v1alpha3","kind":"VirtualService","metadata":{"annotations":{},"name":"forecast-route","namespace":"weather"},"spec":{"hosts":["forecast"],"http":[{"route":[{"destination":{"host":"forecast","subset":"v1"},"weight":0},{"destination":{"host":"forecast","subset":"v2"},"weight":100}]}]}}
  creationTimestamp: "2023-11-07T15:01:36Z"
  generation: 7
  name: forecast-route
  namespace: weather
  resourceVersion: "733998"
  uid: a728357f-78b9-42d7-9341-679750758ddd
spec:
  hosts:
  - forecast
  http:
  - route:
    - destination:
        host: forecast
        subset: v1
      weight: 0
    - destination:
        host: forecast
        subset: v2
      weight: 100

2)在生效策略之前,我们先做个测试:

浏览器访问http://192.168.222.101:3000/dashboard192.168.1.230:3000,点击查询天气
[root@master-1-230 10.2]# nohup  kubectl port-forward  -n weather svc/frontend --address 192.168.1.230 3000:3000 &
[1] 28244
[root@master-1-230 10.2]# nohup: 忽略输入并把输出追加到"nohup.out"
[root@master-1-230 10.2]# netstat  -lnp|grep 3000
tcp        0      0 192.168.1.230:3000      0.0.0.0:*               LISTEN      28244/kubectl
查询frontend示例的istio-proxy日志
 
[root@master-1-230 10.2]# kubectl -n weather logs `kubectl -n weather get po |grep frontend-v1|awk '{print $1}'` -c istio-proxy |grep '/weather'
[2023-11-09T14:24:46.934Z] "GET /weather?locate=Hangzhou HTTP/1.1" 200 - via_upstream - "-" 0 665 28 27 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "4646ceb5-f03d-9dd2-8195-896f1c399478" "forecast:3002" "10.244.154.24:3002" outbound|3002|v2|forecast.weather.svc.cluster.local 10.244.167.139:58828 10.111.255.74:3002 10.244.167.139:34458 - -
[2023-11-09T14:24:55.852Z] "GET /weather?locate=Beijing HTTP/1.1" 200 - via_upstream - "-" 0 646 63 63 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "16062079-8a68-992b-9bb9-716e7bdba35e" "forecast:3002" "10.244.154.24:3002" outbound|3002|v2|forecast.weather.svc.cluster.local 10.244.167.139:58936 10.111.255.74:3002 10.244.167.139:34568 - -
[2023-11-09T14:24:59.372Z] "GET /weather?locate=Shanghai HTTP/1.1" 200 - via_upstream - "-" 0 682 125 124 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "a2042ddd-df6d-9d29-afc0-fec7ca5bfdd9" "forecast:3002" "10.244.154.24:3002" outbound|3002|v2|forecast.weather.svc.cluster.local 10.244.167.139:58982 10.111.255.74:3002 10.244.167.139:34614 - -
[2023-11-09T14:25:16.320Z] "GET /weather?locate=%E4%B8%8A%E6%B5%B7 HTTP/1.1" 200 - via_upstream - "-" 0 665 69 68 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "016f4062-2939-9293-b39f-47ec6bda6440" "forecast:3002" "10.244.154.24:3002" outbound|3002|v2|forecast.weather.svc.cluster.local 10.244.167.139:59194 10.111.255.74:3002 10.244.167.139:34826 - -
[2023-11-09T14:25:17.136Z] "GET /weather?locate=%E4%B8%8A%E6%B5%B7 HTTP/1.1" 200 - via_upstream - "-" 0 665 58 58 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "d10f1d59-c9ca-9d65-9b31-2b42e69a5cc4" "forecast:3002" "10.244.154.24:3002" outbound|3002|v2|forecast.weather.svc.cluster.local 10.244.167.139:59206 10.111.255.74:3002 10.244.167.139:34836 - -
[2023-11-09T14:25:17.857Z] "GET /weather?locate=%E4%B8%8A%E6%B5%B7 HTTP/1.1" 200 - via_upstream - "-" 0 665 12 12 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "e5823607-3ac9-9367-9994-49fa51329a42" "forecast:3002" "10.244.154.24:3002" outbound|3002|v2|forecast.weather.svc.cluster.local 10.244.167.139:59226 10.111.255.74:3002 10.244.167.139:34856 - -
[2023-11-09T14:25:18.307Z] "GET /weather?locate=%E4%B8%8A%E6%B5%B7 HTTP/1.1" 200 - via_upstream - "-" 0 665 44 43 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "a91ac2e7-4485-92ed-9750-04c46fb841cd" "forecast:3002" "10.244.154.24:3002" outbound|3002|v2|forecast.weather.svc.cluster.local 10.244.167.139:59240 10.111.255.74:3002 10.244.167.139:34870 - -
[2023-11-09T14:25:18.771Z] "GET /weather?locate=%E4%B8%8A%E6%B5%B7 HTTP/1.1" 200 - via_upstream - "-" 0 665 12 12 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "cb2d5c56-59bd-9966-9c25-b104f779cbd7" "forecast:3002" "10.244.154.24:3002" outbound|3002|v2|forecast.weather.svc.cluster.local 10.244.167.139:59248 10.111.255.74:3002 10.244.167.139:34880 - -
[2023-11-09T14:25:18.979Z] "GET /weather?locate=%E4%B8%8A%E6%B5%B7 HTTP/1.1" 200 - via_upstream - "-" 0 665 13 12 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "c52ef400-6b85-9151-a28a-1df0d11b854f" "forecast:3002" "10.244.154.24:3002" outbound|3002|v2|forecast.weather.svc.cluster.local 10.244.167.139:59262 10.111.255.74:3002 10.244.167.139:34894 - -
[2023-11-09T14:25:19.256Z] "GET /weather?locate=%E4%B8%8A%E6%B5%B7 HTTP/1.1" 200 - via_upstream - "-" 0 665 11 11 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "62ae2343-cd51-964d-ae87-157f1c840ef3" "forecast:3002" "10.244.154.24:3002" outbound|3002|v2|forecast.weather.svc.cluster.local 10.244.167.139:59274 10.111.255.74:3002 10.244.167.139:34906 - -
[2023-11-09T14:25:19.528Z] "GET /weather?locate=%E4%B8%8A%E6%B5%B7 HTTP/1.1" 200 - via_upstream - "-" 0 665 12 12 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "7f85d5e8-f782-915b-822c-73ad0eae7110" "forecast:3002" "10.244.154.24:3002" outbound|3002|v2|forecast.weather.svc.cluster.local 10.244.167.139:59296 10.111.255.74:3002 10.244.167.139:34926 - -
[2023-11-09T14:25:19.699Z] "GET /weather?locate=%E4%B8%8A%E6%B5%B7 HTTP/1.1" 200 - via_upstream - "-" 0 665 15 15 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "ca4a9641-579f-9518-aca2-70d133740e48" "forecast:3002" "10.244.154.24:3002" outbound|3002|v2|forecast.weather.svc.cluster.local 10.244.167.139:59304 10.111.255.74:3002 10.244.167.139:34936 - -
[2023-11-09T14:25:19.871Z] "GET /weather?locate=%E4%B8%8A%E6%B5%B7 HTTP/1.1" 200 - via_upstream - "-" 0 665 13 13 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "da0deec8-dceb-934c-9f0b-0fd8350c6672" "forecast:3002" "10.244.154.24:3002" outbound|3002|v2|forecast.weather.svc.cluster.local 10.244.167.139:59320 10.111.255.74:3002 10.244.167.139:34952 - -
[2023-11-09T14:25:20.059Z] "GET /weather?locate=%E4%B8%8A%E6%B5%B7 HTTP/1.1" 200 - via_upstream - "-" 0 665 14 13 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "595f8721-f979-9b24-a8a5-c4fec0bd4a73" "forecast:3002" "10.244.154.24:3002" outbound|3002|v2|forecast.weather.svc.cluster.local 10.244.167.139:59336 10.111.255.74:3002 10.244.167.139:34968 - -
[2023-11-09T14:25:20.281Z] "GET /weather?locate=%E4%B8%8A%E6%B5%B7 HTTP/1.1" 200 - via_upstream - "-" 0 665 13 13 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "0ddf6b60-128b-9a60-bb7d-2082dc67a5ec" "forecast:3002" "10.244.154.24:3002" outbound|3002|v2|forecast.weather.svc.cluster.local 10.244.167.139:59344 10.111.255.74:3002 10.244.167.139:34974 - -
[2023-11-09T14:25:20.497Z] "GET /weather?locate=%E4%B8%8A%E6%B5%B7 HTTP/1.1" 200 - via_upstream - "-" 0 665 13 13 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "77e022b4-2704-9e01-8cdf-28d5652d747e" "forecast:3002" "10.244.154.24:3002" outbound|3002|v2|forecast.weather.svc.cluster.local 10.244.167.139:59358 10.111.255.74:3002 10.244.167.139:34988 - -
[2023-11-09T14:25:20.822Z] "GET /weather?locate=%E4%B8%8A%E6%B5%B7 HTTP/1.1" 200 - via_upstream - "-" 0 665 12 11 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "a874c5f4-d19a-9240-938c-dd7e96a10fd7" "forecast:3002" "10.244.154.24:3002" outbound|3002|v2|forecast.weather.svc.cluster.local 10.244.167.139:59368 10.111.255.74:3002 10.244.167.139:35000 - -
[2023-11-09T14:25:20.985Z] "GET /weather?locate=%E4%B8%8A%E6%B5%B7 HTTP/1.1" 200 - via_upstream - "-" 0 665 15 15 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "af0ab9d2-3716-97ae-896d-3d5eaf7c12c9" "forecast:3002" "10.244.154.24:3002" outbound|3002|v2|forecast.weather.svc.cluster.local 10.244.167.139:59386 10.111.255.74:3002 10.244.167.139:35018 - -
[2023-11-09T14:25:21.635Z] "GET /weather?locate=%E4%B8%8A%E6%B5%B7 HTTP/1.1" 200 - via_upstream - "-" 0 665 31 31 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "ff723e8a-fc3c-9cb4-84c2-83e4e611f047" "forecast:3002" "10.244.154.24:3002" outbound|3002|v2|forecast.weather.svc.cluster.local 10.244.167.139:59402 10.111.255.74:3002 10.244.167.139:35034 - -
 发现只有 forecast-v2的日志。
 同时也可以查看forecast-v1以及forecast-v2的istio-proxy日志,可以根据时间来判定
[root@master-1-230 10.2]# kubectl -n weather logs `kubectl -n weather get po |grep forecast-v1|awk '{print $1}'` -c istio-proxy |grep '/weather'
[2023-11-09T13:42:40.913Z] "HEAD /weather?locate=hangzhou HTTP/1.1" 200 - via_upstream - "-" 0 0 11909 11909 "-" "curl/7.52.1" "84dc5087-3a92-9854-b491-0a5033eb4eac" "forecast:3002" "10.244.154.61:3002" inbound|3002|| 127.0.0.6:38103 10.244.154.61:3002 10.244.167.139:57580 outbound_.3002_.v1_.forecast.weather.svc.cluster.local default
[2023-11-09T13:50:00.799Z] "HEAD /weather?locate=hangzhou HTTP/1.1" 200 - via_upstream - "-" 0 0 405 318 "-" "curl/7.52.1" "d8f358d0-ed92-9acc-8036-75504524644c" "forecast:3002" "10.244.154.61:3002" inbound|3002|| 127.0.0.6:34143 10.244.154.61:3002 10.244.167.139:47970 outbound_.3002_.v1_.forecast.weather.svc.cluster.local default
[2023-11-09T13:52:05.992Z] "HEAD /weather?locate=hangzhou HTTP/1.1" 200 - via_upstream - "-" 0 0 6 1 "-" "curl/7.52.1" "8da52a42-fa5a-9136-a6bb-afe32717d1d5" "forecast:3002" "10.244.154.61:3002" inbound|3002|| 127.0.0.6:32996 10.244.154.61:3002 10.244.167.139:49096 outbound_.3002_.v1_.forecast.weather.svc.cluster.local default
[root@master-1-230 10.2]# 


[root@master-1-230 10.2]# kubectl -n weather logs `kubectl -n weather get po |grep forecast-v2|awk '{print $1}'` -c istio-proxy |grep '/weather'
[2023-11-09T14:25:20.286Z] "GET /weather?locate=%E4%B8%8A%E6%B5%B7 HTTP/1.1" 200 - via_upstream - "-" 0 665 10 10 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "0ddf6b60-128b-9a60-bb7d-2082dc67a5ec" "forecast:3002" "10.244.154.24:3002" inbound|3002|| 127.0.0.6:41764 10.244.154.24:3002 10.244.167.139:59344 outbound_.3002_.v2_.forecast.weather.svc.cluster.local default
[2023-11-09T14:25:20.502Z] "GET /weather?locate=%E4%B8%8A%E6%B5%B7 HTTP/1.1" 200 - via_upstream - "-" 0 665 9 9 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "77e022b4-2704-9e01-8cdf-28d5652d747e" "forecast:3002" "10.244.154.24:3002" inbound|3002|| 127.0.0.6:35566 10.244.154.24:3002 10.244.167.139:59358 outbound_.3002_.v2_.forecast.weather.svc.cluster.local default
[2023-11-09T14:25:20.827Z] "GET /weather?locate=%E4%B8%8A%E6%B5%B7 HTTP/1.1" 200 - via_upstream - "-" 0 665 9 8 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "a874c5f4-d19a-9240-938c-dd7e96a10fd7" "forecast:3002" "10.244.154.24:3002" inbound|3002|| 127.0.0.6:42376 10.244.154.24:3002 10.244.167.139:59368 outbound_.3002_.v2_.forecast.weather.svc.cluster.local default
[2023-11-09T14:25:20.991Z] "GET /weather?locate=%E4%B8%8A%E6%B5%B7 HTTP/1.1" 200 - via_upstream - "-" 0 665 11 11 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "af0ab9d2-3716-97ae-896d-3d5eaf7c12c9" "forecast:3002" "10.244.154.24:3002" inbound|3002|| 127.0.0.6:59303 10.244.154.24:3002 10.244.167.139:59386 outbound_.3002_.v2_.forecast.weather.svc.cluster.local default
[2023-11-09T14:25:21.641Z] "GET /weather?locate=%E4%B8%8A%E6%B5%B7 HTTP/1.1" 200 - via_upstream - "-" 0 665 26 26 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "ff723e8a-fc3c-9cb4-84c2-83e4e611f047" "forecast:3002" "10.244.154.24:3002" inbound|3002|| 127.0.0.6:45165 10.244.154.24:3002 10.244.167.139:59402 outbound_.3002_.v2_.forecast.weather.svc.cluster.local default
[root@master-1-230 10.2]# 

3)配置镜像策略

[root@master-1-230 11.14]# pwd
/root/k8s/14.4/cloud-native-istio/11_traffic-management/11.14
[root@master-1-230 11.14]# kubectl apply -f vs-forecast-mirroring.yaml -n weather
virtualservice.networking.istio.io/forecast-route configured

查看策略

[root@master-1-230 11.14]# kubectl -n weather get vs forecast-route -o yaml
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"networking.istio.io/v1alpha3","kind":"VirtualService","metadata":{"annotations":{},"name":"forecast-route","namespace":"weather"},"spec":{"hosts":["forecast"],"http":[{"mirror":{"host":"forecast","subset":"v1"},"route":[{"destination":{"host":"forecast","subset":"v2"},"weight":100}]}]}}
  creationTimestamp: "2023-11-07T15:01:36Z"
  generation: 8
  name: forecast-route
  namespace: weather
  resourceVersion: "735623"
  uid: a728357f-78b9-42d7-9341-679750758ddd
spec:
  hosts:
  - forecast
  http:
  - mirror:
      host: forecast
      subset: v1
    route:
    - destination:
        host: forecast
        subset: v2
      weight: 100
该策略,会将流量都发送到v2(weight: 100),然后会镜像一份到v1。
测试:
浏览器继续访问http://192.168.1.230:3000/dashboard,点击查询天气

查询frontend示例的istio-proxy日志

[root@master-1-230 11.14]# kubectl -n weather logs --tail 10  `kubectl -n weather get po |grep frontend-v1|awk '{print $1}'` -c istio-proxy |grep '/weather'
[2023-11-09T14:30:47.577Z] "GET /weather?locate=%E6%9D%AD%E5%B7%9E HTTP/1.1" 200 - via_upstream - "-" 0 665 41 41 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "353dda34-aad1-9df7-8999-632444dc0537" "forecast:3002" "10.244.154.24:3002" outbound|3002|v2|forecast.weather.svc.cluster.local 10.244.167.139:34948 10.111.255.74:3002 10.244.167.139:38812 - -
[2023-11-09T14:30:48.542Z] "GET /weather?locate=%E6%9D%AD%E5%B7%9E HTTP/1.1" 200 - via_upstream - "-" 0 665 25 24 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "cc90fbc0-0c33-9347-aca1-56fc810df2a6" "forecast:3002" "10.244.154.24:3002" outbound|3002|v2|forecast.weather.svc.cluster.local 10.244.167.139:34970 10.111.255.74:3002 10.244.167.139:38834 - -
[2023-11-09T14:30:49.262Z] "GET /weather?locate=%E6%9D%AD%E5%B7%9E HTTP/1.1" 200 - via_upstream - "-" 0 665 87 86 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "703ef8c9-86e9-9246-9f95-961a17677f07" "forecast:3002" "10.244.154.24:3002" outbound|3002|v2|forecast.weather.svc.cluster.local 10.244.167.139:34992 10.111.255.74:3002 10.244.167.139:38856 - -
[2023-11-09T14:30:49.911Z] "GET /weather?locate=%E6%9D%AD%E5%B7%9E HTTP/1.1" 200 - via_upstream - "-" 0 665 14 14 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "56574593-2499-9bf1-a649-aac18efcb2f9" "forecast:3002" "10.244.154.24:3002" outbound|3002|v2|forecast.weather.svc.cluster.local 10.244.167.139:35016 10.111.255.74:3002 10.244.167.139:38880 - -
[2023-11-09T14:30:50.558Z] "GET /weather?locate=%E6%9D%AD%E5%B7%9E HTTP/1.1" 200 - via_upstream - "-" 0 665 12 12 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "87742b95-7eec-95e5-b58b-52ab10eea189" "forecast:3002" "10.244.154.24:3002" outbound|3002|v2|forecast.weather.svc.cluster.local 10.244.167.139:35034 10.111.255.74:3002 10.244.167.139:38898 - -
只有v2的日志,但是再去查看两个forecast-v1以及forecast-v2的istio-proxy日志,会发现两个版本里都有访问日志
[root@master-1-230 11.14]# kubectl -n weather logs --tail 10  `kubectl -n weather get po |grep forecast-v1|awk '{print $1}'` -c istio-proxy |grep '/weather'
[2023-11-09T14:30:46.634Z] "GET /weather?locate=%E6%9D%AD%E5%B7%9E HTTP/1.1" 200 - via_upstream - "-" 0 25 1 0 "10.244.167.139" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "237c54a6-4d96-99d7-8f8d-395e8df87649" "forecast-shadow:3002" "10.244.154.61:3002" inbound|3002|| 127.0.0.6:52742 10.244.154.61:3002 10.244.167.139:0 outbound_.3002_.v1_.forecast.weather.svc.cluster.local default
[2023-11-09T14:30:46.827Z] "GET /weather?locate=%E6%9D%AD%E5%B7%9E HTTP/1.1" 200 - via_upstream - "-" 0 25 1 0 "10.244.167.139" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "e65b72d6-a5f3-9aba-ad0b-de5dfba9a1f4" "forecast-shadow:3002" "10.244.154.61:3002" inbound|3002|| 127.0.0.6:38057 10.244.154.61:3002 10.244.167.139:0 outbound_.3002_.v1_.forecast.weather.svc.cluster.local default
[2023-11-09T14:30:47.011Z] "GET /weather?locate=%E6%9D%AD%E5%B7%9E HTTP/1.1" 200 - via_upstream - "-" 0 25 2 2 "10.244.167.139" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "98d0385f-9334-9645-b872-ffe9544eb883" "forecast-shadow:3002" "10.244.154.61:3002" inbound|3002|| 127.0.0.6:54292 10.244.154.61:3002 10.244.167.139:0 outbound_.3002_.v1_.forecast.weather.svc.cluster.local default
[2023-11-09T14:30:47.213Z] "GET /weather?locate=%E6%9D%AD%E5%B7%9E HTTP/1.1" 200 - via_upstream - "-" 0 25 2 1 "10.244.167.139" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "857c0f6c-0fd6-989c-9cd5-4e02f72e767e" "forecast-shadow:3002" "10.244.154.61:3002" inbound|3002|| 127.0.0.6:46949 10.244.154.61:3002 10.244.167.139:0 outbound_.3002_.v1_.forecast.weather.svc.cluster.local default
[2023-11-09T14:30:47.437Z] "GET /weather?locate=%E6%9D%AD%E5%B7%9E HTTP/1.1" 200 - via_upstream - "-" 0 25 1 0 "10.244.167.139" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "821a3ae6-9206-9bf9-af23-eb4ca866af85" "forecast-shadow:3002" "10.244.154.61:3002" inbound|3002|| 127.0.0.6:53880 10.244.154.61:3002 10.244.167.139:0 outbound_.3002_.v1_.forecast.weather.svc.cluster.local default
[2023-11-09T14:30:47.582Z] "GET /weather?locate=%E6%9D%AD%E5%B7%9E HTTP/1.1" 200 - via_upstream - "-" 0 25 1 0 "10.244.167.139" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "353dda34-aad1-9df7-8999-632444dc0537" "forecast-shadow:3002" "10.244.154.61:3002" inbound|3002|| 127.0.0.6:34783 10.244.154.61:3002 10.244.167.139:0 outbound_.3002_.v1_.forecast.weather.svc.cluster.local default
[2023-11-09T14:30:48.548Z] "GET /weather?locate=%E6%9D%AD%E5%B7%9E HTTP/1.1" 200 - via_upstream - "-" 0 25 2 2 "10.244.167.139" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "cc90fbc0-0c33-9347-aca1-56fc810df2a6" "forecast-shadow:3002" "10.244.154.61:3002" inbound|3002|| 127.0.0.6:54872 10.244.154.61:3002 10.244.167.139:0 outbound_.3002_.v1_.forecast.weather.svc.cluster.local default
[2023-11-09T14:30:49.314Z] "GET /weather?locate=%E6%9D%AD%E5%B7%9E HTTP/1.1" 200 - via_upstream - "-" 0 25 1 0 "10.244.167.139" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "703ef8c9-86e9-9246-9f95-961a17677f07" "forecast-shadow:3002" "10.244.154.61:3002" inbound|3002|| 127.0.0.6:52370 10.244.154.61:3002 10.244.167.139:0 outbound_.3002_.v1_.forecast.weather.svc.cluster.local default
[2023-11-09T14:30:49.916Z] "GET /weather?locate=%E6%9D%AD%E5%B7%9E HTTP/1.1" 200 - via_upstream - "-" 0 25 1 0 "10.244.167.139" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "56574593-2499-9bf1-a649-aac18efcb2f9" "forecast-shadow:3002" "10.244.154.61:3002" inbound|3002|| 127.0.0.6:36846 10.244.154.61:3002 10.244.167.139:0 outbound_.3002_.v1_.forecast.weather.svc.cluster.local default
[2023-11-09T14:30:50.563Z] "GET /weather?locate=%E6%9D%AD%E5%B7%9E HTTP/1.1" 200 - via_upstream - "-" 0 25 1 0 "10.244.167.139" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "87742b95-7eec-95e5-b58b-52ab10eea189" "forecast-shadow:3002" "10.244.154.61:3002" inbound|3002|| 127.0.0.6:36866 10.244.154.61:3002 10.244.167.139:0 outbound_.3002_.v1_.forecast.weather.svc.cluster.local default
[root@master-1-230 11.14]# 


kubectl -n weather logs  --tail 10 `kubectl -n weather get po |grep forecast-v2|awk '{print $1}'` -c istio-proxy |grep '/weather'
[root@master-1-230 11.14]# kubectl -n weather logs  --tail 10 `kubectl -n weather get po |grep forecast-v2|awk '{print $1}'` -c istio-proxy |grep '/weather'
[2023-11-09T14:30:49.314Z] "GET /weather?locate=%E6%9D%AD%E5%B7%9E HTTP/1.1" 200 - via_upstream - "-" 0 665 37 36 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "703ef8c9-86e9-9246-9f95-961a17677f07" "forecast:3002" "10.244.154.24:3002" inbound|3002|| 127.0.0.6:37945 10.244.154.24:3002 10.244.167.139:34992 outbound_.3002_.v2_.forecast.weather.svc.cluster.local default
[2023-11-09T14:30:49.916Z] "GET /weather?locate=%E6%9D%AD%E5%B7%9E HTTP/1.1" 200 - via_upstream - "-" 0 665 11 11 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "56574593-2499-9bf1-a649-aac18efcb2f9" "forecast:3002" "10.244.154.24:3002" inbound|3002|| 127.0.0.6:40986 10.244.154.24:3002 10.244.167.139:35016 outbound_.3002_.v2_.forecast.weather.svc.cluster.local default
[2023-11-09T14:30:50.563Z] "GET /weather?locate=%E6%9D%AD%E5%B7%9E HTTP/1.1" 200 - via_upstream - "-" 0 665 9 9 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "87742b95-7eec-95e5-b58b-52ab10eea189" "forecast:3002" "10.244.154.24:3002" inbound|3002|| 127.0.0.6:35427 10.244.154.24:3002 10.244.167.139:35034 outbound_.3002_.v2_.forecast.weather.svc.cluster.local default