1.查看集群情况，发现db大小不一样，且差距很大
[root@localhost kube_etcd]# /home/s/bin/etcdctl --endpoints=https://11.0.1.149:2379,https://11.0.1.150:2379,https://11.0.1.151:2379 --cacert=/home/s/cert/kube_etcd/ca.pem --cert=/home/s/cert/kube_etcd/client.pem --key=/home/s/cert/kube_etcd/client-key.pem endpoint status --write-out="table" +-------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+ | ENDPOINT | ID | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS | +-------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+ | https://11.0.1.149:2379 | 6e45a7efe4f43e72 | 3.5.0 | 23 MB | false | false | 2 | 11 | 11 | | | https://11.0.1.150:2379 | d0c21986d57fb7ce | 3.5.0 | 398 MB | true | false | 2 | 11 | 11 | | | https://11.0.1.151:2379 | 83b35c880ec44a22 | 3.5.0 | 22 MB | false | false | 2 | 11 | 11 | | +-------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
2.分析差异数据情况
通过排查，文档内容对比，发现是schedule 、job、events、pod 数据。 应该属于脏数据了
[root@localhost kube_etcd]# /home/s/bin/etcdctl --endpoints=https://11.0.1.149:2379 --cacert=/home/s/cert/kube_etcd/ca.pem --cert=/home/s/cert/kube_etcd/client.pem --key=/home/s/cert/kube_etcd/client-key.pem get / --prefix --keys-only > 149.txt [root@localhost kube_etcd]# /home/s/bin/etcdctl --endpoints=https://11.0.1.150:2379 --cacert=/home/s/cert/kube_etcd/ca.pem --cert=/home/s/cert/kube_etcd/client.pem --key=/home/s/cert/kube_etcd/client-key.pem get / --prefix --keys-only > 150.txt [root@localhost kube_etcd]# cat 150.txt | wc -l 362726 [root@localhost kube_etcd]# cat 149.txt | wc -l 3124 [root@localhost kube_etcd]# cat 150.txt | grep "job|scheduler" | wc -l 0 [root@localhost kube_etcd]# cat 150.txt | grep "job" | wc -l 89725 [root@localhost kube_etcd]# cat 150.txt | grep "scheduler" | wc -l 179779 [root@localhost kube_etcd]# cat 149.txt | grep job | wc -l 11 [root@localhost kube_etcd]# cat 149.txt | grep scheduler | wc -l 321 [root@localhost kube_etcd]# cat 150.txt | grep -v "scheduler" | grep -v job | grep -v ^$ | wc -l 1576 [root@localhost kube_etcd]# cat 149.txt | grep -v "scheduler" | grep -v job | grep -v ^$ | wc -l 1233
[root@localhost kube_etcd]# cat 150.txt | grep -v "scheduler" | grep -v job | grep -v events | grep -v pod | grep -v ^$ | wc -l 781 [root@localhost kube_etcd]# cat 149.txt | grep -v "scheduler" | grep -v job | grep -v events | grep -v pod | grep -v ^$ | wc -l 781
3.尝试碎片整理，发现并不能解决此问题
[root@localhost kube_etcd]# /home/s/bin/etcdctl --endpoints=https://11.0.1.149:2379,https://11.0.1.150:2379,https://11.0.1.151:2379 --cacert=/home/s/cert/kube_etcd/ca.pem --cert=/home/s/cert/kube_etcd/client.pem --key=/home/s/cert/kube_etcd/client-key.pem endpoint status --write-out="table" +-------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+ | ENDPOINT | ID | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS | +-------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+ | https://11.0.1.149:2379 | 6e45a7efe4f43e72 | 3.5.0 | 23 MB | false | false | 2 | 11 | 11 | | | https://11.0.1.150:2379 | d0c21986d57fb7ce | 3.5.0 | 398 MB | true | false | 2 | 11 | 11 | | | https://11.0.1.151:2379 | 83b35c880ec44a22 | 3.5.0 | 22 MB | false | false | 2 | 11 | 11 | | +-------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+ [root@localhost kube_etcd]# /home/s/bin/etcdctl --endpoints=https://11.0.1.149:2379,https://11.0.1.150:2379,https://11.0.1.151:2379 --cacert=/home/s/cert/kube_etcd/ca.pem --cert=/home/s/cert/kube_etcd/client.pem --key=/home/s/cert/kube_etcd/client-key.pem defrag Finished defragmenting etcd member[https://11.0.1.149:2379] {"level":"warn","ts":"2023-08-10T17:18:05.790+0800","logger":"etcd-client","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc0002d8c40/#initially=[https://11.0.1.149:2379;https://11.0.1.150:2379;https://11.0.1.151:2379]","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = context deadline exceeded"} Failed to defragment etcd member[https://11.0.1.150:2379] (context deadline exceeded) Finished defragmenting etcd member[https://11.0.1.151:2379] [root@localhost kube_etcd]# /home/s/bin/etcdctl --endpoints=https://11.0.1.149:2379,https://11.0.1.150:2379,https://11.0.1.151:2379 --cacert=/home/s/cert/kube_etcd/ca.pem --cert=/home/s/cert/kube_etcd/client.pem --key=/home/s/cert/kube_etcd/client-key.pem endpoint status --write-out="table" +-------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+ | ENDPOINT | ID | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS | +-------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+ | https://11.0.1.149:2379 | 6e45a7efe4f43e72 | 3.5.0 | 15 MB | false | false | 2 | 11 | 11 | | | https://11.0.1.150:2379 | d0c21986d57fb7ce | 3.5.0 | 375 MB | true | false | 2 | 11 | 11 | | | https://11.0.1.151:2379 | 83b35c880ec44a22 | 3.5.0 | 15 MB | false | false | 2 | 11 | 11 | |
4.移除150节点并重新加入
a、备份etcd数据（物理备份和snapshot,三台都备份）
cp -a /home/s/data/kube_etcd/ /home/bak/
etcdctl snapshot save /home/bak/etcd.db
b、停服150
systemctl stop kube_etcd
c、移除150节点
etcdctl remove name
[root@localhost kube_etcd]# /home/s/bin/etcdctl --endpoints=https://11.0.1.149:2379,https://11.0.1.150:2379,https://11.0.1.151:2379 --cacert=/home/s/cert/kube_etcd/ca.pem --cert=/home/s/cert/kube_etcd/client.pem --key=/home/s/cert/kube_etcd/client-key.pem member list 6e45a7efe4f43e72, started, 11.0.1.149, https://11.0.1.149:2380, https://11.0.1.149:2379, false 83b35c880ec44a22, started, 11.0.1.151, https://11.0.1.151:2380, https://11.0.1.151:2379, false d0c21986d57fb7ce, started, 11.0.1.150, https://11.0.1.150:2380, https://11.0.1.150:2379, false [root@localhost kube_etcd]# /home/s/bin/etcdctl --endpoints=https://11.0.1.149:2379,https://11.0.1.150:2379,https://11.0.1.151:2379 --cacert=/home/s/cert/kube_etcd/ca.pem --cert=/home/s/cert/kube_etcd/client.pem --key=/home/s/cert/kube_etcd/client-key.pem member remove d0c21986d57fb7ce Member d0c21986d57fb7ce removed from cluster af94405c9fa143f5 [root@localhost kube_etcd]# /home/s/bin/etcdctl --endpoints=https://11.0.1.149:2379,https://11.0.1.150:2379,https://11.0.1.151:2379 --cacert=/home/s/cert/kube_etcd/ca.pem --cert=/home/s/cert/kube_etcd/client.pem --key=/home/s/cert/kube_etcd/client-key.pem member list 6e45a7efe4f43e72, started, 11.0.1.149, https://11.0.1.149:2380, https://11.0.1.149:2379, false 83b35c880ec44a22, started, 11.0.1.151, https://11.0.1.151:2380, https://11.0.1.151:2379, false
d、删除150 member数据
rm -rf member
e、加入etcd集群
[root@localhost kube_etcd]# /home/s/bin/etcdctl --endpoints=https://11.0.1.149:2379,https://11.0.1.150:2379,https://11.0.1.151:2379 --cacert=/home/s/cert/kube_etcd/ca.pem --cert=/home/s/cert/kube_etcd/client.pem --key=/home/s/cert/kube_etcd/client-key.pem member add 11.0.1.150 --peer-urls=https://11.0.1.150:2380 Member 28d53d476e303d1b added to cluster af94405c9fa143f5 ETCD_NAME="11.0.1.150" ETCD_INITIAL_CLUSTER="11.0.1.150=https://11.0.1.150:2380,11.0.1.149=https://11.0.1.149:2380,11.0.1.151=https://11.0.1.151:2380" ETCD_INITIAL_ADVERTISE_PEER_URLS="https://11.0.1.150:2380" ETCD_INITIAL_CLUSTER_STATE="existing"
f、启动150 etcd
注意：修改配置：kube_etcd.conf
initial-cluster-state: 'existing'
systemctl start kube_etcd
为了方便查看日志， 我是先直接 /home/s/bin/etcd --config-file /home/s/etc/kube_etcd/kube_etcd.conf 启动的
[root@localhost kube_etcd]# /home/s/bin/etcd --config-file /home/s/etc/kube_etcd/kube_etcd.conf {"level":"info","ts":"2023-08-10T17:32:24.741+0800","caller":"etcdmain/config.go:337","msg":"loaded server configuration, other configuration command line flags and environment variables will be ignored if provided","path":"/home/s/etc/kube_etcd/kube_etcd.conf"} {"level":"info","ts":"2023-08-10T17:32:24.741+0800","caller":"etcdmain/etcd.go:72","msg":"Running: ","args":["/home/s/bin/etcd","--config-file","/home/s/etc/kube_etcd/kube_etcd.conf"]} {"level":"info","ts":"2023-08-10T17:32:24.741+0800","caller":"embed/etcd.go:131","msg":"configuring peer listeners","listen-peer-urls":["https://11.0.1.150:2380"]} {"level":"info","ts":"2023-08-10T17:32:24.741+0800","caller":"embed/etcd.go:478","msg":"starting with peer TLS","tls-info":"cert = /home/s/cert/kube_etcd/peer.pem, key = /home/s/cert/kube_etcd/peer-key.pem, client-cert=, client-key=, trusted-ca = /home/s/cert/kube_etcd/ca.pem, client-cert-auth = false, crl-file = ","cipher-suites":[]} {"level":"info","ts":"2023-08-10T17:32:24.743+0800","caller":"embed/etcd.go:139","msg":"configuring client listeners","listen-client-urls":["http://localhost:2379","https://11.0.1.150:2379"]} {"level":"warn","ts":"2023-08-10T17:32:24.744+0800","caller":"embed/etcd.go:606","msg":"scheme is HTTP while key and cert files are present; ignoring key and cert files","client-url":"http://localhost:2379"} {"level":"info","ts":"2023-08-10T17:32:24.745+0800","caller":"embed/etcd.go:307","msg":"starting an etcd server","etcd-version":"3.5.0","git-sha":"946a5a6f2","go-version":"go1.16.3","go-os":"linux","go-arch":"amd64","max-cpu-set":1,"max-cpu-available":1,"member-initialized":false,"name":"11.0.1.150","data-dir":"/home/s/data/kube_etcd","wal-dir":"","wal-dir-dedicated":"","member-dir":"/home/s/data/kube_etcd/member","force-new-cluster":false,"heartbeat-interval":"300ms","election-timeout":"4.8s","initial-election-tick-advance":true,"snapshot-count":50000,"snapshot-catchup-entries":5000,"initial-advertise-peer-urls":["https://11.0.1.150:2380"],"listen-peer-urls":["https://11.0.1.150:2380"],"advertise-client-urls":["https://11.0.1.150:2379"],"listen-client-urls":["http://localhost:2379","https://11.0.1.150:2379"],"listen-metrics-urls":[],"cors":["*"],"host-whitelist":["*"],"initial-cluster":"11.0.1.149=https://11.0.1.149:2380,11.0.1.150=https://11.0.1.150:2380,11.0.1.151=https://11.0.1.151:2380","initial-cluster-state":"existing","initial-cluster-token":"etcd-cluster","quota-size-bytes":5368709120,"pre-vote":true,"initial-corrupt-check":false,"corrupt-check-time-interval":"0s","auto-compaction-mode":"periodic","auto-compaction-retention":"1h0m0s","auto-compaction-interval":"1h0m0s","discovery-url":"","discovery-proxy":"","downgrade-check-interval":"5s"} {"level":"warn","ts":1691659944.7454906,"caller":"fileutil/fileutil.go:57","msg":"check file permission","error":"directory \"/home/s/data/kube_etcd\" exist, but the permission is \"drwxr-xr-x\". The recommended permission is \"-rwx------\" to prevent possible unprivileged access to the data"} {"level":"info","ts":"2023-08-10T17:32:24.747+0800","caller":"etcdserver/backend.go:81","msg":"opened backend db","path":"/home/s/data/kube_etcd/member/snap/db","took":"1.206798ms"} {"level":"info","ts":"2023-08-10T17:32:24.770+0800","caller":"etcdserver/raft.go:448","msg":"starting local member","local-member-id":"28d53d476e303d1b","cluster-id":"af94405c9fa143f5"} {"level":"info","ts":"2023-08-10T17:32:24.770+0800","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"28d53d476e303d1b switched to configuration voters=()"} {"level":"info","ts":"2023-08-10T17:32:24.770+0800","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"28d53d476e303d1b became follower at term 0"} {"level":"info","ts":"2023-08-10T17:32:24.770+0800","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"newRaft 28d53d476e303d1b [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]"} {"level":"warn","ts":"2023-08-10T17:32:24.771+0800","caller":"auth/store.go:1220","msg":"simple token is not cryptographically signed"} {"level":"info","ts":"2023-08-10T17:32:24.772+0800","caller":"mvcc/kvstore.go:415","msg":"kvstore restored","current-rev":1} {"level":"info","ts":"2023-08-10T17:32:24.772+0800","caller":"etcdserver/quota.go:117","msg":"enabled backend quota","quota-name":"v3-applier","quota-size-bytes":5368709120,"quota-size":"5.4 GB"} {"level":"info","ts":"2023-08-10T17:32:24.773+0800","caller":"rafthttp/pipeline.go:72","msg":"started HTTP pipelining with remote peer","local-member-id":"28d53d476e303d1b","remote-peer-id":"6e45a7efe4f43e72"} {"level":"info","ts":"2023-08-10T17:32:24.773+0800","caller":"rafthttp/transport.go:286","msg":"added new remote peer","local-member-id":"28d53d476e303d1b","remote-peer-id":"6e45a7efe4f43e72","remote-peer-urls":["https://11.0.1.149:2380"]} {"level":"info","ts":"2023-08-10T17:32:24.773+0800","caller":"rafthttp/pipeline.go:72","msg":"started HTTP pipelining with remote peer","local-member-id":"28d53d476e303d1b","remote-peer-id":"83b35c880ec44a22"} {"level":"info","ts":"2023-08-10T17:32:24.773+0800","caller":"rafthttp/transport.go:286","msg":"added new remote peer","local-member-id":"28d53d476e303d1b","remote-peer-id":"83b35c880ec44a22","remote-peer-urls":["https://11.0.1.151:2380"]} {"level":"info","ts":"2023-08-10T17:32:24.773+0800","caller":"rafthttp/peer.go:133","msg":"starting remote peer","remote-peer-id":"6e45a7efe4f43e72"} {"level":"info","ts":"2023-08-10T17:32:24.773+0800","caller":"rafthttp/pipeline.go:72","msg":"started HTTP pipelining with remote peer","local-member-id":"28d53d476e303d1b","remote-peer-id":"6e45a7efe4f43e72"} {"level":"info","ts":"2023-08-10T17:32:24.777+0800","caller":"rafthttp/peer.go:137","msg":"started remote peer","remote-peer-id":"6e45a7efe4f43e72"} {"level":"info","ts":"2023-08-10T17:32:24.777+0800","caller":"rafthttp/transport.go:317","msg":"added remote peer","local-member-id":"28d53d476e303d1b","remote-peer-id":"6e45a7efe4f43e72","remote-peer-urls":["https://11.0.1.149:2380"]} {"level":"info","ts":"2023-08-10T17:32:24.778+0800","caller":"rafthttp/peer.go:133","msg":"starting remote peer","remote-peer-id":"83b35c880ec44a22"} {"level":"info","ts":"2023-08-10T17:32:24.778+0800","caller":"rafthttp/pipeline.go:72","msg":"started HTTP pipelining with remote peer","local-member-id":"28d53d476e303d1b","remote-peer-id":"83b35c880ec44a22"} {"level":"info","ts":"2023-08-10T17:32:24.780+0800","caller":"rafthttp/peer.go:137","msg":"started remote peer","remote-peer-id":"83b35c880ec44a22"} {"level":"info","ts":"2023-08-10T17:32:24.781+0800","caller":"rafthttp/transport.go:317","msg":"added remote peer","local-member-id":"28d53d476e303d1b","remote-peer-id":"83b35c880ec44a22","remote-peer-urls":["https://11.0.1.151:2380"]} {"level":"info","ts":"2023-08-10T17:32:24.781+0800","caller":"etcdserver/server.go:843","msg":"starting etcd server","local-member-id":"28d53d476e303d1b","local-server-version":"3.5.0","cluster-version":"to_be_decided"} {"level":"info","ts":"2023-08-10T17:32:24.784+0800","caller":"embed/etcd.go:687","msg":"starting with client TLS","tls-info":"cert = /home/s/cert/kube_etcd/peer.pem, key = /home/s/cert/kube_etcd/peer-key.pem, client-cert=, client-key=, trusted-ca = /home/s/cert/kube_etcd/ca.pem, client-cert-auth = false, crl-file = ","cipher-suites":[]} {"level":"warn","ts":"2023-08-10T17:32:24.784+0800","caller":"embed/etcd.go:700","msg":"Flag `enable-v2` is deprecated and will get removed in etcd 3.6."} {"level":"info","ts":"2023-08-10T17:32:24.784+0800","caller":"embed/etcd.go:580","msg":"serving peer traffic","address":"11.0.1.150:2380"} {"level":"info","ts":"2023-08-10T17:32:24.784+0800","caller":"embed/etcd.go:552","msg":"cmux::serve","address":"11.0.1.150:2380"} {"level":"info","ts":"2023-08-10T17:32:24.784+0800","caller":"rafthttp/stream.go:169","msg":"started stream writer with remote peer","local-member-id":"28d53d476e303d1b","remote-peer-id":"6e45a7efe4f43e72"} {"level":"info","ts":"2023-08-10T17:32:24.784+0800","caller":"rafthttp/stream.go:169","msg":"started stream writer with remote peer","local-member-id":"28d53d476e303d1b","remote-peer-id":"6e45a7efe4f43e72"} {"level":"info","ts":"2023-08-10T17:32:24.784+0800","caller":"embed/etcd.go:276","msg":"now serving peer/client/metrics","local-member-id":"28d53d476e303d1b","initial-advertise-peer-urls":["https://11.0.1.150:2380"],"listen-peer-urls":["https://11.0.1.150:2380"],"advertise-client-urls":["https://11.0.1.150:2379"],"listen-client-urls":["http://localhost:2379","https://11.0.1.150:2379"],"listen-metrics-urls":[]} {"level":"info","ts":"2023-08-10T17:32:24.784+0800","caller":"rafthttp/stream.go:395","msg":"started stream reader with remote peer","stream-reader-type":"stream MsgApp v2","local-member-id":"28d53d476e303d1b","remote-peer-id":"6e45a7efe4f43e72"} {"level":"info","ts":"2023-08-10T17:32:24.784+0800","caller":"rafthttp/stream.go:395","msg":"started stream reader with remote peer","stream-reader-type":"stream Message","local-member-id":"28d53d476e303d1b","remote-peer-id":"6e45a7efe4f43e72"} {"level":"info","ts":"2023-08-10T17:32:24.788+0800","caller":"rafthttp/stream.go:169","msg":"started stream writer with remote peer","local-member-id":"28d53d476e303d1b","remote-peer-id":"83b35c880ec44a22"} {"level":"info","ts":"2023-08-10T17:32:24.788+0800","caller":"rafthttp/stream.go:169","msg":"started stream writer with remote peer","local-member-id":"28d53d476e303d1b","remote-peer-id":"83b35c880ec44a22"} {"level":"info","ts":"2023-08-10T17:32:24.788+0800","caller":"rafthttp/stream.go:395","msg":"started stream reader with remote peer","stream-reader-type":"stream MsgApp v2","local-member-id":"28d53d476e303d1b","remote-peer-id":"83b35c880ec44a22"} {"level":"info","ts":"2023-08-10T17:32:24.788+0800","caller":"rafthttp/stream.go:395","msg":"started stream reader with remote peer","stream-reader-type":"stream Message","local-member-id":"28d53d476e303d1b","remote-peer-id":"83b35c880ec44a22"} {"level":"info","ts":"2023-08-10T17:32:24.789+0800","caller":"etcdserver/server.go:744","msg":"starting initial election tick advance","election-ticks":16} {"level":"info","ts":"2023-08-10T17:32:24.790+0800","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"28d53d476e303d1b [term: 0] received a MsgHeartbeat message with higher term from 83b35c880ec44a22 [term: 3]"} {"level":"info","ts":"2023-08-10T17:32:24.790+0800","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"28d53d476e303d1b became follower at term 3"} {"level":"info","ts":"2023-08-10T17:32:24.790+0800","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"raft.node: 28d53d476e303d1b elected leader 83b35c880ec44a22 at term 3"} {"level":"info","ts":"2023-08-10T17:32:24.809+0800","caller":"rafthttp/stream.go:249","msg":"set message encoder","from":"28d53d476e303d1b","to":"6e45a7efe4f43e72","stream-type":"stream MsgApp v2"} {"level":"info","ts":"2023-08-10T17:32:24.809+0800","caller":"rafthttp/peer_status.go:53","msg":"peer became active","peer-id":"6e45a7efe4f43e72"} {"level":"info","ts":"2023-08-10T17:32:24.809+0800","caller":"rafthttp/stream.go:274","msg":"established TCP streaming connection with remote peer","stream-writer-type":"stream MsgApp v2","local-member-id":"28d53d476e303d1b","remote-peer-id":"6e45a7efe4f43e72"} {"level":"info","ts":"2023-08-10T17:32:24.809+0800","caller":"rafthttp/stream.go:249","msg":"set message encoder","from":"28d53d476e303d1b","to":"6e45a7efe4f43e72","stream-type":"stream Message"} {"level":"info","ts":"2023-08-10T17:32:24.809+0800","caller":"rafthttp/stream.go:274","msg":"established TCP streaming connection with remote peer","stream-writer-type":"stream Message","local-member-id":"28d53d476e303d1b","remote-peer-id":"6e45a7efe4f43e72"} {"level":"info","ts":"2023-08-10T17:32:24.816+0800","caller":"rafthttp/stream.go:249","msg":"set message encoder","from":"28d53d476e303d1b","to":"83b35c880ec44a22","stream-type":"stream MsgApp v2"} {"level":"info","ts":"2023-08-10T17:32:24.816+0800","caller":"rafthttp/peer_status.go:53","msg":"peer became active","peer-id":"83b35c880ec44a22"} {"level":"info","ts":"2023-08-10T17:32:24.816+0800","caller":"rafthttp/stream.go:274","msg":"established TCP streaming connection with remote peer","stream-writer-type":"stream MsgApp v2","local-member-id":"28d53d476e303d1b","remote-peer-id":"83b35c880ec44a22"} {"level":"info","ts":"2023-08-10T17:32:24.816+0800","caller":"rafthttp/stream.go:249","msg":"set message encoder","from":"28d53d476e303d1b","to":"83b35c880ec44a22","stream-type":"stream Message"} {"level":"info","ts":"2023-08-10T17:32:24.816+0800","caller":"rafthttp/stream.go:274","msg":"established TCP streaming connection with remote peer","stream-writer-type":"stream Message","local-member-id":"28d53d476e303d1b","remote-peer-id":"83b35c880ec44a22"} {"level":"info","ts":"2023-08-10T17:32:24.823+0800","caller":"rafthttp/stream.go:412","msg":"established TCP streaming connection with remote peer","stream-reader-type":"stream MsgApp v2","local-member-id":"28d53d476e303d1b","remote-peer-id":"83b35c880ec44a22"}
g、查看集群情况
[root@localhost kube_etcd]# /home/s/bin/etcdctl --endpoints=https://11.0.1.149:2379,https://11.0.1.150:2379,https://11.0.1.151:2379 --cacert=/home/s/cert/kube_etcd/ca.pem --cert=/home/s/cert/kube_etcd/client.pem --key=/home/s/cert/kube_etcd/client-key.pem endpoint status --write-out="table" +-------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+ | ENDPOINT | ID | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS | +-------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+ | https://11.0.1.149:2379 | 6e45a7efe4f43e72 | 3.5.0 | 15 MB | false | false | 3 | 22 | 22 | | | https://11.0.1.150:2379 | 28d53d476e303d1b | 3.5.0 | 16 MB | false | false | 3 | 22 | 22 | | | https://11.0.1.151:2379 | 83b35c880ec44a22 | 3.5.0 | 16 MB | true | false | 3 | 22 | 22 | | +-------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+ [root@localhost kube_etcd]# /home/s/bin/etcdctl --endpoints=https://11.0.1.149:2379,https://11.0.1.150:2379,https://11.0.1.151:2379 --cacert=/home/s/cert/kube_etcd/ca.pem --cert=/home/s/cert/kube_etcd/client.pem --key=/home/s/cert/kube_etcd/client-key.pem member list 28d53d476e303d1b, started, 11.0.1.150, https://11.0.1.150:2380, https://11.0.1.150:2379, false 6e45a7efe4f43e72, started, 11.0.1.149, https://11.0.1.149:2380, https://11.0.1.149:2379, false 83b35c880ec44a22, started, 11.0.1.151, https://11.0.1.151:2380, https://11.0.1.151:2379, false