kubectl scale 对运行在k8s 环境中的pod 数量进行扩容(增加)或缩容(减小)。 HPA:（Horizontal Pod Autoscaler）Pod自动弹性伸缩，K8S通过对Pod中运行的容器各项指标（CPU占用、内存占用、网络请求量）的检测，实现对Pod实例个数的动态新增和减少。 一、手动调整Pod数量的方式： 1.改yaml文件改replicas数量 2.在dashboard改deployment的pod值 3.通过kubectl scale命令(临时): kubectl scale deployment linux39-tomcat-app1-deployment --replicas=3 -n linux39 kubectl delete hpa xx -n xx 4.通过kubectl edit命令(临时)： kubectl edit deployment linux39-tomcat-app1-deployment -n linux39 实例： [root@localhost7C ]# kubectl run net-test1 --image=alpine --replicas=4 sleep 360000 [root@localhost7C ]# kubectl scale deployment --replicas=3 net-test1 -n default [root@localhost7C ]#kubectl get deployment -n default net-test1 NAME READY UP-TO-DATE AVAILABLE AGE net-test1 3/3 3 3 57m #scale不支持下面三个命令查看，autoscale支持。 [root@localhost7C ]# kubectl get hpa net-test1 -n default [root@localhost7C ]# kubectl describe hpa net-test1 -n default [root@localhost7C ]# kubectl delete hpa net-test1 -n default 二、HPA自动伸缩 1.HPA控制器简介及实现 k8s从1.1版本开始增加了名称为HPA(Horizontal Pod Autoscaler)的控制器，用于实现基于pod中资源(CPU/Memory)利用率进行对pod的自动扩缩容功能的实现， 早期的版本只能基于Heapster组件实现对CPU利用率做为触发条件，但是在k8s 1.11版本开始使用Metrices Server完成数据采集， 然后将采集到的数据通过API（Aggregated API，汇总API），例如metrics.k8s.io、custom.metrics.k8s.io、external.metrics.k8s.io， 然后再把数据提供给HPA控制器进行查询，以实现基于某个资源利用率对pod进行扩缩容的目的。 控制管理器默认每隔15s（可以通过–horizontal-pod-autoscaler-sync-period修改）查询metrics的资源使用情况 支持以下三种metrics指标类型： ->预定义metrics（比如Pod的CPU）以利用率的方式计算 ->自定义的Pod metrics，以原始值（raw value）的方式计算 ->自定义的object metrics 支持两种metrics查询方式： ->Heapster ->自定义的REST API 支持多metrics 1.通过命令配置扩缩容 kubectl autoscale deployment linux39-tomcat-app1-deployment --min=2 --max=5 --cpu-percent=50 -n linux39 2.yaml文件中定义扩缩容(kind: HorizontalPodAutoscaler),配置说明： apiVersion: autoscaling/v2beta1 #定义API版本 kind: HorizontalPodAutoscaler #对象类型 metadata: #定义对象元数据 namespace: linux36 #创建后隶属的namespace name: linux36-tomcat-app1-podautoscaler #hpa名称 labels: #label标签 app: linux36-tomcat-app1 #hpa的label名称 version: v2beta1 #hpa的api版本 spec: #定义对象具体信息 scaleTargetRef: #定义水平伸缩的目标对象，Deployment、ReplicationController/ReplicaSet apiVersion: apps/v1 #API版本，HorizontalPodAutoscaler.spec.scaleTargetRef.apiVersion kind: Deployment #目标对象类型为deployment(重点) name: linux36-tomcat-app1-deployment #deployment 的具体名称(重点) minReplicas: 2 #最小pod数 maxReplicas: 5 #最大pod数 targetCPUUtilizationPercentage: 30 #设置CPU使用率警戒线百分比，指定对应pod的cpu资源使用率达到30%就触发hpa。 #metrics: #调用metrics数据定义 #- type: Resource #类型为资源 # resource: #定义资源 # name: cpu #资源名称为cpu # targetAverageUtilization: 80 #CPU使用率 #- type: Resource #类型为资源 # resource: #定义资源 # name: memory #资源名称为memory # targetAverageValue: 1024Mi #memory使用率 经验说明： 如果HPA的最小值高于业务yaml文件replicas的值,以hpa中的min的值为准 如果HPA的最小值低于业务yaml文件replicas的值,以hpa中的min的值为准 无状态应用可以hpa,有状态的应用不使用hpa HPA自动伸缩实例： 1.clone代码: git clone https://github.com/kubernetes-incubator/metrics-server.git cd metrics-server/ 2.：准备image： 测试系统自带的指标数据： curl http://localhost:8080/apis/metrics.k8s.io/v1beta1/nodes curl http://localhost:8080/apis/metrics.k8s.io/v1beta1/pods 3.测试指标数据： # kubectl top # kubectl top nodes #报错如下 Error from server (NotFound): the server could not find the requested resource (get services http:heapster:) 解决方案： docker pull k8s.gcr.io/metrics-server-amd64:v0.3.5 #google镜像仓库 docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/metrics-server-amd64:v0.3.5 #阿里云镜像仓库 或者 docker tag k8s.gcr.io/metrics-server-amd64:v0.3.5 harbor.zzhz.com/baseimages/metrics-server-amd64:v0.3.5 docker push harbor.zzhz.com/baseimages/metrics-server-amd64:v0.3.5 4.(可选)修改controller-manager启动参数，重启controller-manager [root@localhost7C ~]# kube-controller-manager --help | grep horizontal-pod-autoscaler [root@localhost7C ~]# vim /etc/systemd/system/kube-controller-manager.service [Unit] Description=Kubernetes Controller Manager Documentation=https://github.com/GoogleCloudPlatform/kubernetes [Service] ExecStart=/usr/bin/kube-controller-manager \ --address=127.0.0.1 \ --allocate-node-cidrs=true \ --cluster-cidr=10.20.0.0/16 \ --cluster-name=kubernetes \ --cluster-signing-cert-file=/etc/kubernetes/ssl/ca.pem \ --cluster-signing-key-file=/etc/kubernetes/ssl/ca-key.pem \ --kubeconfig=/etc/kubernetes/kube-controller-manager.kubeconfig \ --leader-elect=true \ --node-cidr-mask-size=24 \ --root-ca-file=/etc/kubernetes/ssl/ca.pem \ --service-account-private-key-file=/etc/kubernetes/ssl/ca-key.pem \ --service-cluster-ip-range=10.10.0.0/16 \ --use-service-account-credentials=true \ #是否使用其他客户端数据 --horizontal-pod-autoscaler-sync-period=10s \ #可选项目，定义数据采集周期间隔时间 --v=2 Restart=always RestartSec=5 [Install] WantedBy=multi-user.target 5.修改yaml文件 cd metrics-server/deploy/kubernetes [root@localhost7C ~]# ll k8s/metrics/ aggregated-metrics-reader.yaml auth-delegator.yaml auth-reader.yaml metrics-apiservice.yaml metrics-server-deployment.yaml metrics-server-service.yaml resource-reader.yaml [root@localhost7C ]# vim metrics-server-deployment.yaml --- apiVersion: v1 kind: ServiceAccount metadata: name: metrics-server namespace: kube-system --- apiVersion: apps/v1 kind: Deployment metadata: name: metrics-server namespace: kube-system labels: k8s-app: metrics-server spec: selector: matchLabels: k8s-app: metrics-server template: metadata: name: metrics-server labels: k8s-app: metrics-server spec: serviceAccountName: metrics-server volumes: # mount in tmp so we can safely use from-scratch images and/or read-only containers - name: tmp-dir emptyDir: {} containers: - name: metrics-server image: registry.cn-hangzhou.aliyuncs.com/google_containers/metrics-server-amd64:v0.3.6 #镜像 imagePullPolicy: IfNotPresent args: - --cert-dir=/tmp - --secure-port=4443 ports: - name: main-port containerPort: 4443 protocol: TCP securityContext: readOnlyRootFilesystem: true runAsNonRoot: true runAsUser: 1000 volumeMounts: - name: tmp-dir mountPath: /tmp nodeSelector: kubernetes.io/os: linux kubernetes.io/arch: "amd64" 6.部署和查看资源使用情况 [root@localhost7C kubernetes]# kubectl apply -f ./ [root@localhost7C kubernetes]# kubectl top node NAME CPU(cores) CPU% MEMORY(bytes) MEMORY% 192.168.80.120 107m 2% 1140Mi 101% 192.168.80.130 137m 3% 1098Mi 98% 192.168.80.140 108m 2% 1200Mi 84% 192.168.80.150 69m 1% 816Mi 64% 192.168.80.160 65m 1% 844Mi 53% 192.168.80.170 59m 1% 752Mi 43% [root@localhost7C kubernetes]# kubectl top pod -A NAMESPACE NAME CPU(cores) MEMORY(bytes) kube-system kube-dns-69979c4b84-2h6d2 3m 31Mi kube-system kube-flannel-ds-amd64-2262m 3m 17Mi kube-system kube-flannel-ds-amd64-69qjr 3m 15Mi kube-system kube-flannel-ds-amd64-6bsnm 1m 11Mi kube-system kube-flannel-ds-amd64-6cq5q 2m 11Mi kube-system kube-flannel-ds-amd64-ckmzs 2m 14Mi kube-system kube-flannel-ds-amd64-xddjr 3m 11Mi kube-system metrics-server-ccccb9bb6-m6pws 2m 16Mi 指标数据： curl http://localhost:8080/apis/metrics.k8s.io/v1beta1/nodes curl http://localhost:8080/apis/metrics.k8s.io/v1beta1/pods 7.创建示例 [root@localhost7C kubernetes]# cat nginx.yaml kind: Deployment apiVersion: apps/v1 metadata: labels: app: magedu-nginx-deployment-label name: magedu-nginx-deployment namespace: default spec: replicas: 3 selector: matchLabels: app: magedu-nginx-selector template: metadata: labels: app: magedu-nginx-selector spec: containers: - name: magedu-nginx-container image: harbor.zzhz.com/baseimage/nginx:latest #command: ["/apps/tomcat/bin/run_tomcat.sh"] #imagePullPolicy: IfNotPresent imagePullPolicy: Always ports: - containerPort: 80 protocol: TCP name: http resources: limits: cpu: '1' memory: 200Mi requests: cpu: '1' memory: 200Mi --- kind: Service apiVersion: v1 metadata: labels: app: magedu-nginx-service-label name: magedu-nginx-service namespace: default spec: type: NodePort ports: - name: http port: 80 protocol: TCP targetPort: 80 nodePort: 30080 selector: app: magedu-nginx-selector #方式一：通过命令配置扩缩容： #--cpu-percent=1 指定对应pod的cpu资源使用率达到50%就触发hpa #等待一会，可以看到相关的hpa信息（K8s上metrics服务收集所有pod资源的时间间隔大概在60s的时间） kubectl autoscale deployment magedu-nginx-deployment --max=5 --min=2 --cpu-percent=5 -n default 验证信息： [root@localhost7C kubernetes]# kubectl get hpa magedu-nginx-deployment -n default NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE magedu-nginx-deployment Deployment/magedu-nginx-deployment 0%/5% 2 5 3 3m47s #查看详细信息 [root@localhost7C kubernetes]# kubectl describe hpa magedu-nginx-deployment -n default Name: magedu-nginx-deployment Namespace: default Labels: <none> Annotations: <none> CreationTimestamp: Mon, 27 Mar 2023 14:20:23 +0800 Reference: Deployment/magedu-nginx-deployment Metrics: ( current / target ) resource cpu on pods (as a percentage of request): 0% (0) / 5% Min replicas: 2 Max replicas: 5 Deployment pods: 2 current / 2 desired Conditions: Type Status Reason Message ---- ------ ------ ------- AbleToScale True ReadyForNewScale recommended size matches current size ScalingActive True ValidMetricFound the HPA was able to successfully calculate a replica count from cpu resource utilization (percentage of request) ScalingLimited True TooFewReplicas the desired replica count is less than the minimum replica count Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal SuccessfulRescale 69s horizontal-pod-autoscaler New size: 2; reason: All metrics below target 验证信息：变成二个pod. [root@localhost7C kubernetes]# kubectl get hpa magedu-nginx-deployment -n default NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE magedu-nginx-deployment Deployment/magedu-nginx-deployment 0%/5% 2 5 2 7m55s #删除hpa [root@localhost7C kubernetes]# kubectl delete hpa magedu-nginx-deployment -n default #方式二：yaml文件中定义扩缩容配置(kind: HorizontalPodAutoscaler) [root@localhost7C kubernetes]# cat hpa.yaml #apiVersion: autoscaling/v2beta1 apiVersion: autoscaling/v1 kind: HorizontalPodAutoscaler metadata: namespace: default name: magedu-nginx-podautoscaler labels: app: magedu-nginx version: v2beta1 spec: scaleTargetRef: apiVersion: apps/v1 #apiVersion: extensions/v1beta1 kind: Deployment #类型 name: magedu-nginx-deployment #nginx-deployment名 minReplicas: 3 maxReplicas: 5 targetCPUUtilizationPercentage: 4 [root@localhost7C kubernetes]# kubectl apply -f hpa.yaml horizontalpodautoscaler.autoscaling/magedu-nginx-podautoscaler created [root@localhost7C kubernetes]# kubectl describe hpa magedu-nginx-podautoscaler Name: magedu-nginx-podautoscaler Namespace: default Labels: app=magedu-nginx version=v2beta1 Annotations: kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"autoscaling/v1","kind":"HorizontalPodAutoscaler","metadata":{"annotations":{},"labels":{"app":"magedu-nginx","version":"v2b... CreationTimestamp: Mon, 27 Mar 2023 14:31:43 +0800 Reference: Deployment/magedu-nginx-deployment Metrics: ( current / target ) resource cpu on pods (as a percentage of request): <unknown> / 4% Min replicas: 3 Max replicas: 5 Deployment pods: 2 current / 3 desired Conditions: Type Status Reason Message ---- ------ ------ ------- AbleToScale True SucceededRescale the HPA controller was able to update the target scale to 3 Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal SuccessfulRescale 11s horizontal-pod-autoscaler New size: 3; reason: Current number of replicas below Spec.MinReplicas [root@localhost7C kubernetes]# kubectl get hpa magedu-nginx-podautoscaler NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE magedu-nginx-podautoscaler Deployment/magedu-nginx-deployment 0%/4% 3 5 3 88s 参考文档 https://www.cnblogs.com/qiuhom-1874/p/14293237.html