526互联

k8s Kustomize 声明式管理应用

发布时间 2023-09-07 18:42:17作者: 小吉猫

kustomize 介绍

Kustomize 是一个用来定制 Kubernetes 配置的工具。它提供以下功能特性来管理应用配置文件：

1. 从其他来源生成资源
2. 为资源设置贯穿性（Cross-Cutting）字段
3. 组织和定制资源集合

查看kustomize版本

# kubectl version --short --client
Client Version: v1.25.2
Kustomize Version: v4.5.7

kustomize 资源清单

bases <[]string>                                # 列表中每个条目都应能解析为一个包含 kustomization.yaml 文件的目录。该字段在 v2.1.0 中已弃用。
resources <[]string>                            # 待定制的原始资源配置文件列表，将由Kustomize按顺序处理
namespace <string>                              # 设定所有名称空间级别资源所属的目标名称空间
commonLabels <map[string]string>                # 添加到所有资源的通用标签，包括Pod模板及相关的标签选择器
commonAnnotations <map[string]string>           # 添加到所有资源的通用注解
configMapGenerator <[]ConfigMapGeneratorArgs>   # ConfigMap资源生成器列表
- name <String>                                 # ConfigMap资源的名称，会受到namePrefix和nameSuffix的影响
  namespace <String>                            # 资源所在的名称空间，会覆盖Kustomize级别的名称空间设定
  behavior <String>                             # 与上级同名资源的合并策略，可用取值为create/replace/merge
  files <[]String>                              # 从指定的路径加载文件生成ConfigMap，要使用当前项目的相对路径
  literals <[]String>                           # 从指定的key=value格式的直接值生成ConfigMap
  env <String>                                  # 从指定的环境变量文件中加载key=value格式的环境变量作为资源数据
secretGenerator <[]secretGeneratorArgs>         # Secret资源生成器列表
- name <String>                                 # Secret资源的名称，会受到namePrefix和nameSuffix的影响
  namespace <String>                            # 资源所在的名称空间，会覆盖Kustomize级别的名称空间设定
  behavior <String>                             # 与上级同名资源的合并策略，可用取值为create/replace/merge
  files <[]String>                              # 从指定的路径加载文件生成Secret，加载起始于当前项目的相对路径
  literals <[]String>                           # 从指定的key=value格式的直接值生成Secret
  type <String>                                 # Secret资源的类型，且kubernetes.io/tls有特殊的键名要求
generatorOptions <GeneratorOptions>             # 当前kustomization.yaml中的ConfigMap和Secret生成器专用的选项
  labels <map[String]String>                    # 为当前kustomization.yaml中所有生成的资源添加的标签
  annotations <map[String]String>               # 为生成的所有资源添加的注解
  disableNameSuffixHash <Boolean>               # 是否禁用hash名称后缀，默认为启用
namePrefix <string>                             # 统一给所有资源添加的名称前缀
nameSuffix <string>                             # 统一给所有资源添加的名称后缀
images <[]Image>                                # 将所有Pod模板中符合name字段条件的镜像文件修改为指定的镜像
- name <String>                                 # 资源清单中原有的镜像名称，即待替换的镜像
  nameName <String>                             # 要使用的新镜像名称
  newTag <String>                               # 要使用的新镜像的标签
  digest <String>                               # 要使用的新镜像的SHA256校验码
  vars <[]Var>                                  # 指定可替换Pod容器参数中变量的值或容器环境变量的值
- name <String>                                 # 变量的名称，支持以$(name)格式进行引用
  objref <String>                               # 包含了要引用的目标字段的对象的名称
  fieldref <String>                             # 引用的字段名称，默认为metadata.name
patchesJson6902 <[]Json6902>                    # 由各待补对象及其补丁文件所组成的列表
  path <String>                                 # 补丁文件，不含有目标资源对象的信息，支持JSON或YAML格式
  target <Target>                               # 待补资源对象
    group <String>                              # 资源所属的群组
    version <String>                            # API版本
    kind <String>                               # 资源类型
    name <String>                               # 资源对象的名称
    namespace <string>                          # 资源对象所属的名称空间
patchesStrategicMerge <[]string>                # 将补丁补到匹配的资源之上，匹配的方式是根据资源Group/Version/Kind + Name/Namespace判断
vars <[]Var>                                    # 每个条目用来从某资源的字段来析取文字。该字段在 v5.0.0 中已弃用。
configurations <[]string>                       # 列表中每个条目都应能解析为一个包含 Kustomize 转换器配置 的文件
crds <[]string>                                 # 列表中每个条目都应能够解析为 Kubernetes 类别的 OpenAPI 定义文件
replacements:
- source:
    group: string
    version: string
    kind: string
    name: string
    namespace: string
    fieldPath: string
    options:
      delimiter: string
      index: int
      create: bool
  targets:
  - select:
      group: string
      version: string
      kind: string
      name: string
      namespace: string
    reject:
    - group: string
      version: string
      kind: string
      name: string
      namespace: string
    fieldPaths:
    - string
    options:
      delimiter: string
      index: int
      create: bool

kustomize 文件结构

base

~/someApp
├── deployment.yaml
├── kustomization.yaml
└── service.yaml

overlay

~/someApp
├── base
│   ├── deployment.yaml
│   ├── kustomization.yaml
│   └── service.yaml
└── overlays
       ├── development
       │   ├── cpu_count.yaml
       │   ├── kustomization.yaml
       │   └── replica_count.yaml
       └── production
              ├── cpu_count.yaml
              ├── kustomization.yaml
              └── replica_count.yaml

kustomize 使用示例

使用kustomize 部署demoapp。

创建项目目录

# mkdir demoapp/{base,overlays} -pv

base 配置文件

base/depoly-demoapp.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: demoapp
  name: deployment-demoapp
spec:
  replicas: 1
  selector:
    matchLabels:
      app: demoapp
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: demoapp
    spec:
      containers:
      - image: ikubernetes/demoapp:v1.0
        name: demoapp
        env:
        - name: PORT
          value: "8080"

base/service-demoapp.yaml

apiVersion: v1
kind: Service
metadata:
  labels:
    app: demoapp
  name: svc-demoapp
spec:
  ports:
  - name: http-8080
    port: 8080
    protocol: TCP
    targetPort: 8080
  selector:
    app: demoapp
  type: ClusterIP

base/ns-demoapp.yaml

apiVersion: v1
kind: Namespace
metadata:
    name: demoapp

base/kustomization.yaml

namespace: demoapp
resources:
- depoly-demoapp.yaml
- service-demoapp.yaml
- ns-demoapp.yaml

查看生成的YAML文件内容

# kubectl kustomize base/
apiVersion: v1
kind: Namespace
metadata:
  name: demoapp
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: demoapp
  name: svc-demoapp
  namespace: demoapp
spec:
  ports:
  - name: http-8080
    port: 8080
    protocol: TCP
    targetPort: 8080
  selector:
    app: demoapp
  type: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: demoapp
  name: deployment-demoapp
  namespace: demoapp
spec:
  replicas: 1
  selector:
    matchLabels:
      app: demoapp
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: demoapp
    spec:
      containers:
      - env:
        - name: PORT
          value: "8080"
        image: ikubernetes/demoapp:v1.0
        name: demoapp

创建base资源

# kubectl apply -k base/
namespace/demoapp created
service/svc-demoapp created
deployment.apps/deployment-demoapp created

查看base svc资源

# kubectl get svc -n demoapp
NAME          TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
svc-demoapp   ClusterIP   10.100.228.19   <none>        8080/TCP   65s

查看base pod资源

# kubectl get pod -n demoapp
NAME                                  READY   STATUS    RESTARTS   AGE
deployment-demoapp-69f6cf9477-588rt   1/1     Running   0          30s

删除base资源

# kubectl delete -k base/
namespace "demoapp" deleted
service "svc-demoapp" deleted
deployment.apps "deployment-demoapp" deleted

overlays配置文件

创建环境目录

# mkdir overlays/production
生成环境配置目录

overlays/production/application.properties

cat <<EOF > overlays/production/application.properties
FOO_File=Prod_Bar
EOF

overlays/production/.env

cat <<EOF > overlays/production.env
FOO_Env=Prod_Bar
EOF

overlays/production/password.txt

cat <<EOF >./password.txt
username=admin
password=secret
EOF

overlays/production/increase_replicas.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: deployment-demoapp
spec:
  replicas: 3

overlays/production/patch.yaml

- op: replace                    # 替换式补丁
  path: /spec/ports/0/targetPort # .spec.ports[0].targetPort字段
  value: 48080                   # 将上面字段的值修改为此处指定的值
- op: add                        # 添加新的配置项
  path: /spec/ports/1            # .spec.ports[1]列表项
  value:                         # 添加的内容为以下几个字段
    name: https
    protocol: TCP
    port: 443
    targetPort: 48443

overlays/production/set_configmap.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: deployment-demoapp
spec:
  template:
    spec:
      containers:
      - name: demoapp
        volumeMounts:
        - name: config
          mountPath: /config
        - name: password
          mountPath: /secrets
      volumes:
      - name: config
        configMap:
          name: example-configmap-file
      - name: password
        secret:
          secretName: example-secret-1

overlays/production/set_memory.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: deployment-demoapp
spec:
  template:
    spec:
      containers:
      - name: demoapp
        resources:
          limits:
            memory: 128Mi

overlays/production/set_env.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: deployment-demoapp
spec:
  template:
    spec:
      containers:
      - name: demoapp
        env:
        - name: DEMO_SERVICE_NAME
          value: SERVICE_NAME

overlays/production/kustomization.yaml

namespace: demoapp
commonLabels:
  app: prod-demoapp
commonAnnotations:
  oncallPager: 800-555-1212
resources:
- ../../base
configMapGenerator:
- name: example-configmap-file
  files:
  - application.properties
- name: example-configmap-env
  envs:
  - .env
- name: example-configmap-literals
  literals:
  - FOO_literals=Bar
secretGenerator:
- name: example-secret-1
  files:
  - password.txt
- name: example-secret-2
  literals:
  - username=admin
  - password=secret
generatorOptions:
  disableNameSuffixHash: true
  labels:
    type: generated
  annotations:
    note: generated
patchesStrategicMerge:
- increase_replicas.yaml
- set_memory.yaml
- set_configmap.yaml
- set_env.yaml
patchesJson6902:
- target:
    version: v1
    kind: Service
    name: svc-demoapp
  path: patch.yaml
replacements:
- source:
    kind: Service
    name: svc-demoapp
  targets:
  - select:
      kind: Deployment
      name: deployment-demoapp
    fieldPaths:
      - spec.template.spec.containers.[name=demoapp].env.[name=DEMO_SERVICE_NAME].value
namePrefix: prod-
nameSuffix: "-001"

查看overlays/production YAML 文件

# kubectl kustomize  production/
apiVersion: v1
kind: Namespace
metadata:
  annotations:
    oncallPager: 800-555-1212
  labels:
    app: prod-demoapp
  name: demoapp
---
apiVersion: v1
data:
  FOO_Env: Prod_Bar
kind: ConfigMap
metadata:
  annotations:
    note: generated
    oncallPager: 800-555-1212
  labels:
    app: prod-demoapp
    type: generated
  name: prod-example-configmap-env-001
  namespace: demoapp
---
apiVersion: v1
data:
  application.properties: |
    FOO_File=Prod_Bar
kind: ConfigMap
metadata:
  annotations:
    note: generated
    oncallPager: 800-555-1212
  labels:
    app: prod-demoapp
    type: generated
  name: prod-example-configmap-file-001
  namespace: demoapp
---
apiVersion: v1
data:
  FOO_literals: Bar
kind: ConfigMap
metadata:
  annotations:
    note: generated
    oncallPager: 800-555-1212
  labels:
    app: prod-demoapp
    type: generated
  name: prod-example-configmap-literals-001
  namespace: demoapp
---
apiVersion: v1
data:
  password.txt: dXNlcm5hbWU9YWRtaW4KcGFzc3dvcmQ9c2VjcmV0Cg==
kind: Secret
metadata:
  annotations:
    note: generated
    oncallPager: 800-555-1212
  labels:
    app: prod-demoapp
    type: generated
  name: prod-example-secret-1-001
  namespace: demoapp
type: Opaque
---
apiVersion: v1
data:
  password: c2VjcmV0
  username: YWRtaW4=
kind: Secret
metadata:
  annotations:
    note: generated
    oncallPager: 800-555-1212
  labels:
    app: prod-demoapp
    type: generated
  name: prod-example-secret-2-001
  namespace: demoapp
type: Opaque
---
apiVersion: v1
kind: Service
metadata:
  annotations:
    oncallPager: 800-555-1212
  labels:
    app: prod-demoapp
  name: prod-svc-demoapp-001
  namespace: demoapp
spec:
  ports:
  - name: http-8080
    port: 8080
    protocol: TCP
    targetPort: 48080
  - name: https
    port: 443
    protocol: TCP
    targetPort: 48443
  selector:
    app: prod-demoapp
  type: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    oncallPager: 800-555-1212
  labels:
    app: prod-demoapp
  name: prod-deployment-demoapp-001
  namespace: demoapp
spec:
  replicas: 3
  selector:
    matchLabels:
      app: prod-demoapp
  template:
    metadata:
      annotations:
        oncallPager: 800-555-1212
      labels:
        app: prod-demoapp
    spec:
      containers:
      - env:
        - name: DEMO_SERVICE_NAME
          value: prod-svc-demoapp-001
        - name: PORT
          value: "8080"
        image: ikubernetes/demoapp:v1.0
        name: demoapp
        resources:
          limits:
            memory: 128Mi
        volumeMounts:
        - mountPath: /config
          name: config
        - mountPath: /secrets
          name: password
      volumes:
      - configMap:
          name: prod-example-configmap-file-001
        name: config
      - name: password
        secret:
          secretName: prod-example-secret-1-001

创建 overlays/production 资源

# kubectl apply -k  production/
namespace/demoapp created
configmap/prod-example-configmap-env-001 created
configmap/prod-example-configmap-file-001 created
configmap/prod-example-configmap-literals-001 created
secret/prod-example-secret-1-001 created
secret/prod-example-secret-2-001 created
service/prod-svc-demoapp-001 created
deployment.apps/prod-deployment-demoapp-001 created

创建 overlays/production pod 资源

# kubectl get pod -n demoapp
NAME                                           READY   STATUS    RESTARTS   AGE
prod-deployment-demoapp-001-545cfd57b8-82dq7   1/1     Running   0          52s
prod-deployment-demoapp-001-545cfd57b8-b7kd2   1/1     Running   0          52s
prod-deployment-demoapp-001-545cfd57b8-dxksg   1/1     Running   0          52s

删除 overlays/production 资源

# kubectl delete -k production/
namespace "demoapp" deleted
configmap "prod-example-configmap-env-001" deleted
configmap "prod-example-configmap-file-001" deleted
configmap "prod-example-configmap-literals-001" deleted
secret "prod-example-secret-1-001" deleted
secret "prod-example-secret-2-001" deleted
service "prod-svc-demoapp-001" deleted
deployment.apps "prod-deployment-demoapp-001" deleted

参考文档

https://github.com/kubernetes-sigs/kustomize