promethus监控k8s有两种方式，一直是k8s搭建自己监控自己 第二种是外部搭建好了promethus,k8s搭建pod收集信息
yaml地址为 https://github.com/starsliao/Prometheus/tree/master/kubernetes 一般选择2.2.1
按需修改service的模式暴露端口，修改后如下：
apiVersion: v1
kind: Service
metadata:
# annotations:
# prometheus.io/scrape: 'true'
labels:
app.kubernetes.io/name: kube-state-metrics
app.kubernetes.io/version: v2.2.1
name: kube-state-metrics
namespace: ops-monit
spec:
type: NodePort
ports:
- name: http-metrics
port: 8080
targetPort: http-metrics
nodePort: 30866
- name: telemetry
port: 8081
targetPort: telemetry
nodePort: 30867
selector:
app.kubernetes.io/name: kube-state-metrics
创建
kubectl create namespace ops-monit
cd kube-state-metrics
kubectl apply -f .
配置prometheus.yml
- job_name: 'k8s-cadvisor'
scrape_interval: 60s
scrape_timeout: 60s
metrics_path: /metrics/cadvisor
kubernetes_sd_configs: # kubernetes 自动发现
- api_server: https://192.168.1.21:6443 # apiserver 地址
role: node # node 类型的自动发现
namespaces:
names:
- ops-monit
bearer_token_file: k8s.token
tls_config:
insecure_skip_verify: true
bearer_token_file: k8s.token
tls_config:
insecure_skip_verify: true
relabel_configs:
- source_labels: [__address__]
regex: '(.*):10250'
replacement: '${1}:10255'
target_label: __address__
action: replace
- action: labelmap
regex: __meta_kubernetes_node_label_(.+)
metric_relabel_configs:
- source_labels: [instance]
separator: ;
regex: (.+)
target_label: node
replacement: $1
action: replace
- source_labels: [pod_name]
separator: ;
regex: (.+)
target_label: pod
replacement: $1
action: replace
- source_labels: [container_name]
separator: ;
regex: (.+)
target_label: container
replacement: $1
action: replace
- job_name: kube-state-metrics-1
kubernetes_sd_configs:
- api_server: https://192.168.1.21:6443 # apiserver 地址
role: endpoints # node 类型的自动发现
namespaces:
names:
- ops-monit
bearer_token_file: k8s.token
tls_config:
insecure_skip_verify: true
bearer_token_file: k8s.token
tls_config:
insecure_skip_verify: true
relabel_configs:
- action: labelmap
regex: __meta_kubernetes_node_label_(.+)
- separator: ;
regex: (.*)
target_label: __address__
replacement: 192.168.1.21:30866
- source_labels: [__meta_kubernetes_service_label_app_kubernetes_io_name]
regex: kube-state-metrics
replacement: $1
action: keep
- action: labelmap
regex: __meta_kubernetes_service_label_(.+)
- source_labels: [__meta_kubernetes_namespace]
action: replace
target_label: k8s_namespace
- source_labels: [__meta_kubernetes_service_name]
action: replace
target_label: k8s_sname
- job_name: kube-state-metrics-2
kubernetes_sd_configs:
- api_server: https://192.168.1.21:6443 # apiserver 地址
role: endpoints # node 类型的自动发现
namespaces:
names:
- ops-monit
bearer_token_file: k8s.token
tls_config:
insecure_skip_verify: true
bearer_token_file: k8s.token
tls_config:
insecure_skip_verify: true
relabel_configs:
- action: labelmap
regex: __meta_kubernetes_node_label_(.+)
- separator: ;
regex: (.*)
target_label: __address__
replacement: 192.168.1.21:30867
- source_labels: [__meta_kubernetes_service_label_app_kubernetes_io_name]
regex: kube-state-metrics
replacement: $1
action: keep
- action: labelmap
regex: __meta_kubernetes_service_label_(.+)
- source_labels: [__meta_kubernetes_namespace]
action: replace
target_label: k8s_namespace
- source_labels: [__meta_kubernetes_service_name]
action: replace
target_label: k8s_sname
或者简单点
- job_name: "kube-state"
metrics_path: '/metrics'
static_configs:
- targets:
- "192.128.252.128:8080"
minikube或者k3s临时转发port。
kubectl port-forward kube-state-metrics-747cb9cd6-jgfjw -n ops-monit 8080:8080 8081:8081 --address 192.168.252.128
