Prometheus 监控 ingress-nginx 性能指标

0x01 Prometheus 配置

在prometheus-additional.yaml配置文件中，添加监控配置。
参考文章：https://github.com/kubernetes/ingress-nginx/blob/main/deploy/prometheus/prometheus.yaml

- job_name: 'k8s-ingress-nginx'
  kubernetes_sd_configs:
  - role: pod
    namespaces:
      names:
      - ingress-nginx # 只监控 ingress-nginx 命名空间中的 Pod
  relabel_configs:
  - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
    action: keep
    regex: true  # 仅保留 prometheus.io/scrape 注解值为 true 的 Pods
  - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scheme]
    action: replace
    target_label: __scheme__
    regex: (https?)
  - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]
    action: replace
    target_label: __metrics_path__
    regex: (.+)
  - source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]
    action: replace
    target_label: __address__
    regex: ([^:]+)(?::\d+)?;(\d+)
    replacement: $1:$2
  - source_labels: [__meta_kubernetes_service_name]
    regex: prometheus-server
    action: drop

0x02 在 ingress-nginx 添加注释

确保prometheus自动发现的时候，可以获取到监控地址 端口 协议等信息。
Ingress NGINX Controller 默认暴露的指标路径是 http://:10254/metrics。这意味着，默认情况下，/metrics 路径用于提供 NGINX Ingress 控制器的 Prometheus 指标，且该路径通常监听在端口 10254 上。

root@k8s-master:~/prom# kubectl edit  daemonset -n ingress-nginx 
spec:
  template:
    metadata:
      annotations:
        prometheus.io/path: "/metrics"
        prometheus.io/port: "10254"
        prometheus.io/scheme: "http"
        prometheus.io/scrape: "true"

0x03 RBAC

让prometheus可以访问到ingress pod，我这边直接让prometheus可以只读所有资源。

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: prometheus-k8s-readonly
subjects:
  - kind: ServiceAccount
    name: prometheus-k8s
    namespace: monitoring  # 指定命名空间为 monitoring
roleRef:
  kind: ClusterRole
  name: prometheus-k8s-readonly  # 绑定的角色
  apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: prometheus-k8s-readonly
rules:
  # 允许访问所有的 API Group 和资源
  - apiGroups: ["*"]
    resources: ["*"]
    verbs: ["get", "list", "watch"]

0x04 Grafana Dashboards

https://github.com/kubernetes/ingress-nginx/blob/main/deploy/grafana/dashboards/nginx.json
https://github.com/kubernetes/ingress-nginx/blob/main/deploy/grafana/dashboards/request-handling-performance.json