prometheus configmaps添加报警规则硬盘超过85就钉钉报警
时间: 2023-11-07 20:57:49 浏览: 110
要实现这个功能,您需要完成以下步骤:
1. 创建一个 ConfigMap 来存储报警规则。您可以使用 YAML 文件来定义规则,例如:
```
apiVersion: v1
kind: ConfigMap
metadata:
name: alert-rules
namespace: monitoring
data:
rules.yml: |
groups:
- name: Host Metrics
rules:
- alert: HighDiskUsage
expr: 100 - (node_filesystem_avail_bytes{mountpoint="/"} * 100 / node_filesystem_size_bytes{mountpoint="/"}) > 85
for: 5m
labels:
severity: warning
annotations:
summary: "Disk usage is high on {{ $labels.instance }}"
description: "{{ $labels.instance }} has less than 15% disk space available."
```
2. 在 Prometheus 的配置文件中引用该 ConfigMap。例如:
```
apiVersion: monitoring.coreos.com/v1
kind: Prometheus
metadata:
name: prometheus
namespace: monitoring
spec:
serviceAccountName: prometheus
serviceMonitorSelector:
matchLabels:
app: node-exporter
ruleSelector:
matchLabels:
prometheus: prometheus
alerting:
alertmanagers:
- static_configs:
- targets:
- alertmanager.monitoring.svc:9093
configMaps:
- name: alert-rules
```
3. 在 Alertmanager 的配置文件中设置报警接收者。例如:
```
route:
group_by: ['alertname', 'severity']
receiver: 'dingtalk'
routes:
- match:
severity: warning
receiver: 'dingtalk'
receivers:
- name: 'dingtalk'
dingtalk_configs:
- api_secret: <API_SECRET>
api_url: 'https://oapi.dingtalk.com/robot/send?access_token=<ACCESS_TOKEN>'
at_mobiles: '12345678901,12345678902'
```
注意,您需要将 `<API_SECRET>` 和 `<ACCESS_TOKEN>` 替换成您的钉钉机器人的密钥和访问令牌。另外,`at_mobiles` 参数用于指定要通知的手机号码列表。
4. 重新加载 Prometheus 和 Alertmanager 的配置文件。例如:
```
kubectl rollout restart deployment prometheus -n monitoring
kubectl rollout restart deployment alertmanager -n monitoring
```
运行以上命令将重新加载 Prometheus 和 Alertmanager 的配置文件,使其生效。
完成以上步骤后,当硬盘使用率超过 85% 时,Prometheus 将触发报警规则,并将报警发送到 Alertmanager。Alertmanager 将根据报警规则中的匹配条件和接收者配置发送报警信息到指定的钉钉群组或个人。
阅读全文