June 15, 2026 - Open source Prometheus supports webhooks, DingTalk, and email. Set the notification method via the receiver parameter in ack-prometheus-operator. See Alerting configuration. Create an alert rule. Deploy a PrometheusRule CRD in the cluster to define alert rules (see Deploying Prometheus rules).

October 21, 2018 - # Replace job="apiserver" with the actual job name in your Prometheus configuration (kube-prometheus-stack uses "apiserver" by default). The 3% error rate threshold is a rough default; adjust based on your workload.

Alert thresholds depend on the nature of your applications. Some queries may have arbitrary tolerance thresholds. Building an efficient monitoring platform takes time.

prometheus.rules: |- groups: - name: demo alert rules: - alert: High Pod Memory expr: sum(container_memory_usage_bytes) > 1 for: 1m labels: severity: slack annotations: summary: High Memory Usage prometheus.yml: |- global: scrape_interval: 5s evaluation_interval: 5s rule_files: - /etc/prometheus/prometheus.rules alerting: alertmanagers: - scheme: http static_configs: - targets: - "alertmanager.monitoring.svc:9093"

runbook_url: https://github.com/kubernetes-monitoring/kubernetes-mixin/tree/master/runbook.md#alert-name-prometheusoperatordown · expr: | absent(up{job="prometheus-operator",namespace="monitoring"} == 1) for: 15m · labels: severity: critical · - name: kubernetes-apps · rules: - alert: KubePodCrashLooping ·

The PrometheusRule CRD allows to define alerting and recording rules. The operator knows which PrometheusRule objects to select for a given Prometheus based on the spec.ruleSelector field.

April 24, 2026 - apiVersion: monitoring.coreos.com/v1 kind: PrometheusRule metadata: name: kubernetes-essential namespace: monitoring spec: groups: - name: kubernetes.essential rules: - alert: PodCrashLooping expr: rate(kube_pod_container_status_restarts_total[15m]) > 0 for: 1h labels: severity: warning annotations: summary: "Pod {{ $labels.namespace }}/{{ $labels.pod }} is crash looping" - alert: NodeNotReady expr: kube_node_status_condition{condition="Ready",status="true"} == 0 for: 5m labels: severity: critical annotations: summary: "Node {{ $labels.node }} is not ready" - alert: PVCNearlyFull expr: kubelet

September 20, 2023 - The Prometheus Operator provides Kubernetes native deployment and management of Prometheus Alert Rules.

July 28, 2026 - Arc-enabled Kubernetes clusters do not support Platform metric recommended alert rules. Instead, equivalent Prometheus-based alert rules are provided for these clusters.

January 30, 2023 - Before we set up Prometheus, you need to know about two types of Prometheus rules. They are · Alerting rules - Alerts are written in PromQL. They evaluate one or more expressions and fire alerts based on the result.

March 22, 2025 - ✔ Set thresholds based on baselines, not assumptions ✔ Group alerts (e.g., all node issues together) to avoid alert fatigue ✔ Use labels (severity, team) to route alerts properly ✔ Silence alerts during maintenance windows ✔ Send alerts ...

September 10, 2025 - Simulate alert conditions: Temporarily modify your alert rules to trigger based on current conditions. Use Prometheus's API: Send test alerts directly through the Prometheus API. Alertmanager's web UI: Use the Alertmanager web interface to view and manage active alerts. ... Setting up basic alerts for your Kubernetes monitoring stack is a key step in maintaining the reliability and performance of your applications.

groups: - name: example labels: team: myteam rules: - alert: HighRequestLatency expr: job:request_latency_seconds:mean5m{job="myjob"} > 0.5 for: 10m keep_firing_for: 5m labels: severity: page annotations: summary: High request latency

apiVersion: extensions/v1beta1 kind: Ingress metadata: annotations: ingress.kubernetes.io/auth-realm: Authentication Required ingress.kubernetes.io/auth-secret: internal-services-auth ingress.kubernetes.io/auth-type: basic kubernetes.io/ingress.allow-http: "false" name: ingress-monitoring namespace: monitoring spec: tls: - hosts: - prometheus.example.com - grafana.example.com secretName: example-tls rules: - host: prometheus.example.com http: paths: - path: / backend: serviceName: prometheus-svc servicePort: 9090 - path: /alertmanager backend: serviceName: alertmanager servicePort: 9093 - host: grafana.example.com http: paths: - path: / backend: serviceName: grafana servicePort: 3000

July 23, 2024 - Once set up, you can test the alerting by triggering conditions that match our PrometheusRule. This should result in notifications being sent to your configured Slack channel. so first let us just decrease the number of replicas to one so that PrometheusRule is triggered, and then wait for few minutes and check the Prometheus UI again ... We have journeyed through an extensive and enlightening tutorial, aiming to demystify the workings of Prometheus in a Kubernetes environment.

May 16, 2021 - Create alerting rules in Prometheus (Like if any pod is not running) ... If you install a stable/prometheus helm chart, alertmanager is also installed with it. ... Be a part of the Slack workspace. Go to Slack -> Administration -> Manage apps. The Alertmanager uses the Incoming Webhooks feature of Slack, so we need to set that up first. ... Here you need to add the channel for receiving the alerts.

Alert rules in Kubernetes ML clusters ... model serving throughput dropping). Prometheus evaluates these rules every 30 seconds against your cluster's metrics and fires alerts to Alertmanager, which routes them to PagerDuty, Slack, or email....

April 17, 2023 - Before we go into more detail on writing Prometheus alert rules, let's quickly summarize the concepts that this article will cover. Prometheus alert templates provide a way to define standard fields and behavior for multiple alerts. You can define these templates in the Prometheus configuration file.

February 9, 2026 - Prometheus Kubernetes PrometheusRule Alerting Monitoring · The PrometheusRule CRD allows you to define Prometheus alerting and recording rules as Kubernetes resources. Instead of managing rules in ConfigMaps or static files, PrometheusRule ...

March 15, 2025 - Note: If you are following my tutorial ... - targets: - "alertmanager.monitoring.svc:9093" All the alerting rules have to be present on Prometheus config based on your needs....