|
1 | 1 | .. _builtin-alert-enrichment: |
2 | 2 |
|
3 | | -Builtin Alert Enrichment |
| 3 | +Enrich Alerts |
4 | 4 | ######################################## |
5 | 5 |
|
6 | | -Robusta takes Prometheus to the next level by correlating alerts with other observability data. |
| 6 | +Ever feel overwhelmed by alerts that lack context? Robusta enriches alerts automatically and lets you create custom enrichment rules. |
| 7 | + |
| 8 | +.. note:: |
7 | 9 |
|
8 | | -Testing out Prometheus alerts |
| 10 | + **Looking for automatic AI enrichment?** Check out :doc:`HolmesGPT </configuration/holmesgpt/main-features>` for zero-configuration AI-powered alert enrichment that automatically investigates alerts and provides root cause analysis. |
| 11 | + |
| 12 | +Builtin Alert Enrichment |
9 | 13 | ********************************* |
10 | | -1. Deploy a broken pod that will be stuck in pending state: |
| 14 | + |
| 15 | +Robusta automatically enriches Prometheus alerts with relevant Kubernetes context: |
| 16 | + |
| 17 | +* **Pod events** - Recent events related to the affected pod |
| 18 | +* **Pod logs** - Relevant log excerpts from crashing or failing containers |
| 19 | +* **Resource metrics** - CPU, memory, and other resource usage data |
| 20 | +* **Related Kubernetes objects** - Deployments, ReplicaSets, ConfigMaps, etc. |
| 21 | + |
| 22 | +This happens automatically for common Prometheus alerts without any configuration. To extend it to your own Prometheus alerts you can define custom playbooks. |
| 23 | + |
| 24 | +Testing Alert Enrichment |
| 25 | +^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 26 | + |
| 27 | +1. Deploy a broken pod: |
11 | 28 |
|
12 | 29 | .. code-block:: bash |
13 | 30 | :name: cb-apply-pendingpod |
14 | 31 |
|
15 | 32 | kubectl apply -f https://raw.githubusercontent.com/robusta-dev/kubernetes-demos/main/pending_pods/pending_pod_resources.yaml |
16 | 33 |
|
17 | | -2. Trigger a Prometheus alert immediately, skipping the normal delays: |
| 34 | +2. Trigger a Prometheus alert immediately: |
18 | 35 |
|
19 | 36 | .. code-block:: bash |
20 | 37 | :name: cb-trigger-prometheus-alert |
21 | 38 |
|
22 | 39 | robusta playbooks trigger prometheus_alert alert_name=KubePodCrashLooping namespace=default pod_name=example-pod |
23 | 40 |
|
24 | | -.. admonition:: Example Slack Message |
| 41 | +.. admonition:: Example Enriched Alert |
25 | 42 |
|
26 | 43 | .. image:: /images/simulatedprometheusalert.png |
| 44 | + |
| 45 | +Custom Alert Enrichment |
| 46 | +********************************* |
| 47 | + |
| 48 | +Create custom enrichment rules to: |
| 49 | + |
| 50 | +* Reduce MTTR by automatically gathering system state and logs when alerts fire |
| 51 | +* Make faster decisions on which team needs to investigate |
| 52 | +* Link alerts to runbooks and documentation for better knowledge sharing |
| 53 | + |
| 54 | +Get started with these examples: |
| 55 | + |
| 56 | +.. toctree:: |
| 57 | + :maxdepth: 1 |
| 58 | + |
| 59 | + prometheus-examples/bash-alert-enrichment |
| 60 | + prometheus-examples/link-alert-enrichment |
0 commit comments