Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,43 @@ Verify it Works

Send a dummy alert to AlertManager:

.. code-block:: bash
.. tab-set::

robusta demo-alert
.. tab-item:: Robusta CLI

If you have the Robusta CLI installed, you can send a test alert using the following command:

.. code-block:: bash

robusta demo-alert

.. tab-item:: Robusta UI

In the Robusta UI, go to the "Clusters" tab, choose the right cluster and click "Simulate Alert".

.. image:: /images/robusta-ui-simulate-alert-1.png
Comment thread
pavangudiwada marked this conversation as resolved.
:alt: Choose the cluster
:width: 900
:align: center

Then

1. Check **Send alert with no resource**.
2. Provide a name for the alert in the **Alert name (identifier)** field (e.g., "Testing Prod AlertManager").
3. Select **Alert Manager** under the "Send alert to" section.
4. Click the **Simulate Alert** button to send the test alert.

.. image:: /images/robusta-ui-simulate-alert-2.png
:alt: Send Test Alert
:width: 600
:align: center

If everything is setup properly, this alert will reach Robusta. It will show up in the Robusta UI, Slack, and other configured sinks.
Comment thread
pavangudiwada marked this conversation as resolved.

.. note::

It might take a few minutes for the alert to arrive due to AlertManager's `group_wait` and `group_interval` settings. More info `here <https://prometheus.io/docs/alerting/latest/configuration/#:~:text=How%20long%20to%20wait%20before%20sending%20a%20notification%20about%20new%20alerts%20that%0A%23%20are%20added%20to%20a%20group%20of%20alerts%20for%20which%20an%20initial%20notification%20has%0A%23%20already%20been%20sent>`_.

.. details:: I configured AlertManager, but I'm not receiving alerts?
:class: warning

Expand Down
38 changes: 30 additions & 8 deletions docs/configuration/alertmanager-integration/victoria-metrics.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,26 +8,48 @@ You will need to configure two integrations: one to send alerts to Robusta and a
Send Alerts to Robusta
============================

To configure, edit AlertManager's configuration:
Add the following to your Victoria Metrics Alertmanager configuration (e.g., Helm values file or VMAlertmanagerConfig CRD):

.. code-block:: yaml

receivers:
- name: 'robusta'
webhook_configs:
- url: 'http://<ROBUSTA-HELM-RELEASE-NAME>-runner.<NAMESPACE>.svc.cluster.local/api/alerts'
send_resolved: true # (3)

route: # (1)
routes:
- receiver: 'robusta'
group_by: [ '...' ]
group_wait: 1s
group_interval: 1s
matchers:
- severity =~ ".*"
repeat_interval: 4h
continue: true # (2)

.. code-annotations::
1. Put Robusta's route as the first route, to guarantee it receives alerts. If you can't do so, you must guarantee all previous routes has ``continue: true`` set.
2. Keep sending alerts to receivers defined after Robusta.
3. Important, so Robusta knows when alerts are resolved.

.. include:: ./_alertmanager-config.rst

.. include:: ./_testing_integration.rst

Configure Metric Querying
Configure Metrics Querying
====================================

Metrics querying lets Robusta pull metrics and create silences.
Robusta can query metrics and create silences using Victoria Metrics. If both are in the same Kubernetes cluster, Robusta can auto-detect the Victoria Metrics service. To verify, go to the "Apps" tab in Robusta, select an application, and check for usage graphs.

Add the following to ``generated_values.yaml`` and :ref:`update Robusta <Simple Upgrade>`.
If auto-detection fails you must add the ``prometheus_url`` parameter and :ref:`update Robusta <Simple Upgrade>`.

.. code-block:: yaml

globalConfig: # this line should already exist
# add the lines below
alertmanager_url: "http://ALERT_MANAGER_SERVICE_NAME.NAMESPACE.svc.cluster.local:9093"
grafana_url: ""
prometheus_url: "http://VICTORIA_METRICS_SERVICE_NAME.NAMESPACE.svc.cluster.local:8429"
alertmanager_url: "http://<VM_ALERT_MANAGER_SERVICE_NAME>.<NAMESPACE>.svc.cluster.local:9093" # Example:"http://vmalertmanager-victoria-metrics-vm.default.svc.cluster.local:9093/"
prometheus_url: "http://VM_Metrics_SERVICE_NAME.NAMESPACE.svc.cluster.local:8429" # Example:"http://vmsingle-vmks-victoria-metrics-k8s-stack.default.svc.cluster.local:8429"
# Add any labels that are relevant to the specific cluster (optional)
# prometheus_additional_labels:
# cluster: 'CLUSTER_NAME_HERE'
Expand Down
Binary file added docs/images/robusta-ui-simulate-alert-1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/robusta-ui-simulate-alert-2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading