Skip to content

Commit 3fbfd09

Browse files
Add experimental alerting features workflows and notifications pages
Adds six pages under kibana-alerting-experimental/: - workflows-alerting.md: how workflows connect to action policies, runtime execution order - notifications.md: action policy overview, dispatcher behavior, evaluation steps - notifications/create-configure-action-policy.md: creation and configuration guide - notifications/action-policy-reference.md: full field reference - notifications/manage-action-policies.md: management operations - notifications/notification-gating.md: new page covering acknowledge/snooze/deactivate gating Incorporates additions from PR #6395: dispatcher definition, gating steps in notifications.md, and the new notification-gating.md page. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent 8f8c4d0 commit 3fbfd09

7 files changed

Lines changed: 375 additions & 0 deletions

File tree

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
---
2+
navigation_title: Notifications
3+
applies_to:
4+
stack: unavailable
5+
serverless: preview
6+
products:
7+
- id: kibana
8+
description: "How experimental alerting features action policies route alert episodes to notifications: matchers, grouping, frequency, and workflow destinations."
9+
---
10+
11+
# experimental alerting features notifications
12+
13+
After a rule produces alert episodes, action policies decide what to do about them: who gets notified, how often, and through which channel.
14+
15+
This page explains how action policies work. For creating and configuring them step by step, refer to [Create and configure an action policy](notifications/create-configure-action-policy.md).
16+
17+
## What is an action policy [action-policies]
18+
19+
20+
An action policy is a saved object in your space that controls notification routing. It's not attached to a rule. It's global within the space, so when an episode is produced, the system evaluates all enabled policies in the space that are not snoozed and decides which ones apply.
21+
22+
Each policy has four controls:
23+
24+
| Control | What it does |
25+
| --- | --- |
26+
| Matcher (optional KQL) | Filters which episodes this policy applies to. An empty matcher matches all episodes in the space. |
27+
| Dispatch per (grouping) | Controls how episodes batch into notifications: one per episode, one per notification group (Dispatch per **Group**), or one digest for all. |
28+
| Frequency | Controls how often the policy can notify for the same notification group. |
29+
| Destinations | One or more workflows to invoke when all conditions are met. |
30+
31+
## How policies apply to rules
32+
33+
Action policies don't reference rules directly. You scope a policy using KQL over episode and rule fields, for example `rule.labels: "checkout"` or `data.severity: "critical"`. A policy applies to every matching episode in the space, from any rule.
34+
35+
Multiple policies can match the same episode, and each runs independently. There's no precedence or merging between them. If no policy matches an episode, no notification is sent. This is intentional.
36+
37+
## How action policies are evaluated [how-action-policies-evaluated]
38+
39+
40+
{{kib}} runs a background process called the dispatcher that checks for eligible episodes on a short interval (around 10 seconds) and evaluates action policies against them. The dispatcher is separate from the rule schedule. Rules write events on their own cadence, and the dispatcher picks them up asynchronously.
41+
42+
For each enabled policy that is not snoozed, the dispatcher works through the following steps:
43+
44+
1. **Gating:** Is the episode acknowledged, snoozed, or deactivated? If so, skip dispatch. Refer to [Notification gating](notifications/notification-gating.md) to learn more.
45+
2. **Matcher:** Does the episode match the policy's KQL? If not, skip this policy.
46+
3. **Grouping:** How should matching episodes batch into notification groups?
47+
4. **Frequency:** Has a notification already gone out for this notification group recently? If so, wait.
48+
5. **Destinations:** Send to the policy's workflow destinations.
49+
50+
### Notification dispatch outcomes [possible-outcomes]
51+
The dispatcher runs on a short interval (around 5 seconds). Notifications don't arrive on the exact rule schedule. They follow the dispatcher's own cycle.
52+
53+
### Possible outcomes [possible-outcomes]
54+
55+
Each notification attempt results in one of the following outcomes.
56+
57+
| Outcome | What it means |
58+
| --- | --- |
59+
| `dispatched` | A notification was sent. |
60+
| `throttled` | Dispatch was suppressed because the **frequency** interval had not elapsed. |
61+
| `suppressed` | The episode was suppressed before dispatch (acknowledged, snoozed, or deactivated). |
62+
| `unmatched` | No policy matched this episode; no workflow ran. |
63+
| `error` | Processing failed. Check {{kib}} logs. |
64+
65+
You can query these outcomes in Discover through the `.alert-actions` data stream.
66+
67+
## Why policies are separate from rules
68+
69+
Rules don't own policies. A rule can't say "notify team X when it fires." Instead, team X creates an action policy that uses a KQL matcher to pick up matching episodes.
70+
71+
This design means:
72+
- One policy can cover episodes from many rules.
73+
- You can update routing without touching any rule.
74+
- Rules can be created without any notification policy, which is useful for testing.
75+
76+
When you're ready to route notifications, go to [Create and configure an action policy](notifications/create-configure-action-policy.md).
77+
Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,99 @@
1+
---
2+
navigation_title: Action policy reference
3+
applies_to:
4+
stack: unavailable
5+
serverless: preview
6+
products:
7+
- id: kibana
8+
description: "Grouping modes, frequency options, dispatch outcomes, and matcher field reference for action policies in the experimental alerting features."
9+
---
10+
11+
# Action policy reference [action-policy-reference]
12+
13+
14+
Action policies are part of the experimental alerting features in Kibana. Use this page when building action policies. Below, you will find details about valid matcher fields, grouping modes and frequency options, dispatch outcome, and more. For step-by-step guidance, refer to [Create and configure an action policy](create-configure-action-policy.md).
15+
16+
## Matcher fields [matcher-fields]
17+
18+
Use these fields in the **Matcher** expression to filter which episodes a policy applies to. Combine them with standard KQL operators, for example `data.severity: "critical" AND episode_status: "active"`.
19+
20+
| Field | Description | Example |
21+
|---|---|---|
22+
| `episode_status` | Current lifecycle status of the episode. Accepted values: `active`, `inactive`, `pending`, `recovering`. | `episode_status: "active"` |
23+
| `data.*` | Dynamic payload fields sent by the rule. Available fields depend on the rule type and configuration. | `data.severity: "critical"` or `data.host.name: "web-01"` |
24+
| `rule.id` | Unique identifier for the rule that generated the episode. | `rule.id: "rule-001"` |
25+
| `rule.name` | Display name of the rule. | `rule.name: "High CPU"` |
26+
| `rule.labels` | Key-value labels attached to the rule. Use dot notation to target a specific label key. | `rule.labels.env: "production"` |
27+
28+
<!--[CONTENT NEEDED for M2: M2 adds two first-class episode-level severity fields that will be directly matchable in KQL:
29+
30+
- `episode.severity` - The current severity of the episode (most recent evaluation). Enables matching like `episode.severity: "CRITICAL"`.
31+
- `episode.severity_max` - The highest severity seen over the episode's lifetime. Enables matching like `episode.severity_max: "CRITICAL"` to catch episodes that were once critical even if they have since de-escalated.
32+
33+
Add both fields to this table with examples. Update the introductory sentence to include them. Also remove or deprecate the `data.severity` example once `episode.severity` is the preferred approach, otherwise users will get conflicting guidance about which field to use for severity matching.
34+
35+
There is also an open M2 question about whether a severity change mid-episode (de-escalation or escalation) triggers policy re-evaluation. If it does, document the re-evaluation behavior in the frequency options section below, since it interacts with frequency limits.]
36+
-->
37+
38+
## Dispatch per options [notification-grouping]
39+
40+
Controls how the policy batches matching episodes before sending a notification.
41+
42+
| Option | Description | When to use |
43+
|---|---|---|
44+
| Episode | Each episode triggers its own notification independently. Default selection. | You need per-issue visibility and want to handle each problem separately. |
45+
| Group | The policy bundles episodes that share the same value for a specified `data.*` field into one notification per unique value (a **notification group**). | A rule produces many related episodes (for example, one per service or host) and you want to reduce noise by batching them into shared notifications. |
46+
| Digest | The policy combines all matching episodes into a single notification, regardless of what they have in common. | You want a single periodic summary of everything that matched, rather than individual alerts. |
47+
48+
## Frequency [throttle-strategies]
49+
50+
**Frequency** controls how often the policy fires for a given episode or notification group. The available options depend on the **Dispatch per** setting. Not all options are valid for all modes.
51+
52+
| Option | Description | When to use |
53+
|---|---|---|
54+
| On status change | Notifies when the episode status changes (for example, active → recovering). One notification per transition. | You only need to know when something breaks and when it's resolved. No reminders needed. |
55+
| On status change + repeat at interval | Notifies on status change, then resends notifications at a regular interval while the episode remains in the same status. | You want status change alerts plus periodic notifications that a problem is still unresolved, in case it has been missed or pushed aside. |
56+
| At most once every… | Caps notifications at one per episode or notification group within the chosen interval, regardless of rule frequency. | You want to limit alert volume for noisy rules without missing new or ongoing issues. |
57+
| Every evaluation | Notifies on every rule evaluation. Can be noisy. Use sparingly and only with infrequent rule schedules. | You need a full audit trail of every evaluation, or the rule runs infrequently enough that noise isn't a concern. |
58+
59+
<!--[CONTENT NEEDED for M2: An open M2 question is whether a severity change mid-episode (escalation or de-escalation of `episode.severity`) triggers policy re-evaluation independently of episode status changes. If it does, this table needs a new strategy option or a note explaining the interaction between severity changes and the "On status change" option. Confirm the M2 decision before updating.]
60+
-->
61+
62+
### Frequency options for Episode [frequency-when-episode-per_episode]
63+
64+
Available frequency options when you set **Dispatch per** to **Episode**.
65+
66+
| Option | Description | Example |
67+
|---|---|---|
68+
| On status change | Notifies once when the episode opens and once when it recovers. No repeat notifications while it remains active. Best for when you trust your ticketing or incident workflow to track ongoing issues | A host goes down at 9:00am → one notification. Recovers at 11:00am → one notification. No notifications between them. |
69+
| On status change + repeat at interval | Same as On status change, but also sends a reminder at a set interval while the episode is still active. | A host goes down at 9:00am → notification. With a 1h repeat: reminder at 10:00am, 11:00am. Recovers at 11:30am → notification. |
70+
| Every evaluation | Fires on every rule evaluation, regardless of status. Can be noisy on frequent rule schedules. Avoid in production. | A rule running every 5 minutes with one active episode produces up to 288 notifications per day. |
71+
72+
### Frequency options for Group
73+
74+
Available frequency options when you set **Dispatch per** to **Group**.
75+
76+
| Option | Description | Example |
77+
|---|---|---|
78+
| At most once every… | Limits how often each notification group can notify, regardless of how many episodes match or how often the rule runs. | 10 episodes share `data.host.name: "web-01"`. With a 1h limit, you get at most one notification per hour for that notification group. |
79+
| Every evaluation | Fires on every rule evaluation for each unique value in the group-by field. Still noisy on frequent rule schedules. | A rule running every 10 minutes with 5 unique host values produces up to 6 notifications per host per hour. |
80+
81+
### Frequency options for Digest
82+
83+
Available frequency options when you set **Dispatch per** to **Digest**.
84+
85+
| Option | Description | Example |
86+
|---|---|---|
87+
| Every evaluation | The only option for Digest. Fires on every rule run, bundling all matching episodes into one message. Pair with a longer rule schedule to avoid frequent summary messages. | A rule running every 30 minutes with 20 matching episodes produces one summary notification every 30 minutes containing all 20. |
88+
89+
## Dispatch outcomes
90+
91+
The system records each notification attempt with one of the following outcomes. To investigate delivery issues, query the `.alert-actions` data stream in Discover and filter by the `outcome` field.
92+
93+
| Outcome | What happened |
94+
|---|---|
95+
| `dispatched` | The system sent the notification successfully. |
96+
| `throttled` | The system skipped delivery because the **frequency** interval had not elapsed. This is expected behavior, not an error. |
97+
| `suppressed` | Dispatch was blocked before the notification went out—the rule was acknowledged, snoozed, or deactivated. |
98+
| `unmatched` | No action policy matched this episode, so no workflow ran. |
99+
| `error` | An error occurred during processing. Check {{kib}} logs to identify the cause. |
Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
---
2+
navigation_title: Create an action policy
3+
applies_to:
4+
stack: unavailable
5+
serverless: preview
6+
products:
7+
- id: kibana
8+
description: "Create action policies in the experimental alerting features, configure matchers, Dispatch per, Frequency, and workflow destinations."
9+
---
10+
11+
# Create and configure an action policy [create-manage-action-policies]
12+
13+
14+
Action policies are part of the experimental alerting features in Kibana. Rules define what counts as a problem. Action policies define what happens when a problem is detected. They determine which episodes generate notifications, how episodes batch for dispatch, **frequency** limits on notifications, and where they're routed.
15+
16+
Because policies are separate from rules and global within a space, you can update notification behavior across many rules at once without touching detection logic, and you can route the same alerts differently depending on severity or source. You create and manage policies from the **Action policies** page, not from the rule form.
17+
18+
For matcher fields, grouping modes, frequency options, and dispatch outcomes, refer to [Action policy reference](action-policy-reference.md).
19+
20+
<!--
21+
## Create an action policy
22+
23+
[CONTENT NEEDED for M2: UI. Once the navigation and action policy settings have been confirmed, add instructions for creating an action policy.]
24+
-->
25+
26+
## Policy fields [policy-fields]
27+
28+
### Matcher [matcher]
29+
30+
31+
An optional KQL expression that filters which episodes this policy applies to. An empty matcher matches every episode in the space.
32+
33+
Use matchers to route different episodes to different policies, for example, one policy for `data.severity: "critical"` episodes routed to PagerDuty and another for warnings routed to Slack. For available fields and examples, refer to [Matcher fields](action-policy-reference.md#matcher-fields).
34+
35+
<!--[CONTENT NEEDED for M2: The `data.severity: "critical"` example above will become the legacy approach once M2 ships. M2 promotes severity to `episode.severity` and `episode.severity_max` as first-class episode fields. Update this example to use `episode.severity: "CRITICAL"` and update the cross-reference to include the new fields. Also decide whether to retain `data.severity` as an alternative for rules that haven't migrated, or to remove it from guidance entirely.]
36+
-->
37+
38+
### Grouping and frequency [reduce-noise-grouping]
39+
40+
41+
**Dispatch per** controls how episodes batch into notifications. **Frequency** controls how often the policy can notify for each batch.
42+
43+
:::{table}
44+
:widths: 4-4-4
45+
46+
| Dispatch per | What it does | Available Frequency options |
47+
|---|---|---|
48+
| Episode | One notification per episode. | - On status change <br> - On status change + repeat at interval <br> - Every evaluation |
49+
| Group | Bundle episodes that share a field value. Specify **Group by** (for example `data.service.name` or `data.host.name`). | - At most once every… <br> - Every evaluation |
50+
| Digest | One notification for all matching episodes combined. | Every evaluation |
51+
52+
:::
53+
54+
For detailed descriptions, frequency options, and examples for each mode, refer to [Dispatch per options](action-policy-reference.md#notification-grouping).
55+
56+
### Frequency [throttle]
57+
58+
59+
**Frequency** limits how often the policy can fire for a given notification group. The interval resets from the last time the policy fired, so successive notifications stay at least `interval` apart. Set a duration such as `1h` or `30m`. For available options by **Dispatch per** mode, refer to [Frequency](action-policy-reference.md#throttle-strategies).
60+
61+
### Destinations
62+
63+
One or more workflows to invoke when the policy matches. Use the search field to find and attach workflows.
64+
65+
### Snooze
66+
67+
An optional time window during which the policy doesn't dispatch. Useful for planned maintenance or quiet periods without disabling the policy entirely.
Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
---
2+
navigation_title: Manage action policies
3+
applies_to:
4+
stack: unavailable
5+
serverless: preview
6+
products:
7+
- id: kibana
8+
description: "Enable, disable, snooze, maintenance windows, bulk actions, and API key rotation for action policies in the experimental alerting features."
9+
---
10+
11+
# Manage action policies
12+
13+
Managing action policies is part of the experimental alerting features in Kibana. After you create an action policy, you can control when it runs, pause it temporarily, and keep its credentials current. This page covers those management tasks.
14+
15+
## Enable, snooze, and maintenance
16+
17+
You can disable a policy so it is not evaluated for new episodes. You can snooze a policy for a defined window so that it does not dispatch notifications during that period. Policies that are not enabled or are snoozed are skipped when the dispatcher evaluates policies.
18+
19+
### Maintenance windows [maintenance-windows]
20+
21+
22+
Maintenance windows are scheduled periods during which a policy does not dispatch notifications. They are configured on the action policy alongside snooze and other policy controls, not on the rule. Rule evaluation continues and alert episodes can still be recorded in `.rule-events`. Only dispatch through that policy pauses. Use maintenance windows for planned deployments, infrastructure changes, or recurring quiet periods.
23+
24+
## Update API keys
25+
26+
You can rotate the API key used to run a policy's workflows without changing matchers or destinations. Use the **Update API key** action on one policy or for multiple selected policies.
27+
28+
::::{important} Production considerations
29+
When you update or delete an action policy, previous API keys used for execution are marked for invalidation and removed on a schedule managed by {{kib}}. Allow for a short delay before new keys are used for dispatch.
30+
::::
31+
32+
## Bulk actions
33+
34+
On the action policies list, select one or more policies to enable, disable, snooze, and do more in bulk. **Select all** selects every policy on the current page of results. Clear the selection before changing filters if you need a different set.

0 commit comments

Comments
 (0)