diff --git a/docset.yml b/docset.yml index 9d1881f8a4..4db6f43a7e 100644 --- a/docset.yml +++ b/docset.yml @@ -149,8 +149,8 @@ subs: ls-pipelines-app: "Logstash Pipelines" maint-windows-app: "Maintenance Windows" maint-windows-cap: "Maintenance windows" - alerting-v2: "experimental alerting features" - alerting-v2-cap: "Experimental alerting features" + alerting-v2-system: "experimental alerting system" + alerting-v2-system-cap: "Experimental alerting system" custom-roles-app: "Custom Roles" data-source: "data view" data-sources: "data views" diff --git a/explore-analyze/alerting.md b/explore-analyze/alerting.md index b0aa40b604..630ebaee87 100644 --- a/explore-analyze/alerting.md +++ b/explore-analyze/alerting.md @@ -8,28 +8,55 @@ applies_to: products: - id: kibana - id: cloud-serverless + - id: elasticsearch + - id: cloud-hosted +navigation_title: Alerting +description: Watch your data and respond to conditions automatically with Elastic alerting. Compare Kibana alerting, the experimental ES|QL-based alerting system, and Watcher to find the right fit. --- -# Alerting +# Alerting [alerting-overview] -Alerting tools in Elasticsearch and Kibana provide functionality to monitor data and notify you about significant changes or events in real time. This page provides an overview of how the key components work. +Elastic alerting helps you watch your data and respond when something needs attention, whether that is a metric crossing a limit, an asset leaving an area on a map, or an unusual pattern in your time series. You set the conditions and how people should be notified. Elastic runs the checks for you. -## Alerts +Elastic offers three alerting systems. For most new projects and projects on the most recent {{kib}} versions, the {{alerting-v2-system}} is the recommended system. If you're not sure which fits your situation, refer to [Compare alerting systems](alerting/compare-alerting-systems.md). -Alerts are notifications generated when specific conditions are met. These notifications are sent to you through channels that you previously set such as email, Slack, webhooks, PagerDuty, and so on. +## {{alerting-v2-system-cap}} -Alerts are created based on rules, which define the criteria for triggering them. Rules monitor the data indexed in Elasticsearch and evaluate conditions on a defined schedule to identify matches. For example, a threshold rule can generate an alert when a value crosses a specific threshold, while a machine learning rule activates an alert when an anomaly detection job identifies an anomaly. +```{applies_to} +stack: experimental 9.5+ +serverless: experimental +``` + +The {{alerting-v2-system}} is built on {{esql}}. You write the query that defines what to watch for, choose how alert episodes are tracked per series, and control notifications through action policies that handle routing, frequency, and notification batching. The {{alerting-v2-system}} also adds alert episode lifecycle tracking, per-series snooze, and rules on alert episodes for correlation and escalation. It is a strong fit when you want full control over what data travels with each alert episode and how your team is notified. + +:::{note} +The {{alerting-v2-system}} runs next to {{kib}} alerting on {{serverless-full}} and {{stack}} 9.5 and later. You don't have to move everything at once. You can copy or rebuild rules when you're ready, and your existing {{kib}} alerting rules won't be affected. +::: + +[Get started with the {{alerting-v2-system}} →](alerting/experimental-alerting-system.md) + +## {{kib}} alerting + +```{applies_to} +stack: ga +serverless: ga +``` + +{{kib}} alerting gives you ready-made rule types that work with applications such as APM, metrics, security, and uptime monitoring. You set conditions on a schedule you choose and send notifications through common channels (email, chat apps, webhooks, on-call tools, and more). Setup uses forms and clear steps, so you do not need to learn a query language first. It is a strong fit when you want broad coverage out of the box. + +[Get started with {{kib}} alerting →](alerting/alerts.md) ## Watcher + ```{applies_to} +stack: ga serverless: unavailable ``` -You can use Watcher for alerting and monitoring specific conditions in your data. It enables you to define rules and take automated actions when certain criteria are met. Watcher is a powerful alerting tool for custom use cases and more complex alerting logic. It allows advanced scripting using Painless to define complex conditions and transformations. +Watcher is for unusual or highly tailored setups where you need scripts, chained steps, or close control over {{es}} APIs. It does not use the main {{kib}} rules UI used by {{kib}} alerting. It is available on the {{stack}} only, not in {{serverless-full}}. :::{tip} -For most use cases, you should use Kibana Alerts instead of Watcher. Kibana Alerts allows rich integrations across use cases like APM, metrics, security, and uptime. Prepackaged rule types simplify setup and hide the details of complex, domain-specific detections, while providing a consistent interface across Kibana. - -Watcher is not available in {{serverless-full}}. +For most teams, {{kib}} alerting or the {{alerting-v2-system}} is easier to adopt than Watcher. Both work within {{kib}}'s rules UI and don't require writing {{es}} watch definitions. ::: +[Get started with Watcher →](alerting/watcher.md) diff --git a/explore-analyze/alerting/alerts.md b/explore-analyze/alerting/alerts.md index 972d944c84..883fd206bc 100644 --- a/explore-analyze/alerting/alerts.md +++ b/explore-analyze/alerting/alerts.md @@ -16,9 +16,15 @@ description: "Overview of Kibana alerting: rules, alerts, actions, connectors, a # {{kib}} alerting [alerts] {{kib}} alerting is the built-in alerting system in {{kib}}. It lets you define rules that check your data on a schedule, create alerts when conditions are met, and trigger actions through connectors (email, Slack, webhooks, and more). It is available on all deployments. - + +:::{note} +:applies_to: {"stack": "experimental 9.5+", "serverless": "experimental"} +For the {{alerting-v2-system}} built on {{esql}}, refer to [{{alerting-v2-system-cap}}](experimental-alerting-system.md). +::: + +::::{note} +In this doc, *alert* refers to a tracked occurrence of a rule condition. If you're using the {{alerting-v2-system}}, the equivalent concept is called an *alert episode*. The two terms describe similar ideas in different systems and are not interchangeable. +:::: ## {{rules-ui}} [rules] diff --git a/explore-analyze/alerting/compare-alerting-systems.md b/explore-analyze/alerting/compare-alerting-systems.md new file mode 100644 index 0000000000..63acca9d28 --- /dev/null +++ b/explore-analyze/alerting/compare-alerting-systems.md @@ -0,0 +1,53 @@ +--- +navigation_title: Compare alerting systems +applies_to: + stack: ga + serverless: ga +products: + - id: kibana + - id: cloud-serverless + - id: elasticsearch + - id: cloud-hosted +description: Compare Kibana alerting, the experimental ES|QL-based alerting system, and Watcher by use case and deployment type to select the right tool for your monitoring needs. +--- + +# Compare alerting systems [compare-alerting-systems] + +Elastic offers three alerting systems, each suited to different use cases and workflows. For most new projects and projects on the most recent {{kib}} versions, the {{alerting-v2-system}} is the recommended system. Use this page to compare them by goal, feature, and availability. + +## Select by use case + +| Goal | Suggested system | Availability | +|---|---|---| +| Monitor metrics, logs, or uptime with ready-made rules and no query language | [{{kib}} alerting](alerts.md) | {applies_to}`stack: ga` {applies_to}`serverless: ga` | +| Use rules built for {{elastic-sec}}, {{observability}}, APM, or Maps | [{{kib}} alerting](alerts.md) | {applies_to}`stack: ga` {applies_to}`serverless: ga` | +| Write {{esql}} to define exactly what to detect and what data each alert episode carries | [{{alerting-v2-system-cap}}](experimental-alerting-system.md) | {applies_to}`serverless: experimental` {applies_to}`stack: experimental 9.5+` | +| Query alert history in Discover or build dashboards from alert data | [{{alerting-v2-system-cap}}](experimental-alerting-system.md) | {applies_to}`serverless: experimental` {applies_to}`stack: experimental 9.5+` | +| Manage notification routing, grouping, and throttling in one place, reusable across rules | [{{alerting-v2-system-cap}}](experimental-alerting-system.md) | {applies_to}`serverless: experimental` {applies_to}`stack: experimental 9.5+` | +| Build highly custom notification logic with reusable, configurable workflows | [{{alerting-v2-system-cap}}](experimental-alerting-system.md) | {applies_to}`serverless: experimental` {applies_to}`stack: experimental 9.5+` | +| Build highly custom logic with scripting and chained inputs | [Watcher](watcher.md) | {applies_to}`stack: ga` {applies_to}`serverless: unavailable` | + +## Compare at a glance + +| | {{kib}} alerting | {{alerting-v2-system-cap}} | Watcher | +|---|---|---|---| +| **Best for** | Teams using built-in rule types with form-based setup | Teams that need full control over detection and notification routing | Custom alerting logic requiring scripting | +| **Rule definition** | Select a rule type and fill in parameters | Write an {{esql}} query or use a rule builder with form-based setup | Write a JSON watch definition | +| **Alert data** | In-place updates, limited query support | Append-only events queryable with {{esql}} in Discover | Watch history index | +| **Notifications** | Configured per action on each rule | Centralized action policies, reusable across rules; supports action-level throttling and conditions | Action-level throttling and conditions | +| **Noise reduction** | Snooze per rule, maintenance windows | Per-episode acknowledge or deactivate, per-series snooze, maintenance windows, match condition routing in action policies | Action conditions and throttling | +| **Available on {{serverless-full}}** | Yes | Yes, {applies_to}`serverless: experimental` | No | +| **Available on {{stack}}** | Yes | Yes, {applies_to}`stack: experimental 9.5+` | Yes | + + diff --git a/explore-analyze/alerting/experimental-alerting-system.md b/explore-analyze/alerting/experimental-alerting-system.md new file mode 100644 index 0000000000..0b721f8905 --- /dev/null +++ b/explore-analyze/alerting/experimental-alerting-system.md @@ -0,0 +1,88 @@ +--- +navigation_title: Experimental alerting system +applies_to: + stack: experimental 9.5+ + serverless: experimental +products: + - id: kibana + - id: cloud-serverless +description: The experimental Kibana alerting system uses ES|QL rules to detect conditions, track problems as alert episodes, and route notifications through reusable action policies. +--- + +# {{alerting-v2-system-cap}} overview [experimental-alerting-system-overview] + +The {{alerting-v2-system}} in {{kib}} watches your {{es}} data continuously, so your team doesn't have to. You define the conditions that matter, such as when to open an issue, who should know, and how often to notify them. The system handles the rest. + +::::{note} +In the generally available {{kib}} alerting system, the term *alert* refers to a tracked occurrence of a rule condition. In the {{alerting-v2-system}}, the equivalent concept is called an *alert episode*. The two terms describe similar ideas in different systems and are not interchangeable. +:::: + +## The core idea [experimental-alerting-system-core-idea] + +The {{alerting-v2-system}} separates *detecting* a problem from *acting* on it: + +- **Detecting** - Rules focus purely on what to watch for in your data and on collecting breach and recovery events. +- **Acting** - Action policies handle who gets notified, when, and how, independently of any rule. + +You can build and test detection logic before wiring up any notifications, and update notification routing across all rules in one place without editing the rules themselves. + +## The four building blocks + +The {{alerting-v2-system}} is built around four objects: rules, alert episodes, action policies, and workflows, each with a distinct role. + +### Rules + +A rule defines what to watch for in your data and how often to check. Every rule runs in one of two modes: alert or signal. + +- **Alert** - Opens an alert episode when the rule finds a match and closes it when the condition clears, notifying your team at each episode state change. Helpful when you want to follow a problem from first detection to resolution. +- **Signal** - Records rule query results over time without opening episodes or sending notifications. Helps you build a baseline, spot trends, or collect evidence before deciding whether something is worth alerting on. + + + +### Alert episodes + +In Alert mode, the rule opens one alert episode per problem and keeps it open until the condition clears. The alert episode moves through states (pending, active, recovering, inactive) giving you one lifecycle to triage rather than a separate item per rule check. + + + +### Action policies + +An action policy is the gating layer between an alert episode and a workflow. It decides whether and when to invoke a workflow by evaluating suppression, match conditions, and frequency. Policy configuration determines the scope. A policy can apply to alert episodes from a specific rule, multiple rules, or all rules in the space. + + + +### Workflows + +A workflow is what actually sends the message or runs the automation, for example, posting to Slack, sending an email, calling a webhook. The {{alerting-v2-system}} invokes workflows through action policies that you configure to trigger on a schedule or on a state change on an alert episode. + + + +## How the pieces fit together [experimental-alerting-system-how-pieces-fit-together] + +At the simplest level: + +1. A rule checks your data on a schedule. +2. The rule's query returns results when data matching its conditions is found. +3. The rule's mode determines what happens next: + - Alert - The rule opens an alert episode to track the problem. An action policy can route it to a workflow to perform an action or send a notification. + - Signal - Each result is recorded for querying later. Nothing else happens. + +For a more detailed explanation of each stage, refer to [How the {{alerting-v2-system}} works](kibana-alerting-experimental/how-it-works.md). + +## Next steps + +To understand how the {{alerting-v2-system}} fits into {{kib}}'s alerting options, refer to [Alerting](../alerting.md) or [Compare alerting systems](compare-alerting-systems.md). + + \ No newline at end of file diff --git a/explore-analyze/alerting/kibana-alerting-experimental/glossary.md b/explore-analyze/alerting/kibana-alerting-experimental/glossary.md new file mode 100644 index 0000000000..446314007a --- /dev/null +++ b/explore-analyze/alerting/kibana-alerting-experimental/glossary.md @@ -0,0 +1,71 @@ +--- +navigation_title: Glossary +applies_to: + stack: experimental 9.5+ + serverless: experimental +products: + - id: kibana + - id: cloud-serverless +description: Definitions of key terms used throughout the experimental Kibana alerting system documentation. +--- + +# {{alerting-v2-system-cap}} glossary [experimental-alerting-system-glossary] + +These terms appear throughout the {{alerting-v2-system}} docs. If a term is unclear while reading, check its definition here before going further. + +**Action policy** +: A configuration that controls whether and how often an alert episode triggers a notification, including which alerts qualify and how to avoid sending too many notifications. A single action policy can apply to one rule, several rules, or all rules in the space. + + +**Alert episode** +: The complete record of one problem tracked in Alert mode, from first detection to recovery, moving through states (pending, active, recovering, inactive). + +**Breach** +: A single instance when a rule's query finds a match, which may or may not open an alert episode depending on how the rule is configured. + +**Dispatcher** +: The background process that evaluates action policies against active alert episodes on a short interval (around 5 seconds), independent of the rule schedule. + + +**{{esql}}** +: The query language every rule uses to search your data. To learn more, refer to the [{{esql}} reference](elasticsearch://reference/query-languages/esql.md). + +**Notification** +: The message or action delivered when an alert episode matches an action policy and a workflow sends it, such as a Slack message, an email, or a webhook call. + + +**Rule** +: The definition of what to watch for in your data, how often to check, and what counts as a match; runs on a schedule and produces signals (Signal mode) or tracks alert episodes (Alert mode). + + +**Rule event** +: A record written to `.rule-events` every time a rule runs and its query finds a match; in Signal mode it is a signal, in Alert mode it belongs to an alert episode. + +**Severity** +: A label attached to alert episodes to indicate urgency; available as a filter in action policies so critical episodes can be routed differently from low-priority ones. + + +**Signal** +: A rule event recorded in Signal mode; stored and queryable in Discover but doesn't open an alert episode or trigger notifications. + +**Threshold** +: The condition a rule uses to decide when something is worth alerting on, including how many times the condition must be met before an alert episode opens or closes. + + +**Workflow** +: The automation that sends a message or runs an action when an action policy decides a notification should go out, such as posting to Slack, sending an email, or calling a webhook. + diff --git a/explore-analyze/alerting/kibana-alerting-experimental/how-it-works.md b/explore-analyze/alerting/kibana-alerting-experimental/how-it-works.md new file mode 100644 index 0000000000..37271fb2b8 --- /dev/null +++ b/explore-analyze/alerting/kibana-alerting-experimental/how-it-works.md @@ -0,0 +1,66 @@ +--- +navigation_title: How it works +applies_to: + stack: experimental 9.5+ + serverless: experimental +products: + - id: kibana + - id: cloud-serverless +description: A detailed walkthrough of how Alert mode and Signal mode rules process data, produce rule events, and drive alert episodes, action policies, and notifications in the experimental alerting system. +--- + +# How the {{alerting-v2-system}} works [experimental-alerting-system-how-it-works] + +This page walks through what happens at each step after a rule runs, and broken down by mode. Use it to understand how the different components of the {{alerting-v2-system}} interact. + +## Rule runs in Alert mode [experimental-alerting-system-how-alert-mode-works] + +In Alert mode, the rule doesn't just record that a condition was found. It opens an alert episode that persists and tracks the problem until the condition clears. Each time the rule runs, it writes a rule event that can advance the episode's lifecycle state. An action policy sits between the episode and your team, deciding whether and when to trigger a workflow. + +| Step | Actor | Action | +|------|-------|--------| +| 1 | Rule | Runs on schedule and evaluates {{esql}} against your data | +| 2 | Rule | Query returns results → A rule event is written to `.rule-events` | +| 3 | System | Creates an alert episode and sets its initial state to `pending`; episode advances to `active` once the activation threshold is met | +| 4 | Action policy | Evaluates the episode against its conditions (checks for suppression, match conditions, and frequency) | +| 5 | Action policy | If conditions are met, triggers a workflow | +| 6 | Workflow | Sends notification or runs automation | +| 7 | Rule | Condition clears → New rule event written → Episode moves to `recovering` → `inactive` | +| 8 | Action policy | Evaluates recovery event and triggers a workflow if conditions are met | +| 9 | Workflow | Sends the recovery notification | + +:::{note} +Steps 4–6 and 8–9 run on a separate background process that polls roughly every 5 seconds. Action policy evaluation is not triggered synchronously by the rule's own execution. There is always at least one dispatcher polling cycle between a rule run and any resulting notification. +::: + +### Example: Latency monitoring in Alert mode + +An SRE team wants to know when checkout service latency degrades, and notify the on-call team when it does. The team creates an Alert mode rule: + +1. The rule runs an {{esql}} query every five minutes, checking p95 checkout service latency. +2. When p95 exceeds 2 seconds for more than one consecutive check, the rule opens an alert episode. +3. An action policy with a `rule.tags: "checkout"` matcher skips low-severity episodes and sends a Slack message through an on-call workflow. + +The engineer investigates, fixes a slow query, and the alert episode recovers automatically. + +## Rule runs in Signal mode [experimental-alerting-system-how-signal-mode-works] + +In Signal mode, the rule acts purely as a data producer. Each time the rule runs and its query returns results, it writes a rule event to `.rule-events` and stops. Signals accumulate over time and are immediately queryable in Discover for incident investigation, or as inputs to Alert mode rules that detect correlated activity across multiple signals. + +| Step | Actor | Action | +|------|-------|--------| +| 1 | Rule | Runs on schedule and evaluates {{esql}} against your data | +| 2 | Rule | Query returns results → Writes a rule event (signal) to `.rule-events` | +| 3 | System | Signal is immediately queryable in Discover, dashboards, and {{esql}} | + +No alert episode is opened. No action policy evaluates the result. No notification is sent. + +### Example: Tracking administrator API calls in Signal mode + +A security team wants to track calls to a rarely-used administrator API endpoint, but individual calls aren't suspicious enough to page anyone. To start collecting data without generating noise, the team creates a Signal mode rule: + +1. The rule runs an {{esql}} query on a schedule, checking for calls to the administrator API endpoint. +2. Each time the query returns results, the rule writes a signal to `.rule-events`. +3. The signals accumulate silently and are immediately queryable in Discover. + +After a few weeks, the accumulated signals become useful in two ways. The team can write an Alert mode rule that combines admin API calls with other signals (such as a spike in error rates) to catch correlated activity that neither signal would surface on its own. When an outage happens, the team can query the signal history as evidence directly in Discover, without reconstructing the original query or worrying that the source data has become stale. \ No newline at end of file diff --git a/explore-analyze/toc.yml b/explore-analyze/toc.yml index 6ba6d30986..86981bd3b5 100644 --- a/explore-analyze/toc.yml +++ b/explore-analyze/toc.yml @@ -385,6 +385,11 @@ toc: - file: report-and-share/reporting-troubleshooting-pdf.md - file: alerting.md children: + - file: alerting/compare-alerting-systems.md + - file: alerting/experimental-alerting-system.md + children: + - file: alerting/kibana-alerting-experimental/how-it-works.md + - file: alerting/kibana-alerting-experimental/glossary.md - file: alerting/alerts.md children: - file: alerting/alerts/alerting-getting-started.md