Ingest new documentation

Author: netdatabot

docs/Alerts & Notifications/Alert Configuration Reference.mdx

@@ -1038,7 +1038,7 @@ Several stock health configurations use host variables to reference dimensions f
 
 ##### Prometheus Collector Variables
 
-For metrics collected by the go.d `prometheus` collector, each unique Prometheus label set usually produces a separate chart. The chart ID is built from the metric name followed by `-label=value` pairs for every label (e.g. `kubelet_volume_stats_used_bytes-persistentvolumeclaim=my-pvc`). In the Netdata chart registry, the prefix comes from the go.d job `FullName`: it is `prometheus.<metric_name>-<label_set>` only when the job name is literally `prometheus`; otherwise it is `prometheus_<job_name>.<metric_name>-<label_set>` (for example, `prometheus_local.<metric_name>-<label_set>` or `prometheus_kubelet.<metric_name>-<label_set>`). For summary and histogram metric families, the collector may also emit related chart IDs such as `<id>`, `<id>_sum`, and `<id>_count`, so verify the exact chart ID you want to reference.
+For metrics collected by the go.d `prometheus` collector, each unique Prometheus label set usually produces a separate chart. The chart ID is built from the metric name followed by `-label=value` pairs for every label (e.g. `kubelet_volume_stats_used_bytes-persistentvolumeclaim=my-pvc`); characters in a label value that are not chart-ID-safe, such as `.`, are replaced with `_` in the chart ID, while the chart's label keeps the original value (so `addr="10.0.0.1"` yields `…-addr=10_0_0_1`). In the Netdata chart registry, the prefix comes from the go.d job `FullName`: it is `prometheus.<metric_name>-<label_set>` only when the job name is literally `prometheus`; otherwise it is `prometheus_<job_name>.<metric_name>-<label_set>` (for example, `prometheus_local.<metric_name>-<label_set>` or `prometheus_kubelet.<metric_name>-<label_set>`). Summary and histogram families also emit separate `_sum` and `_count` charts; the suffix is part of the metric name, so the IDs are `<metric_name>_sum-<label_set>` and `<metric_name>_count-<label_set>` (just `<metric_name>_sum` / `<metric_name>_count` when the series has no labels), while histogram buckets are dimensions of the base `<metric_name>` chart. Verify the exact chart ID you want to reference.
 
 Because Prometheus chart IDs typically contain hyphens and `=` characters, use the `${...}` brace form to reference them in `calc`/`warn`/`crit` expressions — the unbraced `$var` form stops parsing at `-`. Apply the same rule for both the common `prometheus_<job_name>` prefix and the special-case plain `prometheus` prefix, including any `_sum` or `_count` chart variants.
 

docs/Alerts & Notifications/Alerts & Notifications.mdx

@@ -3,7 +3,7 @@ custom_edit_url: "https://github.com/netdata/netdata/edit/master/src/health/READ
 sidebar_label: "Alerts & Notifications"
 learn_status: "Published"
 learn_rel_path: "Alerts & Notifications"
-sidebar_position: "140"
+sidebar_position: "150"
 learn_link: "https://learn.netdata.cloud/docs/alerts-&-notifications"
 slug: "/alerts-&-notifications"
 ---

docs/Alerts & Notifications/Notifications/Agent Dispatched Notifications/Agent Notifications Reference.mdx

@@ -100,6 +100,7 @@ Use the `edit-config` script to safely edit configuration files. It automaticall
 :::
 
 1. Open the Agent's health notification config:
+
    ```bash
    sudo ./edit-config health_alarm_notify.conf
    ```
@@ -109,6 +110,7 @@ Use the `edit-config` script to safely edit configuration files. It automaticall
 3. Define recipients per **role** (see below).
 
 4. Restart the Agent for changes to take effect:
+
    ```bash
    sudo systemctl restart netdata
    ```
@@ -286,7 +288,7 @@ role_recipients_email[sysadmin]="disabled"
 If left empty, the default recipient for that method is used.
 </details>
 
-<details>
+<details id="alert-severity-filtering">
 <summary><strong>Alert Severity Filtering</strong></summary><br/>
 
 You can limit certain recipients to only receive **critical** alerts:
@@ -298,11 +300,47 @@ role_recipients_email[sysadmin]="user1@example.com user2@example.com|critical"
 This setup:
 
 - Sends all alerts to `user1@example.com`
-- Sends only critical-related alerts to `user2@example.com`
+- Sends notifications to `user2@example.com` only once the alarm reaches CRITICAL, then continues sending status changes (including WARNING and CLEAR) until the alarm is cleared.
 
 Works for all supported methods: email, Slack, Telegram, Twilio, Discord, etc.
 </details>
 
+<details>
+<summary><strong>Controlling Recovered (CLEAR) Notifications</strong></summary><br/>
+
+When an alert returns to normal, Netdata sends a **CLEAR** (recovered) notification. You can control when and whether these are sent.
+
+**Default behavior:** Netdata suppresses CLEAR notifications when the alert was never in a WARNING or CRITICAL state. If `old_status` was not WARNING or CRITICAL and the alert transitions to CLEAR, no notification is sent. This prevents noise from alerts that flap without ever reaching a problem state.
+
+**Enable CLEAR for all transitions:** If your downstream system handles deduplication, set `clear_alarm_always` in `health_alarm_notify.conf` to override the default suppression and send a CLEAR notification regardless of the previous status:
+
+```ini
+clear_alarm_always='YES'
+```
+
+**Filter by CRITICAL history with the `|critical` modifier:** As described in [Alert Severity Filtering](#alert-severity-filtering) above, `|critical` forwards notifications only for alerts that have reached CRITICAL status. This affects both WARNING and CLEAR:
+
+- **WARNING** notifications are suppressed unless the alarm has previously reached CRITICAL.
+- **CLEAR** notifications are only sent when the alert previously passed through CRITICAL. If the alert only went through WARNING → CLEAR, the CLEAR is not forwarded.
+
+```ini
+role_recipients_email[sysadmin]="admin@example.com|critical"
+```
+
+**Suppress all CLEAR notifications:** Use the `|noclear` modifier to completely block CLEAR notifications for a recipient while still receiving WARNING and CRITICAL alerts:
+
+```ini
+role_recipients_email[sysadmin]="admin@example.com|noclear"
+```
+
+You can combine modifiers. This example notifies only for alarms that have reached CRITICAL (WARNING is suppressed until then), and excludes CLEAR notifications entirely:
+
+```ini
+role_recipients_email[sysadmin]="admin@example.com|critical|noclear"
+```
+
+</details>
+
 <details>
 <summary><strong>Proxy Settings</strong></summary><br/>
 
@@ -411,21 +449,25 @@ Here are solutions for common alert notification issues:
 ### Email Notifications Not Working
 
 1. Verify your email configuration:
+
    ```bash
    grep -E "SEND_EMAIL|DEFAULT_RECIPIENT_EMAIL" /etc/netdata/health_alarm_notify.conf
    ```
 
 2. Check if the system can send mail:
+
    ```bash
    echo "Test" | mail -s "Test Email" your@email.com
    ```
 
 3. Look for errors in the Netdata log:
+
    ```bash
    tail -f /var/log/netdata/error.log | grep "alarm notify"
    ```
 
 4. Test with debugging enabled:
+
    ```bash
    sudo su -s /bin/bash netdata
    export NETDATA_ALARM_NOTIFY_DEBUG=1
@@ -435,11 +477,13 @@ Here are solutions for common alert notification issues:
 ### Slack Notifications Failing
 
 1. Verify your webhook URL is correct:
+
    ```bash
    grep -E "SLACK_WEBHOOK_URL" /etc/netdata/health_alarm_notify.conf
    ```
 
 2. Check for network connectivity to Slack:
+
    ```bash
    curl -X POST -H "Content-type: application/json" --data '{"text":"Test"}' YOUR_WEBHOOK_URL
    ```
@@ -449,11 +493,13 @@ Here are solutions for common alert notification issues:
 ### PagerDuty Integration Issues
 
 1. Verify your service key:
+
    ```bash
    grep -E "PAGERDUTY_SERVICE_KEY" /etc/netdata/health_alarm_notify.conf
    ```
 
 2. Test the PagerDuty API directly:
+
    ```bash
    curl -H "Content-Type: application/json" -X POST -d '{"service_key":"YOUR_SERVICE_KEY","event_type":"trigger","description":"Test"}' https://events.pagerduty.com/generic/2010-04-15/create_event.json
    ```

docs/Alerts & Notifications/Notifications/Centralized Cloud Notifications/Webhook.mdx

@@ -105,13 +105,13 @@ By default, the following headers will be sent in the HTTP request
 
 Netdata webhook integration supports 3 different authentication mechanisms:
 
-##### Mutual TLS authentication (recommended)
+##### Mutual TLS authentication
 
-In mutual Transport Layer Security (mTLS) authentication, the client and the server authenticate each other using X.509 certificates. This ensures that the client is connecting to the intended server, and that the server is only accepting connections from authorized clients.
+Netdata always sends a client certificate with every webhook request, regardless of which authentication method is selected in the UI. This means mTLS is available on all webhook integrations by default — no additional configuration is needed on the Netdata side to enable it.
 
-This is the default authentication mechanism used if no other method is selected.
+The authentication method you select (no auth, basic, or bearer) controls only whether an Authorization header is included in the request. It does not affect the client certificate behavior.
 
-To take advantage of mutual TLS, you can configure your server to verify Netdata's client certificate. In order to achieve this, the Netdata client sending the notification supports mutual TLS (mTLS) to identify itself with a client certificate that your server can validate.
+If you want to verify Netdata's client certificate on your end, configure your server to validate it using the Netdata CA certificate below.
 
 The steps to perform this validation are as follows:
 

docs/Collecting Metrics/Collectors configuration.mdx

@@ -58,7 +58,7 @@ You can modify how often collectors gather metrics to optimize CPU usage. This c
 1. Open `netdata.conf` using [`edit-config`](/docs/netdata-agent/configuration#edit-configuration-files).
 2. Set the `update every` value (default is `1`, meaning one-second intervals):
     ```text
-    [global]
+    [db]
         update every = 2
     ```
 

docs/Collecting Metrics/Collectors/Applications/Alamos FE2 server.mdx

@@ -94,11 +94,14 @@ The following options can be defined globally: update_every, autodetection_retry
 |  | autodetection_retry | Autodetection retry interval (seconds). Set 0 to disable. | 0 | no |
 | **Target** | url | Target endpoint URL. |  | yes |
 |  | timeout | HTTP request timeout (seconds). | 10 | no |
+|  | expected_prefix | If set, the job's check passes only when at least one scraped metric name starts with this prefix. Guards against scraping an unexpected endpoint. |  | no |
+| **Customization** | app | Application name used as the app segment of chart contexts (`prometheus.<app>.<metric>`). When unset, it is taken from a matched profile, otherwise it falls back to the job name. |  | no |
 | **Filters** | [selector](#option-filters-selector) | Time series selector (filter). |  | no |
 | **Limits** | max_time_series | Global time series limit. If an endpoint returns more time series than this, the data is not processed. | 2000 | no |
 |  | max_time_series_per_metric | Per-metric time series limit. Metrics with more time series than this are skipped. | 200 | no |
 | **Customization** | [fallback_type](#option-customization-fallback-type) | Fallback type rules for untyped metrics. |  | no |
-|  | label_prefix | Optional prefix added to all labels of all charts. Labels will be formatted as `prefix_name`. |  | no |
+|  | [relabeling](#option-customization-relabeling) | Prometheus-compatible metric relabeling, applied before charts are built. |  | no |
+|  | [profiles](#option-customization-profiles) | Curated, exporter-specific chart profiles. Disable with mode `none`. | auto | no |
 | **HTTP Auth** | username | Username for Basic HTTP authentication. |  | no |
 |  | password | Password for Basic HTTP authentication. |  | no |
 |  | bearer_token_file | Path to a file containing a bearer token (used for `Authorization: Bearer`). |  | no |
@@ -155,6 +158,57 @@ fallback_type:
 ```
 
 
+<a id="option-customization-relabeling"></a>
+##### relabeling
+
+A list of relabeling blocks. Each block applies a list of Prometheus
+`metric_relabel_configs` rules to the metrics whose name matches `match`. See the
+[relabeling reference](https://github.com/netdata/netdata/blob/master/src/go/plugin/go.d/collector/prometheus/relabel/README.md)
+for the full action set and more examples.
+
+- `match`: Netdata simple patterns matched against the full metric name — including
+  any `_bucket`/`_sum`/`_count` suffix, so prefer globs like `app_lat*` over an exact
+  `app_lat` (space-separated; `*` matches any sequence, `?` any character, a leading
+  `!` negates). Use `*` to target every metric. Required.
+- `metric_relabel_configs`: Prometheus relabel rules (`source_labels`, `separator`,
+  `regex`, `modulus`, `target_label`, `replacement`, `action`), applied in order to
+  the scraped samples before charts are built.
+
+Relabeling that would corrupt a histogram or summary — splitting it, dropping a
+component, mutating the `le`/`quantile` label, or merging two families — is rejected.
+
+```yaml
+relabeling:
+  - match: 'http_*'
+    metric_relabel_configs:
+      - source_labels: [code]
+        regex: '(\d)\d\d'
+        target_label: code_class
+        replacement: '${1}xx'
+```
+
+
+<a id="option-customization-profiles"></a>
+##### profiles
+
+Profiles ship curated charts for recognized exporters. `profiles.mode` selects them:
+
+- `auto` (default): every profile whose `match` hits at least one scraped metric.
+- `exact`: only the profiles named in `mode_exact.entries` (each must match, or the job fails its check).
+- `combined`: `auto` plus the profiles named in `mode_combined.entries`.
+- `none`: no profiles — generic autogen charts only (the pre-profile behavior).
+
+Only the block matching the selected mode (`mode_exact` or `mode_combined`) is read; entries under the other block are ignored. Metrics not covered by a selected profile keep their generic autogen charts.
+
+```yaml
+profiles:
+  mode: exact
+  mode_exact:
+    entries:
+      - name: haproxy
+```
+
+
 
 </details>
 
@@ -286,6 +340,55 @@ jobs:
 ```
 </details>
 
+###### Metric relabeling
+
+Derive a `code_class` label (2xx, 4xx, ...) on metrics named `http_*`.
+
+<details open>
+<summary>Config</summary>
+
+```yaml
+jobs:
+  - name: local
+    url: http://127.0.0.1:9090/metrics
+    relabeling:
+      - match: 'http_*'
+        metric_relabel_configs:
+          - source_labels: [code]
+            regex: '(\d)\d\d'
+            target_label: code_class
+            replacement: '${1}xx'
+
+```
+</details>
+
+###### Rename labels that collide with Netdata's reserved labels
+
+When these metrics are re-exported in Prometheus format, Netdata adds its own `instance`,
+`family`, `chart`, and `dimension` labels. If the scraped endpoint already uses one of those
+names, the re-export emits a duplicate label and a downstream Prometheus rejects the scrape.
+Rename the colliding labels to avoid it (the use case the former `label_prefix` option served).
+
+
+<details open>
+<summary>Config</summary>
+
+```yaml
+jobs:
+  - name: coredns
+    url: http://127.0.0.1:9153/metrics
+    relabeling:
+      - match: '*'
+        metric_relabel_configs:
+          - regex: '(instance|family)'
+            action: labelmap
+            replacement: 'coredns_$1'
+          - regex: '(instance|family)'
+            action: labeldrop
+
+```
+</details>
+
 
 
 ## Alerts