You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[anomalydetection] Unify scorer, make it a correlator (4) (#52492)
### What does this PR do?
Unifies the anomaly scorer into a single `anomalyScorer` struct implementing the `Correlator` interface in a single file. We can now benchmark the scorer. This removes the helper reporting logs, it's merged into this struct.
This contains the main score logic and also the subscribe logic which is used to create events (high severity = event).
> [!NOTE]
> We use "the severity becomes high" and "the severity isn't high anymore" as start / end of correlation. This will be reworked in #52493 to directly emit event at the start.
The config can now enable/disable logs and reporter events (telemetry always enabled if component enabled).
New config keys for output:
```yaml
anomaly_detection:
anomaly_scorer:
enabled: true
output:
logs: true
correlation_events: false
```
#### Diagram
```mermaid
flowchart LR
IN["ProcessAnomaly(a)"] --> PEND["pending map"]
PEND --> ADV
subgraph ADV["Advance(t) — mu held"]
EWMA["merge → evict → bucket → EWMA"]
end
ADV -->|"[]secEWMA\nmu released"| SUBS
subgraph SUBS["Per-subscription state machines — subsMu"]
FSM["Low / Medium / High FSM\n+ cooldown per sub"]
FSM -->|"transition"| CB["Listener.OnSeverityTransition(evt)"]
end
CB -->|"self-sub\n(internal watcher)"| IW["gauges · logs · episode tracking"]
IW --> AC["ActiveCorrelations()"]
ADV --> SC["ScoreState() / LastScore()"]
SUBS --> SB["Subscribe(cfg) → func()\nexposes via SubscribeScorer on Component"]
```
### Motivation
Nice score for kafka scenario, no false positives (using bocpd+tukey_biweight+holt_residual):
<img width="531" height="304" alt="Screenshot 2026-06-19 at 11 20 44" src="https://github.com/user-attachments/assets/ace05855-359a-4064-b42c-994b6baf5a1e" />
### Describe how you validated your changes
### Additional Notes
Co-authored-by: celian.raimbault <celian.raimbault@datadoghq.com>
|`impl/patterns/`| Tokenizer + clusterer used by log pattern extractor |
46
47
47
48
### Component catalog (defaults)
@@ -58,8 +59,27 @@ Registered in `impl/component_catalog.go`. Enabled by default unless noted:
58
59
| Detector |`cusum`, `scanmw`, `scanwelch`, `holt_residual`, `tukey_biweight`| off |
59
60
| Correlator |`time_cluster`| on |
60
61
| Correlator |`cross_signal`, `passthrough`| off |
62
+
| Correlator |`anomaly_scorer`| off |
63
+
64
+
Toggle detectors/correlators/extractors via `anomaly_detection.detectors.<name>.enabled` in datadog.yaml.
65
+
66
+
The `anomaly_scorer` correlator has a **dedicated config namespace** under `anomaly_detection.anomaly_scorer.*` (not `detectors.*`) with an `output` sub-section controlling logs and correlation events:
67
+
68
+
```yaml
69
+
anomaly_detection:
70
+
anomaly_scorer:
71
+
enabled: true
72
+
alpha: 0.3
73
+
window_secs: 30
74
+
low_threshold: 0.030
75
+
high_threshold: 0.060
76
+
output:
77
+
logs: true
78
+
correlation_events: false
79
+
cooldown_secs: 300
80
+
```
61
81
62
-
Toggle via `anomaly_detection.detectors.<name>.enabled` in datadog.yaml.
82
+
The scorer is also available standalone (without the engine) via `NewAnomalyScorer` in `impl/` for testbench replay.
0 commit comments