Skip to content

Commit 2976a07

Browse files
committed
docs(monitoring): mention labels and typical setup
1 parent 614bb66 commit 2976a07

1 file changed

Lines changed: 26 additions & 4 deletions

File tree

docs/04-For Operators/05-monitoring.md

Lines changed: 26 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -22,11 +22,33 @@ configuration parameters for the control plane in the control plane's
2222
[logging](https://github.com/metal-stack/metal-roles/blob/master/control-plane/roles/logging/README.md)
2323
role.
2424

25-
In the partitions, Alloy is deployed inside a systemd-managed Docker
26-
container. Configuration parameters can be found in the partition's
25+
In the partitions, Alloy can be deployed inside a systemd-managed Docker
26+
container on management servers and switches. Configuration parameters can be found in the partition's
2727
[alloy](https://github.com/metal-stack/metal-roles/blob/master/partition/roles/alloy/README.md)
28-
role. Which hosts Alloy collects from can be configured via the
29-
`prometheus_alloy_targets` variable.
28+
role.
29+
30+
### Partition Log Sources
31+
32+
Alloy is configured through snippets that define what logs are collected. The following snippets are typically used:
33+
34+
| Host type | Snippet | Description | Key labels |
35+
| ---------------------- | -------------- | ------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------- |
36+
| Leaves, spines, exits | `journal` | Collects logs from the systemd journal; auto-discovers both volatile (`/run/log/journal`) and persistent (`/var/log/journal`) storage | `job=systemd-journal`, `unit`, `level` |
37+
| Management servers | `journal-file` | Collects logs from the persistent systemd journal at a configurable path; supports migrating cursor position from promtail | `job=systemd-journal`, `unit`, `level` |
38+
| Hosts without journald | `syslog` | Tails `/var/log/syslog` | `job=syslog` |
39+
| Hosts running Docker | `docker` | Collects logs from all Docker containers via the Docker socket | `job=docker`, `container` |
40+
41+
All log entries carry the `host` and `partition` labels regardless of snippet, which makes it easy to filter logs in Grafana Explore by host or partition.
42+
43+
### Querying Logs in Grafana
44+
45+
Logs can be explored in Grafana using the **Explore** view with the Loki data source. Useful label filters:
46+
47+
- `{partition="<partition-id>"}` — all logs from a partition
48+
- `{host="<hostname>"}` — all logs from a specific host
49+
- `{job="docker", container="<name>"}` — logs from a specific Docker container
50+
- `{job="systemd-journal", unit="<unit>.service"}` — logs from a specific systemd unit
51+
- `{job="systemd-journal", level="error"}` — error-level journal entries across all units
3052

3153
:::note Migrating from promtail
3254

0 commit comments

Comments
 (0)