Skip to content

Commit 86bb29c

Browse files
authored
[filebeat] fix running on kubernets docs (#47599)
* [filebeat] adjust running on kubernetes docs Adjust the docs to correct the inputs configuration snippets: - remove them from the OpenShift section as they are valid for k8s and OpenSHift - fix the issues on the snippets
1 parent b56b8c5 commit 86bb29c

1 file changed

Lines changed: 46 additions & 37 deletions

File tree

docs/reference/filebeat/running-on-kubernetes.md

Lines changed: 46 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,52 @@ curl -L -O https://raw.githubusercontent.com/elastic/beats/{{ version.stack | M.
3232
**If you are using Kubernetes 1.7 or earlier:** Filebeat uses a hostPath volume to persist internal data. It’s located under `/var/lib/filebeat-data`. The manifest uses folder autocreation (`DirectoryOrCreate`), which was introduced in Kubernetes 1.8. You need to remove `type: DirectoryOrCreate` from the manifest, and create the host folder yourself.
3333
::::
3434

35+
To support runtime environments different from Docker, like CRI-O or containerd, configure the `paths` as follows:
36+
37+
### A single filestream input for all container logs
38+
39+
```yaml
40+
filebeat.inputs:
41+
- type: filestream
42+
id: container-logs <1>
43+
prospector.scanner.symlinks: true <2>
44+
parsers:
45+
- container: ~
46+
paths:
47+
- /var/log/containers/*.log <3>
48+
processors:
49+
- add_kubernetes_metadata:
50+
host: ${NODE_NAME}
51+
matchers:
52+
- logs_path:
53+
logs_path: /var/log/containers/
54+
```
55+
1. All `filestream` inputs require a unique ID. One input will be created for all container logs.
56+
2. Container logs use symlinks, so they need to be enabled.
57+
3. Path for all container logs.
58+
59+
### One filestream input per container using autodiscover:
60+
61+
```yaml
62+
filebeat.autodiscover:
63+
providers:
64+
- type: kubernetes
65+
node: ${NODE_NAME}
66+
hints.enabled: true
67+
hints.default_config:
68+
type: filestream
69+
id: container-${data.kubernetes.container.id} <1>
70+
prospector.scanner.symlinks: true <2>
71+
parsers:
72+
- container: ~
73+
paths:
74+
- /var/log/containers/*-${data.kubernetes.container.id}.log <3>
75+
```
76+
77+
1. All `filestream` inputs require a unique ID.
78+
2. Container logs use symlinks, so they need to be enabled.
79+
3. A path for each container, so the input will only ingest the logs from its
80+
container.
3581

3682
## Settings [_settings]
3783

@@ -90,43 +136,6 @@ If you are using Red Hat OpenShift, you need to specify additional settings in t
90136

91137
This command sets the node selector for the project to an empty string. If you don’t run this command, the default node selector will skip control plane nodes.
92138

93-
In order to support runtime environments with Openshift (for example, CRI-O, containerd), you need to configure the following path:
94-
95-
```yaml
96-
filebeat.inputs:
97-
- type: filestream
98-
id: container-${data.kubernetes.container.id} <1>
99-
prospector.scanner.symlinks: true <2>
100-
parsers:
101-
- container: ~
102-
paths: <3>
103-
- /var/log/containers/*.log
104-
```
105-
1. All `filestream` inputs require a unique ID.
106-
2. Container logs use symlinks, so they need to be enabled.
107-
3. The same path needs to be configured in the autodiscover settings, if enabled:
108-
109-
```yaml
110-
filebeat.autodiscover:
111-
providers:
112-
- type: kubernetes
113-
node: ${NODE_NAME}
114-
hints.enabled: true
115-
hints.default_config:
116-
type: filestream
117-
id: container-${data.kubernetes.container.id}
118-
prospector.scanner.symlinks: true
119-
parsers:
120-
- container: ~
121-
paths:
122-
- /var/log/containers/*.log
123-
```
124-
125-
::::{note}
126-
`/var/log/containers/\*.log` is normally a symlink to `/var/log/pods/*/*.log`, so `paths` can be edited accordingly.
127-
::::
128-
129-
130139
## Load {{kib}} dashboards [_load_kib_dashboards]
131140

132141
Filebeat comes packaged with various pre-built {{kib}} dashboards that you can use to visualize logs from your Kubernetes environment.

0 commit comments

Comments
 (0)