You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: crowdsec-docs/unversioned/user_guides/log_centralization.md
+29-29Lines changed: 29 additions & 29 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,35 +8,35 @@ sidebar_position: 10
8
8
9
9
If you expose services on the internet from multiple servers, setting up crowdsec on all of them might make the overall setup more complex.
10
10
11
-
In order to keep things simpler, you can use a central server that will receive all your logs, and only run a single instance of crowdsec on this server
11
+
To simplify things, you can use a central server to receive all your logs and only run a single instance of crowdsec on this server.
12
12
13
-
In this guide, our goal is to centralize 2 types of logs:
13
+
In this guide, our goal is to centralize two types of logs:
14
14
- Nginx logs
15
15
- SSH auth logs
16
16
17
-
We'll configure nginx to directly forward the access logs to our central rsyslog server.<br/>
18
-
For the auth logs, we'll configure a local rsyslog on each web server to forward them to our central server.
17
+
We'll configure nginx to forward the access logs to our central rsyslog server.<br/>
18
+
We'll configure a local rsyslog for the auth logs on each web server and forward them to our central server.
19
19
20
-
On the central server, a rsyslog server will receive those logs and write them to files.<br/>
21
-
On this same server, the Security Engine will analyze those logs to detect malicious behaviors in them.<br/>
22
-
Finally, we will have a Firewall Remediation Component running on each of our web server to block the malicious IPs.
20
+
On the central server, a rsyslog server will receive those logs and write them into files.<br/>
21
+
The Security Engine will analyze those logs on this same server to detect malicious behaviours.<br/>
22
+
Finally, we will have a Firewall Remediation Component running on each web server to block the malicious IPs.
- If you have a firewall, you will need to allow communication on 514/UDP (syslog) and 8080/TCP (crowdsec LAPI) from the web servers to the central server
31
-
- By default, rsyslog is a clear-text protocol. If you all the machines interact over LAN, this is probably not an issue, but they are communication over internet, you will probably want to setup TLS on the syslog server.
30
+
- If you have a firewall, you will need to allow communication on 514/UDP (Syslog) and 8080/TCP (crowdsec LAPI) from the web servers to the central server
31
+
- By default, rsyslog is a clear-text protocol. If all the machines interact over LAN, this is probably not an issue, but if they communicate over the internet, you will probably want to set up TLS on the syslog server.
32
32
33
33
## Rsyslog Server Setup
34
34
35
35
Let's start by setting up our central rsyslog.
36
36
37
37
If rsyslog is not installed, you can install it with `apt install rsyslog` (assuming a debian-like distribution).
38
38
39
-
First step is to configure rsyslog with an UDP listener and a template to write the received logs to disk.
39
+
The first step is to configure rsyslog with a UDP listener and a template to write the received logs to disk.
40
40
41
41
Create the file `/etc/rsyslog.d/10_remote.conf` with the following content:
You will also need to edit `/etc/rsyslog.conf` to make sure `$RepeatedMsgReduction` is set to `off` (some distributions set it to `on` by defautl, but this is rarely recommended, especially when consuming potentially a high volume of logs)
69
+
You will also need to edit `/etc/rsyslog.conf` to make sure `$RepeatedMsgReduction` is set to `off` (some distributions set it to `on` by default, but this is rarely recommended, especially when consuming potentially a high volume of logs)
70
70
71
71
Finally, restart rsyslog to use the new configuration:
72
72
```bash
73
73
systemctl restart rsyslog
74
74
```
75
75
76
-
We will also setup logrotate to avoid filling our disk with the logs. Create a file `/etc/logrotate.d/remote-logs` with the following content:
76
+
We will also set up Logrotate to avoid filling our disk with the logs. Create a file `/etc/logrotate.d/remote-logs` with the following content:
77
77
```
78
78
/var/log/remote-logs/*/*.log {
79
79
daily
@@ -89,7 +89,7 @@ We will also setup logrotate to avoid filling our disk with the logs. Create a f
89
89
}
90
90
```
91
91
92
-
This will keep 7 days of compressed logs.
92
+
This configuration will keep 7 days of compressed logs.
As nginx supports multiple `access_log` and `error_log` directives, you can keep the existing directives to still have a local copy of the logs.
104
+
As nginx supports multiple `access_log` and `error_log` directives, you can keep the existing directives to keep a local copy of the logs.
105
105
106
106
### Auth logs
107
107
@@ -115,7 +115,7 @@ Restart the rsyslog client:
115
115
$ systemctl restart rsyslog
116
116
```
117
117
118
-
## Crowdsec Setup
118
+
## CrowdSec Setup
119
119
120
120
### Central Server
121
121
@@ -131,7 +131,7 @@ Next, we install crowdsec:
131
131
$ sudo apt install crowdsec
132
132
```
133
133
134
-
Crowdsec will automatically detect we are running on a linux server, and install the base linux collection.
134
+
CrowdSec will automatically detect we are running on a Linux server and install the base Linux collection.
135
135
136
136
But because our logs are not in a standard location, we need to configure the acquisition to tell crowdsec where our logs are.
137
137
@@ -143,15 +143,15 @@ labels:
143
143
type: syslog
144
144
```
145
145
146
-
We now need to do the same thing for the auth logs, create a file `/etc/crowdsec/acquis.d/ssh.yaml` with the following content:
146
+
Repeat for auth logs, create a file `/etc/crowdsec/acquis.d/ssh.yaml` with the following content:
147
147
```
148
148
filenames:
149
149
- /var/log/remote-logs/auth/*.log
150
150
labels:
151
151
type: syslog
152
152
```
153
153
154
-
Note that we are setting the type label to `syslog`. This will instruct crowdsec to use the `syslog` parser to extract the actual type from the log itself.
154
+
Note that we are setting the type label to `syslog`, instructing crowdsec to use the `syslog` parser to extract the actual type from the log itself.
155
155
156
156
Then, we need to install the nginx collection for crowdsec to be able to detect attacks:
Crowdsec by itself will only detect bad behaviors and take decisions against IPs, but will not block them.
176
+
CrowdSec, by itself, will only detect bad behaviors and make decisions about IPs; it will not block them.
177
177
178
-
In order to block an IP, you need to install a [remediation component](/unversioned/bouncers/intro.md).
178
+
To block an IP, you need to install a [remediation component](/unversioned/bouncers/intro.md).
179
179
180
-
For the purpose of this guide, we'll be using the [firewall remediation component](/unversioned/bouncers/firewall.mdx) that will add local firewall rules to block malicious IPs.
180
+
For this guide, we'll be using the [firewall remediation component](/unversioned/bouncers/firewall.mdx) that will add local firewall rules to block malicious IPs.
We'll scan one of our web servers, and because both of them are querying the same crowdsec instance, if one of them is attacked, the attacker will also be blocked on the other.
225
+
We'll scan one of our web servers, and because both of them are querying the same crowdsec instance if one detects the attack, the other server will also block the attacker.
226
226
227
227
```bash
228
228
$ nikto -h 52.50.157.217
229
229
```
230
230
231
-
After the scan is done, try to access the 2 servers with curl:
231
+
After the scan is done, try to access the two servers with curl:
0 commit comments