You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: crowdsec-docs/unversioned/user_guides/interactive_se_install/03_acquisition copy.mdx
+46-10Lines changed: 46 additions & 10 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -11,28 +11,64 @@ import CodeBlock from '@theme/CodeBlock';
11
11
12
12
# Acquisition
13
13
14
-
blabla
14
+
Acquisition setup indicates how to retrieve the logs (file, syslog,...) and what parser to use for those logs.
15
+
The configuration for all acquisitions is located in /etc/crowdsec/acquis.yaml, you can choose to split and store them in /etc/crowdsec/acquis.d/<anyName>.yaml (both method can coexist)
15
16
16
17
## Setting up acquisition datasources for detection
17
18
19
+
TODO: explain how to infer the type give link to collection of prefilled with nicely named placeholders example for all datasource. give 2 examples for each: one minimal and one with all optional fields (+how to use wildcard and exclude for files for example) - can we make a gtp prompt ?
18
20
19
21
### Instructions
20
-
...
22
+
... per datasource ?
21
23
22
24
### Verification
23
25
24
-
Let's check ...
26
+
Let's check that parsing is working for all installed parsers and logs.
25
27
26
-
#### CrowdSec installation health
27
-
28
-
> [] Check ...
28
+
#### Check acquisition: all necessary logs are read and parsed properly
29
29
30
+
> [] Files are properly read and parsed
31
+
- Use your services normally: navigate on your website, log in via ssh ...
32
+
- Check the metrics
30
33
```bash
31
-
...
34
+
sudo cscli metrics
32
35
```
33
-
- You should see ...
34
-
- ...
35
-
36
+
Alternatively to select only the metrics we're interrested in here:
37
+
```bash
38
+
sudo cscli metrics show acquisition parsers
39
+
```
40
+
- You should see names of the log files you configured in the acquisition section, and the number of lines parsed for each of them.
41
+
- The number of "Lines parsed" should be non-zero for each of the files you configured in the acquisition section. and for most cases should be equal to the number of "lines read".
42
+
- The parsers metrics show you what parsers were successfully used. Look for the name of the parsers you installed in the previous step
36
43
44
+
#### Run on existing logs if you have some
37
45
46
+
> [] Check alerts in past logs
47
+
- Most services open to the internet for more than a few days will have some malicious activity in their logs.
48
+
- You can run the Security Engine on existing logs to check if it detects any malicious activity.
49
+
- ... blabla we have DSN commands in datasources docs.
50
+
- Try...
51
+
- You will probably see alerts raised when doing this command:
52
+
```bash
53
+
sudo cscli alerts list
54
+
```
55
+
- The alerts will also show up in the console within a few minutes if you enrolled your Security Engine.
56
+
38
57
### Troubleshooting
58
+
If you never see any alerts: However unlikely it's possible that you're not being hit by attacks but first lets check what could be wrong.
59
+
60
+
<details>
61
+
<summary>IPs in your logs not properly X-forwarded for</summary>
62
+
63
+
- Check that the logs you are reading have the real Source IP and not the one of one of your proxies or load balancers.
64
+
- Local/Private IPs are whitelisted by default, so log with non x-forwarded IPs wont trigger alerts. //more details here.. link ...
65
+
- If you have a reverse proxy, make sure to use the `X-Forwarded-For` header in your logs.
66
+
- If you have a load balancer, make sure to use the `X-Real-IP` header in your logs.
67
+
</details>
68
+
<details>
69
+
<summary>You are using custom log formats</summary>
70
+
71
+
The parsers we provide are made to parse default log formats of the supported services.
72
+
If you are using custom log formats, you will need to create your own parsers or modify the existing ones to match your log format.
73
+
- You can find more information on how to create your own parsers in the [CrowdSec documentation](https://doc.crowdsec.net/docs/next/log_processor/parsers/intro).
0 commit comments