Skip to content

Commit 429b4e9

Browse files
author
jdv
committed
section 3 first draft
1 parent 741677e commit 429b4e9

1 file changed

Lines changed: 46 additions & 10 deletions

File tree

crowdsec-docs/unversioned/user_guides/interactive_se_install/03_acquisition copy.mdx

Lines changed: 46 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -11,28 +11,64 @@ import CodeBlock from '@theme/CodeBlock';
1111

1212
# Acquisition
1313

14-
blabla
14+
Acquisition setup indicates how to retrieve the logs (file, syslog,...) and what parser to use for those logs.
15+
The configuration for all acquisitions is located in /etc/crowdsec/acquis.yaml, you can choose to split and store them in /etc/crowdsec/acquis.d/<anyName>.yaml (both method can coexist)
1516

1617
## Setting up acquisition datasources for detection
1718

19+
TODO: explain how to infer the type give link to collection of prefilled with nicely named placeholders example for all datasource. give 2 examples for each: one minimal and one with all optional fields (+how to use wildcard and exclude for files for example) - can we make a gtp prompt ?
1820

1921
### Instructions
20-
...
22+
... per datasource ?
2123

2224
### Verification
2325

24-
Let's check ...
26+
Let's check that parsing is working for all installed parsers and logs.
2527

26-
#### CrowdSec installation health
27-
28-
> [ ] Check ...
28+
#### Check acquisition: all necessary logs are read and parsed properly
2929

30+
> [ ] Files are properly read and parsed
31+
- Use your services normally: navigate on your website, log in via ssh ...
32+
- Check the metrics
3033
```bash
31-
...
34+
sudo cscli metrics
3235
```
33-
- You should see ...
34-
- ...
35-
36+
Alternatively to select only the metrics we're interrested in here:
37+
```bash
38+
sudo cscli metrics show acquisition parsers
39+
```
40+
- You should see names of the log files you configured in the acquisition section, and the number of lines parsed for each of them.
41+
- The number of "Lines parsed" should be non-zero for each of the files you configured in the acquisition section. and for most cases should be equal to the number of "lines read".
42+
- The parsers metrics show you what parsers were successfully used. Look for the name of the parsers you installed in the previous step
3643

44+
#### Run on existing logs if you have some
3745

46+
> [ ] Check alerts in past logs
47+
- Most services open to the internet for more than a few days will have some malicious activity in their logs.
48+
- You can run the Security Engine on existing logs to check if it detects any malicious activity.
49+
- ... blabla we have DSN commands in datasources docs.
50+
- Try...
51+
- You will probably see alerts raised when doing this command:
52+
```bash
53+
sudo cscli alerts list
54+
```
55+
- The alerts will also show up in the console within a few minutes if you enrolled your Security Engine.
56+
3857
### Troubleshooting
58+
If you never see any alerts: However unlikely it's possible that you're not being hit by attacks but first lets check what could be wrong.
59+
60+
<details>
61+
<summary>IPs in your logs not properly X-forwarded for</summary>
62+
63+
- Check that the logs you are reading have the real Source IP and not the one of one of your proxies or load balancers.
64+
- Local/Private IPs are whitelisted by default, so log with non x-forwarded IPs wont trigger alerts. //more details here.. link ...
65+
- If you have a reverse proxy, make sure to use the `X-Forwarded-For` header in your logs.
66+
- If you have a load balancer, make sure to use the `X-Real-IP` header in your logs.
67+
</details>
68+
<details>
69+
<summary>You are using custom log formats</summary>
70+
71+
The parsers we provide are made to parse default log formats of the supported services.
72+
If you are using custom log formats, you will need to create your own parsers or modify the existing ones to match your log format.
73+
- You can find more information on how to create your own parsers in the [CrowdSec documentation](https://doc.crowdsec.net/docs/next/log_processor/parsers/intro).
74+
</details>

0 commit comments

Comments
 (0)