Skip to content

add doc about custom hub items #1061

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
sabban wants to merge 2 commits into
base: main
Choose a base branch
from
Open

add doc about custom hub items #1061

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
2d8fab6
add doc about custom hub items
sabban Apr 8, 2026
aa5cadd
update
sabban Apr 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
297 changes: 297 additions & 0 deletions crowdsec-docs/unversioned/getting_started/installation/kubernetes.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,303 @@ crowdsec-agent-kf9fr 1/1 Running 0 34s
crowdsec-lapi-777c469947-jbk9q 1/1 Running 0 34s
```

### Adding custom parsers, scenarios, postoverflows, and AppSec files

You can ship your own parser, scenario, postoverflow, AppSec configuration, AppSec rule, AppSec scenario, and AppSec postoverflow files through Helm by using:

- `config.parsers`
- `config.scenarios`
- `config.postoverflows`
- `appsec.configs`
- `appsec.rules`
- `appsec.scenarios`
- `appsec.postoverflows`

Those values are mounted into the relevant pods as files and passed to CrowdSec as-is.

`config.parsers` is split by parser stage:

- `config.parsers.s00-raw`
- `config.parsers.s01-parse`
- `config.parsers.s02-enrich`

`config.scenarios` is a flat map of scenario files.

`config.postoverflows` is split by postoverflow stage:

- `config.postoverflows.s00-enrich`
- `config.postoverflows.s01-whitelist`

`appsec.configs`, `appsec.rules`, and `appsec.scenarios` are flat maps of AppSec files.

`appsec.postoverflows` is split by postoverflow stage:

- `appsec.postoverflows.s00-enrich`
- `appsec.postoverflows.s01-whitelist`

What is allowed:

- `config.parsers.<stage>.<fileName>: |` with a string value
- `config.scenarios.<fileName>: |` with a string value
- `config.postoverflows.<stage>.<fileName>: |` with a string value
- `appsec.configs.<fileName>: |` with a string value
- `appsec.rules.<fileName>: |` with a string value
- `appsec.scenarios.<fileName>: |` with a string value
- `appsec.postoverflows.<stage>.<fileName>: |` with a string value
- file names with or without a `.yaml` / `.yml` suffix
- keys such as `my-parser`, `my-parser.yaml`, `test_scenario`, `custom.rule.yaml`, `my-appsec-config`, `my-rule.yml`, `my-postoverflow`, `my-appsec-scenario.yaml`

What is not allowed:

- nested objects below the file name
- using a non-string value for a file entry
- adding custom stages under `config.parsers`
- adding custom stages under `config.postoverflows` or `appsec.postoverflows`
- forgetting to escape dots when using `--set` or `--set-file`
- enabling `appsec.configs`, `appsec.rules`, `appsec.scenarios`, or `appsec.postoverflows` without also enabling the AppSec component

This is valid in a values file:

```yaml title="crowdsec-custom-items.yaml"
config:
parsers:
s01-parse:
test-parser: |
name: mycorp/test-parser
description: "Parse my custom application logs"
filter: "evt.Parsed.program == 'myapp'"
onsuccess: next_stage
nodes:
- grok:
pattern: '^%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:level} %{GREEDYDATA:message}$'
apply_on: message

another-parser.yaml: |
name: mycorp/another-parser
description: "A parser stored with an explicit .yaml suffix"
filter: "evt.Parsed.program == 'another-app'"
onsuccess: next_stage

scenarios:
test-scenario: |
type: leaky
name: mycorp/test-scenario
description: "Detect repeated errors from my application"
filter: "evt.Meta.service == 'myapp' && evt.Parsed.level == 'ERROR'"
groupby: evt.Meta.source_ip
capacity: 5
leakspeed: 1m
blackhole: 5m

another-scenario.yaml: |
type: trigger
name: mycorp/another-scenario
description: "Raise an alert on a specific event"
filter: "evt.Meta.log_type == 'special_event'"
blackhole: 10m

postoverflows:
s01-whitelist:
trusted-ip.yaml: |
name: mycorp/trusted-ip
description: "Whitelist a known source after overflow"
whitelist:
reason: "trusted source"
expression:
- evt.Overflow.Source_ip == "203.0.113.10"

appsec:
enabled: true
configs:
my-appsec-config: |
name: mycorp/my-appsec-config
default_remediation: ban
inband_rules:
- mycorp/my-rule

rules:
my-rule.yaml: |
name: mycorp/my-rule
zones:
- URI
transform:
- lowercase
match:
type: contains
value: /admin
action: ban

scenarios:
my-appsec-scenario: |
type: trigger
name: mycorp/my-appsec-scenario
description: "Generate an alert when the custom AppSec rule matches"
filter: evt.Meta.log_type == 'appsec'
blackhole: 2m

postoverflows:
s01-whitelist:
my-appsec-whitelist.yaml: |
name: mycorp/my-appsec-whitelist
description: "Whitelist a trusted source for AppSec alerts"
whitelist:
reason: "trusted appsec source"
expression:
- evt.Overflow.Source_ip == "203.0.113.11"
```

Minimal valid parser file example:

```yaml title="parser.yaml"
name: mycorp/minimal-parser
description: "Parse a simple custom log format"
filter: "evt.Parsed.program == 'myapp'"
onsuccess: next_stage
nodes:
- grok:
pattern: '^%{WORD:action} %{IP:source_ip}$'
apply_on: message
statics:
- meta: log_type
value: myapp_event
```

The same parser embedded as a `config.parsers` item:

```yaml
config:
parsers:
s01-parse:
minimal-parser: |
name: mycorp/minimal-parser
description: "Parse a simple custom log format"
filter: "evt.Parsed.program == 'myapp'"
onsuccess: next_stage
nodes:
- grok:
pattern: '^%{WORD:action} %{IP:source_ip}$'
apply_on: message
statics:
- meta: log_type
value: myapp_event
```

This is not valid:

```yaml
config:
parsers:
s01-parse:
test-parser:
yaml: |
name: mycorp/test-parser
```

The example above is invalid because `test-parser` becomes an object with a `yaml` field. Each file entry must be a single string.

The same rule applies to AppSec files. For example, this is not valid either:

```yaml
appsec:
configs:
my-appsec-config:
yaml: |
name: mycorp/my-appsec-config
```

Like parser and scenario files, AppSec file names can include `.yaml` or `.yml`, uppercase characters, underscores, or dots. The chart normalizes those keys into Kubernetes-safe volume names before mounting them, so the same filename flexibility is available for `config.postoverflows`, `appsec.configs`, `appsec.rules`, `appsec.scenarios`, and `appsec.postoverflows`.

#### Using a dedicated values file

If you have several custom items, the simplest approach is to keep them in a dedicated values file and pass it with `-f`:

```bash
helm upgrade --install crowdsec crowdsec/crowdsec \
-n crowdsec \
-f crowdsec-values.yaml \
-f crowdsec-custom-items.yaml
```

This works well when:

- you manage several parsers, scenarios, or AppSec files together
- you want everything in Git
- you do not want shell escaping issues in the command line

#### Using `--set-file`

You can also load file contents directly from local files:

```bash
helm upgrade --install crowdsec crowdsec/crowdsec \
-n crowdsec \
-f crowdsec-values.yaml \
--set-file "config.parsers.s01-parse.test-parser=/tmp/crowdsec-custom-items/parser.yaml" \
--set-file "config.scenarios.test-scenario=/tmp/crowdsec-custom-items/scenario.yaml" \
--set-file "config.postoverflows.s01-whitelist.my-whitelist=/tmp/crowdsec-custom-items/postoverflow.yaml" \
--set 'appsec.enabled=true' \
--set-file "appsec.configs.my-appsec-config=/tmp/crowdsec-custom-items/appsec-config.yaml" \
--set-file "appsec.rules.my-rule=/tmp/crowdsec-custom-items/appsec-rule.yaml" \
--set-file "appsec.scenarios.my-appsec-scenario=/tmp/crowdsec-custom-items/appsec-scenario.yaml" \
--set-file "appsec.postoverflows.s01-whitelist.my-appsec-whitelist=/tmp/crowdsec-custom-items/appsec-postoverflow.yaml"
```

If you want the key itself to contain `.yaml`, you must escape the dot in the Helm path:

```bash
helm upgrade --install crowdsec crowdsec/crowdsec \
-n crowdsec \
-f crowdsec-values.yaml \
--set-file "config.parsers.s01-parse.test-parser\.yaml=/tmp/crowdsec-custom-items/parser.yaml" \
--set-file "config.scenarios.test-scenario\.yaml=/tmp/crowdsec-custom-items/scenario.yaml" \
--set-file "config.postoverflows.s01-whitelist.my-whitelist\.yaml=/tmp/crowdsec-custom-items/postoverflow.yaml" \
--set 'appsec.enabled=true' \
--set-file "appsec.configs.my-appsec-config\.yaml=/tmp/crowdsec-custom-items/appsec-config.yaml" \
--set-file "appsec.rules.my-rule\.yaml=/tmp/crowdsec-custom-items/appsec-rule.yaml" \
--set-file "appsec.scenarios.my-appsec-scenario\.yaml=/tmp/crowdsec-custom-items/appsec-scenario.yaml" \
--set-file "appsec.postoverflows.s01-whitelist.my-appsec-whitelist\.yaml=/tmp/crowdsec-custom-items/appsec-postoverflow.yaml"
```

This is important because Helm uses dots as path separators. Without escaping, this:

```bash
--set-file "config.parsers.s01-parse.test-parser.yaml=/tmp/crowdsec-custom-items/parser.yaml"
```

is interpreted as:

```yaml
config:
parsers:
s01-parse:
test-parser:
yaml: <file-content>
```

which is not valid for CrowdSec chart values.

Practical rules:

- `config.parsers.s01-parse.test-parser` is valid
- `config.parsers.s01-parse.test-parser\.yaml` is valid
- `config.scenarios.test-scenario` is valid
- `config.scenarios.test-scenario\.yaml` is valid
- `config.postoverflows.s01-whitelist.my-whitelist` is valid
- `config.postoverflows.s01-whitelist.my-whitelist\.yaml` is valid
- `appsec.configs.my-appsec-config` is valid
- `appsec.configs.my-appsec-config\.yaml` is valid
- `appsec.rules.my-rule` is valid
- `appsec.rules.my-rule\.yaml` is valid
- `appsec.scenarios.my-appsec-scenario` is valid
- `appsec.scenarios.my-appsec-scenario\.yaml` is valid
- `appsec.postoverflows.s01-whitelist.my-appsec-whitelist` is valid
- `appsec.postoverflows.s01-whitelist.my-appsec-whitelist\.yaml` is valid
- `config.parsers.s03-custom.test-parser` is not valid
- `config.postoverflows.s02-custom.my-whitelist` is not valid
- `config.parsers.s01-parse.test-parser.yaml` without escaping is not valid
- `appsec.scenarios.my-appsec-scenario.yaml` without escaping is not valid
- `appsec.rules.my-rule.yaml` without escaping is not valid

### A word about source IPs

For CrowdSec to work in Kubernetes, it must see the real client IP. Otherwise
Expand Down
7 changes: 5 additions & 2 deletions crowdsec-docs/versioned_docs/version-v1.7/configuration/values_parameters.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -366,8 +366,11 @@ configuration values, their defaults, and their purpose.
| `appsec.replicas`<a id="appsec-replicas"></a> | Number of replicas for the AppSec Deployment | `1` |
| `appsec.strategy`<a id="appsec-strategy"></a> | Deployment strategy for AppSec | `{}` |
| `appsec.acquisitions`<a id="appsec-acquisitions"></a> | AppSec acquisitions (datasource listeners), e.g. appsec listener on 7422 | `[]` |
| `appsec.configs`<a id="appsec-configs"></a> | AppSec configs (key = filename, value = file content) | `{}` |
| `appsec.rules`<a id="appsec-rules"></a> | AppSec rule files (key = filename, value = file content) | `{}` |
| `appsec.scenarios`<a id="appsec-scenarios"></a> | AppSec scenario files (key = filename, value = file content; `.yaml` / `.yml` suffix optional, chart normalizes keys to Kubernetes-safe volume names) | `{}` |
| `appsec.postoverflows.s00-enrich`<a id="appsec-postoverflows-s00-enrich"></a> | AppSec postoverflow enrichment files (key = filename, value = file content; chart normalizes keys to Kubernetes-safe volume names) | `{}` |
| `appsec.postoverflows.s01-whitelist`<a id="appsec-postoverflows-s01-whitelist"></a> | AppSec postoverflow whitelist files (key = filename, value = file content; chart normalizes keys to Kubernetes-safe volume names) | `{}` |
| `appsec.configs`<a id="appsec-configs"></a> | AppSec configs (key = filename, value = file content; `.yaml` / `.yml` suffix optional, chart normalizes keys to Kubernetes-safe volume names) | `{}` |
| `appsec.rules`<a id="appsec-rules"></a> | AppSec rule files (key = filename, value = file content; `.yaml` / `.yml` suffix optional, chart normalizes keys to Kubernetes-safe volume names) | `{}` |
| `appsec.priorityClassName`<a id="appsec-priorityclassname"></a> | Priority class name for AppSec pods | `""` |
| `appsec.deployAnnotations`<a id="appsec-deployannotations"></a> | Annotations added to the AppSec Deployment | `{}` |
| `appsec.podAnnotations`<a id="appsec-podannotations"></a> | Annotations added to AppSec pods | `{}` |
Expand Down
Loading