Skip to content

Update KFL reference pages to match kfl2 specification #94

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
alongir wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Update KFL reference pages to match kfl2 specification #94

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/pages/en/display_filters.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ http && request.headers["content-type"] == "application/json"
```
timestamp >= now() # Live traffic only
timestamp > timestamp("2024-02-01T14:00:00Z") # After specific time
elapsed_time <= 300000000 # Last 5 minutes
timestamp > now() - duration("5m") # Last 5 minutes
```

---
Expand Down
212 changes: 35 additions & 177 deletions src/pages/en/filtering.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,206 +1,64 @@
---
title: Display Filters (KFL - Kubeshark Filtering Language)
description: The filter input enables filtering results using Kubeshark Filter Language (KFL).
title: Display Filters (KFL)
description: Filter traffic using Kubeshark Filter Language (KFL), powered by CEL expressions.
layout: ../../layouts/MainLayout.astro
---

> KFL is a rich filtering language (not a query language)
**Kubeshark Filter Language (KFL)** is a display filter language powered by [Common Expression Language (CEL)](https://github.com/google/cel-go). It filters network traffic according to specific rules — instead of searching for the needle in the haystack, it filters out the haystack to reveal the needle.

Inspired by Wireshark's display filters. Its primary purpose is to filter network traffic according to specific rules. Instead of searching for the needle in the haystack, it filters out the haystack to reveal the needle.
For example, to only see HTTP responses with client error status codes (400–499):

**Kubeshark Filter Language (KFL)** is a language that enables you filter traffic that matches a boolean expression. For example; to only see the items with HTTP response status codes (`400` – `499`), enter:

```python
http and response.status == r"4.*"
```cel
http && status_code >= 400 && status_code < 500
```

and click the **Apply** button. Your traffic stream will look like this:
Click the **Apply** button to filter the traffic stream:

![Filter example](/filter-applied.png)

## KFL vs. Pod Targeting (Display vs. Capture Filters)
## KFL vs. Capture Filters

KFL should not be confused with [Capture Filters](/en/pod_targeting) as they serve different purposes:

KFL should not be confused with [Pod Targeting](/en/pod_targeting) as they serve different purposes. KFL statements only affect the data presented in the Dashboard, hence they are referred as Display FIlters. Pod Targeting rules constitute CApture Filters. These rules determine which pods are targeted and, consequently, which traffic is tapped.
| Aspect | Display Filters (KFL) | Capture Filters |
|--------|------------------------|-----------------|
| Purpose | Filter what is displayed | Filter what is captured |
| Impact | Dashboard view only | Resource consumption |
| Scope | Single browser tab | Cluster-wide |
| Syntax | CEL expressions | Helm values / Dashboard |

For those familiar with Wireshark, KFL can be likened to Wireshark's Display Filters, and Pod Targeting to Wireshark's BPF (Berkeley Packet Filter) filters.
For those familiar with Wireshark, KFL is analogous to Wireshark's Display Filters, and Capture Filters to Wireshark's BPF (Berkeley Packet Filter) filters.

## Queryable UI Elements

When you hover over UI elements and they display a green plus sign, it means this element can be added to your query. Selecting an element with a green plus sign will add this element to the query. For example, selecting this queryable element:
When you hover over UI elements and they display a green **+** sign, the element can be added to your query. Clicking appends the corresponding filter expression.

![Queryable UI Elements Example](/filter-ui-example.png)

adds `response.status == 201` to your query and only displays `HTTP 201` responses in the live traffic streaming.

## KFL Syntax Reference

**Kubeshark Filter Language (KFL)** is the language implemented inside [`kubeshark/worker`](https://github.com/kubeshark/worker) that enables the user to filter the traffic efficiently and precisely.

```python
http and request.method == "GET" and request.path != "/example" and (request.query.a > 42 or request.headers["x"] == "y")
```

The language as a whole evaluates into a boolean outcome, always. Such that the record which makes the boolean `true` is a record that matches the filter.

There are certain helper methods that can do more than reduce into a boolean value.

### Literals

The language supports the following literals:

- Nil `nil`
- Boolean `true` or `false`
- Number `42`, `3.14` etc.
- String `"hello world"`
- Regex `r"prefix.*"`

### Operators

Operations can be grouped (precedence) using parentheses `(...)`

> *Note: Operators are evaluated from left to right, excluding parentheses `(...)`*

The language supports the following operators:

#### Logical

`and`, `or`

> *Note: `false and`, `true or` are evaluated fast*

#### Equality

`==`, `!=`

#### Comparison

`>=`, `>`, `<=`, `<`

#### Unary

`!`, `-`

### Helpers

Helpers in KFL are method invocations that enable filtering capability which cannot be provided through the syntax. These are the available helpers in KFL:

#### `startsWith(string)`

Returns `true` if the given selector's value starts with the `string`. e.g. `brand.name.startsWith("Chev")`

#### `endsWith(string)`

Returns `true` if the given selector's value ends with the `string`. e.g. `brand.name.endsWith("let")`

#### `contains(string)`

Returns `true` if the given selector's value contains the `string`. e.g. `brand.name.contains("ro")`

#### `datetime(string) integer`

Returns the UNIX timestamp `integer` which is the equivalent of the time that's provided by the `string`. If the given date-time string is not in a recognized format then it evaluates to `false`. The format must be same as `"1/2/2006, 3:04:05 PM"` (`en-US`) e.g. `timestamp > datetime("10/19/2021, 6:29:02 PM")`

It's equal to the `time.Now().UnixNano() / int64(time.Millisecond)` in Go. e.g. `1635190131000`

#### `limit(integer)`
This makes it easy to build complex filters without typing the full expression.

Limits the number of records that are streamed back as a result of a query. Always evaluates to `true`.
## Quick Examples

#### `json()`
```cel
# HTTP server errors
http && status_code >= 500

A decoding helper that decodes the given JSON field if it's possible. It's used through chain calls and the JSONPath that's wanted to be accessed can be chained to the end like; `response.body.json().brand.name == "Chevrolet"`
# Traffic to a specific namespace
dst.pod.namespace == "production"

#### `xml()`
# DNS queries for a domain
dns && "example.com" in dns_questions

A decoding helper that decodes the given XML field if it's possible. It's used through chain calls and the dot-notation path that's wanted to be accessed can be chained to the end like; `response.body.xml().brand.name == "Chevrolet"`
# Filter by pod labels (safe access)
map_get(local_labels, "app", "") == "payments"

#### `redact(string...)`
# Slow requests (over 5 seconds)
http && elapsed_time > 5000000

A record altering helper that takes N number of `string` typed arguments. The arguments are dot-notation paths and it replaces the value on each matching path with `[REDACTED]` string.

The `json()` and `xml()` helpers are supported inside the arguments of the `redact` helper.

#### `now() integer`

Returns the UNIX timestamp `integer` which is the equivalent of the time now.

#### `seconds(integer) integer`

Returns the UNIX timestamp `integer` which is the equivalent of the time `integer` seconds before or after from now according to the sign of the argument.

#### `minutes(integer) integer`

Returns the UNIX timestamp `integer` which is the equivalent of the time `integer` minutes before or after from now according to the sign of the argument.

#### `hours(integer) integer`

Returns the UNIX timestamp `integer` which is the equivalent of the time `integer` hours before or after from now according to the sign of the argument.

#### `days(integer) integer`

Returns the UNIX timestamp `integer` which is the equivalent of the time `integer` days before or after from now according to the sign of the argument.

#### `weeks(integer) integer`

Returns the UNIX timestamp `integer` which is the equivalent of the time `integer` weeks before or after from now according to the sign of the argument.

#### `months(integer) integer`

Returns the UNIX timestamp `integer` which is the equivalent of the time `integer` months before or after from now according to the sign of the argument.

#### `years(integer) integer`

Returns the UNIX timestamp `integer` which is the equivalent of the time `integer` years before or after from now according to the sign of the argument.

> *Note: Calling an undefined helper makes the whole expression collapse and evaluate to `false`.*

### Selectors

Selectors in KFL are JSONPath(s) that refer to the path in a JSON document. In KFL, every record is a JSON document. Any selector that does not match a path is evaluated to `false`.

Following are some selector examples:

- `brand.name` basic selector
- `request.path[1]` index selector
- `request.headers["a"] == "b"` key selector

Selectors can be combined with a subset of helpers such that they evaluate into a boolean without any operator. e.g. `brand.name.startsWith("Chev")` These are the list of helpers that can be used with selectors:

- `startsWith(string)`
- `endsWith(string)`
- `contains(string)`

Such that instead of writing a boolean expression like `brand.name == "Chevrolet"`, one of the following can be written:

```python
brand.name.startsWith("Chev")
brand.name.endsWith("let")
brand.name.contains("ro")
# Recent traffic
timestamp > now() - duration("5m")
```

A selector can be compared to another selector as well. Such that for these given JSON document, filters like those can be written:

JSON: `{"model":"Camaro","brand":{"name":"Chevrolet"},"year":2021,"salesYear":2021}` Filter: `year == salesYear`

JSON: `{"model":"Camaro","brand":{"name":"Chevrolet"},"year":2021,"salesYear":2020}` Filter: `year != salesYear`

> *Note: A selector (JSONPath) that couldn't be found in the JSON document makes the whole expression collapse and evaluate to `false`.*

#### Wildcard Selector

Wildcard(`*`) selectors are evaluated into arrays and the elements in the array are checked using the respective operator against whether any or all elements are matching the given criteria or not. So, for example;

- for a JSON like `{"request":{"path":["api","v1","example"]}}` the query `request.path.* == "v1"` returns `true`
- for a JSON like `{"request":{"path":["api","v1","example"]}}` the query `request.path.* == "v2"` returns `false`
- for a JSON like `{"request":{"path":["api","v1","example"]}}` the query `request.path.* != "v2"` returns `true`
- for a JSON like `{"request":{"path":[1, 2, 3]}}` the query `request.path.* > 2` returns `true`
- for a JSON like `{"request":{"path":[1, 2, 3]}}` the query `request.path.* > 4` returns `false`

The wildcard selector can be used to compare two JSON arrays. So, for a JSON like `{"request":{"path":[1, 2, 3]},"response":{"header":[1, 2, 3]}}` the query `request.path.* == response.header.*` checks for full value equality and returns `true`.

Empty arrays are evaluated to `false`, the rest is `true`:

- for a JSON like `{"request":{"path":[{"x":1}, {"x":2}, {"x":3}]}}` the query `request.path.*.x and true` returns `true`. The part of the query `request.path.*.x` evaluates to `[1, 2, 3]`.
- for a JSON like `{"request":{"path":[]}}` the query `request.path.* and true` returns `true`
## Full Reference

Please check out [the unit tests](https://github.com/up9inc/basenine/blob/main/server/lib/eval_test.go) of the language for more examples.
For the complete KFL syntax, all supported variables, and advanced examples, see the [KFL Reference](/en/v2/kfl2).
Loading