Skip to content

filtering.md

Latest commit

 

History

History
78 lines (57 loc) · 3.06 KB

filtering.md

File metadata and controls

78 lines (57 loc) · 3.06 KB
Raw

FLP filtering language

Flowlogs-pipeline uses a simple query language to filter network flows:

(srcnamespace="netobserv" OR (srcnamespace="ingress" AND dstnamespace="netobserv")) AND srckind!="service"

The syntax includes:

  • Logical boolean operators (case insensitive)
    • and
    • or
  • Comparison operators
    • equals =
    • not equals !=
    • matches regexp =~
    • not matches regexp !~
    • greater than (or equal) > / >=
    • less than (or equal) < / <=
  • Unary operations
    • field is present: with(field)
    • field is absent: without(field)
  • Parenthesis-based priority

API integration

The language is currently integrated in the "keep_entry" transform/filtering API. Example:

    transform:
      type: filter
      filter:
        rules:
        - type: keep_entry_query
          keepEntryQuery: (namespace="A" and with(workload)) or service=~"abc.+"
          keepEntrySampling: 10 # Optionally, a sampling interval can be associated with the filter

Integration with the NetObserv operator and CLI

See also the list of field names that are available for queries, and the API documentation.

Operator

In the NetObserv operator, the filtering query language is used in FlowCollector spec.processor.filters. Example:

spec:
  processor:
    filters:
      - query: |
          (SrcK8S_Namespace="netobserv" OR (SrcK8S_Namespace="openshift-ingress" AND DstK8S_Namespace="netobserv"))
        outputTarget: Loki  # The filter can target a specific output (such as Loki logs or exported data), or all outputs.
        sampling: 10        # Optionally, a sampling interval can be associated with the filter

CLI

In the NetObserv CLI, the filtering query language is used in the --query parameter. Example:

kubectl netobserv flows --query='SrcK8S_Namespace="netobserv" OR (SrcK8S_Namespace="openshift-ingress" AND DstK8S_Namespace="netobserv")'

Internals

This language is designed using Yacc / goyacc.

The definition file describes the syntax based on a list of tokens. It is derived to a go source file using goyacc, which defines constants for the tokens, among other things. The lexer file defines structures and helpers that can be used from expr.y, the logic used to interpret the language in a structured way, and is also where actual characters/strings are mapped to syntax tokens. Finally, eval.go runs the desired query on actual data.

When adding features to the language, you'll likely have to change expr.y and lexer.go.

To regenerate expr.y.go, run:

make goyacc