Skip to content

Commit 7f39c3a

Browse files
author
LorenzoTettamanti
committed
Updated documentation: Added quick start guide for new users
1 parent 7ad0d47 commit 7f39c3a

3 files changed

Lines changed: 393 additions & 5 deletions

File tree

‎Doc/docs/cfcli/overview.md‎

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -71,8 +71,3 @@ This document describes the available commands and provides a quick reference ta
7171
| `cfcli monitoring connections` | Monitoring / Status | Displays logged connections from the Identity service |
7272

7373
---
74-
75-
## Examples
76-
### How to interact with the cortexflow agent
77-
!!! warning
78-
This part is work in progress

‎Doc/docs/cfcli/quick-start-guide.md‎

Lines changed: 387 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,387 @@
1+
2+
## CortexFlow Agent: Quick Start Guide
3+
!!! warning
4+
Up to now the only supported cluster environment is Kubernetes
5+
6+
This section contains a quick tutorial that shows how to set up a local Kubernetes cluster using two popular tools and how to use the CortexFlow agent to return the latest detected TCP packages.
7+
The first tool is [minikube](https://minikube.sigs.k8s.io/docs/start/?arch=%2Flinux%2Fx86-64%2Fstable%2Fbinary+download), a popular tool for setting up a single-node local cluster. The second tool is [Kind](https://kind.sigs.k8s.io/), a tool for running local Kubernetes clusters using Docker containers.
8+
9+
Since CortexFlow works in both environments, you can choose which one suits you best. In both examples, we will use Calico CNI.
10+
## Cluster setup (with Calico)
11+
!!! note
12+
If you already have a working cluster you can skip this section
13+
14+
=== "Minikube "
15+
To setup a single node minikube cluster, first you need to install minikube from the [Minikube Download Page](https://minikube.sigs.k8s.io/docs/start)
16+
17+
Then you can create a cluster with Calico CNI using the command:
18+
```bash
19+
minikube start --cni=calico
20+
```
21+
You can **verify** the Calico installation using this command:
22+
```bash
23+
watch kubectl get pods -l k8s-app=calico-node -A
24+
```
25+
This is the expected output:
26+
```bash
27+
NAMESPACE NAME READY STATUS RESTARTS AGE
28+
kube-system calico-node-2xcf4 1/1 Running 0 57s
29+
kube-system calico-node-gkqkg 1/1 Running 0 57s
30+
kube-system calico-node-j44hp 1/1 Running 0 57s
31+
```
32+
33+
=== "Kind "
34+
The first step to setup a cluster with Kind is to install Kind from the [Kind Repository](https://github.com/kubernetes-sigs/kind)
35+
If you have go and docker,podman or nerdctl you can install kind using the command:
36+
```bash
37+
go install sigs.k8s.io/kind@v0.30.0
38+
```
39+
Then you can create a cluster with the [Calico CNI](https://github.com/projectcalico/calico):
40+
41+
- Disable the default CNI to use Calico CNI by running the following command:
42+
43+
```bash
44+
cat > values.yaml <<EOF
45+
kind: Cluster
46+
apiVersion: kind.x-k8s.io/v1alpha4
47+
nodes:
48+
- role: control-plane
49+
- role: worker
50+
- role: worker
51+
networking:
52+
disableDefaultCNI: true
53+
podSubnet: 192.168.0.0/16
54+
EOF
55+
```
56+
57+
Here we are setting a cluster with one master node and due worker nodes. If you need to set a cluster with only one master node and one worker node you can use this command:
58+
59+
```bash
60+
cat > values.yaml <<EOF
61+
kind: Cluster
62+
apiVersion: kind.x-k8s.io/v1alpha4
63+
nodes:
64+
- role: control-plane
65+
- role: worker
66+
networking:
67+
disableDefaultCNI: true
68+
podSubnet: 192.168.0.0/16
69+
EOF
70+
```
71+
72+
73+
- Start your Kind cluster with one control plane and two worker nodes by running the following command:
74+
```bash
75+
kind create cluster --config values.yaml --name dev
76+
```
77+
78+
- Confirm that you now have three nodes in your cluster by running the following command:
79+
```bash
80+
kubectl get nodes -o wide
81+
```
82+
This is the expected output for a three nodes cluster (master-worker-worker):
83+
```bash
84+
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
85+
dev-control-plane NotReady control-plane 4m v1.25.0 172.18.0.2 <none> Ubuntu 22.04.1 LTS 5.10.0-17-amd64 containerd://1.6.7
86+
dev-worker NotReady <none> 4m v1.25.0 172.18.0.4 <none> Ubuntu 22.04.1 LTS 5.10.0-17-amd64 containerd://1.6.7
87+
dev-worker2 NotReady <none> 4m v1.25.0 172.18.0.3 <none> Ubuntu 22.04.1 LTS 5.10.0-17-amd64 containerd://1.6.7
88+
```
89+
90+
- Now you have to install the calico operator:
91+
```bash
92+
kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.30.3/manifests/operator-crds.yaml
93+
kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.30.3/manifests/tigera-operator.yaml
94+
```
95+
- Now you need to setup the custom resources. You can deep dive into the configuration options using the [installation reference](https://docs.tigera.io/calico/latest/reference/installation/api) :
96+
```bash
97+
kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.30.3/manifests/custom-resources.yaml
98+
```
99+
- The last step is to install Calico using the manifest file:
100+
```bash
101+
kubectl apply -f https://raw.githubusercontent.com/projectcalico/calico/v3.30.3/manifests/calico.yaml
102+
```
103+
104+
You can **verify** the Calico installation using this command:
105+
```bash
106+
watch kubectl get pods -l k8s-app=calico-node -A
107+
```
108+
This is the expected output:
109+
```bash
110+
NAMESPACE NAME READY STATUS RESTARTS AGE
111+
kube-system calico-node-2xcf4 1/1 Running 0 57s
112+
kube-system calico-node-gkqkg 1/1 Running 0 57s
113+
kube-system calico-node-j44hp 1/1 Running 0 57s
114+
```
115+
116+
117+
### Common issues while using eBPF in a local setup:
118+
Since CortexBrain uses BPF maps to manage monitoring data with the pinning operation ([What's Pinning?](https://docs.ebpf.io/linux/concepts/pinning/)) and minikube doesn't mount the the BPF file system automatically we need to do it manually:
119+
120+
- First enter into the minikube node with SSH:
121+
```bash
122+
minikube ssh
123+
```
124+
125+
- Now you need to mount the BPF file system:
126+
```bash
127+
mount -t bpf bpffs /sys/fs/bpf
128+
```
129+
- Since the agent needs to do read and write operations you need to check if the /sys/fs/bpf is in read-write mode:
130+
```bash
131+
ls -ld /sys/fs/bpf
132+
```
133+
This is the expected output:
134+
```bash
135+
drwx-----T 3 root root 0 Sep 4 16:34 /sys/fs/bpf
136+
```
137+
Now the system can create sub-directories and pin maps into the /sys/fs/bpf directory to temporary store all the acquired data
138+
139+
In case your /sys/fs/bpf doesn't have the right permissions you can use this command to setup the **drwx** permissions:
140+
```bash
141+
chmod 700 /sys/fs/bpf
142+
```
143+
144+
145+
## Agent setup
146+
147+
=== "Minikube"
148+
To set up the CortexFlow Agent, first you have to install the core components. The easiest way to install all the core components is by using `cfcli`.
149+
150+
- Firstly, install the CLI using the Cargo package manager:
151+
```bash
152+
cargo install cfcli
153+
```
154+
- Then check where cfcli is installed
155+
```bash
156+
which cfcli
157+
```
158+
You will get a path like this one */home/[USERNAME]/.cargo/bin/cfcli*
159+
160+
- The third step is to install the core components
161+
```bash
162+
cfcli install cortexflow
163+
```
164+
For this step, this is the expected output:
165+
```bash
166+
[SYSTEM] Founded config files
167+
[SYSTEM] Readed configs for env variable: "Kubernetes"
168+
=====> Preparing cortexflow installation
169+
=====> Creating the config files
170+
=====> Insert your cluster environment (e.g. Kubernetes)
171+
```
172+
Here you need to enter your cluster environment (e.g. Kubernetes, Docker Swarm, etc...)
173+
After that, the installation will take place and will take no longer than 1 minute.
174+
<details>
175+
<summary> This is the expected output</summary>
176+
```bash
177+
[SYSTEM] Configuration files saved in path :"/home/cortexflow/.config/cfcli/config.yaml"
178+
179+
180+
[SYSTEM] Readed configs for env variable: "Kubernetes"
181+
=====> Copying installation files
182+
✅ Copied file from https://raw.githubusercontent.com/CortexFlow/CortexBrain/refs/heads/main/core/src/testing/configmap.yaml
183+
✅ Copied file from https://raw.githubusercontent.com/CortexFlow/CortexBrain/refs/heads/main/core/src/testing/configmap-role.yaml
184+
✅ Copied file from https://raw.githubusercontent.com/CortexFlow/CortexBrain/refs/heads/main/core/src/testing/rolebinding.yaml
185+
✅ Copied file from https://raw.githubusercontent.com/CortexFlow/CortexBrain/refs/heads/main/core/src/testing/cortexflow-rolebinding.yaml
186+
✅ Copied file from https://raw.githubusercontent.com/CortexFlow/CortexBrain/refs/heads/main/core/src/testing/identity.yaml
187+
✅ Copied file from https://raw.githubusercontent.com/CortexFlow/CortexBrain/refs/heads/feature/ebpf-core/core/src/testing/agent.yaml
188+
189+
190+
=====> Creating cortexflow namespace
191+
=====> Installing cortexflow components
192+
=====> (1/ 6) Applying configmap.yaml
193+
✅ Applied configmap.yaml
194+
=====> (2/ 6) Applying configmap-role.yaml
195+
✅ Applied configmap-role.yaml
196+
=====> (3/ 6) Applying rolebinding.yaml
197+
✅ Applied rolebinding.yaml
198+
=====> (4/ 6) Applying cortexflow-rolebinding.yaml
199+
✅ Applied cortexflow-rolebinding.yaml
200+
=====> (5/ 6) Applying identity.yaml
201+
✅ Applied identity.yaml
202+
=====> (6/ 6) Applying agent.yaml
203+
✅ Applied agent.yaml
204+
205+
206+
=====> Removing temporary installation files
207+
✅ Removed file configmap.yaml
208+
✅ Removed file configmap-role.yaml
209+
✅ Removed file rolebinding.yaml
210+
✅ Removed file cortexflow-rolebinding.yaml
211+
✅ Removed file identity.yaml
212+
✅ Removed file agent.yaml
213+
=====> installation completed
214+
215+
```
216+
217+
</details>
218+
219+
- Now you can check if the install has been successful using the `status` command:
220+
```bash
221+
cfcli status
222+
```
223+
```bash
224+
🔍 CortexFlow Status Report
225+
==================================================
226+
227+
📦 Namespace Status:
228+
✅ cortexflow namespace: EXISTS
229+
230+
🚀 Pods Status:
231+
✅ cortexflow-agent-ffbb95665-l47dw: Running (1/1)
232+
✅ cortexflow-identity-7579cd5974-4c9hv: Running (2/2)
233+
234+
🌐 Services Status:
235+
🔗 cortexflow-agent: ClusterIP (10.96.88.219)
236+
237+
==================================================
238+
```
239+
- The last step is to do a port-forward to let us access the API through the CLI
240+
```bash
241+
kubectl port-forward svc/cortexflow-agent 9090:9090 -n cortexflow
242+
```
243+
244+
Now the CLI can access the agent service, and you can start calling the agent API with the cfcli `monitoring` commands. At first, let's see which endpoints we can access using the list command
245+
246+
```bash
247+
cfcli monitoring list
248+
```
249+
This is the expected output with the list of agent functions:
250+
```bash
251+
=====> Connected to CortexFlow Server Reflection
252+
Available services:
253+
ActiveConnections
254+
```
255+
Now we can use the `monitoring connections` command to get the latest detected TCP packets.
256+
```bash
257+
cfcli monitoring connections
258+
```
259+
```bash
260+
[SYSTEM] Founded config files
261+
[SYSTEM] Readed configs for env variable: "Kubernetes"
262+
=====> Connecting to cortexflow Client
263+
=====> Connected to CortexFlow Client
264+
{"\"35655\"": "\"143.171.168.192\"", "\"48872\"": "\"133.171.168.192\"", "\"35623\"": "\"148.171.168.192\"", "\"48807\"": "\"173.171.168.192\"", "\"60011\"": "\"136.171.168.192\"", "\"48551\"": "\"163.171.168.192\"", "\"48582\"": "\"129.171.168.192\"", "\"48580\"": "\"133.171.168.192\"", "\"100228\"": "\"147.171.168.192\"", "\"46616\"": "\"133.171.168.192\"", "\"36079\"": "\"136.171.168.192\"", "\"36077\"": "\"136.171.168.192\"", "\"43845\"": "\"131.171.168.192\"", "\"35619\"": "\"136.171.168.192\"", "\"45042\"": "\"135.171.168.192\"", "\"42669\"": "\"148.171.168.192\"", "\"49747\"": "\"143.171.168.192\"", "\"45305\"": "\"147.171.168.192\"", "\"45280\"": "\"147.171.168.192\"", "0": "0", "\"45281\"": "\"147.171.168.192\"", "\"36682\"": "\"136.171.168.192\"", "\"35631\"": "\"148.171.168.192\"", "\"42722\"": "\"148.171.168.192\"", "\"44074\"": "\"136.171.168.192\"", "\"45\"": "\"133.171.168.192\"", "\"44335\"": "\"136.171.168.192\"", "\"35625\"": "\"148.171.168.192\"", "\"32\"": "\"156.171.168.192\"", "\"36073\"": "\"135.171.168.192\"", "\"49748\"": "\"143.171.168.192\"", "\"45282\"": "\"147.171.168.192\"", "\"49380\"": "\"129.171.168.192\"", "\"35620\"": "\"148.171.168.192\"", "\"90399\"": "\"158.171.168.192\"", "\"49077\"": "\"143.171.168.192\"", "\"45312\"": "\"147.171.168.192\"", "\"49383\"": "\"133.171.168.192\"", "\"48581\"": "\"133.171.168.192\"", "\"48809\"": "\"173.171.168.192\"", "\"49379\"": "\"156.171.168.192\"", "\"0\"": "\"173.171.168.192\"", "\"42725\"": "\"148.171.168.192\"", "\"42721\"": "\"148.171.168.192\"", "\"44075\"": "\"136.171.168.192\"", "\"41299\"": "\"135.171.168.192\"", "\"16\"": "\"143.171.168.192\"", "\"44071\"": "\"136.171.168.192\"", "\"42720\"": "\"148.171.168.192\""}
265+
```
266+
267+
268+
=== "Kind "
269+
To set up the CortexFlow Agent, first you have to install the core components. The easiest way to install all the core components is by using `cfcli`.
270+
271+
- Firstly, install the CLI using the Cargo package manager:
272+
```bash
273+
cargo install cfcli
274+
```
275+
- Then check where cfcli is installed
276+
```bash
277+
which cfcli
278+
```
279+
You will get a path like this one */home/[USERNAME]/.cargo/bin/cfcli*
280+
281+
- The third step is to install the core components
282+
```bash
283+
cfcli install cortexflow
284+
```
285+
For this step, this is the expected output:
286+
```bash
287+
[SYSTEM] Founded config files
288+
[SYSTEM] Readed configs for env variable: "Kubernetes"
289+
=====> Preparing cortexflow installation
290+
=====> Creating the config files
291+
=====> Insert your cluster environment (e.g. Kubernetes)
292+
```
293+
Here you need to enter your cluster environment (e.g. Kubernetes, Docker Swarm, etc...)
294+
After that, the installation will take place and will take no longer than 1 minute.
295+
<details>
296+
<summary> This is the expected output</summary>
297+
```bash
298+
[SYSTEM] Configuration files saved in path :"/home/cortexflow/.config/cfcli/config.yaml"
299+
300+
301+
[SYSTEM] Readed configs for env variable: "Kubernetes"
302+
=====> Copying installation files
303+
✅ Copied file from https://raw.githubusercontent.com/CortexFlow/CortexBrain/refs/heads/main/core/src/testing/configmap.yaml
304+
✅ Copied file from https://raw.githubusercontent.com/CortexFlow/CortexBrain/refs/heads/main/core/src/testing/configmap-role.yaml
305+
✅ Copied file from https://raw.githubusercontent.com/CortexFlow/CortexBrain/refs/heads/main/core/src/testing/rolebinding.yaml
306+
✅ Copied file from https://raw.githubusercontent.com/CortexFlow/CortexBrain/refs/heads/main/core/src/testing/cortexflow-rolebinding.yaml
307+
✅ Copied file from https://raw.githubusercontent.com/CortexFlow/CortexBrain/refs/heads/main/core/src/testing/identity.yaml
308+
✅ Copied file from https://raw.githubusercontent.com/CortexFlow/CortexBrain/refs/heads/feature/ebpf-core/core/src/testing/agent.yaml
309+
310+
311+
=====> Creating cortexflow namespace
312+
=====> Installing cortexflow components
313+
=====> (1/ 6) Applying configmap.yaml
314+
✅ Applied configmap.yaml
315+
=====> (2/ 6) Applying configmap-role.yaml
316+
✅ Applied configmap-role.yaml
317+
=====> (3/ 6) Applying rolebinding.yaml
318+
✅ Applied rolebinding.yaml
319+
=====> (4/ 6) Applying cortexflow-rolebinding.yaml
320+
✅ Applied cortexflow-rolebinding.yaml
321+
=====> (5/ 6) Applying identity.yaml
322+
✅ Applied identity.yaml
323+
=====> (6/ 6) Applying agent.yaml
324+
✅ Applied agent.yaml
325+
326+
327+
=====> Removing temporary installation files
328+
✅ Removed file configmap.yaml
329+
✅ Removed file configmap-role.yaml
330+
✅ Removed file rolebinding.yaml
331+
✅ Removed file cortexflow-rolebinding.yaml
332+
✅ Removed file identity.yaml
333+
✅ Removed file agent.yaml
334+
=====> installation completed
335+
336+
```
337+
338+
</details>
339+
340+
- Now you can check if the install has been successful using the `status` command:
341+
```bash
342+
cfcli status
343+
```
344+
```bash
345+
🔍 CortexFlow Status Report
346+
==================================================
347+
348+
📦 Namespace Status:
349+
✅ cortexflow namespace: EXISTS
350+
351+
🚀 Pods Status:
352+
✅ cortexflow-agent-ffbb95665-l47dw: Running (1/1)
353+
✅ cortexflow-identity-7579cd5974-4c9hv: Running (2/2)
354+
355+
🌐 Services Status:
356+
🔗 cortexflow-agent: ClusterIP (10.96.88.219)
357+
358+
==================================================
359+
```
360+
- The last step is to do a port-forward to let us access the API through the CLI
361+
```bash
362+
kubectl port-forward svc/cortexflow-agent 9090:9090 -n cortexflow
363+
```
364+
365+
Now the CLI can access the agent service, and you can start calling the agent API with the cfcli `monitoring` commands. At first, let's see which endpoints we can access using the list command
366+
367+
```bash
368+
cfcli monitoring list
369+
```
370+
This is the expected output with the list of agent functions:
371+
```bash
372+
=====> Connected to CortexFlow Server Reflection
373+
Available services:
374+
ActiveConnections
375+
```
376+
Now we can use the `monitoring connections` command to get the latest detected TCP packets.
377+
```bash
378+
cfcli monitoring connections
379+
```
380+
```bash
381+
[SYSTEM] Founded config files
382+
[SYSTEM] Readed configs for env variable: "Kubernetes"
383+
=====> Connecting to cortexflow Client
384+
=====> Connected to CortexFlow Client
385+
{"\"35655\"": "\"143.171.168.192\"", "\"48872\"": "\"133.171.168.192\"", "\"35623\"": "\"148.171.168.192\"", "\"48807\"": "\"173.171.168.192\"", "\"60011\"": "\"136.171.168.192\"", "\"48551\"": "\"163.171.168.192\"", "\"48582\"": "\"129.171.168.192\"", "\"48580\"": "\"133.171.168.192\"", "\"100228\"": "\"147.171.168.192\"", "\"46616\"": "\"133.171.168.192\"", "\"36079\"": "\"136.171.168.192\"", "\"36077\"": "\"136.171.168.192\"", "\"43845\"": "\"131.171.168.192\"", "\"35619\"": "\"136.171.168.192\"", "\"45042\"": "\"135.171.168.192\"", "\"42669\"": "\"148.171.168.192\"", "\"49747\"": "\"143.171.168.192\"", "\"45305\"": "\"147.171.168.192\"", "\"45280\"": "\"147.171.168.192\"", "0": "0", "\"45281\"": "\"147.171.168.192\"", "\"36682\"": "\"136.171.168.192\"", "\"35631\"": "\"148.171.168.192\"", "\"42722\"": "\"148.171.168.192\"", "\"44074\"": "\"136.171.168.192\"", "\"45\"": "\"133.171.168.192\"", "\"44335\"": "\"136.171.168.192\"", "\"35625\"": "\"148.171.168.192\"", "\"32\"": "\"156.171.168.192\"", "\"36073\"": "\"135.171.168.192\"", "\"49748\"": "\"143.171.168.192\"", "\"45282\"": "\"147.171.168.192\"", "\"49380\"": "\"129.171.168.192\"", "\"35620\"": "\"148.171.168.192\"", "\"90399\"": "\"158.171.168.192\"", "\"49077\"": "\"143.171.168.192\"", "\"45312\"": "\"147.171.168.192\"", "\"49383\"": "\"133.171.168.192\"", "\"48581\"": "\"133.171.168.192\"", "\"48809\"": "\"173.171.168.192\"", "\"49379\"": "\"156.171.168.192\"", "\"0\"": "\"173.171.168.192\"", "\"42725\"": "\"148.171.168.192\"", "\"42721\"": "\"148.171.168.192\"", "\"44075\"": "\"136.171.168.192\"", "\"41299\"": "\"135.171.168.192\"", "\"16\"": "\"143.171.168.192\"", "\"44071\"": "\"136.171.168.192\"", "\"42720\"": "\"148.171.168.192\""}
386+
```
387+

0 commit comments

Comments
 (0)