Skip to content

Commit 1dee38f

Browse files
slovernslovern
authored
Merge pull request #108073 from kquinn1204/OCPBUGS-31038-reopen-88557
OCPBUGS-31038 Openshift multus docs mention plugin chaining
2 parents a5fd17a + a244379 commit 1dee38f

4 files changed

Lines changed: 303 additions & 0 deletions

File tree

‎_topic_maps/_topic_map.yml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1675,6 +1675,8 @@ Topics:
16751675
File: configuring-master-interface
16761676
- Name: Removing an additional network
16771677
File: removing-additional-network
1678+
- Name: Enabling multi-networking for advanced use cases with CNI Plugin chaining
1679+
File: about-chaining
16781680
- Name: About virtual routing and forwarding
16791681
File: about-virtual-routing-and-forwarding
16801682
- Name: Assigning a secondary network to a VRF

‎modules/cnf-about-cni-chaining.adoc‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
// Module included in the following assemblies:
2+
//
3+
// networking/multiple_networks/secondary_networks/about-chaining.adoc
4+
5+
:_mod-docs-content-type: CONCEPT
6+
[id="cnf-about-cni-chaining_{context}"]
7+
= About CNI chaining
8+
9+
[role="_abstract"]
10+
CNI plugin chaining allows pods to use multiple network interfaces. This enables advanced configurations such as traffic isolation and prioritized routing through granular traffic policies.
11+
12+
By using CNI plugin chaining, different types of traffic can be isolated to meet performance, security, and compliance requirements, providing greater flexibility in network design and traffic management.
13+
14+
Some scenarios where this might be useful include:
15+
16+
* *Multi-Network topologies*: Enables you to attach pods to multiple networks, each with its own traffic policy, where relevant.
17+
* *Traffic isolation*: Provides separate networks for management, storage, and application traffic to ensure each has the appropriate security and QoS settings.
18+
* *Custom routing rules*: Ensures that specific traffic, for example SIP traffic, always uses a designated network interface, while other traffic follows the default network.
19+
* *Enhanced network performance*: Allows you to prioritize certain traffic types or manage congestion by directing them through dedicated network interfaces.

‎modules/plugin-chaining-with-multus-cni.adoc‎

Lines changed: 268 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,268 @@
1+
// Module included in the following assemblies:
2+
//
3+
// networking/multiple_networks/secondary_networks/about-chaining.adoc
4+
5+
:_mod-docs-content-type: PROCEDURE
6+
[id="configuring-plugin-chaining-with-multus-cni_{context}"]
7+
= Configuring plugin chaining with the route-override CNI plugin
8+
9+
[role="_abstract"]
10+
Plugin chaining allows you to configure multiple CNI plugins to be applied sequentially to the same network interface, where each plugin in the chain processes the interface in order.
11+
12+
When you define a `NetworkAttachmentDefinition` (NAD) with a `plugins` array, the first plugin can create the interface, and a second plugin can modify its routing configuration.
13+
14+
The `route-override` CNI plugin is commonly used as the second plugin in a chain to modify the routing configuration of an interface created by the first plugin. It supports the following operations:
15+
16+
* `addroutes:` Add static routes to direct traffic for specific destination networks through the interface.
17+
* `delroutes:` Remove specific routes from the interface.
18+
* `flushroutes:` Remove all routes from the interface.
19+
* `flushgateway:` Remove the default gateway route from the interface.
20+
21+
The following example demonstrates plugin chaining by configuring a pod with two additional network interfaces, each on a separate VLAN with custom routing:
22+
23+
* `eth1` on the `192.168.100.0/24` network (VLAN 100), with a static route directing `10.0.0.0/8` traffic through this interface.
24+
* `eth2` on the `192.168.200.0/24` network (VLAN 200), with a static route directing `172.16.0.0/12` traffic through this interface.
25+
26+
Each interface uses a chain of two plugins: `macvlan` to create the interface on a VLAN, and `route-override` to add static routes that direct specific traffic through that interface.
27+
28+
.Prerequisites
29+
30+
* Install the OpenShift CLI (`oc`).
31+
* An account with `cluster-admin` privileges.
32+
33+
.Procedure
34+
35+
. Create a namespace for the example by running the following command:
36+
+
37+
[source,terminal]
38+
----
39+
$ oc create namespace chain-example
40+
----
41+
42+
. Create the first NetworkAttachmentDefinition (NAD) with a chained plugin configuration.
43+
44+
.. Create a YAML file, such as `management.yaml`, to define a NAD that configures a new interface, `eth1`, on VLAN 100 with the following configuration:
45+
+
46+
[source,yaml]
47+
----
48+
apiVersion: k8s.cni.cncf.io/v1
49+
kind: NetworkAttachmentDefinition
50+
metadata:
51+
name: management-net
52+
namespace: chain-example
53+
spec:
54+
config: '{
55+
"cniVersion": "1.0.0",
56+
"name": "management-net",
57+
"plugins": [
58+
{
59+
"type": "macvlan",
60+
"master": "br-ex",
61+
"vlan": 100,
62+
"mode": "bridge",
63+
"ipam": {
64+
"type": "static",
65+
"addresses": [
66+
{
67+
"address": "192.168.100.10/24",
68+
"gateway": "192.168.100.1"
69+
}
70+
]
71+
}
72+
},
73+
{
74+
"type": "route-override",
75+
"addroutes": [
76+
{
77+
"dst": "10.0.0.0/8",
78+
"gw": "192.168.100.1"
79+
}
80+
]
81+
}
82+
]
83+
}'
84+
----
85+
86+
. Create the NAD by running the following command:
87+
+
88+
[source,terminal]
89+
----
90+
$ oc apply -f management.yaml
91+
----
92+
93+
. Create the second NAD with a chained plugin configuration.
94+
95+
.. Create a YAML file, such as `sip.yaml`, to define a NAD that configures a new interface, `eth2`, on VLAN 200 with the following configuration:
96+
+
97+
[source,yaml]
98+
----
99+
apiVersion: k8s.cni.cncf.io/v1
100+
kind: NetworkAttachmentDefinition
101+
metadata:
102+
name: sip-net
103+
namespace: chain-example
104+
spec:
105+
config: '{
106+
"cniVersion": "1.0.0",
107+
"name": "sip-net",
108+
"plugins": [
109+
{
110+
"type": "macvlan",
111+
"master": "br-ex",
112+
"vlan": 200,
113+
"mode": "bridge",
114+
"ipam": {
115+
"type": "static",
116+
"addresses": [
117+
{
118+
"address": "192.168.200.10/24",
119+
"gateway": "192.168.200.1"
120+
}
121+
]
122+
}
123+
},
124+
{
125+
"type": "route-override",
126+
"addroutes": [
127+
{
128+
"dst": "172.16.0.0/12",
129+
"gw": "192.168.200.1"
130+
}
131+
]
132+
}
133+
]
134+
}'
135+
----
136+
137+
. Create the NAD by running the following command:
138+
+
139+
[source,terminal]
140+
----
141+
$ oc apply -f sip.yaml
142+
----
143+
144+
. Attach the `NetworkAttachmentDefinition` resources to a pod by creating a pod definition file, such as `pod.yaml`, with the following configuration:
145+
+
146+
[source,yaml]
147+
----
148+
apiVersion: v1
149+
kind: Pod
150+
metadata:
151+
name: chain-test-pod
152+
namespace: chain-example
153+
labels:
154+
app: chain-test
155+
annotations:
156+
k8s.v1.cni.cncf.io/networks: '[
157+
{ "name": "management-net", "interface": "eth1" },
158+
{ "name": "sip-net", "interface": "eth2" }
159+
]'
160+
spec:
161+
securityContext:
162+
runAsNonRoot: true
163+
seccompProfile:
164+
type: RuntimeDefault
165+
containers:
166+
- name: test-container
167+
image: registry.access.redhat.com/ubi9/ubi:latest
168+
command: ["sleep", "infinity"]
169+
securityContext:
170+
allowPrivilegeEscalation: false
171+
capabilities:
172+
drop: ["ALL"]
173+
----
174+
175+
. Create the pod by running the following command:
176+
+
177+
[source,terminal]
178+
----
179+
$ oc apply -f pod.yaml
180+
----
181+
182+
. Verify the pod is running with the following command:
183+
+
184+
[source,terminal]
185+
----
186+
$ oc wait --for=condition=Ready pod/chain-test-pod -n chain-example --timeout=120s
187+
----
188+
+
189+
Example output:
190+
+
191+
[source,terminal]
192+
----
193+
pod/chain-test-pod condition met
194+
----
195+
196+
.Verification
197+
198+
. Run the following command to list all network interfaces and their assigned IP addresses inside the pod. This verifies that the pod has the additional interfaces configured by plugin chaining:
199+
+
200+
[source,terminal]
201+
----
202+
$ oc exec chain-test-pod -n chain-example -- ip a
203+
----
204+
+
205+
Example output:
206+
+
207+
[source,terminal]
208+
----
209+
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
210+
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
211+
inet 127.0.0.1/8 scope host lo
212+
valid_lft forever preferred_lft forever
213+
inet6 ::1/128 scope host
214+
valid_lft forever preferred_lft forever
215+
2: eth0@if31: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 8901 qdisc noqueue state UP
216+
link/ether 0a:58:0a:83:02:19 brd ff:ff:ff:ff:ff:ff link-netnsid 0
217+
inet 10.131.2.25/23 brd 10.131.3.255 scope global eth0
218+
valid_lft forever preferred_lft forever
219+
inet6 fe80::858:aff:fe83:219/64 scope link
220+
valid_lft forever preferred_lft forever
221+
3: eth1@if5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc noqueue state UP qlen 1000
222+
link/ether aa:25:73:ff:a7:00 brd ff:ff:ff:ff:ff:ff link-netnsid 0
223+
inet 192.168.100.10/24 brd 192.168.100.255 scope global eth1
224+
valid_lft forever preferred_lft forever
225+
inet6 fe80::a825:73ff:feff:a700/64 scope link
226+
valid_lft forever preferred_lft forever
227+
4: eth2@if5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc noqueue state UP qlen 1000
228+
link/ether aa:a4:6c:4e:e8:97 brd ff:ff:ff:ff:ff:ff link-netnsid 0
229+
inet 192.168.200.10/24 brd 192.168.200.255 scope global eth2
230+
valid_lft forever preferred_lft forever
231+
inet6 fe80::a8a4:6cff:fe4e:e897/64 scope link
232+
valid_lft forever preferred_lft forever
233+
----
234+
+
235+
This output shows the pod has three network interfaces:
236+
237+
* `eth0:` The default interface, connected to the cluster network.
238+
* `eth1:` The first additional interface from `management-net`, with IP `192.168.100.10`.
239+
* `eth2:` The second additional interface from `sip-net`, with IP `192.168.200.10`.
240+
241+
. Run the following command to verify that the `route-override` plugin added the expected static routes:
242+
+
243+
[source,terminal]
244+
----
245+
$ oc exec chain-test-pod -n chain-example -- ip route
246+
----
247+
+
248+
Example output:
249+
+
250+
[source,terminal]
251+
----
252+
default via 10.132.0.1 dev eth0
253+
10.0.0.0/8 via 192.168.100.1 dev eth1
254+
10.132.0.0/23 dev eth0 proto kernel scope link src 10.132.1.97
255+
10.132.0.0/14 via 10.132.0.1 dev eth0
256+
100.64.0.0/16 via 10.132.0.1 dev eth0
257+
169.254.0.5 via 10.132.0.1 dev eth0
258+
172.16.0.0/12 via 192.168.200.1 dev eth2
259+
172.30.0.0/16 via 10.132.0.1 dev eth0
260+
192.168.100.0/24 dev eth1 proto kernel scope link src 192.168.100.10
261+
192.168.200.0/24 dev eth2 proto kernel scope link src 192.168.200.10
262+
----
263+
+
264+
This output confirms that the `route-override` plugin in each chain added the expected static routes:
265+
266+
* For `10.0.0.0/8 via 192.168.100.1 dev eth1`, traffic destined for `10.0.0.0/8` is routed through `eth1` via the `management-net` gateway. This route was added by the `route-override` plugin in the `management-net` chain.
267+
* For `172.16.0.0/12 via 192.168.200.1 dev eth2`, traffic destined for `172.16.0.0/12` is routed through `eth2` via the `sip-net` gateway. This route was added by the `route-override` plugin in the `sip-net` chain.
268+
* The connected subnet routes (`192.168.100.0/24` and `192.168.200.0/24`) were created by the `macvlan` plugin, while the default route uses `eth0`, the cluster network interface.

‎networking/multiple_networks/secondary_networks/about-chaining.adoc‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
:_mod-docs-content-type: ASSEMBLY
2+
[id="enabling-multi-networking-for-advanced-use-cases-with-cni-plugin-chaining"]
3+
= Enabling multi-networking for advanced use cases with CNI plugin chaining
4+
include::_attributes/common-attributes.adoc[]
5+
:context: enabling-multi-networking-for-advanced-use-cases-with-cni-plugin-chaining
6+
7+
[role="_abstract"]
8+
You can use Container Network Interface (CNI) plugin chaining to enable advanced multi-networking use cases for your pods.
9+
10+
toc::[]
11+
12+
include::modules/cnf-about-cni-chaining.adoc[leveloffset=+1]
13+
14+
include::modules/plugin-chaining-with-multus-cni.adoc[leveloffset=+1]

0 commit comments

Comments
 (0)