Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 19 additions & 14 deletions automation/vars/multi-namespace-skmo.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ vas:
build_output: namespace.yaml

- name: nncp-configuration # stage 1
path: examples/va/multi-namespace/control-plane/networking/nncp
path: examples/va/multi-namespace-skmo/control-plane/networking/nncp
wait_conditions:
# We don't wait for these NNCPs at this stage, because we'll wait for
# both namespaces in the next stage so that they can deploy in parallel
Expand All @@ -26,24 +26,24 @@ vas:
--timeout=5m
values:
- name: network-values
src_file: values.yaml
src_file: ../../../../multi-namespace/control-plane/networking/nncp/values.yaml
build_output: nncp.yaml

- name: nncp-configuration2 # stage 2
path: examples/va/multi-namespace/control-plane2/networking/nncp
path: examples/va/multi-namespace-skmo/control-plane2/networking/nncp
wait_conditions:
- >-
oc -n openstack wait nncp
oc -n openstack2 wait nncp
-l osp/nncm-config-type=standard
--for jsonpath='{.status.conditions[0].reason}'=SuccessfullyConfigured
--timeout=5m
values:
- name: network-values2
src_file: values.yaml
src_file: ../../../../multi-namespace/control-plane2/networking/nncp/values.yaml
build_output: nncp2.yaml

- name: network-configuration # stage 3
path: examples/va/multi-namespace/control-plane/networking
path: examples/va/multi-namespace-skmo/control-plane/networking
wait_conditions:
- >-
oc -n metallb-system wait pod
Expand All @@ -52,11 +52,11 @@ vas:
--timeout=5m
values:
- name: network-values
src_file: nncp/values.yaml
src_file: ../../../multi-namespace/control-plane/networking/nncp/values.yaml
build_output: network.yaml

- name: network-configuration2 # stage 4
path: examples/va/multi-namespace/control-plane2/networking
path: examples/va/multi-namespace-skmo/control-plane2/networking
wait_conditions:
- >-
oc -n metallb-system wait pod
Expand All @@ -65,7 +65,7 @@ vas:
--timeout=5m
values:
- name: network-values2
src_file: nncp/values.yaml
src_file: ../../../multi-namespace/control-plane2/networking/nncp/values.yaml
build_output: network2.yaml

- pre_stage_run: # stage 5
Expand Down Expand Up @@ -222,7 +222,7 @@ vas:
inventory: "${HOME}/ci-framework-data/artifacts/zuul_inventory.yml"

- name: edpm-nodeset # stage 7
path: examples/va/multi-namespace/edpm/nodeset
path: examples/va/multi-namespace-skmo/edpm/nodeset
wait_conditions:
# We don't wait for this namespace's OpenStackDataPlaneNodeSet at
# this stage, because we'll wait for both namespaces in the next
Expand All @@ -233,7 +233,7 @@ vas:
--timeout=5m
values:
- name: edpm-nodeset-values
src_file: values.yaml
src_file: ../../../multi-namespace/edpm/nodeset/values.yaml
build_output: nodeset.yaml

- pre_stage_run: # stage 8
Expand All @@ -242,7 +242,7 @@ vas:
source: "../../playbooks/multi-namespace/ns2_osdp_services.yaml"
inventory: "${HOME}/ci-framework-data/artifacts/zuul_inventory.yml"
name: edpm-nodeset2
path: examples/va/multi-namespace/edpm2/nodeset
path: examples/va/multi-namespace-skmo/edpm2/nodeset
wait_conditions:
- >-
oc -n openstack wait
Expand All @@ -254,10 +254,15 @@ vas:
--timeout=10m
values:
- name: edpm-nodeset2-values
src_file: values.yaml
src_file: ../../../multi-namespace/edpm2/nodeset/values.yaml
build_output: nodeset2.yaml

- name: edpm-deployment # stage 9
- pre_stage_run: # stage 9
- name: Recreate EDPM deployments when NodeSet config hash drifted
type: playbook
source: "skmo/recreate-edpm-deployment-if-stale.yaml"
inventory: "${HOME}/ci-framework-data/artifacts/zuul_inventory.yml"
name: edpm-deployment
path: examples/va/multi-namespace/edpm
wait_conditions:
# We don't wait for this namespace's OpenStackDataPlaneDeployment at
Expand Down
7 changes: 3 additions & 4 deletions examples/va/multi-namespace-skmo/application-credentials.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,11 +107,10 @@ coordination to automate this redeployment.

## Enabling Application Credentials

AC is **enabled by default** — `control-plane2/application-credentials.yaml`
is included in `kustomization.yaml` so it is applied on the very first OSCP
deploy. Services start with `v3applicationcredential` auth from day one.
AC is **enabled** in `control-plane2/application-credentials.yaml`
is included in `kustomization.yaml`.

Because the OSCP is created with AC already active, the OSCP `Ready` condition
The OSCP `Ready` condition
implicitly gates on all `KeystoneApplicationCredential` CRs being Ready (the
service operators finish reconciling only once they have an
`ApplicationCredentialSecret` set). EDPM compute nodes therefore receive AC
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
---
apiVersion: v1
kind: ConfigMap
metadata:
name: edpm-nodeset-values
data:
nodeset:
ansible:
ansibleVars:
edpm_ovn_bridge_mappings:
- "datacentre:br-ex"
- "octavia:octbr"
edpm_network_config_template: |
---
{% set mtu_list = [ctlplane_mtu] %}
{% for network in nodeset_networks %}
{% set _ = mtu_list.append(lookup('vars', networks_lower[network] ~ '_mtu')) %}
{%- endfor %}
{% set min_viable_mtu = mtu_list | max %}
network_config:
- type: interface
name: nic1
use_dhcp: true
mtu: {{ min_viable_mtu }}
- type: ovs_bridge
name: {{ neutron_physical_bridge_name }}
mtu: {{ min_viable_mtu }}
use_dhcp: false
dns_servers: {{ ctlplane_dns_nameservers }}
domain: {{ dns_search_domains }}
addresses:
- ip_netmask: {{ ctlplane_ip }}/{{ ctlplane_cidr }}
routes: {{ ctlplane_host_routes }}
members:
- type: interface
name: nic2
mtu: {{ min_viable_mtu }}
# force the MAC address of the bridge to this interface
primary: true
{% for network in nodeset_networks %}
- type: vlan
mtu: {{ lookup('vars', networks_lower[network] ~ '_mtu') }}
vlan_id: {{ lookup('vars', networks_lower[network] ~ '_vlan_id') }}
addresses:
- ip_netmask:
{{ lookup('vars', networks_lower[network] ~ '_ip') }}/{{ lookup('vars', networks_lower[network] ~ '_cidr') }}
routes: {{ lookup('vars', networks_lower[network] ~ '_host_routes') }}
{% endfor %}
- type: ovs_bridge
name: octbr
mtu: {{ min_viable_mtu }}
use_dhcp: false
members:
- type: vlan
mtu: {{ min_viable_mtu }}
vlan_id: 23
device: nic2
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
# OVN Octavia compute bridge overlay for EDPM nodesets.
apiVersion: kustomize.config.k8s.io/v1alpha1
kind: Component

patches:
- target:
kind: ConfigMap
name: edpm-nodeset-values
path: edpm-octavia-ansible-vars.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
# Provide octavia-network-values ConfigMap for use by replacements.
apiVersion: kustomize.config.k8s.io/v1alpha1
kind: Component

resources:
- octavia-network-values.yaml

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

*values.yaml should ideally exist in the examples dir and not in va, dt or lib dirs. The *values.yaml files are exposed in the examples dir as the interfaces for user customization.

Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# local-config: octavia keys for replacements (static oc kustomize; ci-gen also adds octavia at deploy)
---
apiVersion: v1
kind: ConfigMap
metadata:
name: octavia-network-values
annotations:
config.kubernetes.io/local-config: "true"
data:
octavia:
dnsDomain: octavia.example.com
mtu: 1500
vlan: 23

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This vlan: 23 is defined here and consumed in [1]. But if this value was changed to something else, I see that it would still be hard-coded to 23 here [2]. Not a blocking issue, but just something to be aware of.

[1] https://github.com/openstack-k8s-operators/architecture/pull/759/changes#diff-f56986a4376344555d922a11476626568461efa2c013fcf4ecd8720cbe518209R141-R150
[2] https://github.com/openstack-k8s-operators/architecture/pull/759/changes#diff-78a76258db67b22a42639a642b302d2e51593283370201c3da20951ebbaad255R56

base_iface: enp7s0
subnets:
- allocationRanges:
- end: 172.23.0.250
start: 172.23.0.100
cidr: 172.23.0.0/24
name: subnet1
vlan: 23
net-attach-def: |
{
"cniVersion": "0.3.1",
"name": "octavia",
"type": "bridge",
"bridge": "octbr",
"ipam": {
"type": "whereabouts",
"range": "172.23.0.0/24",
"range_start": "172.23.0.30",
"range_end": "172.23.0.70",
"routes": [
{
"dst": "172.24.0.0/16",
"gw": "172.23.0.150"
}
]
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
# Patch service-values ConfigMap with Octavia service parameters.
apiVersion: kustomize.config.k8s.io/v1alpha1
kind: Component

patches:
- target:
kind: ConfigMap
name: service-values
path: octavia-service-values.yaml

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

*values.yaml should ideally exist in the examples dir and not in va, dt or lib dirs. The *values.yaml files are exposed in the examples dir as the interfaces for user customization. Perhaps you had a specific reason for doing this though, as I see it was done a few times with different *values.yaml files. Could you explain the rationale behind it?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No rationale, I moved them to va/ during the first refactoring to build the octavia-controlplane component chain, but didn't realize it belongs examples dir by convention, will move.

Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# Octavia keys for service-values ConfigMap (shared by central and leaf control planes)
---
apiVersion: v1
kind: ConfigMap
metadata:
name: service-values
annotations:
config.kubernetes.io/local-config: "true"
data:
ovn:
ovnController:
nicMappings:
octavia: octbr
octavia:
enabled: true
amphoraImageContainerImage: quay.io/gthiemonge/octavia-amphora-image
apacheContainerImage: registry.redhat.io/ubi9/httpd-24:latest
octaviaAPI:
networkAttachments:
- internalapi
octaviaHousekeeping:
networkAttachments:
- octavia
octaviaHealthManager:
networkAttachments:
- octavia
octaviaWorker:
networkAttachments:
- octavia

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Octavia replacement blocks in this file and examples/va/multi-namespace-skmo/control-plane2/kustomization.yaml are identical. While not a blocking issue, it might be cleaner if that config could be extracted to a higher-level common kustomization.yaml that both sub-dirs could then include as a resource/component. In fact, it could be placed in va/multi-namespace-skmo and moved out of examples entirely, which would make the inclusion cleaner too.

Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,11 @@ kind: Component

components:
- ../../multi-namespace/control-plane
- ../../../../va/multi-namespace-skmo/octavia-controlplane

resources:
- networking/octavia-netattach.yaml
- octavia-ca-passphrase.yaml

patches:
- target:
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- ../../../multi-namespace/control-plane/networking
- octavia-netattach.yaml

components:
- ../../../../../va/multi-namespace-skmo/networking-octavia

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All the config here is identical across all three nodes. Do we need to use a label selector? Could we just drop the labelSelector and apply to all nodes? Or are there OCP nodes we don't want to target, and hence the specific selectors?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at it, the selectors really seem redundant. However I'm not sure about @vakwetu originial idea/purpose (maybe for future testing expansion), so I'd keep it if it's not a blocker.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No worries. We can always improve it later in a follow-up if @vakwetu agrees.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vakwetu confirmed that labelSelector has no purpose, so I removed it

Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

# ci_gen_kustomize_values writes env-specific network-values to the base
# values.yaml before build (see automation vars src_file).
resources:
- ../../../../multi-namespace/control-plane/networking/nncp

components:
- ../../../components/octavia-network-values

patches:
- target:
kind: NodeNetworkConfigurationPolicy
patch: |-
- op: add
path: /spec/desiredState/interfaces/-
value:
description: Octavia vlan host interface
name: octavia
state: up
type: vlan
vlan:
base-iface: _replaced_
id: _replaced_
- target:
kind: NodeNetworkConfigurationPolicy
patch: |-
- op: add
path: /spec/desiredState/interfaces/-
value:
description: Octavia bridge
mtu: 1500
name: octbr
type: linux-bridge
bridge:
options:
stp:
enabled: false
port:
- name: octavia

replacements:
- source:
kind: ConfigMap
name: octavia-network-values
fieldPath: data.octavia.base_iface
targets:
- select:
kind: NodeNetworkConfigurationPolicy
fieldPaths:
- spec.desiredState.interfaces.[name=octavia].vlan.base-iface
- source:
kind: ConfigMap
name: octavia-network-values
fieldPath: data.octavia.vlan
targets:
- select:
kind: NodeNetworkConfigurationPolicy
fieldPaths:
- spec.desiredState.interfaces.[name=octavia].vlan.id

@abays abays Jun 29, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When this is built (via [1]), there is no associated NamespaceTransformer. As a result, it has no metadata.namespace, which could be a problem.

[1] https://github.com/openstack-k8s-operators/architecture/pull/759/changes#diff-7a39d8689d0ab40481d5adb8501faa7ba4317ce05bb6b3754f1b371f886db5d8

Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
---
apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
name: octavia
namespace: openstack
labels:
osp/net: octavia
osp/net-attach-def-type: standard
spec:
config: |
_replaced_
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
apiVersion: v1
kind: Secret
metadata:
name: octavia-ca-passphrase
type: Opaque
stringData:
server-ca-passphrase: "12345678"
Loading
Loading