Skip to content

Commit 9e3fbb3

Browse files
mnasiadkamnasiadka
authored
Merge pull request #285 from stackhpc/upstream/master-2026-06-01
Synchronise master with upstream
2 parents 3bbf80a + e9abc26 commit 9e3fbb3

102 files changed

Lines changed: 3615 additions & 728 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎doc/source/admin/config-neutron-server-processes.rst‎

Lines changed: 163 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,163 @@
1+
.. _config-neutron-server-processes:
2+
3+
================================
4+
Neutron server-side processes
5+
================================
6+
7+
On controller nodes, what is commonly referred to as ``neutron-server`` is
8+
actually a set of cooperating processes. Each process has a dedicated role:
9+
serving the REST API, handling RPC from agents, running periodic tasks, or
10+
performing ML2/OVN maintenance work.
11+
12+
Since OpenStack Epoxy (2025.1), the Neutron API is served exclusively through
13+
a WSGI server (typically uWSGI). The companion processes listed below are
14+
started as separate services alongside the API.
15+
16+
17+
Overview
18+
--------
19+
20+
The following table summarizes the Neutron server-side processes:
21+
22+
.. list-table::
23+
:header-rows: 1
24+
:widths: 15 30 20 25
25+
26+
* - Process
27+
- Executable / entry point
28+
- When it runs
29+
- Key configuration
30+
* - API
31+
- uWSGI workers loading ``neutron.wsgi.api:application``
32+
- Always
33+
- uWSGI ``processes``, :ref:`config-wsgi`
34+
* - RPC
35+
- ``neutron-rpc-server`` (spawns ``rpc worker`` and
36+
``rpc reports worker`` child processes)
37+
- When agents require RPC
38+
- ``api_workers``, ``rpc_workers``, ``rpc_state_report_workers``
39+
* - Periodic
40+
- ``neutron-periodic-workers`` (runs plugin periodic tasks as threads)
41+
- With any ML2 mechanism driver and WSGI API
42+
- ``periodic_interval``, ``periodic_fuzzy_delay``
43+
* - Maintenance
44+
- ``neutron-ovn-maintenance-worker``
45+
- ML2/OVN deployments only
46+
- See :ref:`maintenance_worker`
47+
48+
All of these processes load the same ``neutron.conf`` and plugin configuration
49+
files (for example ``ml2_conf.ini``). Only these server-side processes connect
50+
to the Neutron database; agents must not connect directly.
51+
52+
53+
API workers
54+
-----------
55+
56+
The API workers serve the Neutron REST API. Each worker is a uWSGI process
57+
that loads the WSGI application entry point
58+
``neutron.wsgi.api:application``.
59+
60+
The number of API workers is configured in the uWSGI configuration file
61+
(``processes`` option). Deployers should also review the worker tuning
62+
guidance in :doc:`config-wsgi`.
63+
64+
.. note::
65+
66+
ML2/OVN requires the uWSGI ``start-time = %t`` parameter so that API
67+
workers can register themselves in the OVN hash ring during
68+
initialization. See :doc:`config-wsgi` for details.
69+
70+
71+
RPC workers
72+
-------------
73+
74+
The ``neutron-rpc-server`` service handles RPC communication between the
75+
Neutron server and agents (for example DHCP, L3, and OVS agents). When
76+
started, it spawns two types of child worker processes:
77+
78+
* **rpc worker** — handles general RPC requests from agents and other
79+
OpenStack services. The number of processes is controlled by
80+
``[DEFAULT] rpc_workers``.
81+
82+
* **rpc reports worker** — dedicated to processing agent state reports
83+
(heartbeats). The number of processes is controlled by
84+
``[DEFAULT] rpc_state_report_workers``.
85+
86+
When the Neutron API is served by an external web server (such as Apache with
87+
``mod_wsgi``), RPC listeners cannot run inside the API process. In that case,
88+
``neutron-rpc-server`` must be started as a separate service. See
89+
:doc:`config-wsgi` for an example command.
90+
91+
.. note::
92+
93+
If the ML2/OVN mechanism driver is used without additional agents that
94+
require RPC, both ``rpc_workers`` and ``rpc_state_report_workers`` can be
95+
set to ``0``. See :doc:`ovn/rpc` for more information.
96+
97+
98+
Periodic workers
99+
----------------
100+
101+
The ``neutron-periodic-workers`` service runs periodic tasks registered by
102+
the ML2 plugin and service plugins (for example DHCP agent scheduling,
103+
quota synchronization, and agent health checks). These tasks are collected
104+
via each plugin's ``get_workers()`` method and executed as threads inside
105+
a single process (internally referred to as the **services worker**).
106+
107+
This process is spawned for any deployment using the Neutron API WSGI module
108+
with an ML2 mechanism driver.
109+
110+
111+
Maintenance worker
112+
------------------
113+
114+
The ``neutron-ovn-maintenance-worker`` service is required only when using
115+
the ML2/OVN mechanism driver. It synchronizes the Neutron and OVN databases
116+
and runs periodic inconsistency checks.
117+
118+
For details on what the maintenance worker does and how plugins can register
119+
additional periodic tasks, see :ref:`maintenance_worker`.
120+
121+
122+
Process identification
123+
----------------------
124+
125+
Neutron child processes set descriptive process titles (for example
126+
``api worker``, ``rpc worker``, ``rpc reports worker``, ``periodic worker``,
127+
``services worker``, and ``maintenance worker``) to make them easy to identify
128+
with tools such as ``ps``.
129+
130+
This behavior is controlled by the ``[DEFAULT] setproctitle`` option in
131+
``neutron.conf``. When set to ``brief``, process titles are shorter and easier
132+
to read. The default is ``on``, which appends the original command string for
133+
backwards compatibility with scripts that match on the old process names.
134+
135+
136+
Configuration reference
137+
-----------------------
138+
139+
The following ``neutron.conf`` options control worker counts and periodic
140+
task behavior:
141+
142+
``api_workers``
143+
Number of API worker processes. If not set, defaults to the number of CPU
144+
cores, capped by available memory.
145+
146+
``rpc_workers``
147+
Number of RPC worker processes. If not set, defaults to half the number of
148+
API workers. Set to ``0`` to disable RPC workers entirely.
149+
150+
``rpc_state_report_workers``
151+
Number of RPC worker processes dedicated to agent state reports. Defaults
152+
to ``1``. Set to ``0`` to disable the dedicated state-report workers.
153+
154+
``periodic_interval``
155+
Seconds between running periodic tasks. Defaults to ``40``.
156+
157+
``periodic_fuzzy_delay``
158+
Random delay range (in seconds) when starting the periodic task scheduler,
159+
to reduce stampeding across workers. Defaults to ``5``. Set to ``0`` to
160+
disable.
161+
162+
For the full list of options and their defaults, see
163+
:doc:`../configuration/neutron`.

‎doc/source/admin/config-wsgi.rst‎

Lines changed: 65 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -72,13 +72,72 @@ serve this job:
7272
7373
.. end
7474
75+
Start Neutron periodic workers
76+
------------------------------
77+
78+
When Neutron API is served by a web server, ML2 plugin periodic tasks are
79+
started in a dedicated process:
80+
81+
.. code-block:: console
82+
83+
# /usr/bin/neutron-periodic-workers --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini
84+
85+
.. end
86+
87+
This process collects plugin workers registered through ``get_workers()`` and
88+
starts them according to their ``worker_process_count`` value:
89+
90+
* Workers with ``worker_process_count=0`` are grouped inside a single
91+
``AllServicesNeutronWorker`` and executed as threads in one child process.
92+
This is appropriate for interval-based ``PeriodicWorker`` tasks such as quota
93+
cleanup, L3 garbage collection, and agent health checks. These workers call
94+
``start()`` and return; their ``wait()`` method only blocks while the
95+
periodic loop is running, which is safe inside the grouped thread model.
96+
* Workers with ``worker_process_count>0`` are started in separate child
97+
processes, one per worker definition. Plugin workers whose ``wait()`` method
98+
blocks indefinitely on a long-running service must use
99+
``worker_process_count=1``, the same model as ``RpcWorker``. Examples include
100+
the L3 and metering ``RpcWorker`` instances and the BGP worker.
101+
102+
Plugin authors returning workers from ``get_workers()`` must set
103+
``worker_process_count=1`` for any worker that blocks on ``wait()`` after
104+
``start()``. Grouping such workers with ``worker_process_count=0`` would
105+
prevent other grouped workers from making progress.
106+
107+
The following worker types are not started by this process:
108+
109+
* ``MaintenanceWorker`` from the ML2/OVN mechanism driver. This worker is
110+
started by ``neutron-ovn-maintenance-worker``.
111+
* ``AllServicesNeutronWorker`` instances, if returned directly by a plugin.
112+
113+
Start Neutron OVN maintenance worker
114+
------------------------------------
115+
116+
When the ML2/OVN mechanism driver is used, the database maintenance task runs
117+
in a dedicated process:
118+
119+
.. code-block:: console
120+
121+
# /usr/bin/neutron-ovn-maintenance-worker --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini
122+
123+
.. end
124+
75125
Neutron Worker Processes
76126
------------------------
77127

78-
Neutron will attempt to spawn a number of child processes for handling API
79-
and RPC requests. The number of API workers is set to the number of CPU
80-
cores, further limited by available memory, and the number of RPC workers
81-
is set to half that number.
128+
In a WSGI deployment, Neutron splits background work across several
129+
executables in addition to the API server:
130+
131+
* ``neutron-rpc-server``: handles RPC requests from agents.
132+
* ``neutron-periodic-workers``: handles ML2 plugin periodic and thread
133+
workers.
134+
* ``neutron-ovn-maintenance-worker``: handles the ML2/OVN database
135+
maintenance task when that mechanism driver is enabled.
136+
137+
The RPC server will attempt to spawn a number of child processes for handling
138+
RPC requests. The number of API workers is set to the number of CPU cores,
139+
further limited by available memory, and the number of RPC workers is set to
140+
half that number.
82141

83142
It is strongly recommended that all deployers set these values themselves,
84143
via the api_workers and rpc_workers configuration parameters.
@@ -96,7 +155,7 @@ on the hypervisors, or rpc message timeout exceptions in agent logs
96155
(for example, "broken pipe" errors).
97156

98157
There is also the rpc_state_report_workers option, which determines
99-
the number fo RPC worker processes dedicated to process state reports
158+
the number of RPC worker processes dedicated to process state reports
100159
from the various agents. This may be increased to resolve frequent delay
101160
in processing agents heartbeats.
102161

@@ -109,4 +168,4 @@ in processing agents heartbeats.
109168
ML2/OVN uses the ``[uwsgi]start-time = %t`` parameter to create the OVN hash
110169
ring registers during the initialization process. This value is populated
111170
by the uWSGi process with the start time. For more information, check
112-
`Configuring uWSGI <https://uwsgi-docs.readthedocs.io/en/latest/Configuration.html>_`.
171+
`Configuring uWSGI <https://uwsgi-docs.readthedocs.io/en/latest/Configuration.html>`_.

‎doc/source/admin/config.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,7 @@ Configuration
4545
config-subnet-onboard
4646
config-subnet-pools
4747
config-trunking
48+
config-neutron-server-processes
4849
config-wsgi
4950

5051
.. note::

‎doc/source/admin/notifications.rst‎

Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
.. _admin-notifications:
2+
3+
=============
4+
Notifications
5+
=============
6+
7+
Like other OpenStack services, Neutron emits notifications to the message bus
8+
using the ``Notifier`` class provided by :oslo.messaging-doc:`oslo.messaging
9+
<reference/notifier.html>`. From the consumer point of view, a notification
10+
consists of an envelope with a fixed structure defined by oslo.messaging and a
11+
payload defined by Neutron.
12+
13+
Notification envelope
14+
~~~~~~~~~~~~~~~~~~~~~
15+
16+
The wire format of a notification is the following::
17+
18+
{
19+
"priority": <string, selected from a predefined list by the sender>,
20+
"event_type": <string, defined by the sender>,
21+
"timestamp": <string, the isotime of when the notification was emitted>,
22+
"publisher_id": <string, defined by the sender>,
23+
"message_id": <uuid, generated by oslo>,
24+
"payload": <json serialized dict, defined by the sender>
25+
}
26+
27+
Neutron emits legacy (unversioned) notifications only. The payload is a
28+
free-form dictionary that mirrors API request or response bodies. Unlike Nova,
29+
Neutron does not provide versioned notification payloads or a backward
30+
compatibility contract for them.
31+
32+
When notifications are sent
33+
~~~~~~~~~~~~~~~~~~~~~~~~~~~
34+
35+
Most notifications are emitted by the Pecan WSGI ``NotifierHook`` when the
36+
Neutron API processes create, update, or delete operations on REST resources.
37+
38+
* Notifications are sent for ``POST``, ``PUT``, and ``DELETE`` requests only.
39+
* Member actions (for example,
40+
``PUT /routers/{router_id}/add_router_interface``) do not trigger these
41+
notifications.
42+
* For each operation, a ``*.start`` notification is sent before the plugin
43+
runs and a matching ``*.end`` notification is sent after a successful
44+
response (HTTP status code 300 or lower). If the operation fails, only the
45+
``*.start`` notification is emitted.
46+
47+
Additional notifications are sent directly by specific plugins and agents for
48+
router interfaces, resource tags, agent scheduling, metering, and usage
49+
auditing. See :doc:`/reference/notifications` for the full event catalog and
50+
payload details.
51+
52+
Publisher identifiers
53+
~~~~~~~~~~~~~~~~~~~~~
54+
55+
Neutron uses ``neutron_lib.rpc.get_notifier()`` to obtain notifier instances.
56+
The ``publisher_id`` field in the envelope is built from the service name and
57+
host, for example ``network.myhost`` or ``router.myhost``. The most common
58+
publisher identifiers are:
59+
60+
* ``network.<host>`` — API resource CRUD, router interfaces, tags, DHCP agent
61+
scheduling, usage audit
62+
* ``router.<host>`` — L3 agent scheduling events
63+
* ``metering.<host>`` — metering agent traffic reports
64+
65+
Configuration
66+
~~~~~~~~~~~~~
67+
68+
Notifications are configured through oslo.messaging options in
69+
``neutron.conf``. To enable notifications, set a driver other than ``noop``:
70+
71+
.. code-block:: ini
72+
73+
[DEFAULT]
74+
transport_url = rabbit://guest:guest@localhost:5672/
75+
76+
[oslo_messaging_notifications]
77+
driver = messagingv2
78+
topics = notifications
79+
80+
The available drivers are:
81+
82+
* ``messaging`` — send notifications using the 1.0 message format
83+
* ``messagingv2`` — send notifications using the 2.0 message format (with a
84+
message envelope)
85+
* ``routing`` — configurable routing notifier (by priority or ``event_type``)
86+
* ``log`` — publish notifications using Python logging infrastructure
87+
* ``test`` — store notifications in memory for test verification
88+
* ``noop`` — disable sending notifications entirely
89+
90+
The default topic is ``notifications``. Messages are published to
91+
``<topic>.<priority>`` (for example, ``notifications.INFO``).
92+
93+
To support the DHCP agent, the ``messagingv2`` driver must be enabled so that
94+
RPC-based agent notifiers can receive resource change events.
95+
96+
Notifications can be disabled entirely by setting
97+
:oslo.config:option:`oslo_messaging_notifications.driver` to ``noop``.
98+
99+
Reference
100+
~~~~~~~~~
101+
102+
A list of notification events and example payloads can be found in
103+
:doc:`/reference/notifications`.

‎doc/source/admin/ops.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ Operations
77
.. toctree::
88
:maxdepth: 2
99

10+
notifications
1011
ops-ip-availability
1112
ops-resource-tags
1213
ops-quotas

0 commit comments

Comments
 (0)