Merge pull request #1699 from rackerlabs/resync-ironic-nautobot

haseebsyed12 · web-flow · commit 0f9a69790dd1 · 2026-02-26T18:06:17.000Z
feat: scheduled resync-ironic-nautobot
diff --git a/components/site-workflows/cronworkflows/resync-ironic-nautobot.yaml b/components/site-workflows/cronworkflows/resync-ironic-nautobot.yaml
@@ -0,0 +1,26 @@
+---
+apiVersion: argoproj.io/v1alpha1
+kind: CronWorkflow
+metadata:
+  name: resync-ironic-nautobot
+  annotations:
+    workflows.argoproj.io/title: Scheduled resync of Ironic nodes to Nautobot
+    workflows.argoproj.io/description: |
+      Periodically resyncs all Ironic nodes to Nautobot to catch any drift.
+      Runs daily at 2:00 AM UTC by default.
+
+      To manually trigger:
+      ```
+      argo -n argo-events submit --from cronworkflow/resync-ironic-nautobot
+      ```
+
+      Defined in `components/site-workflows/cronworkflows/resync-ironic-nautobot.yaml`
+spec:
+  schedule: "0 2 * * *"
+  timezone: "UTC"
+  concurrencyPolicy: "Forbid"
+  successfulJobsHistoryLimit: 3
+  failedJobsHistoryLimit: 3
+  workflowSpec:
+    workflowTemplateRef:
+      name: resync-ironic-nautobot
diff --git a/components/site-workflows/kustomization.yaml b/components/site-workflows/kustomization.yaml
@@ -24,6 +24,8 @@ resources:
   - sensors/sensor-ironic-node-port.yaml
   - sensors/sensor-ironic-node-portgroup.yaml
   - sensors/sensor-ironic-oslo-event.yaml
+  # CronWorkflows
+  - cronworkflows/resync-ironic-nautobot.yaml
 
 helmCharts:
   - name: nautobot-token
diff --git a/docs/component-argo-workflows.md b/docs/component-argo-workflows.md
@@ -6,13 +6,14 @@ set of WorkflowTemplates below.
 
 ## WorkflowTemplates
 
-| WorkflowTemplate      | Description                                               | Input                   | Output     |   |
-|---------------------- |-----------------------------------------------------------|-------------------------|------------|---|
-| get-device-nautobot   | Return Device Information from Nautobot                   | device_id               | device     |   |
-| get-bmc-creds         | Get the credentials for the target Device                 | device_id               | secret     | * |
-| get-bmc-ip            | Get BMC IP address for target Device                      | device_id               | ip         |   |
-| nautobot-api          | HTTP Template Workflow to query the Nautobot API          | method,nautobot_url,uri | result     |   |
-| bmc-sync-creds        | Sync's a devices BMC password with what we have on record | device_id               |            |   |
+| WorkflowTemplate        | Description                                               | Input                   | Output     |   |
+|-------------------------|-----------------------------------------------------------|-------------------------|------------|---|
+| get-device-nautobot     | Return Device Information from Nautobot                   | device_id               | device     |   |
+| get-bmc-creds           | Get the credentials for the target Device                 | device_id               | secret     | * |
+| get-bmc-ip              | Get BMC IP address for target Device                      | device_id               | ip         |   |
+| nautobot-api            | HTTP Template Workflow to query the Nautobot API          | method,nautobot_url,uri | result     |   |
+| bmc-sync-creds          | Sync's a devices BMC password with what we have on record | device_id               |            |   |
+| resync-ironic-nautobot  | Resync Ironic nodes to Nautobot                           | node (optional)         |            |   |
 
 \* WorkflowTemplate which requires a manual / custom implementation.
 
diff --git a/docs/operator-guide/index.md b/docs/operator-guide/index.md
@@ -37,6 +37,7 @@ be the regular project area.
 
 - [Gateway API Migration Guide](gateway-api.md) - Migration from ingress-nginx to Kubernetes Gateway API with Envoy Gateway
 - [Argo Workflows](workflows.md) - Workflow orchestration and troubleshooting
+- [Ironic to Nautobot Sync](ironic-nautobot-sync.md) - Event-driven sync and bulk resync operations
 - [Monitoring Stack](monitoring.md) - Prometheus and Grafana monitoring
 
 [cli]: <../user-guide/openstack-cli.md>
diff --git a/docs/operator-guide/ironic-nautobot-sync.md b/docs/operator-guide/ironic-nautobot-sync.md
@@ -0,0 +1,110 @@
+# Ironic to Nautobot Synchronization
+
+This guide explains how OpenStack Ironic data is synchronized to Nautobot
+and how to handle situations when they get out of sync.
+
+## Event-Driven Sync
+
+Under normal operation, Ironic data is automatically synchronized to Nautobot
+via Oslo notifications. When changes occur in Ironic, events are published to
+RabbitMQ and processed by Argo Events workflows.
+
+### How It Works
+
+1. Ironic publishes Oslo notifications to RabbitMQ when nodes change
+2. An Argo Events EventSource consumes messages from the `ironic` queue
+3. A Sensor filters for relevant events and triggers workflows
+4. The `openstack-oslo-event` workflow processes the event and updates Nautobot
+
+### Supported Events
+
+The following Ironic events trigger Nautobot updates:
+
+| Event Type | Action |
+|------------|--------|
+| `baremetal.node.create.end` | Creates device in Nautobot |
+| `baremetal.node.update.end` | Updates device in Nautobot |
+| `baremetal.node.delete.end` | Deletes device from Nautobot |
+| `baremetal.node.provision_set.end` | Updates device status and syncs inspection data |
+| `baremetal.port.create.end` | Creates interface in Nautobot |
+| `baremetal.port.update.end` | Updates interface in Nautobot |
+| `baremetal.port.delete.end` | Deletes interface from Nautobot |
+
+### Data Synchronized
+
+For each Ironic node, the following data is synced to Nautobot:
+
+- Device name (generated from manufacturer and service tag)
+- Serial number
+- Manufacturer and model
+- Hardware specs (memory, CPUs, local storage)
+- Provision state (mapped to Nautobot status)
+- Location and rack (derived from connected switches)
+- Tenant (from Ironic lessee field)
+- Network interfaces and their connections
+
+## Bulk Resync
+
+When Nautobot gets out of sync with Ironic, you can perform a bulk resync.
+
+### Scheduled Resync (CronWorkflow)
+
+A CronWorkflow runs daily at 2:00 AM UTC to catch any drift between Ironic
+and Nautobot. This provides a safety net for missed events.
+
+Check the schedule:
+
+```bash
+argo -n argo-events cron list
+```
+
+Manually trigger the scheduled workflow:
+
+```bash
+argo -n argo-events submit --from cronworkflow/resync-ironic-nautobot
+```
+
+Suspend/resume the schedule:
+
+```bash
+argo -n argo-events cron suspend resync-ironic-nautobot
+argo -n argo-events cron resume resync-ironic-nautobot
+```
+
+### On-Demand Resync (WorkflowTemplate)
+
+Resync all nodes:
+
+```bash
+argo -n argo-events submit --from workflowtemplate/resync-ironic-nautobot
+```
+
+Resync a specific node:
+
+```bash
+argo -n argo-events submit --from workflowtemplate/resync-ironic-nautobot \
+  -p node="<node-uuid>"
+```
+
+### Using CLI Directly
+
+For debugging or running outside the cluster:
+
+```bash
+# Resync all nodes
+resync-ironic-nautobot \
+  --nautobot_url https://nautobot.example.com \
+  --nautobot_token <token>
+
+# Resync a specific node
+resync-ironic-nautobot \
+  --node <uuid> \
+  --nautobot_url https://nautobot.example.com \
+  --nautobot_token <token>
+
+# Dry run to see what would be synced
+resync-ironic-nautobot \
+  --dry-run \
+  --nautobot_url https://nautobot.example.com \
+  --nautobot_token <token>
+```
diff --git a/mkdocs.yml b/mkdocs.yml
@@ -174,6 +174,7 @@ nav:
     - 'Infrastructure':
       - operator-guide/argocd-helm-chart.md
       - operator-guide/workflows.md
+      - operator-guide/ironic-nautobot-sync.md
       - operator-guide/monitoring.md
       - operator-guide/gateway-api.md
       - operator-guide/bmc-password.md
diff --git a/python/understack-workflows/pyproject.toml b/python/understack-workflows/pyproject.toml
@@ -38,6 +38,7 @@ get-raid-devices = "understack_workflows.main.get_raid_devices:main"
 netapp-configure-interfaces = "understack_workflows.main.netapp_configure_net:main"
 netapp-create-svm = "understack_workflows.main.netapp_create_svm:main"
 openstack-oslo-event = "understack_workflows.main.openstack_oslo_event:main"
+resync-ironic-nautobot = "understack_workflows.main.resync_ironic_to_nautobot:main"
 sync-keystone = "understack_workflows.main.sync_keystone:main"
 sync-network-segment-range = "understack_workflows.main.sync_ucvni_group_range:main"
 undersync-switch = "understack_workflows.main.undersync_switch:main"
diff --git a/python/understack-workflows/tests/test_nautobot_device_interface_sync.py b/python/understack-workflows/tests/test_nautobot_device_interface_sync.py
@@ -73,7 +73,7 @@ def test_integrated_interface(self):
         assert _get_interface_type("NIC.Integrated.1-1") == "25gbase-x-sfp28"
 
     def test_unknown_interface(self):
-        assert _get_interface_type("eth0") == "unknown"
+        assert _get_interface_type("eth0") == "other"
 
 
 class TestGetInterfaceDescription:
@@ -228,7 +228,7 @@ def test_interface_info_defaults(self):
         assert iface.enabled is True
         assert iface.mgmt_only is False
         assert iface.pxe_enabled is False
-        assert iface.interface_type == "unknown"
+        assert iface.interface_type == "other"
 
 
 class TestCreateNautobotInterface:
diff --git a/python/understack-workflows/tests/test_resync_ironic_to_nautobot.py b/python/understack-workflows/tests/test_resync_ironic_to_nautobot.py
@@ -0,0 +1,168 @@
+"""Tests for resync_ironic_to_nautobot module."""
+
+from unittest.mock import MagicMock
+from unittest.mock import patch
+
+from understack_workflows.main.resync_ironic_to_nautobot import SyncResult
+from understack_workflows.main.resync_ironic_to_nautobot import argument_parser
+from understack_workflows.main.resync_ironic_to_nautobot import main
+from understack_workflows.main.resync_ironic_to_nautobot import sync_nodes
+
+
+class TestSyncResult:
+    """Test cases for SyncResult dataclass."""
+
+    def test_defaults(self):
+        result = SyncResult()
+        assert result.total == 0
+        assert result.failed == 0
+        assert result.succeeded == 0
+
+    def test_succeeded_calculation(self):
+        result = SyncResult(total=10, failed=3)
+        assert result.succeeded == 7
+
+    def test_all_failed(self):
+        result = SyncResult(total=5, failed=5)
+        assert result.succeeded == 0
+
+    def test_none_failed(self):
+        result = SyncResult(total=5, failed=0)
+        assert result.succeeded == 5
+
+
+class TestArgumentParser:
+    """Test cases for argument_parser function."""
+
+    def test_default_args(self):
+        parser = argument_parser()
+        args = parser.parse_args([])
+        assert args.node is None
+        assert args.dry_run is False
+
+    def test_node_arg(self):
+        parser = argument_parser()
+        args = parser.parse_args(["--node", "test-uuid"])
+        assert args.node == "test-uuid"
+
+    def test_dry_run_arg(self):
+        parser = argument_parser()
+        args = parser.parse_args(["--dry-run"])
+        assert args.dry_run is True
+
+
+class TestSyncNodes:
+    """Test cases for sync_nodes function."""
+
+    @patch("understack_workflows.main.resync_ironic_to_nautobot.IronicClient")
+    @patch(
+        "understack_workflows.main.resync_ironic_to_nautobot.sync_device_to_nautobot"
+    )
+    def test_sync_all_nodes_success(self, mock_sync, mock_ironic_class):
+        mock_ironic = MagicMock()
+        mock_ironic_class.return_value = mock_ironic
+        mock_node1 = MagicMock(uuid="uuid-1", name="node-1")
+        mock_node2 = MagicMock(uuid="uuid-2", name="node-2")
+        mock_ironic.list_nodes.return_value = [mock_node1, mock_node2]
+        mock_sync.return_value = 0
+
+        nautobot = MagicMock()
+        result = sync_nodes(nautobot)
+
+        assert result.total == 2
+        assert result.failed == 0
+        assert mock_sync.call_count == 2
+
+    @patch("understack_workflows.main.resync_ironic_to_nautobot.IronicClient")
+    @patch(
+        "understack_workflows.main.resync_ironic_to_nautobot.sync_device_to_nautobot"
+    )
+    def test_sync_single_node(self, mock_sync, mock_ironic_class):
+        mock_ironic = MagicMock()
+        mock_ironic_class.return_value = mock_ironic
+        mock_node = MagicMock(uuid="uuid-1", name="node-1")
+        mock_ironic.get_node.return_value = mock_node
+        mock_sync.return_value = 0
+
+        nautobot = MagicMock()
+        result = sync_nodes(nautobot, node_uuid="uuid-1")
+
+        assert result.total == 1
+        assert result.failed == 0
+        mock_ironic.get_node.assert_called_once_with("uuid-1")
+
+    @patch("understack_workflows.main.resync_ironic_to_nautobot.IronicClient")
+    @patch(
+        "understack_workflows.main.resync_ironic_to_nautobot.sync_device_to_nautobot"
+    )
+    def test_sync_with_failures(self, mock_sync, mock_ironic_class):
+        mock_ironic = MagicMock()
+        mock_ironic_class.return_value = mock_ironic
+        mock_node1 = MagicMock(uuid="uuid-1", name="node-1")
+        mock_node2 = MagicMock(uuid="uuid-2", name="node-2")
+        mock_ironic.list_nodes.return_value = [mock_node1, mock_node2]
+        mock_sync.side_effect = [0, 1]  # First succeeds, second fails
+
+        nautobot = MagicMock()
+        result = sync_nodes(nautobot)
+
+        assert result.total == 2
+        assert result.failed == 1
+        assert result.succeeded == 1
+
+    @patch("understack_workflows.main.resync_ironic_to_nautobot.IronicClient")
+    def test_dry_run_skips_sync(self, mock_ironic_class):
+        mock_ironic = MagicMock()
+        mock_ironic_class.return_value = mock_ironic
+        mock_node = MagicMock(uuid="uuid-1", name="node-1")
+        mock_ironic.list_nodes.return_value = [mock_node]
+
+        nautobot = MagicMock()
+        result = sync_nodes(nautobot, dry_run=True)
+
+        assert result.total == 1
+        assert result.failed == 0
+
+
+class TestMain:
+    """Test cases for main function."""
+
+    @patch("understack_workflows.main.resync_ironic_to_nautobot.sync_nodes")
+    @patch("understack_workflows.main.resync_ironic_to_nautobot.pynautobot")
+    @patch("understack_workflows.main.resync_ironic_to_nautobot.credential")
+    @patch("understack_workflows.main.resync_ironic_to_nautobot.setup_logger")
+    @patch("understack_workflows.main.resync_ironic_to_nautobot.argument_parser")
+    def test_main_success(
+        self, mock_parser, mock_logger, mock_cred, mock_pynb, mock_sync
+    ):
+        mock_args = MagicMock()
+        mock_args.nautobot_token = "token"
+        mock_args.nautobot_url = "http://nautobot"
+        mock_args.node = None
+        mock_args.dry_run = False
+        mock_parser.return_value.parse_args.return_value = mock_args
+        mock_sync.return_value = SyncResult(total=5, failed=0)
+
+        result = main()
+
+        assert result == 0
+
+    @patch("understack_workflows.main.resync_ironic_to_nautobot.sync_nodes")
+    @patch("understack_workflows.main.resync_ironic_to_nautobot.pynautobot")
+    @patch("understack_workflows.main.resync_ironic_to_nautobot.credential")
+    @patch("understack_workflows.main.resync_ironic_to_nautobot.setup_logger")
+    @patch("understack_workflows.main.resync_ironic_to_nautobot.argument_parser")
+    def test_main_with_failures(
+        self, mock_parser, mock_logger, mock_cred, mock_pynb, mock_sync
+    ):
+        mock_args = MagicMock()
+        mock_args.nautobot_token = "token"
+        mock_args.nautobot_url = "http://nautobot"
+        mock_args.node = None
+        mock_args.dry_run = False
+        mock_parser.return_value.parse_args.return_value = mock_args
+        mock_sync.return_value = SyncResult(total=5, failed=2)
+
+        result = main()
+
+        assert result == 1
diff --git a/python/understack-workflows/understack_workflows/main/resync_ironic_to_nautobot.py b/python/understack-workflows/understack_workflows/main/resync_ironic_to_nautobot.py
diff --git a/python/understack-workflows/understack_workflows/oslo_event/nautobot_device_interface_sync.py b/python/understack-workflows/understack_workflows/oslo_event/nautobot_device_interface_sync.py
diff --git a/workflows/argo-events/kustomization.yaml b/workflows/argo-events/kustomization.yaml
diff --git a/workflows/argo-events/workflowtemplates/resync-ironic-nautobot.yaml b/workflows/argo-events/workflowtemplates/resync-ironic-nautobot.yaml
