|
| 1 | +# Hospital VM Onboarding Runbook |
| 2 | + |
| 3 | +> **Script**: `scripts/powershell/arc-setup.ps1` |
| 4 | +> **When to use**: Onboarding a new NHS hospital gateway VM to Azure Arc for the first time |
| 5 | +
|
| 6 | +--- |
| 7 | + |
| 8 | +## Prerequisites |
| 9 | + |
| 10 | +- [ ] Hospital IT has provisioned the gateway VM (Windows Server 2022 or later) |
| 11 | +- [ ] VM has outbound internet access to: |
| 12 | + - `*.arc.azure.com` |
| 13 | + - `*.his.arc.azure.com` |
| 14 | + - `relay-manbrs-<env>.servicebus.windows.net` |
| 15 | +- [ ] Trust ODS code confirmed via the [ODS portal](https://odsportal.nhsbsa.nhs.uk/) |
| 16 | +- [ ] PACS vendor confirmed (`sectra` | `fujifilm` | `agfa` | `philips` | `carestream`) |
| 17 | +- [ ] NHS region confirmed (`nw` | `neyh` | `mids` | `eoe` | `lon` | `se` | `sw`) |
| 18 | +- [ ] Deployment ring agreed with the programme team |
| 19 | +- [ ] `arc-onboarding-spn-client-id` and `arc-onboarding-spn-client-secret` retrieved from Key Vault |
| 20 | + |
| 21 | +--- |
| 22 | + |
| 23 | +## Step 1 — Determine site parameters |
| 24 | + |
| 25 | +| Parameter | Format | Example | |
| 26 | +|-----------|--------|---------| |
| 27 | +| `SiteCode` | `gw-<ODSCode>-<instance>` | `gw-RVJ-01` | |
| 28 | +| `SiteName` | Trust name, hyphen-separated, no spaces | `North-Bristol-NHS-Trust` | |
| 29 | +| `NHSRegion` | One of: `nw` `neyh` `mids` `eoe` `lon` `se` `sw` | `sw` | |
| 30 | +| `PacsVendor` | One of: `sectra` `fujifilm` `agfa` `philips` `carestream` | `sectra` | |
| 31 | +| `SiteType` | `static` or `mobile` | `static` | |
| 32 | +| `DeploymentRing` | `ring1`–`ring4` (see below) | `ring1` | |
| 33 | + |
| 34 | +**Ring assignments:** |
| 35 | + |
| 36 | +| Ring | Sites | |
| 37 | +|------|-------| |
| 38 | +| ring0 | Test VM only (`mbsgw-review`) | |
| 39 | +| ring1 | 1 site per PACS vendor (first real sites) | |
| 40 | +| ring2 | 1 site per NHS region | |
| 41 | +| ring3 | Remaining static sites | |
| 42 | +| ring4 | Mobile units | |
| 43 | + |
| 44 | +## Step 2 — Run Arc onboarding script on the gateway VM |
| 45 | + |
| 46 | +Copy `scripts/powershell/arc-setup.ps1` to the VM and run from an **elevated PowerShell session**: |
| 47 | + |
| 48 | +```powershell |
| 49 | +.\arc-setup.ps1 ` |
| 50 | + -SubscriptionId "<spoke-subscription-id>" ` |
| 51 | + -TenantId "<tenant-id>" ` |
| 52 | + -ResourceGroup "rg-manbgw-<env>-uks-arc-enabled-servers" ` |
| 53 | + -Location "uksouth" ` |
| 54 | + -ServicePrincipalId "<arc-onboarding-spn-client-id>" ` |
| 55 | + -ServicePrincipalSecret "<arc-onboarding-spn-client-secret>" ` |
| 56 | + -SiteCode "gw-RVJ-01" ` |
| 57 | + -SiteName "North-Bristol-NHS-Trust" ` |
| 58 | + -NHSRegion "sw" ` |
| 59 | + -PacsVendor "sectra" ` |
| 60 | + -SiteType "static" ` |
| 61 | + -DeploymentRing "ring1" |
| 62 | +``` |
| 63 | + |
| 64 | +The script will: |
| 65 | +1. Install the Azure Arc agent (`azcmagent`) if not already present |
| 66 | +2. Stamp site metadata as tags on the Arc machine resource |
| 67 | +3. Connect the VM to Azure Arc with `--resource-name` set to `SiteCode` |
| 68 | + |
| 69 | +Logs are written to `C:\ArcSetup\ArcSetup.log`. |
| 70 | + |
| 71 | +**Verify**: In the Azure portal, navigate to `rg-manbgw-<env>-uks-arc-enabled-servers` → Azure Arc machines → `gw-RVJ-01`. Status should be **Connected**. |
| 72 | + |
| 73 | +## Step 3 — Trigger Terraform to provision the Hybrid Connection |
| 74 | + |
| 75 | +Run the ADO pipeline **Deploy Arc Infrastructure - \<env\>** manually. Terraform discovers the new Arc machine and creates: |
| 76 | + |
| 77 | +- `hc-gw-RVJ-01` in the relay namespace (`relay-manbrs-<env>`) |
| 78 | +- `listen` auth rule on that Hybrid Connection |
| 79 | + |
| 80 | +**Verify**: In the Azure portal, navigate to `relay-manbrs-<env>` → Hybrid Connections → `hc-gw-RVJ-01` is present. |
| 81 | + |
| 82 | +## Step 4 — Deploy the gateway application |
| 83 | + |
| 84 | +Run the ADO pipeline **Deploy Gateway - \<env\>** with: |
| 85 | + |
| 86 | +``` |
| 87 | +targetSiteCode : gw-RVJ-01 |
| 88 | +releaseTag : latest (or a specific tag, e.g. v1.2.3) |
| 89 | +``` |
| 90 | + |
| 91 | +The pipeline: |
| 92 | +1. Retrieves the listen SAS key for `hc-gw-RVJ-01` |
| 93 | +2. Sends an Arc Run Command to `gw-RVJ-01` that writes `.env` and runs `deploy.ps1` |
| 94 | +3. Polls for completion and reports success or failure |
| 95 | + |
| 96 | +## Step 5 — Smoke test |
| 97 | + |
| 98 | +Run from the gateway VM or via Arc Run Command: |
| 99 | + |
| 100 | +```powershell |
| 101 | +Get-Service Gateway-PACS, Gateway-MWL, Gateway-Upload, Gateway-Relay | Select-Object Name, Status |
| 102 | +``` |
| 103 | + |
| 104 | +Expected: all four services **Running**. |
| 105 | + |
| 106 | +Check Log Analytics Workspace for an initial heartbeat within 5 minutes of service start. |
| 107 | + |
| 108 | +--- |
| 109 | + |
| 110 | +## Parameters reference |
| 111 | + |
| 112 | +| Parameter | Required | Default | Description | |
| 113 | +|-----------|----------|---------|-------------| |
| 114 | +| `-SubscriptionId` | Yes | — | Azure spoke subscription ID | |
| 115 | +| `-TenantId` | Yes | — | Azure Entra tenant ID | |
| 116 | +| `-ResourceGroup` | Yes | — | Arc-enabled servers resource group | |
| 117 | +| `-Location` | Yes | — | Azure region (always `uksouth`) | |
| 118 | +| `-ServicePrincipalId` | Yes | — | Arc onboarding SPN client ID | |
| 119 | +| `-ServicePrincipalSecret` | Yes | — | Arc onboarding SPN client secret | |
| 120 | +| `-SiteCode` | No | *(hostname)* | Arc resource name and tag; format `gw-<ODSCode>-<instance>` | |
| 121 | +| `-SiteName` | No | *(not set)* | Human-readable trust name; no spaces | |
| 122 | +| `-NHSRegion` | No | *(not set)* | NHS region code | |
| 123 | +| `-PacsVendor` | No | *(not set)* | PACS system vendor | |
| 124 | +| `-SiteType` | No | `static` | `static` or `mobile` | |
| 125 | +| `-DeploymentRing` | No | `ring0` | Rollout ring (`ring0`–`ring4`) | |
| 126 | + |
| 127 | +--- |
| 128 | + |
| 129 | +## Troubleshooting |
| 130 | + |
| 131 | +### Arc agent fails to connect |
| 132 | + |
| 133 | +Check `C:\ArcSetup\ArcSetup.log` on the VM. Common causes: |
| 134 | + |
| 135 | +- **Firewall blocking outbound** — confirm the VM can reach `*.arc.azure.com` on port 443 |
| 136 | +- **SPN credentials wrong** — verify client ID and secret from Key Vault are current |
| 137 | +- **VM already registered** — if the machine was previously connected under a different name, disconnect first: `azcmagent disconnect` |
| 138 | + |
| 139 | +### Arc machine shows as Disconnected after onboarding |
| 140 | + |
| 141 | +The agent may have lost connectivity. Check: |
| 142 | + |
| 143 | +```powershell |
| 144 | +azcmagent show |
| 145 | +``` |
| 146 | + |
| 147 | +If disconnected, re-run the script. It is safe to re-run — the agent will reconnect and update tags. |
| 148 | + |
| 149 | +### Hybrid Connection not created after Terraform run |
| 150 | + |
| 151 | +The Arc machine must appear in the resource group before Terraform can create the HC. Confirm the machine is **Connected** in the portal (Step 2 verify), then re-run the pipeline. |
| 152 | + |
| 153 | +### Gateway services not starting after deploy |
| 154 | + |
| 155 | +Check the deployment log on the VM: |
| 156 | + |
| 157 | +```powershell |
| 158 | +Get-ChildItem "C:\Program Files\NHS\ManageBreastScreeningGateway\logs\deployments\deploy-*" | |
| 159 | + Sort-Object LastWriteTime -Descending | Select-Object -First 1 | |
| 160 | + ForEach-Object { Get-Content $_.FullName } |
| 161 | +``` |
0 commit comments