Skip to content

Commit 468a169

Browse files
Merge pull request #117 from NHSDigital/DTOSS-13037/automate-arc-app-role-assignment
feat(DTOSS-13037): automate Arc machine app role assignment and enterprise app ownership
2 parents dbf13c4 + 49d7ad9 commit 468a169

9 files changed

Lines changed: 184 additions & 5 deletions

File tree

docs/deployment/deployment-pipeline.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -422,6 +422,8 @@ The ADO managed identity holds:
422422

423423
The managed identity does **not** hold `Key Vault Secrets Officer` — it cannot write secrets. Cloud API tokens must be provisioned separately by the team managing the cloud application.
424424

425+
The managed identity also does **not** hold `AppRoleAssignment.ReadWrite.All`. Microsoft Graph requires this explicit application permission to create app role assignments — being an owner of the enterprise app service principal is insufficient in application auth context. Granting this tenant-wide permission is not permitted under the organisation's Entra policy. Arc machine managed identity app role assignments are therefore granted manually by running `make <env> resource-group-init` under a user account with ownership of `spn-manbrs-web-api-<env>` — see [`scripts/bash/assign_arc_app_roles.sh`](../../scripts/bash/assign_arc_app_roles.sh) and the [Onboard Hospital VM runbook](./runbooks/onboard-hospital-vm.md#step-4--grant-api-access).
426+
425427
### Separation of environments
426428

427429
Each environment (`dev`, `preprod`, `prod`) has its own:

docs/deployment/infrastructure/create-environment.md

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,11 @@ az provider show --namespace Microsoft.GuestConfiguration --query registrationSt
5252
- Name: `spn-azure-arc-onboarding-screening-[environment]`
5353
- If it does not exist, raise a request with the platform team to create it
5454

55+
- Confirm the web API enterprise application exists:
56+
- Name: `spn-manbrs-web-api-[environment]`
57+
- If it does not exist, raise a ServiceNow request for the platform team to create it — the `resource-group-init` Bicep deployment and the `assign_arc_app_roles.sh` script both depend on it
58+
- [Form for ServiceNow](https://nhsdigitallive.service-now.com/nhs_digital?id=sc_cat_item&sys_id=28f3ab4f1bf3ca1078ac4337b04bcb78&sysparm_category=114fced51bdae1502eee65b9bd4bcbdc)
59+
5560
## Code
5661

5762
- Create the configuration files in `infrastructure/environments/[environment]/`:
@@ -233,3 +238,11 @@ Terraform automatically discovers Arc machines registered in the Arc-enabled ser
233238
```bash
234239
make [environment] terraform-apply
235240
```
241+
242+
After Terraform provisions the Hybrid Connection, re-run `resource-group-init` to assign the `Gateway.Access` app role to the new machine's managed identity:
243+
244+
```bash
245+
make [environment] resource-group-init
246+
```
247+
248+
> **Why is this a separate step?** The pipeline managed identity cannot create app role assignments — `AppRoleAssignment.ReadWrite.All` is required in application auth context regardless of SP ownership, and this permission is not permitted under the organisation's Entra policy. Running `resource-group-init` under a user account with ownership of `spn-manbrs-web-api-[environment]` is sufficient. Without this step the gateway will fail to authenticate against the cloud web API. See [Onboard Hospital VM — Step 4](../runbooks/onboard-hospital-vm.md#step-4--grant-api-access).

docs/deployment/runbooks/onboard-hospital-vm.md

Lines changed: 19 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -100,7 +100,24 @@ Run the ADO pipeline **Deploy Arc Infrastructure - \<env\>** manually. Terraform
100100

101101
**Verify**: In the Azure portal, navigate to `relay-manbrs-<env>` → Hybrid Connections → `hc-gw-hull-university-teaching-hospitals-nhs-trust-rwa-01` is present.
102102

103-
## Step 4 — Deploy the gateway application
103+
## Step 4 — Grant API access
104+
105+
> [!IMPORTANT]
106+
> Without this step the gateway services will start but fail to authenticate against the cloud web API. The Arc machine's managed identity must be assigned the `Gateway.Access` app role on `spn-manbrs-web-api-<env>` before the first deployment.
107+
108+
Run from a developer machine with Owner access to the enterprise application:
109+
110+
```bash
111+
make <env> resource-group-init
112+
```
113+
114+
This calls [`scripts/bash/assign_arc_app_roles.sh`](../../../scripts/bash/assign_arc_app_roles.sh) which discovers all Arc machines in `rg-mbsgw-<env>-uks-arc-enabled-servers` and assigns the `Gateway.Access` role to each machine's managed identity.
115+
116+
> **Why is this manual?** The pipeline managed identity (`mi-mbsgw-<env>-adotoaz-uks`) cannot create app role assignments via the pipeline — Microsoft Graph requires `AppRoleAssignment.ReadWrite.All` in application auth context regardless of SP ownership, and this permission cannot be granted under the organisation's Entra policy. Running `resource-group-init` under a user account with ownership of the enterprise app SP is sufficient. See [Deployment Pipeline — Section 12](../deployment-pipeline.md#12-pipeline-identities-and-permissions) for details.
117+
118+
**Verify**: In the Azure portal navigate to **Enterprise Applications → spn-manbrs-web-api-\<env\> → Users and groups**. The Arc machine (`gw-hull-university-teaching-hospitals-nhs-trust-rwa-01`) should appear with the `Gateway.Access` role.
119+
120+
## Step 5 — Deploy the gateway application
104121

105122
Run the ADO pipeline **Deploy Gateway - \<env\>** with:
106123

@@ -115,7 +132,7 @@ The pipeline:
115132
2. Sends an Arc Run Command to `gw-hull-university-teaching-hospitals-nhs-trust-rwa-01` that writes `.env` and runs `deploy.ps1`
116133
3. Polls for completion and reports success or failure
117134

118-
## Step 5 — Smoke test
135+
## Step 6 — Smoke test
119136

120137
Run from the gateway VM or via Arc Run Command:
121138

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
{
2+
"extensions": {
3+
"microsoftGraphV1": "br:mcr.microsoft.com/bicep/extensions/microsoftgraph/v1.0:1.0.0"
4+
}
5+
}
Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
extension microsoftGraphV1
2+
3+
param enterpriseAppClientId string
4+
param enterpriseAppName string
5+
param groupMemberIds array = []
6+
7+
resource enterpriseAppSp 'Microsoft.Graph/servicePrincipals@v1.0' = {
8+
appId: enterpriseAppClientId
9+
owners: {
10+
relationships: groupMemberIds
11+
}
12+
}
13+
14+
resource enterpriseApp 'Microsoft.Graph/applications@v1.0' = {
15+
uniqueName: enterpriseAppName
16+
displayName: enterpriseAppName
17+
owners: {
18+
relationships: groupMemberIds
19+
}
20+
}

infrastructure/terraform/resource_group_init/main.bicep

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,8 @@ param storageAccountRGName string
77
param storageAccountName string
88
param appShortName string
99
param userGroupPrincipalID string
10+
param enterpriseAppClientId string
11+
param groupMemberIds array = []
1012

1113
var hubMap = {
1214
dev: 'dev'
@@ -21,6 +23,7 @@ var privateDNSZoneRGName = 'rg-hub-${hubMap[envConfig]}-uks-private-dns-zones'
2123
var managedIdentityRGName = 'rg-mi-${envConfig}-uks'
2224
var infraResourceGroupName = 'rg-mbsgw-${envConfig}-infra'
2325
var keyVaultName = 'kv-mbsgw-${envConfig}-inf'
26+
var enterpriseAppName = 'spn-manbrs-web-api-${envConfig}'
2427

2528
var miADOtoAZname = 'mi-${appShortName}-${envConfig}-adotoaz-uks'
2629
var miGHtoADOname = 'mi-${appShortName}-${envConfig}-ghtoado-uks'
@@ -207,6 +210,16 @@ resource groupNetworkContributorAssignment 'Microsoft.Authorization/roleAssignme
207210
}
208211
}
209212

213+
module pipelineMiAppRole 'enterpriseAppOwner.bicep' = {
214+
name: 'pipelineMiAppRoleAssignment'
215+
scope: managedIdentityRG
216+
params: {
217+
enterpriseAppClientId: enterpriseAppClientId
218+
enterpriseAppName: enterpriseAppName
219+
groupMemberIds: groupMemberIds
220+
}
221+
}
222+
210223
output miPrincipalID string = managedIdentiyADOtoAZ.outputs.miPrincipalID
211224
output miName string = miADOtoAZname
212225
output keyVaultPrivateDNSZone string = keyVaultPrivateDNSZone.outputs.privateDNSZoneID
Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
#!/usr/bin/env bash
2+
# Assigns the Gateway.Access app role to the managed identity of each Arc-enabled server
3+
# in the environment, allowing the gateway to authenticate against the web API.
4+
#
5+
# This is intentionally a manual script rather than automated via Terraform because the
6+
# pipeline managed identity (mi-manbrs-<env>-adotoaz-uks) lacks the AppRoleAssignment.ReadWrite.All
7+
# Graph application permission required to create app role assignments. In the delegated auth
8+
# context (user running this script), ownership of the service principal is sufficient.
9+
#
10+
# Run this script after onboarding new Arc-enabled servers to assign the Gateway.Access role
11+
# to their managed identities.
12+
#
13+
# Usage: ./scripts/bash/assign_arc_app_roles.sh <env> <subscription>
14+
# Example: ./scripts/bash/assign_arc_app_roles.sh review "Breast Screening - Manage Breast Screening - Dev"
15+
set -eu
16+
17+
ENV_CONFIG="$1"
18+
SUBSCRIPTION="$2"
19+
20+
enterpriseAppName="spn-manbrs-web-api-${ENV_CONFIG}"
21+
rgName="rg-mbsgw-${ENV_CONFIG}-uks-arc-enabled-servers"
22+
appRoleValue="Gateway.Access"
23+
24+
az extension add --name connectedmachine --allow-preview true --yes 2>/dev/null || true
25+
26+
echo "Fetching enterprise app details for: $enterpriseAppName"
27+
spObjectId=$(az ad sp list --filter "displayName eq '${enterpriseAppName}'" --query "[0].id" -o tsv)
28+
appRoleId=$(az ad sp list --filter "displayName eq '${enterpriseAppName}'" --query "[0].appRoles[?value=='${appRoleValue}'].id | [0]" -o tsv)
29+
30+
if [ -z "$spObjectId" ]; then
31+
echo "Error: Enterprise app '$enterpriseAppName' not found"
32+
exit 1
33+
fi
34+
35+
echo "SP object ID: $spObjectId"
36+
echo "App role ($appRoleValue): $appRoleId"
37+
38+
echo "Listing Arc machines in: $rgName"
39+
arcMachines=$(az connectedmachine list --resource-group "$rgName" --subscription "$SUBSCRIPTION" --query "[].name" -o tsv)
40+
41+
if [ -z "$arcMachines" ]; then
42+
echo "No Arc machines found in $rgName"
43+
exit 0
44+
fi
45+
46+
while IFS= read -r machine; do
47+
[ -z "$machine" ] && continue
48+
49+
miPrincipalId=$(az connectedmachine show \
50+
--resource-group "$rgName" \
51+
--name "$machine" \
52+
--subscription "$SUBSCRIPTION" \
53+
--query "identity.principalId" -o tsv)
54+
55+
if [ -z "$miPrincipalId" ]; then
56+
echo " Warning: $machine has no managed identity principal ID, skipping."
57+
continue
58+
fi
59+
60+
echo "Assigning $appRoleValue to $machine (MI: $miPrincipalId)..."
61+
if ! output=$(az rest --method POST \
62+
--uri "https://graph.microsoft.com/v1.0/servicePrincipals/${spObjectId}/appRoleAssignedTo" \
63+
--headers "Content-Type=application/json" \
64+
--body "{\"principalId\": \"${miPrincipalId}\", \"resourceId\": \"${spObjectId}\", \"appRoleId\": \"${appRoleId}\"}" 2>&1); then
65+
if echo "$output" | grep -q "Permission being assigned already exists"; then
66+
echo " Already assigned, skipping."
67+
else
68+
echo "Error: $output"
69+
exit 1
70+
fi
71+
else
72+
echo " Successfully assigned."
73+
fi
74+
done <<< "$arcMachines"
75+
76+
echo "Done."

scripts/bash/resource_group_init.sh

Lines changed: 34 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,12 +22,43 @@ fi
2222

2323
echo "Found group Object ID: $userGroupPrincipalID"
2424

25+
echo "Fetching members of group: $userGroupName"
26+
groupMemberIds=$(az ad group member list --group "$userGroupName" --query "[].id" -o json)
27+
echo "Found group members: $groupMemberIds"
28+
29+
enterpriseAppName="spn-manbrs-web-api-${ENV_CONFIG}"
30+
echo "Fetching appId for enterprise app: $enterpriseAppName"
31+
enterpriseAppClientId=$(az ad sp list --filter "displayName eq '${enterpriseAppName}'" --query "[0].appId" -o tsv)
32+
33+
if [ -z "$enterpriseAppClientId" ]; then
34+
echo "Error: Enterprise app '$enterpriseAppName' not found in Entra ID"
35+
exit 1
36+
fi
37+
38+
echo "Found enterprise app client ID: $enterpriseAppClientId"
39+
40+
echo "Checking uniqueName on application registration $enterpriseAppName..."
41+
appObjectId=$(az ad app show --id "$enterpriseAppClientId" --query id -o tsv)
42+
currentUniqueName=$(az rest --method GET \
43+
--uri "https://graph.microsoft.com/v1.0/applications/${appObjectId}?\$select=uniqueName" \
44+
--query uniqueName -o tsv 2>/dev/null || echo "")
45+
if [ -z "$currentUniqueName" ]; then
46+
echo "Setting uniqueName to: $enterpriseAppName"
47+
az rest --method PATCH \
48+
--uri "https://graph.microsoft.com/v1.0/applications/${appObjectId}" \
49+
--headers "Content-Type=application/json" \
50+
--body "{\"uniqueName\": \"${enterpriseAppName}\"}"
51+
else
52+
echo "uniqueName already set to: $currentUniqueName"
53+
fi
54+
2555
echo "Deploy to hub subscription $HUB_SUBSCRIPTION_ID..."
2656
az deployment sub create --location "$REGION" --template-file infrastructure/terraform/resource_group_init/main.bicep \
2757
--subscription "$HUB_SUBSCRIPTION_ID" \
2858
--parameters enableSoftDelete="$ENABLE_SOFT_DELETE" envConfig="$ENV_CONFIG" region="$REGION" \
2959
storageAccountRGName="$STORAGE_ACCOUNT_RG" storageAccountName="$STORAGE_ACCOUNT_NAME" \
30-
appShortName="$APP_SHORT_NAME" userGroupPrincipalID="$userGroupPrincipalID" --what-if
60+
appShortName="$APP_SHORT_NAME" userGroupPrincipalID="$userGroupPrincipalID" \
61+
enterpriseAppClientId="$enterpriseAppClientId" groupMemberIds="$groupMemberIds" --what-if
3162

3263
read -r -p "Are you sure you want to execute the deployment? (y/n): " confirm
3364
[[ "$confirm" != "y" ]] && exit 0
@@ -36,7 +67,8 @@ output=$(az deployment sub create --location "$REGION" --template-file infrastru
3667
--subscription "$HUB_SUBSCRIPTION_ID" \
3768
--parameters enableSoftDelete="$ENABLE_SOFT_DELETE" envConfig="$ENV_CONFIG" region="$REGION" \
3869
storageAccountRGName="$STORAGE_ACCOUNT_RG" storageAccountName="$STORAGE_ACCOUNT_NAME" \
39-
appShortName="$APP_SHORT_NAME" userGroupPrincipalID="$userGroupPrincipalID")
70+
appShortName="$APP_SHORT_NAME" userGroupPrincipalID="$userGroupPrincipalID" \
71+
enterpriseAppClientId="$enterpriseAppClientId" groupMemberIds="$groupMemberIds")
4072

4173
echo "$output"
4274

scripts/terraform/terraform.mk

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,9 +21,10 @@ ci: # Skip manual approvals when running in CI - make ci <env> <action>
2121
set-azure-account: # Set the Azure account for the environment - make <env> set-azure-account
2222
[ "${SKIP_AZURE_LOGIN}" != "true" ] && az account set -s ${AZURE_SUBSCRIPTION} || true
2323

24-
resource-group-init: set-azure-account get-subscription-ids # Initialise the resources required by terraform - make <env> resource-group-init
24+
resource-group-init: set-azure-account get-subscription-ids # Initialise resources required by terraform and assign Gateway.Access role to Arc machine managed identities - make <env> resource-group-init
2525
$(eval STORAGE_ACCOUNT_NAME=sa${APP_SHORT_NAME}${ENV_CONFIG}tfstate)
2626
scripts/bash/resource_group_init.sh "${REGION}" "${HUB_SUBSCRIPTION_ID}" "${ENABLE_SOFT_DELETE}" "${ENV_CONFIG}" "${STORAGE_ACCOUNT_RG}" "${STORAGE_ACCOUNT_NAME}" "${APP_SHORT_NAME}" "${ARM_SUBSCRIPTION_ID}"
27+
scripts/bash/assign_arc_app_roles.sh "${ENV_CONFIG}" "${ARM_SUBSCRIPTION_ID}"
2728

2829
get-subscription-ids: # Retrieve the hub subscription ID based on the subscription name in ${HUB_SUBSCRIPTION} - make <env> get-subscription-ids
2930
$(eval HUB_SUBSCRIPTION_ID=$(shell az account show --query id --output tsv --name ${HUB_SUBSCRIPTION}))

0 commit comments

Comments
 (0)