diff --git a/docs/deployment/deployment-pipeline.md b/docs/deployment/deployment-pipeline.md index d8eb8816..2f16099f 100644 --- a/docs/deployment/deployment-pipeline.md +++ b/docs/deployment/deployment-pipeline.md @@ -422,6 +422,8 @@ The ADO managed identity holds: The managed identity does **not** hold `Key Vault Secrets Officer` — it cannot write secrets. Cloud API tokens must be provisioned separately by the team managing the cloud application. +The managed identity also does **not** hold `AppRoleAssignment.ReadWrite.All`. Microsoft Graph requires this explicit application permission to create app role assignments — being an owner of the enterprise app service principal is insufficient in application auth context. Granting this tenant-wide permission is not permitted under the organisation's Entra policy. Arc machine managed identity app role assignments are therefore granted manually by running `make resource-group-init` under a user account with ownership of `spn-manbrs-web-api-` — see [`scripts/bash/assign_arc_app_roles.sh`](../../scripts/bash/assign_arc_app_roles.sh) and the [Onboard Hospital VM runbook](./runbooks/onboard-hospital-vm.md#step-4--grant-api-access). + ### Separation of environments Each environment (`dev`, `preprod`, `prod`) has its own: diff --git a/docs/deployment/infrastructure/create-environment.md b/docs/deployment/infrastructure/create-environment.md index aa6d7a65..0bff7bcb 100644 --- a/docs/deployment/infrastructure/create-environment.md +++ b/docs/deployment/infrastructure/create-environment.md @@ -52,6 +52,11 @@ az provider show --namespace Microsoft.GuestConfiguration --query registrationSt - Name: `spn-azure-arc-onboarding-screening-[environment]` - If it does not exist, raise a request with the platform team to create it +- Confirm the web API enterprise application exists: + - Name: `spn-manbrs-web-api-[environment]` + - If it does not exist, raise a ServiceNow request for the platform team to create it — the `resource-group-init` Bicep deployment and the `assign_arc_app_roles.sh` script both depend on it + - [Form for ServiceNow](https://nhsdigitallive.service-now.com/nhs_digital?id=sc_cat_item&sys_id=28f3ab4f1bf3ca1078ac4337b04bcb78&sysparm_category=114fced51bdae1502eee65b9bd4bcbdc) + ## Code - Create the configuration files in `infrastructure/environments/[environment]/`: @@ -233,3 +238,11 @@ Terraform automatically discovers Arc machines registered in the Arc-enabled ser ```bash make [environment] terraform-apply ``` + +After Terraform provisions the Hybrid Connection, re-run `resource-group-init` to assign the `Gateway.Access` app role to the new machine's managed identity: + +```bash +make [environment] resource-group-init +``` + +> **Why is this a separate step?** The pipeline managed identity cannot create app role assignments — `AppRoleAssignment.ReadWrite.All` is required in application auth context regardless of SP ownership, and this permission is not permitted under the organisation's Entra policy. Running `resource-group-init` under a user account with ownership of `spn-manbrs-web-api-[environment]` is sufficient. Without this step the gateway will fail to authenticate against the cloud web API. See [Onboard Hospital VM — Step 4](../runbooks/onboard-hospital-vm.md#step-4--grant-api-access). diff --git a/docs/deployment/runbooks/onboard-hospital-vm.md b/docs/deployment/runbooks/onboard-hospital-vm.md index 62182602..6e88ee66 100644 --- a/docs/deployment/runbooks/onboard-hospital-vm.md +++ b/docs/deployment/runbooks/onboard-hospital-vm.md @@ -100,7 +100,24 @@ Run the ADO pipeline **Deploy Arc Infrastructure - \** manually. Terraform **Verify**: In the Azure portal, navigate to `relay-manbrs-` → Hybrid Connections → `hc-gw-hull-university-teaching-hospitals-nhs-trust-rwa-01` is present. -## Step 4 — Deploy the gateway application +## Step 4 — Grant API access + +> [!IMPORTANT] +> Without this step the gateway services will start but fail to authenticate against the cloud web API. The Arc machine's managed identity must be assigned the `Gateway.Access` app role on `spn-manbrs-web-api-` before the first deployment. + +Run from a developer machine with Owner access to the enterprise application: + +```bash +make resource-group-init +``` + +This calls [`scripts/bash/assign_arc_app_roles.sh`](../../../scripts/bash/assign_arc_app_roles.sh) which discovers all Arc machines in `rg-mbsgw--uks-arc-enabled-servers` and assigns the `Gateway.Access` role to each machine's managed identity. + +> **Why is this manual?** The pipeline managed identity (`mi-mbsgw--adotoaz-uks`) cannot create app role assignments via the pipeline — Microsoft Graph requires `AppRoleAssignment.ReadWrite.All` in application auth context regardless of SP ownership, and this permission cannot be granted under the organisation's Entra policy. Running `resource-group-init` under a user account with ownership of the enterprise app SP is sufficient. See [Deployment Pipeline — Section 12](../deployment-pipeline.md#12-pipeline-identities-and-permissions) for details. + +**Verify**: In the Azure portal navigate to **Enterprise Applications → spn-manbrs-web-api-\ → Users and groups**. The Arc machine (`gw-hull-university-teaching-hospitals-nhs-trust-rwa-01`) should appear with the `Gateway.Access` role. + +## Step 5 — Deploy the gateway application Run the ADO pipeline **Deploy Gateway - \** with: @@ -115,7 +132,7 @@ The pipeline: 2. Sends an Arc Run Command to `gw-hull-university-teaching-hospitals-nhs-trust-rwa-01` that writes `.env` and runs `deploy.ps1` 3. Polls for completion and reports success or failure -## Step 5 — Smoke test +## Step 6 — Smoke test Run from the gateway VM or via Arc Run Command: diff --git a/infrastructure/terraform/resource_group_init/bicepconfig.json b/infrastructure/terraform/resource_group_init/bicepconfig.json new file mode 100644 index 00000000..cd15f3f3 --- /dev/null +++ b/infrastructure/terraform/resource_group_init/bicepconfig.json @@ -0,0 +1,5 @@ +{ + "extensions": { + "microsoftGraphV1": "br:mcr.microsoft.com/bicep/extensions/microsoftgraph/v1.0:1.0.0" + } +} diff --git a/infrastructure/terraform/resource_group_init/enterpriseAppOwner.bicep b/infrastructure/terraform/resource_group_init/enterpriseAppOwner.bicep new file mode 100644 index 00000000..a23c1a13 --- /dev/null +++ b/infrastructure/terraform/resource_group_init/enterpriseAppOwner.bicep @@ -0,0 +1,20 @@ +extension microsoftGraphV1 + +param enterpriseAppClientId string +param enterpriseAppName string +param groupMemberIds array = [] + +resource enterpriseAppSp 'Microsoft.Graph/servicePrincipals@v1.0' = { + appId: enterpriseAppClientId + owners: { + relationships: groupMemberIds + } +} + +resource enterpriseApp 'Microsoft.Graph/applications@v1.0' = { + uniqueName: enterpriseAppName + displayName: enterpriseAppName + owners: { + relationships: groupMemberIds + } +} diff --git a/infrastructure/terraform/resource_group_init/main.bicep b/infrastructure/terraform/resource_group_init/main.bicep index 3fade281..34cdda73 100644 --- a/infrastructure/terraform/resource_group_init/main.bicep +++ b/infrastructure/terraform/resource_group_init/main.bicep @@ -7,6 +7,8 @@ param storageAccountRGName string param storageAccountName string param appShortName string param userGroupPrincipalID string +param enterpriseAppClientId string +param groupMemberIds array = [] var hubMap = { dev: 'dev' @@ -21,6 +23,7 @@ var privateDNSZoneRGName = 'rg-hub-${hubMap[envConfig]}-uks-private-dns-zones' var managedIdentityRGName = 'rg-mi-${envConfig}-uks' var infraResourceGroupName = 'rg-mbsgw-${envConfig}-infra' var keyVaultName = 'kv-mbsgw-${envConfig}-inf' +var enterpriseAppName = 'spn-manbrs-web-api-${envConfig}' var miADOtoAZname = 'mi-${appShortName}-${envConfig}-adotoaz-uks' var miGHtoADOname = 'mi-${appShortName}-${envConfig}-ghtoado-uks' @@ -207,6 +210,16 @@ resource groupNetworkContributorAssignment 'Microsoft.Authorization/roleAssignme } } +module pipelineMiAppRole 'enterpriseAppOwner.bicep' = { + name: 'pipelineMiAppRoleAssignment' + scope: managedIdentityRG + params: { + enterpriseAppClientId: enterpriseAppClientId + enterpriseAppName: enterpriseAppName + groupMemberIds: groupMemberIds + } +} + output miPrincipalID string = managedIdentiyADOtoAZ.outputs.miPrincipalID output miName string = miADOtoAZname output keyVaultPrivateDNSZone string = keyVaultPrivateDNSZone.outputs.privateDNSZoneID diff --git a/scripts/bash/assign_arc_app_roles.sh b/scripts/bash/assign_arc_app_roles.sh new file mode 100755 index 00000000..9d7713de --- /dev/null +++ b/scripts/bash/assign_arc_app_roles.sh @@ -0,0 +1,76 @@ +#!/usr/bin/env bash +# Assigns the Gateway.Access app role to the managed identity of each Arc-enabled server +# in the environment, allowing the gateway to authenticate against the web API. +# +# This is intentionally a manual script rather than automated via Terraform because the +# pipeline managed identity (mi-manbrs--adotoaz-uks) lacks the AppRoleAssignment.ReadWrite.All +# Graph application permission required to create app role assignments. In the delegated auth +# context (user running this script), ownership of the service principal is sufficient. +# +# Run this script after onboarding new Arc-enabled servers to assign the Gateway.Access role +# to their managed identities. +# +# Usage: ./scripts/bash/assign_arc_app_roles.sh +# Example: ./scripts/bash/assign_arc_app_roles.sh review "Breast Screening - Manage Breast Screening - Dev" +set -eu + +ENV_CONFIG="$1" +SUBSCRIPTION="$2" + +enterpriseAppName="spn-manbrs-web-api-${ENV_CONFIG}" +rgName="rg-mbsgw-${ENV_CONFIG}-uks-arc-enabled-servers" +appRoleValue="Gateway.Access" + +az extension add --name connectedmachine --allow-preview true --yes 2>/dev/null || true + +echo "Fetching enterprise app details for: $enterpriseAppName" +spObjectId=$(az ad sp list --filter "displayName eq '${enterpriseAppName}'" --query "[0].id" -o tsv) +appRoleId=$(az ad sp list --filter "displayName eq '${enterpriseAppName}'" --query "[0].appRoles[?value=='${appRoleValue}'].id | [0]" -o tsv) + +if [ -z "$spObjectId" ]; then + echo "Error: Enterprise app '$enterpriseAppName' not found" + exit 1 +fi + +echo "SP object ID: $spObjectId" +echo "App role ($appRoleValue): $appRoleId" + +echo "Listing Arc machines in: $rgName" +arcMachines=$(az connectedmachine list --resource-group "$rgName" --subscription "$SUBSCRIPTION" --query "[].name" -o tsv) + +if [ -z "$arcMachines" ]; then + echo "No Arc machines found in $rgName" + exit 0 +fi + +while IFS= read -r machine; do + [ -z "$machine" ] && continue + + miPrincipalId=$(az connectedmachine show \ + --resource-group "$rgName" \ + --name "$machine" \ + --subscription "$SUBSCRIPTION" \ + --query "identity.principalId" -o tsv) + + if [ -z "$miPrincipalId" ]; then + echo " Warning: $machine has no managed identity principal ID, skipping." + continue + fi + + echo "Assigning $appRoleValue to $machine (MI: $miPrincipalId)..." + if ! output=$(az rest --method POST \ + --uri "https://graph.microsoft.com/v1.0/servicePrincipals/${spObjectId}/appRoleAssignedTo" \ + --headers "Content-Type=application/json" \ + --body "{\"principalId\": \"${miPrincipalId}\", \"resourceId\": \"${spObjectId}\", \"appRoleId\": \"${appRoleId}\"}" 2>&1); then + if echo "$output" | grep -q "Permission being assigned already exists"; then + echo " Already assigned, skipping." + else + echo "Error: $output" + exit 1 + fi + else + echo " Successfully assigned." + fi +done <<< "$arcMachines" + +echo "Done." diff --git a/scripts/bash/resource_group_init.sh b/scripts/bash/resource_group_init.sh index 22a10d93..93a4bcb7 100755 --- a/scripts/bash/resource_group_init.sh +++ b/scripts/bash/resource_group_init.sh @@ -22,12 +22,43 @@ fi echo "Found group Object ID: $userGroupPrincipalID" +echo "Fetching members of group: $userGroupName" +groupMemberIds=$(az ad group member list --group "$userGroupName" --query "[].id" -o json) +echo "Found group members: $groupMemberIds" + +enterpriseAppName="spn-manbrs-web-api-${ENV_CONFIG}" +echo "Fetching appId for enterprise app: $enterpriseAppName" +enterpriseAppClientId=$(az ad sp list --filter "displayName eq '${enterpriseAppName}'" --query "[0].appId" -o tsv) + +if [ -z "$enterpriseAppClientId" ]; then + echo "Error: Enterprise app '$enterpriseAppName' not found in Entra ID" + exit 1 +fi + +echo "Found enterprise app client ID: $enterpriseAppClientId" + +echo "Checking uniqueName on application registration $enterpriseAppName..." +appObjectId=$(az ad app show --id "$enterpriseAppClientId" --query id -o tsv) +currentUniqueName=$(az rest --method GET \ + --uri "https://graph.microsoft.com/v1.0/applications/${appObjectId}?\$select=uniqueName" \ + --query uniqueName -o tsv 2>/dev/null || echo "") +if [ -z "$currentUniqueName" ]; then + echo "Setting uniqueName to: $enterpriseAppName" + az rest --method PATCH \ + --uri "https://graph.microsoft.com/v1.0/applications/${appObjectId}" \ + --headers "Content-Type=application/json" \ + --body "{\"uniqueName\": \"${enterpriseAppName}\"}" +else + echo "uniqueName already set to: $currentUniqueName" +fi + echo "Deploy to hub subscription $HUB_SUBSCRIPTION_ID..." az deployment sub create --location "$REGION" --template-file infrastructure/terraform/resource_group_init/main.bicep \ --subscription "$HUB_SUBSCRIPTION_ID" \ --parameters enableSoftDelete="$ENABLE_SOFT_DELETE" envConfig="$ENV_CONFIG" region="$REGION" \ storageAccountRGName="$STORAGE_ACCOUNT_RG" storageAccountName="$STORAGE_ACCOUNT_NAME" \ - appShortName="$APP_SHORT_NAME" userGroupPrincipalID="$userGroupPrincipalID" --what-if + appShortName="$APP_SHORT_NAME" userGroupPrincipalID="$userGroupPrincipalID" \ + enterpriseAppClientId="$enterpriseAppClientId" groupMemberIds="$groupMemberIds" --what-if read -r -p "Are you sure you want to execute the deployment? (y/n): " confirm [[ "$confirm" != "y" ]] && exit 0 @@ -36,7 +67,8 @@ output=$(az deployment sub create --location "$REGION" --template-file infrastru --subscription "$HUB_SUBSCRIPTION_ID" \ --parameters enableSoftDelete="$ENABLE_SOFT_DELETE" envConfig="$ENV_CONFIG" region="$REGION" \ storageAccountRGName="$STORAGE_ACCOUNT_RG" storageAccountName="$STORAGE_ACCOUNT_NAME" \ - appShortName="$APP_SHORT_NAME" userGroupPrincipalID="$userGroupPrincipalID") + appShortName="$APP_SHORT_NAME" userGroupPrincipalID="$userGroupPrincipalID" \ + enterpriseAppClientId="$enterpriseAppClientId" groupMemberIds="$groupMemberIds") echo "$output" diff --git a/scripts/terraform/terraform.mk b/scripts/terraform/terraform.mk index ed9f5947..e058dc47 100644 --- a/scripts/terraform/terraform.mk +++ b/scripts/terraform/terraform.mk @@ -21,9 +21,10 @@ ci: # Skip manual approvals when running in CI - make ci set-azure-account: # Set the Azure account for the environment - make set-azure-account [ "${SKIP_AZURE_LOGIN}" != "true" ] && az account set -s ${AZURE_SUBSCRIPTION} || true -resource-group-init: set-azure-account get-subscription-ids # Initialise the resources required by terraform - make resource-group-init +resource-group-init: set-azure-account get-subscription-ids # Initialise resources required by terraform and assign Gateway.Access role to Arc machine managed identities - make resource-group-init $(eval STORAGE_ACCOUNT_NAME=sa${APP_SHORT_NAME}${ENV_CONFIG}tfstate) scripts/bash/resource_group_init.sh "${REGION}" "${HUB_SUBSCRIPTION_ID}" "${ENABLE_SOFT_DELETE}" "${ENV_CONFIG}" "${STORAGE_ACCOUNT_RG}" "${STORAGE_ACCOUNT_NAME}" "${APP_SHORT_NAME}" "${ARM_SUBSCRIPTION_ID}" + scripts/bash/assign_arc_app_roles.sh "${ENV_CONFIG}" "${ARM_SUBSCRIPTION_ID}" get-subscription-ids: # Retrieve the hub subscription ID based on the subscription name in ${HUB_SUBSCRIPTION} - make get-subscription-ids $(eval HUB_SUBSCRIPTION_ID=$(shell az account show --query id --output tsv --name ${HUB_SUBSCRIPTION}))