feat/addFabricSensitivityLabels#24
Conversation
- Fix Purview config to match actual script expectations - Remove Foundry from dataSources (conceptual cleanup) - Replace OneLake 'roots' with proper Fabric hierarchy - Add Fabric sensitivity labeling and semantic models - Add inline Content Safety blocklists - Document required Purview scanning permissions - Fix tagging script to use Update-AzTag
Updates to allow minimal paramters for azd up command. This will corr…
…and-permissions Feature/improve spec structure and permissions
There was a problem hiding this comment.
Pull request overview
This PR introduces Fabric sensitivity label support, as announced at Tech Connect. It refactors the example spec (docs/spec-example.json) to a flatter, more explicit structure and upgrades the resource-tagging script to use Update-AzTag instead of Set-AzResource.
Changes:
- Replaces
Set-AzResourcewithUpdate-AzTag -Operation Mergein25-Tag-ResourcesFromSpec.ps1to avoid resource-deserialization failures. - Significantly restructures
docs/spec-example.json: flattens the top-level config, expands the Fabric workspaces block (addingsensitivityLabels,lakehouses,semanticModels), and alignsdefenderForAI/dlpPolicy/labels/retentionPolicies/activityExportto current script expectations.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
scripts/governance/dspmPurview/25-Tag-ResourcesFromSpec.ps1 |
Switches to Update-AzTag -Operation Merge for more robust tag patching |
docs/spec-example.json |
Comprehensive restructure including new Fabric sensitivity label definitions, updated scan/blocklist/retention rule schemas, and flattened top-level fields |
The script change in 25-Tag-ResourcesFromSpec.ps1 is a sound improvement. However, the restructured docs/spec-example.json introduces several field-name mismatches with the live PowerShell scripts that consume the spec. Since docs/spec-local-reference.md explicitly recommends copying this file directly into spec.local.json, these mismatches will silently break real user workflows.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| "retentionPolicies": [ | ||
| { | ||
| "name": "Retain-AI-Audit-7Years", | ||
| "rules": [ | ||
| { | ||
| "duration": "P7Y", | ||
| "action": "Keep" | ||
| } | ||
| ], | ||
| "locations": { | ||
| "Exchange": "All", | ||
| "SharePoint": "All" | ||
| } | ||
| } |
There was a problem hiding this comment.
The retentionPolicies[].rules[] entries in the updated example spec use duration (as a string, e.g. "P7Y") and have no name field, but 14-Create-RetentionPolicy.ps1 reads $r.name (used as the rule name for Get-RetentionComplianceRule and New-RetentionComplianceRule) and $r.durationDays (passed as -RetentionDuration). Using the example as-is will cause the retention rule to be created with an empty name and a null duration, silently producing an invalid or incorrectly configured compliance rule. The example should use name (string) and durationDays (integer) to match the script's field expectations.
| "fabric": { | ||
| "enabled": false, | ||
| "_comment": "Set enabled to true and populate these if scanning Fabric/OneLake", | ||
| "workspaceId": "", | ||
| "lakehouseName": "", | ||
| "oneLakeEndpoint": "" | ||
| "workspaces": [ | ||
| { | ||
| "name": "contoso-data-workspace", | ||
| "workspaceId": "12345678-90ab-cdef-1234-567890abcdef", | ||
| "capacityId": "98765432-10ba-fedc-4321-ba0987654321", | ||
| "scanName": "fabric-workspace-scan", | ||
| "sensitivityLabel": "Fabric-Confidential-Data", | ||
| "requiredPermissions": { | ||
| "principalType": "Purview Service Principal", | ||
| "roles": ["Fabric Workspace Viewer"], | ||
| "notes": "Grant in Fabric workspace settings under 'Manage Access'" | ||
| }, | ||
| "lakehouses": [ | ||
| { | ||
| "name": "contoso-training-lakehouse", | ||
| "lakehouseId": "abcdef12-3456-7890-abcd-ef1234567890", | ||
| "scanName": "training-data-scan", | ||
| "sensitivityLabel": "Fabric-AI-Training-Data" | ||
| }, | ||
| { | ||
| "name": "contoso-pii-lakehouse", | ||
| "lakehouseId": "fedcba98-7654-3210-fedc-ba9876543210", | ||
| "scanName": "pii-data-scan", | ||
| "sensitivityLabel": "Fabric-PII-Restricted" | ||
| } | ||
| ], | ||
| "semanticModels": [ | ||
| { | ||
| "name": "contoso-customer-insights-model", | ||
| "modelId": "11111111-2222-3333-4444-555555555555", | ||
| "scanName": "customer-model-scan", | ||
| "sensitivityLabel": "Fabric-Confidential-Data" | ||
| }, | ||
| { | ||
| "name": "contoso-ai-training-metrics-model", | ||
| "modelId": "66666666-7777-8888-9999-aaaaaaaaaaaa", | ||
| "scanName": "ai-metrics-scan", | ||
| "sensitivityLabel": "Fabric-AI-Training-Data" | ||
| }, | ||
| { | ||
| "name": "contoso-public-dashboard-model", | ||
| "modelId": "bbbbbbbb-cccc-dddd-eeee-ffffffffffff", | ||
| "scanName": "public-dashboard-scan", | ||
| "sensitivityLabel": "Fabric-Internal-Use" | ||
| } | ||
| ] | ||
| } | ||
| ], |
There was a problem hiding this comment.
The PR adds a richly-structured fabric.sensitivityLabels array and a new fabric.workspaces format (with workspaceId, capacityId, lakehouses, semanticModels, and sensitivityLabel fields) to spec-example.json, but docs/spec-local-reference.md is not updated to document any of these new fields. The reference doc still describes the old, simpler fabric.workspaces[].resourceId shape (which is now replaced by workspaceId/capacityId) and says nothing about fabric.sensitivityLabels. Since this doc is the primary field reference for operators, it should be updated to cover the new Fabric sensitivity labeling structure.
| "dataSourceName": "contoso-training-data-storage", | ||
| "scanName": "weekly-storage-scan", | ||
| "rulesetType": "System", | ||
| "rulesetName": "AzureStorage" | ||
| }, | ||
| { | ||
| "dataSourceName": "contoso-customer-database", | ||
| "scanName": "weekly-database-scan", | ||
| "rulesetType": "System", | ||
| "rulesetName": "AzureSqlDatabase" | ||
| } |
There was a problem hiding this comment.
The scans entries in the updated example use dataSourceName and scanName as property names, but 04-Run-Scan.ps1 reads $s.dataSource and $s.name respectively. Since spec-local-reference.md explicitly recommends copying docs/spec-example.json to spec.local.json, any user who follows that guidance will silently pass empty values to the Purview scan API (because PowerShell resolves missing properties as null rather than throwing), causing scan registrations to fail or create incorrectly-named scan definitions. The field names in the example spec's scans array should be changed to dataSource and name to match what 04-Run-Scan.ps1 expects.
| "textBlocklists": [ | ||
| { | ||
| "blocklistName": "contoso-blocked-terms", | ||
| "description": "Corporate blocked terms list", | ||
| "terms": [ | ||
| "confidential-project-codename", | ||
| "internal-auth-token", | ||
| "proprietary-algorithm", | ||
| "customer-ssn", | ||
| "trade-secret" | ||
| ] | ||
| }, | ||
| { | ||
| "blocklistName": "contoso-competitor-terms", | ||
| "description": "Competitor and sensitive business terms", | ||
| "terms": [ | ||
| "competitor-name-1", | ||
| "acquisition-target", | ||
| "unannounced-product", | ||
| "merger-discussions" | ||
| ] | ||
| } | ||
| ], | ||
| "harmSeverityThreshold": 2 |
There was a problem hiding this comment.
The textBlocklists entries in the example use blocklistName (to identify the blocklist) and terms (for the list of items), but 31-Foundry-ConfigureContentSafety.ps1 reads $bl.name (line 34) and $bl.items (line 36) respectively. Because PowerShell resolves missing properties as null, $bl.name will be empty and the blocklist REST call will use a blank path segment, and no items will be pushed since $bl.items will also be null. The example's blocklist entries should use name instead of blocklistName and items instead of terms to match the script's expectations.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| "retentionPolicies": [ | ||
| { | ||
| "name": "Retain-AI-Audit-7Years", | ||
| "rules": [ | ||
| { | ||
| "duration": "P7Y", | ||
| "action": "Keep" | ||
| } | ||
| ], |
There was a problem hiding this comment.
The retention rule object in the spec is missing the required name field and uses duration: "P7Y" (ISO 8601 string), but 14-Create-RetentionPolicy.ps1 (lines 13-14) reads $r.name and $r.durationDays (an integer in days). With the current structure: (1) the rule will never be found by Get-RetentionComplianceRule because $r.name is null, causing a null identity lookup, and (2) the -RetentionDuration parameter will receive $null instead of a number of days. Each rule entry needs a name field and durationDays set to an integer (e.g., 2557 for 7 years), not the ISO 8601 duration string.
Updates from feedback at Tech Connect. Added Fabric Sensitivity Labels functionality. Updated readme files and documentation to align with functionality. Updated script names to remove ambiguity. This pr will be the release once QA has concluded for Q3 release