From b09e48ee9c49b30cb9ea510b022245151187b6d1 Mon Sep 17 00:00:00 2001 From: Copilot <223556219+Copilot@users.noreply.github.com> Date: Sat, 25 Jul 2026 18:45:12 +0200 Subject: [PATCH 01/20] docs: position Custo as central distributor engine Rewrite README to explain source-of-truth separation between MSXOrg/docs standards, initiative docs, and this distribution runtime. Add repo-root AGENTS.md with operator runbook for enabling live rollout. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- AGENTS.md | 37 +++++++++++++++++++++++++++ README.md | 75 ++++++++++++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 111 insertions(+), 1 deletion(-) create mode 100644 AGENTS.md diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..1d89497 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,37 @@ +# Agents + +## Main directive + +Everything here is a work in progress and can be improved. If you find a problem, fix it if it's +small; otherwise register it as an issue in this repo. + +Read [MSXOrg/docs AGENTS.md](https://github.com/MSXOrg/docs/blob/main/AGENTS.md) first for +ecosystem-wide onboarding. This file only covers what's specific to Custo. + +## What this repo is + +Custo is the distribution runtime described in [README.md](README.md). Read that first for the +architecture and the source-of-truth separation between standards, initiative docs, and this +runtime. + +## Operator runbook: enabling live rollout + +The MVP file set (`Repos/Module/AGENTS.md/`) and workflow are in place, but the scheduled sync +cannot run against real repositories until an operator completes these steps: + +1. **Configure GitHub App credentials.** Add `CUSTO_BOT_CLIENT_ID` and `CUSTO_BOT_PRIVATE_KEY` + as repository secrets on `MSXOrg/Custo`, using the same `PSModule's Custo` GitHub App + (app id `1320343`, slug `psmodule-s-custo`) already installed on the `PSModule` organization. + If that app's credentials are not available, generate a new private key for it or provision an + equivalent app with `contents:write`, `pull_requests:write`, and + `repository_custom_properties:read` on target repositories. +2. **Confirm app installation scope.** Verify the app is installed on the `PSModule` organization + with access to the module repositories that should receive `AGENTS.md` + (`gh api orgs/PSModule/installations`). +3. **Set custom properties on target repositories.** Each subscribing module repository needs + `Type = Module` and `SubscribeTo` including `AGENTS.md` set at the repository level. +4. **Dry-run via `workflow_dispatch`.** Trigger the `Sync Managed Files` workflow manually first + and review the summary before relying on the daily schedule. + +Until step 1 is complete, the workflow will fail at the authentication step +(`Connect-GitHub App`) — this is the current, expected blocker for this MVP. diff --git a/README.md b/README.md index f1bda8f..e508c36 100644 --- a/README.md +++ b/README.md @@ -1,2 +1,75 @@ # Custo -Central managed-file distribution and compliance orchestration for MSX initiatives + +Custo is the central managed-file distribution engine for MSX initiatives. It syncs shared, +centrally-owned files into subscribing repositories by pull request, so every repository +inherits the same governance, linting, and agent-context files without copy-pasting them by hand. + +## Source of truth separation + +- **Standards** — *what* every initiative must define and *why* — live in + [`MSXOrg/docs`](https://github.com/MSXOrg/docs), specifically the + [Organization Standard](https://msxorg.github.io/docs/Ways-of-Working/Organization-Standard/). +- **Initiative implementation guidance** — how a specific initiative applies those standards — + lives in each initiative's own docs repository, for example + [`PSModule/docs`](https://github.com/PSModule/docs). +- **Distribution runtime** — *how* managed files are actually delivered to repositories — lives + here, in Custo. + +Custo does not decide what files initiatives should manage. It only distributes what each +initiative's file sets declare, following the [managed files contract](https://msxorg.github.io/docs/Ways-of-Working/Organization-Standard/#managed-files) +defined in MSXOrg/docs. + +Custo replaces [`PSModule/Distributor`](https://github.com/PSModule/Distributor) as the runtime +engine for the PSModule initiative, generalized so other initiatives can reuse it instead of +building their own distributor from scratch. + +## How it works + +File sets are organized by target organization, repository type, and selection: + +```text +Repos/{Type}/{Selection}/ +``` + +- **Type** — groups repositories by kind (`Module`, `Action`, `Template`, `Workflow`, ...). +- **Selection** — an individual file set repositories opt into via the `SubscribeTo` custom + property. Each selection folder mirrors the root of a target repository. + +Target organizations are declared in [`config/targets.json`](config/targets.json), not hardcoded +in the sync script. This keeps Custo org-agnostic: adding a new initiative organization is a +config change, not a code change. + +The [`scripts/Sync-Files.ps1`](scripts/Sync-Files.ps1) script, run by the +[`Sync Managed Files`](.github/workflows/sync-files.yml) workflow: + +1. Reads the target organizations from `config/targets.json`. +2. Discovers file sets under `Repos/`. +3. For each target org, queries repositories for their `Type` and `SubscribeTo` custom + properties. +4. Clones subscribing repositories, copies the relevant files, and opens or updates a + `managed-files/update` pull request when changes are detected. + +## MVP rollout scope + +The first managed resource is **`AGENTS.md`** for PowerShell module repositories +(`Repos/Module/AGENTS.md/AGENTS.md`), targeting the `PSModule` organization by default. The file +is a thin pointer into the central docs rather than a duplicated process document, matching the +pattern already used by [`PSModule/Template-PSModule`](https://github.com/PSModule/Template-PSModule) +and [`PSModule/memory`](https://github.com/PSModule/memory). + +Additional file sets and target organizations are added incrementally after this MVP is proven. + +## Required secrets + +The sync workflow authenticates as a GitHub App with `contents`, `pull_requests`, and +`repository_custom_properties` access on target repositories. Configure these repository secrets +before enabling the scheduled sync: + +- `CUSTO_BOT_CLIENT_ID` +- `CUSTO_BOT_PRIVATE_KEY` + +See [`AGENTS.md`](AGENTS.md) for operator runbook steps and current rollout blockers. + +## License + +MIT License — see [LICENSE](LICENSE). From c9c8f31ebb0e90c745e21733589981ebeeded1f9 Mon Sep 17 00:00:00 2001 From: Copilot <223556219+Copilot@users.noreply.github.com> Date: Sat, 25 Jul 2026 18:46:55 +0200 Subject: [PATCH 02/20] feat: add org-agnostic sync engine Port Sync-Files.ps1 from PSModule/Distributor, generalized to read target organizations from config/targets.json instead of a hardcoded 'PSModule' owner. Add a JSON schema for the targets config. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- config/targets.json | 6 + config/targets.schema.json | 20 ++ scripts/Sync-Files.ps1 | 451 +++++++++++++++++++++++++++++++++++++ 3 files changed, 477 insertions(+) create mode 100644 config/targets.json create mode 100644 config/targets.schema.json create mode 100644 scripts/Sync-Files.ps1 diff --git a/config/targets.json b/config/targets.json new file mode 100644 index 0000000..b3480b7 --- /dev/null +++ b/config/targets.json @@ -0,0 +1,6 @@ +{ + "$schema": "./targets.schema.json", + "organizations": [ + "PSModule" + ] +} diff --git a/config/targets.schema.json b/config/targets.schema.json new file mode 100644 index 0000000..01e58dd --- /dev/null +++ b/config/targets.schema.json @@ -0,0 +1,20 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "title": "Custo target organizations", + "description": "Organizations that the Sync Managed Files workflow discovers subscribing repositories in.", + "type": "object", + "required": ["organizations"], + "properties": { + "organizations": { + "type": "array", + "description": "GitHub organization logins to scan for subscribing repositories.", + "items": { + "type": "string", + "minLength": 1 + }, + "minItems": 1, + "uniqueItems": true + } + }, + "additionalProperties": false +} diff --git a/scripts/Sync-Files.ps1 b/scripts/Sync-Files.ps1 new file mode 100644 index 0000000..f73e7e1 --- /dev/null +++ b/scripts/Sync-Files.ps1 @@ -0,0 +1,451 @@ +#!/usr/bin/env pwsh +<# +.SYNOPSIS + Syncs managed files from this repository to subscribing repositories across configured + target organizations. + +.DESCRIPTION + This script: + 1. Authenticates as a GitHub App for repo-level operations. + 2. Discovers available file sets from the Repos/ directory structure. + 3. Reads the target organizations from config/targets.json. + 4. For each target organization, queries repositories for their Type and SubscribeTo + custom properties. + 5. For each subscribing repository: + - Clones the repository + - Copies managed files from the appropriate file sets + - Detects changes using git + - Creates or updates a pull request if changes are detected + 6. Outputs a summary of actions taken. + +.NOTES + Requires the GitHub PowerShell module and GitHub App authentication via GitHub-Script action. + Target organizations are configuration, not code - see config/targets.json. This keeps the + script usable by any MSX initiative, not just PSModule. +#> + +[CmdletBinding()] +param() + +$ErrorActionPreference = 'Stop' + +# Track summary information +$script:Summary = @{ + TotalReposProcessed = 0 + PRsCreated = 0 + PRsUpdated = 0 + ReposAlreadyInSync = 0 + ReposSkipped = 0 + Errors = @() +} + +#region Helper Functions + +function Get-TargetOrganization { + <# + .SYNOPSIS + Reads the target organization list from config/targets.json. + #> + [CmdletBinding()] + param( + [Parameter(Mandatory)] + [string]$ConfigPath + ) + + if (-not (Test-Path $ConfigPath)) { + throw "Targets config not found at: $ConfigPath" + } + + $config = Get-Content -Path $ConfigPath -Raw | ConvertFrom-Json + + if (-not $config.organizations -or $config.organizations.Count -eq 0) { + throw "No organizations configured in: $ConfigPath" + } + + return @($config.organizations) +} + +function Get-FileSets { + <# + .SYNOPSIS + Discovers available file sets from the Repos/ directory structure. + #> + [CmdletBinding()] + param( + [Parameter(Mandatory)] + [string]$ReposPath + ) + + $fileSets = @{} + + if (-not (Test-Path $ReposPath)) { + throw "Repos directory not found at: $ReposPath" + } + + $typeDirs = Get-ChildItem -Path $ReposPath -Directory + $fileSetTable = @() + + foreach ($typeDir in $typeDirs) { + $typeName = $typeDir.Name + $fileSets[$typeName] = @{} + + $selectionDirs = Get-ChildItem -Path $typeDir.FullName -Directory + + foreach ($selectionDir in $selectionDirs) { + $selectionName = $selectionDir.Name + $files = Get-ChildItem -Path $selectionDir.FullName -File -Recurse + + $fileList = @() + foreach ($file in $files) { + $relativePath = $file.FullName.Substring($selectionDir.FullName.Length + 1) + $fileList += @{ + SourcePath = $file.FullName + RelativePath = $relativePath + } + } + + $fileSets[$typeName][$selectionName] = $fileList + $fileSetTable += [PSCustomObject]@{ + Type = $typeName + FileSet = $selectionName + Files = $fileList.Count + } + } + } + + $fileSetTable | Format-Table -AutoSize | Out-String + + return $fileSets +} + +function Get-SubscribingRepository { + <# + .SYNOPSIS + Queries an organization's repositories for their Type and SubscribeTo custom properties. + #> + [CmdletBinding()] + param( + [Parameter(Mandatory)] + [string]$Owner, + + [Parameter(Mandatory)] + [object]$Context + ) + + $repos = Get-GitHubRepository -Owner $Owner -Context $Context + + $subscribingRepos = @() + + foreach ($repo in $repos) { + $customProps = $repo.CustomProperties + + if (-not $customProps) { + continue + } + + $type = ($customProps | Where-Object Name -EQ 'Type').Value + $subscribeTo = ($customProps | Where-Object Name -EQ 'SubscribeTo').Value + + if (-not $type -or -not $subscribeTo) { + continue + } + + if ($subscribeTo -is [string]) { + $subscribeTo = @($subscribeTo) + } + + if ($subscribeTo.Count -eq 0) { + continue + } + + $subscribingRepos += @{ + Name = $repo.Name + Owner = $repo.Owner.Login + FullName = $repo.FullName + Type = $type + SubscribeTo = $subscribeTo + DefaultBranch = $repo.DefaultBranch + } + } + + $subscribingRepos | ForEach-Object { + [PSCustomObject]@{ + Owner = $_.Owner + Repo = $_.Name + Type = $_.Type + SubscribeTo = $_.SubscribeTo -join ', ' + } + } | Format-Table -AutoSize | Out-String + + return $subscribingRepos +} + +function Sync-RepositoryFile { + <# + .SYNOPSIS + Syncs files to a single repository. + #> + [Diagnostics.CodeAnalysis.SuppressMessageAttribute( + 'PSAvoidUsingWriteHost', '', Scope = 'Function', + Justification = 'Intended for logging in GitHub Actions runners.' + )] + [CmdletBinding()] + param( + [Parameter(Mandatory)] + [hashtable]$Repository, + + [Parameter(Mandatory)] + [hashtable]$FileSets, + + [Parameter(Mandatory)] + [string]$TempPath, + + [Parameter(Mandatory)] + [string]$BranchName, + + [Parameter(Mandatory)] + [string]$CommitMessage, + + [Parameter(Mandatory)] + [string]$PRTitle, + + [Parameter(Mandatory)] + [string]$PRBody, + + [Parameter(Mandatory)] + [string]$PRLabel, + + [Parameter(Mandatory)] + [object]$Context + ) + + $repoFullName = $Repository.FullName + $owner = $Repository.Owner + $repoName = $Repository.Name + $type = $Repository.Type + $subscribeTo = $Repository.SubscribeTo + + $script:Summary.TotalReposProcessed++ + + # Validate before opening a log group - skipped repos stay quiet + if (-not $FileSets.ContainsKey($type)) { + Write-Host "⚠️ $repoFullName - Type folder '$type' not found, skipping" + $script:Summary.ReposSkipped++ + return + } + + $filesToSync = @() + foreach ($selection in $subscribeTo) { + if (-not $FileSets[$type].ContainsKey($selection)) { + Write-Host "⚠️ $repoFullName - Selection '$selection' not found under '$type'" + continue + } + $filesToSync += $FileSets[$type][$selection] + } + + if ($filesToSync.Count -eq 0) { + Write-Host "⚠️ $repoFullName - No matching files, skipping" + $script:Summary.ReposSkipped++ + return + } + + # All real work inside a log group + LogGroup "📦 $repoFullName" { + foreach ($selection in $subscribeTo) { + if ($FileSets[$type].ContainsKey($selection)) { + Write-Host " + $type/$selection ($($FileSets[$type][$selection].Count) files)" + } + } + + $clonePath = Join-Path $TempPath "clone-$repoName-$(Get-Random)" + New-Item -Path $clonePath -ItemType Directory -Force | Out-Null + + try { + $cloneUrl = "https://github.com/$repoFullName.git" + $gitCloneResult = git clone --depth 1 $cloneUrl $clonePath 2>&1 + if ($LASTEXITCODE -ne 0) { + throw "Git clone failed: $gitCloneResult" + } + + Push-Location $clonePath + try { + Set-GitHubGitConfig -Context $Context + + # Branch setup + $remoteBranches = git branch -r 2>&1 + if ($remoteBranches -match "origin/$BranchName") { + git fetch origin $BranchName 2>&1 | Out-Null + git checkout $BranchName 2>&1 | Out-Null + } else { + git checkout -b $BranchName 2>&1 | Out-Null + } + + # Copy files + foreach ($fileInfo in $filesToSync) { + $targetPath = Join-Path $clonePath $fileInfo.RelativePath + $targetDir = Split-Path $targetPath -Parent + if (-not (Test-Path $targetDir)) { + New-Item -Path $targetDir -ItemType Directory -Force | Out-Null + } + Copy-Item -Path $fileInfo.SourcePath -Destination $targetPath -Force + } + + # Detect changes + $status = git status --porcelain 2>&1 + if ([string]::IsNullOrWhiteSpace($status)) { + Write-Host '✅ Already in sync' + $script:Summary.ReposAlreadyInSync++ + return + } + + $status -split "`n" | ForEach-Object { Write-Host " $_" } + + # Commit and push + git add --all 2>&1 | Out-Null + git commit -m $CommitMessage 2>&1 | Out-Null + $pushResult = git push --force --set-upstream origin $BranchName 2>&1 + if ($LASTEXITCODE -ne 0) { + throw "Git push failed: $pushResult" + } + + # Create or update PR + $existingPRs = (Invoke-GitHubAPI -Method GET -Endpoint "/repos/$owner/$repoName/pulls" -Body @{ + head = "${owner}:${BranchName}" + state = 'open' + } -Context $Context).Response + + if ($existingPRs.Count -gt 0) { + Write-Host "✅ Updated PR #$($existingPRs[0].number) - $($existingPRs[0].html_url)" + $script:Summary.PRsUpdated++ + } else { + $pr = (Invoke-GitHubAPI -Method POST -Endpoint "/repos/$owner/$repoName/pulls" -Body @{ + title = $PRTitle + head = $BranchName + base = $Repository.DefaultBranch + body = $PRBody + } -Context $Context).Response + + try { + Invoke-GitHubAPI -Method POST -Endpoint "/repos/$owner/$repoName/issues/$($pr.number)/labels" -Body @{ + labels = @($PRLabel) + } -Context $Context | Out-Null + } catch { + Write-Host "⚠️ Failed to add label: $_" + } + + Write-Host "✅ Created PR #$($pr.number) - $($pr.html_url)" + $script:Summary.PRsCreated++ + } + + } finally { + Pop-Location + } + + } catch { + Write-Host "❌ $_" + $script:Summary.Errors += "$repoFullName : $_" + } finally { + if (Test-Path $clonePath) { + Remove-Item -Path $clonePath -Recurse -Force -ErrorAction SilentlyContinue + } + } + } +} + +#endregion + +#region Main Script + +try { + LogGroup '🔑 Authenticate' { + $context = Connect-GitHubApp -PassThru + $context | Format-List | Out-String + } + + LogGroup '📂 Discover file sets' { + $reposPath = Join-Path $PSScriptRoot '../Repos' + $reposPath = Resolve-Path $reposPath + $fileSets = Get-FileSets -ReposPath $reposPath + } + + if ($fileSets.Count -eq 0) { + Write-Host '⚠️ No file sets found - nothing to do' + exit 0 + } + + LogGroup '🎯 Read target organizations' { + $targetsPath = Join-Path $PSScriptRoot '../config/targets.json' + $targetsPath = Resolve-Path $targetsPath + $organizations = Get-TargetOrganization -ConfigPath $targetsPath + Write-Host "Target organizations: $($organizations -join ', ')" + } + + $subscribingRepos = @() + foreach ($org in $organizations) { + LogGroup "🔍 Find subscribing repositories in $org" { + $orgRepos = Get-SubscribingRepository -Owner $org -Context $context + Write-Host "Found $($orgRepos.Count) subscribing repositories in $org" + $subscribingRepos += $orgRepos + } + } + + if ($subscribingRepos.Count -eq 0) { + Write-Host '⚠️ No subscribing repositories found - nothing to do' + exit 0 + } + + # Sync files to each repository + $tempPath = Join-Path ([System.IO.Path]::GetTempPath()) "custo-sync-$(Get-Random)" + New-Item -Path $tempPath -ItemType Directory -Force | Out-Null + + $branchName = 'managed-files/update' + $commitMessage = 'chore: sync managed files' + $prTitle = '⚙️ [Maintenance]: Sync managed files' + $prLabel = 'NoRelease' + $prBody = @' +This pull request was automatically created by the [Custo](https://github.com/MSXOrg/Custo) workflow that keeps shared files in sync across the organization's repositories. + +The files in this PR are centrally managed. Any local changes to these files will be overwritten on the next sync. To propose changes, update the source files in the Custo repo instead. +'@ + + try { + foreach ($repo in $subscribingRepos) { + Sync-RepositoryFile -Repository $repo ` + -FileSets $fileSets ` + -TempPath $tempPath ` + -BranchName $branchName ` + -CommitMessage $commitMessage ` + -PRTitle $prTitle ` + -PRBody $prBody ` + -PRLabel $prLabel ` + -Context $context + } + } finally { + if (Test-Path $tempPath) { + Remove-Item -Path $tempPath -Recurse -Force -ErrorAction SilentlyContinue + } + } + + # Summary + Write-Host '' + Write-Host '📊 Summary' + Write-Host " Processed: $($script:Summary.TotalReposProcessed)" + Write-Host " Created: $($script:Summary.PRsCreated)" + Write-Host " Updated: $($script:Summary.PRsUpdated)" + Write-Host " In sync: $($script:Summary.ReposAlreadyInSync)" + Write-Host " Skipped: $($script:Summary.ReposSkipped)" + + if ($script:Summary.Errors.Count -gt 0) { + Write-Host " Errors: $($script:Summary.Errors.Count)" + foreach ($err in $script:Summary.Errors) { + Write-Host " ❌ $err" + } + } + +} catch { + Write-Host "❌ Fatal: $_" + Write-Host $_.ScriptStackTrace + exit 1 +} + +#endregion From ddb0ef1ec3855dcaa5790506c72c5407c248a541 Mon Sep 17 00:00:00 2001 From: Copilot <223556219+Copilot@users.noreply.github.com> Date: Sat, 25 Jul 2026 18:47:09 +0200 Subject: [PATCH 03/20] feat: add scheduled sync workflow Daily cron plus workflow_dispatch, mirroring PSModule/Distributor's GitHub App auth pattern via CUSTO_BOT_CLIENT_ID/PRIVATE_KEY secrets. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- .github/workflows/sync-files.yml | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) create mode 100644 .github/workflows/sync-files.yml diff --git a/.github/workflows/sync-files.yml b/.github/workflows/sync-files.yml new file mode 100644 index 0000000..a2a9955 --- /dev/null +++ b/.github/workflows/sync-files.yml @@ -0,0 +1,24 @@ +name: Sync Managed Files + +on: + schedule: + - cron: '0 6 * * *' # Daily at 06:00 UTC + workflow_dispatch: + +permissions: + contents: read + +jobs: + sync-files: + name: Sync files to subscribing repositories + runs-on: ubuntu-latest + steps: + - name: Checkout repository + uses: actions/checkout@eef61447b9ff4aafe5dcd4e0bbf5d482be7e7871 # v4.2.1 + + - name: Sync managed files + uses: PSModule/GitHub-Script@e3b0111c93df3686061cb2c65054f9216ed265e5 # main + with: + Script: ./scripts/Sync-Files.ps1 + ClientID: ${{ secrets.CUSTO_BOT_CLIENT_ID }} + PrivateKey: ${{ secrets.CUSTO_BOT_PRIVATE_KEY }} From 426e424a68056df0ad4ad04e8cb1c6198d4822f4 Mon Sep 17 00:00:00 2001 From: Copilot <223556219+Copilot@users.noreply.github.com> Date: Sat, 25 Jul 2026 18:47:21 +0200 Subject: [PATCH 04/20] feat: add MVP AGENTS.md file set for Module repositories First managed resource: a thin pointer AGENTS.md for PSModule module repos that references MSXOrg/docs and PSModule/docs instead of duplicating process text. Repositories opt in via Type=Module and SubscribeTo including 'AGENTS.md'. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- Repos/Module/AGENTS.md/AGENTS.md | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) create mode 100644 Repos/Module/AGENTS.md/AGENTS.md diff --git a/Repos/Module/AGENTS.md/AGENTS.md b/Repos/Module/AGENTS.md/AGENTS.md new file mode 100644 index 0000000..0b8c95a --- /dev/null +++ b/Repos/Module/AGENTS.md/AGENTS.md @@ -0,0 +1,22 @@ +# Agents + +## Main directive + +Everything here is a work in progress and can be improved. If you find a problem, fix it if it's +small; otherwise register it as an issue in this repo. + +## Ecosystem context + +Read [MSXOrg/docs AGENTS.md](https://github.com/MSXOrg/docs/blob/main/AGENTS.md) for the +ecosystem-wide onboarding steps shared by every MSX repository. + +Read [PSModule/docs](https://github.com/PSModule/docs) for how the PSModule initiative applies +those standards, starting with the +[PowerShell module standard](https://msxorg.github.io/docs/) and +[repository defaults](https://github.com/PSModule/docs/blob/main/src/docs/Modules/Repository-Defaults.md). + +## This file is managed + +This `AGENTS.md` is distributed by [`MSXOrg/Custo`](https://github.com/MSXOrg/Custo). Local edits +are overwritten on the next sync. To change its content, update the source file in Custo's +`Repos/Module/AGENTS.md/AGENTS.md` instead. From 0938def67478cbcd10273e0d9bd2c2b886741c82 Mon Sep 17 00:00:00 2001 From: Copilot <223556219+Copilot@users.noreply.github.com> Date: Sat, 25 Jul 2026 19:18:50 +0200 Subject: [PATCH 05/20] fix: use -ApiEndpoint for Invoke-GitHubAPI Update Sync-Files.ps1 to use the current PSModule/GitHub parameter name (-ApiEndpoint) for PR lookup, PR creation, and label assignment. Behavior is unchanged. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- scripts/Sync-Files.ps1 | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/scripts/Sync-Files.ps1 b/scripts/Sync-Files.ps1 index f73e7e1..545a777 100644 --- a/scripts/Sync-Files.ps1 +++ b/scripts/Sync-Files.ps1 @@ -309,7 +309,7 @@ function Sync-RepositoryFile { } # Create or update PR - $existingPRs = (Invoke-GitHubAPI -Method GET -Endpoint "/repos/$owner/$repoName/pulls" -Body @{ + $existingPRs = (Invoke-GitHubAPI -Method GET -ApiEndpoint "/repos/$owner/$repoName/pulls" -Body @{ head = "${owner}:${BranchName}" state = 'open' } -Context $Context).Response @@ -318,7 +318,7 @@ function Sync-RepositoryFile { Write-Host "✅ Updated PR #$($existingPRs[0].number) - $($existingPRs[0].html_url)" $script:Summary.PRsUpdated++ } else { - $pr = (Invoke-GitHubAPI -Method POST -Endpoint "/repos/$owner/$repoName/pulls" -Body @{ + $pr = (Invoke-GitHubAPI -Method POST -ApiEndpoint "/repos/$owner/$repoName/pulls" -Body @{ title = $PRTitle head = $BranchName base = $Repository.DefaultBranch @@ -326,7 +326,7 @@ function Sync-RepositoryFile { } -Context $Context).Response try { - Invoke-GitHubAPI -Method POST -Endpoint "/repos/$owner/$repoName/issues/$($pr.number)/labels" -Body @{ + Invoke-GitHubAPI -Method POST -ApiEndpoint "/repos/$owner/$repoName/issues/$($pr.number)/labels" -Body @{ labels = @($PRLabel) } -Context $Context | Out-Null } catch { From 8f23d8e4e659248f28fe29492094dd2637e781d9 Mon Sep 17 00:00:00 2001 From: Copilot <223556219+Copilot@users.noreply.github.com> Date: Sat, 25 Jul 2026 20:33:35 +0200 Subject: [PATCH 06/20] feat: discover subscribing repos across all accessible orgs Switch sync target resolution to scope-based discovery. Add all-access mode (default) that enumerates all repositories visible to the app installation token and reads Type/SubscribeTo custom properties per repository. Keep organizations mode available for explicit filtering. Update targets schema/json and README accordingly. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- README.md | 23 +++--- config/targets.json | 4 +- config/targets.schema.json | 29 ++++++-- scripts/Sync-Files.ps1 | 139 +++++++++++++++++++++++++++++++------ 4 files changed, 156 insertions(+), 39 deletions(-) diff --git a/README.md b/README.md index e508c36..66d521a 100644 --- a/README.md +++ b/README.md @@ -35,29 +35,30 @@ Repos/{Type}/{Selection}/ - **Selection** — an individual file set repositories opt into via the `SubscribeTo` custom property. Each selection folder mirrors the root of a target repository. -Target organizations are declared in [`config/targets.json`](config/targets.json), not hardcoded -in the sync script. This keeps Custo org-agnostic: adding a new initiative organization is a -config change, not a code change. +Target discovery scope is declared in [`config/targets.json`](config/targets.json), not hardcoded +in the sync script. The default scope is `all-access`, which scans all repositories visible to the +current GitHub App installation token. This keeps Custo org-agnostic and lets one runtime process +all organizations and repositories it can access. The [`scripts/Sync-Files.ps1`](scripts/Sync-Files.ps1) script, run by the [`Sync Managed Files`](.github/workflows/sync-files.yml) workflow: -1. Reads the target organizations from `config/targets.json`. +1. Reads repository discovery scope from `config/targets.json`. 2. Discovers file sets under `Repos/`. -3. For each target org, queries repositories for their `Type` and `SubscribeTo` custom - properties. +3. Discovers subscribing repositories from all accessible repositories (or explicit organizations + when configured) and reads their `Type` and `SubscribeTo` custom properties. 4. Clones subscribing repositories, copies the relevant files, and opens or updates a `managed-files/update` pull request when changes are detected. ## MVP rollout scope The first managed resource is **`AGENTS.md`** for PowerShell module repositories -(`Repos/Module/AGENTS.md/AGENTS.md`), targeting the `PSModule` organization by default. The file -is a thin pointer into the central docs rather than a duplicated process document, matching the -pattern already used by [`PSModule/Template-PSModule`](https://github.com/PSModule/Template-PSModule) -and [`PSModule/memory`](https://github.com/PSModule/memory). +(`Repos/Module/AGENTS.md/AGENTS.md`). The file is a thin pointer into the central docs rather than +a duplicated process document, matching the pattern already used by +[`PSModule/Template-PSModule`](https://github.com/PSModule/Template-PSModule) and +[`PSModule/memory`](https://github.com/PSModule/memory). -Additional file sets and target organizations are added incrementally after this MVP is proven. +Additional file sets and rollout targets are added incrementally after this MVP is proven. ## Required secrets diff --git a/config/targets.json b/config/targets.json index b3480b7..820aefd 100644 --- a/config/targets.json +++ b/config/targets.json @@ -1,6 +1,4 @@ { "$schema": "./targets.schema.json", - "organizations": [ - "PSModule" - ] + "scope": "all-access" } diff --git a/config/targets.schema.json b/config/targets.schema.json index 01e58dd..1e011a2 100644 --- a/config/targets.schema.json +++ b/config/targets.schema.json @@ -1,20 +1,39 @@ { "$schema": "https://json-schema.org/draft/2020-12/schema", "title": "Custo target organizations", - "description": "Organizations that the Sync Managed Files workflow discovers subscribing repositories in.", + "description": "Repository discovery scope for the Sync Managed Files workflow.", "type": "object", - "required": ["organizations"], + "required": ["scope"], "properties": { + "scope": { + "type": "string", + "enum": ["all-access", "organizations"], + "description": "all-access discovers all repositories visible to the current app installation token; organizations restricts discovery to listed organizations." + }, "organizations": { "type": "array", - "description": "GitHub organization logins to scan for subscribing repositories.", + "description": "GitHub organization logins to scan when scope is organizations.", "items": { "type": "string", "minLength": 1 }, - "minItems": 1, "uniqueItems": true } - }, + }, + "allOf": [ + { + "if": { + "properties": { "scope": { "const": "organizations" } } + }, + "then": { + "required": ["organizations"], + "properties": { + "organizations": { + "minItems": 1 + } + } + } + } + ], "additionalProperties": false } diff --git a/scripts/Sync-Files.ps1 b/scripts/Sync-Files.ps1 index 545a777..2d32938 100644 --- a/scripts/Sync-Files.ps1 +++ b/scripts/Sync-Files.ps1 @@ -2,15 +2,16 @@ <# .SYNOPSIS Syncs managed files from this repository to subscribing repositories across configured - target organizations. + discovery scope. .DESCRIPTION This script: 1. Authenticates as a GitHub App for repo-level operations. 2. Discovers available file sets from the Repos/ directory structure. - 3. Reads the target organizations from config/targets.json. - 4. For each target organization, queries repositories for their Type and SubscribeTo - custom properties. + 3. Reads target discovery mode from config/targets.json. + 4. Discovers subscribing repositories from either: + - all repositories visible to the current installation token, or + - explicit organizations from config. 5. For each subscribing repository: - Clones the repository - Copies managed files from the appropriate file sets @@ -20,8 +21,7 @@ .NOTES Requires the GitHub PowerShell module and GitHub App authentication via GitHub-Script action. - Target organizations are configuration, not code - see config/targets.json. This keeps the - script usable by any MSX initiative, not just PSModule. + Target discovery scope is configuration, not code - see config/targets.json. #> [CmdletBinding()] @@ -41,10 +41,10 @@ $script:Summary = @{ #region Helper Functions -function Get-TargetOrganization { +function Get-TargetScope { <# .SYNOPSIS - Reads the target organization list from config/targets.json. + Reads repository discovery scope from config/targets.json. #> [CmdletBinding()] param( @@ -58,11 +58,19 @@ function Get-TargetOrganization { $config = Get-Content -Path $ConfigPath -Raw | ConvertFrom-Json - if (-not $config.organizations -or $config.organizations.Count -eq 0) { - throw "No organizations configured in: $ConfigPath" + if (-not $config.scope) { + throw "Missing required 'scope' in: $ConfigPath" } - return @($config.organizations) + if ($config.scope -eq 'organizations') { + if (-not $config.organizations -or $config.organizations.Count -eq 0) { + throw "Scope 'organizations' requires a non-empty organizations list in: $ConfigPath" + } + } elseif ($config.scope -ne 'all-access') { + throw "Unsupported scope '$($config.scope)' in: $ConfigPath" + } + + return $config } function Get-FileSets { @@ -180,6 +188,89 @@ function Get-SubscribingRepository { return $subscribingRepos } +function Get-SubscribingRepositoryFromAllAccess { + <# + .SYNOPSIS + Discovers subscribing repositories from all repositories visible to the + current installation token. + #> + [CmdletBinding()] + param( + [Parameter(Mandatory)] + [object]$Context + ) + + $allRepos = @() + $page = 1 + $perPage = 100 + + while ($true) { + $response = (Invoke-GitHubAPI -Method GET -ApiEndpoint '/installation/repositories' -Body @{ + per_page = $perPage + page = $page + } -Context $Context).Response + + if (-not $response.repositories -or $response.repositories.Count -eq 0) { + break + } + + $allRepos += $response.repositories + + if ($response.repositories.Count -lt $perPage) { + break + } + + $page++ + } + + $subscribingRepos = @() + + foreach ($repo in $allRepos) { + $owner = $repo.owner.login + $repoName = $repo.name + + $customProps = (Invoke-GitHubAPI -Method GET -ApiEndpoint "/repos/$owner/$repoName/properties/values" -Context $Context).Response + + $typeProp = $customProps | Where-Object { $_.property_name -eq 'Type' } + $subscribeToProp = $customProps | Where-Object { $_.property_name -eq 'SubscribeTo' } + + $type = $typeProp.value + $subscribeTo = $subscribeToProp.value + + if (-not $type -or -not $subscribeTo) { + continue + } + + if ($subscribeTo -is [string]) { + $subscribeTo = @($subscribeTo) + } + + if ($subscribeTo.Count -eq 0) { + continue + } + + $subscribingRepos += @{ + Name = $repoName + Owner = $owner + FullName = $repo.full_name + Type = $type + SubscribeTo = $subscribeTo + DefaultBranch = $repo.default_branch + } + } + + $subscribingRepos | ForEach-Object { + [PSCustomObject]@{ + Owner = $_.Owner + Repo = $_.Name + Type = $_.Type + SubscribeTo = $_.SubscribeTo -join ', ' + } + } | Format-Table -AutoSize | Out-String + + return $subscribingRepos +} + function Sync-RepositoryFile { <# .SYNOPSIS @@ -373,19 +464,27 @@ try { exit 0 } - LogGroup '🎯 Read target organizations' { + LogGroup '🎯 Read target discovery scope' { $targetsPath = Join-Path $PSScriptRoot '../config/targets.json' $targetsPath = Resolve-Path $targetsPath - $organizations = Get-TargetOrganization -ConfigPath $targetsPath - Write-Host "Target organizations: $($organizations -join ', ')" + $targetScope = Get-TargetScope -ConfigPath $targetsPath + Write-Host "Target scope: $($targetScope.scope)" + if ($targetScope.scope -eq 'organizations') { + Write-Host "Target organizations: $($targetScope.organizations -join ', ')" + } } - $subscribingRepos = @() - foreach ($org in $organizations) { - LogGroup "🔍 Find subscribing repositories in $org" { - $orgRepos = Get-SubscribingRepository -Owner $org -Context $context - Write-Host "Found $($orgRepos.Count) subscribing repositories in $org" - $subscribingRepos += $orgRepos + LogGroup '🔍 Find subscribing repositories' { + if ($targetScope.scope -eq 'all-access') { + $subscribingRepos = Get-SubscribingRepositoryFromAllAccess -Context $context + Write-Host "Found $($subscribingRepos.Count) subscribing repositories across all accessible repositories" + } else { + $subscribingRepos = @() + foreach ($org in $targetScope.organizations) { + $orgRepos = Get-SubscribingRepository -Owner $org -Context $context + Write-Host "Found $($orgRepos.Count) subscribing repositories in $org" + $subscribingRepos += $orgRepos + } } } From eee57a60658774bc4b8065fb6109b1a783b373cb Mon Sep 17 00:00:00 2001 From: Copilot <223556219+Copilot@users.noreply.github.com> Date: Sat, 25 Jul 2026 20:35:42 +0200 Subject: [PATCH 07/20] feat: sync enterprise custom-property schema from file sets Add enterprise-level custom-property management for Type and SubscribeTo. Allowed values are now generated from the Repos tree and upserted via enterprise properties/schema endpoints before repository sync starts. Make this behavior config-driven in targets.json and document required app permissions. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- AGENTS.md | 5 ++- README.md | 18 +++++--- config/targets.json | 9 +++- config/targets.schema.json | 53 ++++++++++++++++++++++ scripts/Sync-Files.ps1 | 92 +++++++++++++++++++++++++++++++++++++- 5 files changed, 169 insertions(+), 8 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 1d89497..5629c8a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -24,12 +24,15 @@ cannot run against real repositories until an operator completes these steps: (app id `1320343`, slug `psmodule-s-custo`) already installed on the `PSModule` organization. If that app's credentials are not available, generate a new private key for it or provision an equivalent app with `contents:write`, `pull_requests:write`, and - `repository_custom_properties:read` on target repositories. + `repository_custom_properties:read` on target repositories, plus enterprise custom-properties + write/admin permission if enterprise schema sync is enabled. 2. **Confirm app installation scope.** Verify the app is installed on the `PSModule` organization with access to the module repositories that should receive `AGENTS.md` (`gh api orgs/PSModule/installations`). 3. **Set custom properties on target repositories.** Each subscribing module repository needs `Type = Module` and `SubscribeTo` including `AGENTS.md` set at the repository level. + Custo now maintains allowed values at enterprise level from `Repos/` file sets when + `customProperties.enabled=true` in `config/targets.json`. 4. **Dry-run via `workflow_dispatch`.** Trigger the `Sync Managed Files` workflow manually first and review the summary before relying on the daily schedule. diff --git a/README.md b/README.md index 66d521a..778cd0a 100644 --- a/README.md +++ b/README.md @@ -45,9 +45,11 @@ The [`scripts/Sync-Files.ps1`](scripts/Sync-Files.ps1) script, run by the 1. Reads repository discovery scope from `config/targets.json`. 2. Discovers file sets under `Repos/`. -3. Discovers subscribing repositories from all accessible repositories (or explicit organizations +3. Syncs enterprise-level custom-property schema definitions (`Type` and `SubscribeTo`) and + allowed values from the discovered file-set tree when `customProperties.enabled=true`. +4. Discovers subscribing repositories from all accessible repositories (or explicit organizations when configured) and reads their `Type` and `SubscribeTo` custom properties. -4. Clones subscribing repositories, copies the relevant files, and opens or updates a +5. Clones subscribing repositories, copies the relevant files, and opens or updates a `managed-files/update` pull request when changes are detected. ## MVP rollout scope @@ -62,9 +64,15 @@ Additional file sets and rollout targets are added incrementally after this MVP ## Required secrets -The sync workflow authenticates as a GitHub App with `contents`, `pull_requests`, and -`repository_custom_properties` access on target repositories. Configure these repository secrets -before enabling the scheduled sync: +The sync workflow authenticates as a GitHub App. For repository sync plus enterprise custom +property maintenance, the app needs: + +- `contents:write` +- `pull_requests:write` +- `repository_custom_properties:read` +- enterprise custom-properties write/admin access (for `/enterprises/{enterprise}/properties/schema`) + +Configure these repository secrets before enabling the scheduled sync: - `CUSTO_BOT_CLIENT_ID` - `CUSTO_BOT_PRIVATE_KEY` diff --git a/config/targets.json b/config/targets.json index 820aefd..0ff49bf 100644 --- a/config/targets.json +++ b/config/targets.json @@ -1,4 +1,11 @@ { "$schema": "./targets.schema.json", - "scope": "all-access" + "scope": "all-access", + "customProperties": { + "enabled": true, + "scope": "enterprise", + "enterprise": "MSXOrg", + "typePropertyName": "Type", + "subscriptionPropertyName": "SubscribeTo" + } } diff --git a/config/targets.schema.json b/config/targets.schema.json index 1e011a2..9d43b70 100644 --- a/config/targets.schema.json +++ b/config/targets.schema.json @@ -18,6 +18,39 @@ "minLength": 1 }, "uniqueItems": true + }, + "customProperties": { + "type": "object", + "description": "Controls automatic custom-property schema synchronization.", + "properties": { + "enabled": { + "type": "boolean", + "description": "When true, Custo synchronizes custom-property definitions before repository sync.", + "default": false + }, + "scope": { + "type": "string", + "enum": ["enterprise"], + "description": "Where to maintain custom-property schema definitions." + }, + "enterprise": { + "type": "string", + "minLength": 1, + "description": "Enterprise slug used for enterprise-level custom-property schema." + }, + "typePropertyName": { + "type": "string", + "minLength": 1, + "description": "Custom-property name for repository type. Defaults to Type." + }, + "subscriptionPropertyName": { + "type": "string", + "minLength": 1, + "description": "Custom-property name for subscribed file sets. Defaults to SubscribeTo." + } + }, + "required": ["enabled"], + "additionalProperties": false } }, "allOf": [ @@ -33,6 +66,26 @@ } } } + }, + { + "if": { + "properties": { + "customProperties": { + "properties": { "enabled": { "const": true } }, + "required": ["enabled"] + } + } + }, + "then": { + "properties": { + "customProperties": { + "required": ["scope", "enterprise"], + "properties": { + "scope": { "const": "enterprise" } + } + } + } + } } ], "additionalProperties": false diff --git a/scripts/Sync-Files.ps1 b/scripts/Sync-Files.ps1 index 2d32938..1ce2ae4 100644 --- a/scripts/Sync-Files.ps1 +++ b/scripts/Sync-Files.ps1 @@ -8,7 +8,8 @@ This script: 1. Authenticates as a GitHub App for repo-level operations. 2. Discovers available file sets from the Repos/ directory structure. - 3. Reads target discovery mode from config/targets.json. + 3. Syncs custom-property schema definitions (Type/SubscribeTo) when configured. + 4. Reads target discovery mode from config/targets.json. 4. Discovers subscribing repositories from either: - all repositories visible to the current installation token, or - explicit organizations from config. @@ -126,6 +127,63 @@ function Get-FileSets { return $fileSets } +function Sync-EnterpriseCustomPropertySchema { + <# + .SYNOPSIS + Ensures enterprise-level custom property definitions exist with allowed values + derived from the Repos/ file-set tree. + #> + [CmdletBinding()] + param( + [Parameter(Mandatory)] + [string]$Enterprise, + + [Parameter(Mandatory)] + [hashtable]$FileSets, + + [Parameter(Mandatory)] + [object]$Context, + + [string]$TypePropertyName = 'Type', + [string]$SubscriptionPropertyName = 'SubscribeTo' + ) + + $typeValues = @($FileSets.Keys | Sort-Object -Unique) + if ($typeValues.Count -eq 0) { + throw 'Cannot sync custom-property schema: no repository types discovered in Repos/.' + } + + $subscriptionValues = @( + $FileSets.Values | + ForEach-Object { $_.Keys } | + Sort-Object -Unique + ) + + if ($subscriptionValues.Count -eq 0) { + throw 'Cannot sync custom-property schema: no file-set selections discovered in Repos/.' + } + + Invoke-GitHubAPI -Method PUT -ApiEndpoint "/enterprises/$Enterprise/properties/schema/$TypePropertyName" -Body @{ + value_type = 'single_select' + required = $false + default_value = $null + description = 'Repository type used by Custo managed-file distribution.' + allowed_values = $typeValues + } -Context $Context | Out-Null + + Invoke-GitHubAPI -Method PUT -ApiEndpoint "/enterprises/$Enterprise/properties/schema/$SubscriptionPropertyName" -Body @{ + value_type = 'multi_select' + required = $false + default_value = $null + description = 'Managed file sets the repository subscribes to from Custo.' + allowed_values = $subscriptionValues + } -Context $Context | Out-Null + + Write-Host "✅ Synced enterprise custom-property schema on '$Enterprise'" + Write-Host " - ${TypePropertyName}: $($typeValues -join ', ')" + Write-Host " - ${SubscriptionPropertyName}: $($subscriptionValues -join ', ')" +} + function Get-SubscribingRepository { <# .SYNOPSIS @@ -474,6 +532,38 @@ try { } } + if ($targetScope.customProperties.enabled -eq $true) { + LogGroup '🧭 Sync custom-property schema' { + if ($targetScope.customProperties.scope -ne 'enterprise') { + throw "Unsupported customProperties.scope '$($targetScope.customProperties.scope)'." + } + if (-not $targetScope.customProperties.enterprise) { + throw "customProperties.enterprise is required when customProperties.enabled is true." + } + + $typePropertyName = if ($targetScope.customProperties.typePropertyName) { + $targetScope.customProperties.typePropertyName + } else { + 'Type' + } + + $subscriptionPropertyName = if ($targetScope.customProperties.subscriptionPropertyName) { + $targetScope.customProperties.subscriptionPropertyName + } else { + 'SubscribeTo' + } + + Sync-EnterpriseCustomPropertySchema ` + -Enterprise $targetScope.customProperties.enterprise ` + -FileSets $fileSets ` + -Context $context ` + -TypePropertyName $typePropertyName ` + -SubscriptionPropertyName $subscriptionPropertyName + } + } else { + Write-Host 'ℹ️ customProperties sync disabled by config' + } + LogGroup '🔍 Find subscribing repositories' { if ($targetScope.scope -eq 'all-access') { $subscribingRepos = Get-SubscribingRepositoryFromAllAccess -Context $context From 3ecaa2d26e6ac54ec1a8ead555f85d650bbe96ba Mon Sep 17 00:00:00 2001 From: Copilot <223556219+Copilot@users.noreply.github.com> Date: Sat, 25 Jul 2026 20:40:47 +0200 Subject: [PATCH 08/20] refactor: make sync pipeline policy-first by enterprise then org Introduce an explicit policy engine phase that runs enterprise controls before repository sync. In all-access mode, discovery is now grouped by organization to enforce enterprise -> organization -> repositories flow. Update README pipeline wording to match the new order. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- README.md | 7 +- scripts/Sync-Files.ps1 | 178 ++++++++++++++++++++++++++++------------- 2 files changed, 125 insertions(+), 60 deletions(-) diff --git a/README.md b/README.md index 778cd0a..a1ddb07 100644 --- a/README.md +++ b/README.md @@ -45,11 +45,12 @@ The [`scripts/Sync-Files.ps1`](scripts/Sync-Files.ps1) script, run by the 1. Reads repository discovery scope from `config/targets.json`. 2. Discovers file sets under `Repos/`. -3. Syncs enterprise-level custom-property schema definitions (`Type` and `SubscribeTo`) and +3. Applies policy controls in order: **Enterprise** first, then **Organizations and their repositories**. +4. Syncs enterprise-level custom-property schema definitions (`Type` and `SubscribeTo`) and allowed values from the discovered file-set tree when `customProperties.enabled=true`. -4. Discovers subscribing repositories from all accessible repositories (or explicit organizations +5. Discovers subscribing repositories from all accessible repositories (or explicit organizations when configured) and reads their `Type` and `SubscribeTo` custom properties. -5. Clones subscribing repositories, copies the relevant files, and opens or updates a +6. Clones subscribing repositories, copies the relevant files, and opens or updates a `managed-files/update` pull request when changes are detected. ## MVP rollout scope diff --git a/scripts/Sync-Files.ps1 b/scripts/Sync-Files.ps1 index 1ce2ae4..113849f 100644 --- a/scripts/Sync-Files.ps1 +++ b/scripts/Sync-Files.ps1 @@ -8,17 +8,19 @@ This script: 1. Authenticates as a GitHub App for repo-level operations. 2. Discovers available file sets from the Repos/ directory structure. - 3. Syncs custom-property schema definitions (Type/SubscribeTo) when configured. - 4. Reads target discovery mode from config/targets.json. - 4. Discovers subscribing repositories from either: + 3. Reads target discovery mode from config/targets.json. + 4. Applies policy controls in order: + - enterprise custom-property schema + - organization and repository targeting + 5. Discovers subscribing repositories from either: - all repositories visible to the current installation token, or - explicit organizations from config. - 5. For each subscribing repository: + 6. For each subscribing repository: - Clones the repository - Copies managed files from the appropriate file sets - Detects changes using git - Creates or updates a pull request if changes are detected - 6. Outputs a summary of actions taken. + 7. Outputs a summary of actions taken. .NOTES Requires the GitHub PowerShell module and GitHub App authentication via GitHub-Script action. @@ -246,11 +248,11 @@ function Get-SubscribingRepository { return $subscribingRepos } -function Get-SubscribingRepositoryFromAllAccess { +function Get-SubscribingRepositoryByOrganizationFromAllAccess { <# .SYNOPSIS Discovers subscribing repositories from all repositories visible to the - current installation token. + current installation token, grouped by owning organization. #> [CmdletBinding()] param( @@ -281,9 +283,13 @@ function Get-SubscribingRepositoryFromAllAccess { $page++ } - $subscribingRepos = @() + $reposByOrg = @{} foreach ($repo in $allRepos) { + if ($repo.owner.type -ne 'Organization') { + continue + } + $owner = $repo.owner.login $repoName = $repo.name @@ -307,7 +313,7 @@ function Get-SubscribingRepositoryFromAllAccess { continue } - $subscribingRepos += @{ + $repoEntry = @{ Name = $repoName Owner = $owner FullName = $repo.full_name @@ -315,17 +321,114 @@ function Get-SubscribingRepositoryFromAllAccess { SubscribeTo = $subscribeTo DefaultBranch = $repo.default_branch } + + if (-not $reposByOrg.ContainsKey($owner)) { + $reposByOrg[$owner] = @() + } + + $reposByOrg[$owner] += $repoEntry } - $subscribingRepos | ForEach-Object { - [PSCustomObject]@{ - Owner = $_.Owner - Repo = $_.Name - Type = $_.Type - SubscribeTo = $_.SubscribeTo -join ', ' + $reposByOrg.GetEnumerator() | ForEach-Object { + $org = $_.Key + $_.Value | ForEach-Object { + [PSCustomObject]@{ + Owner = $org + Repo = $_.Name + Type = $_.Type + SubscribeTo = $_.SubscribeTo -join ', ' + } } } | Format-Table -AutoSize | Out-String + return $reposByOrg +} + +function Invoke-PolicyEngine { + <# + .SYNOPSIS + Applies top-level policy controls before repository sync. + .DESCRIPTION + Current policy order: + 1) Enterprise policy layer + 2) Organization and repository synchronization + #> + [CmdletBinding()] + param( + [Parameter(Mandatory)] + [pscustomobject]$TargetScope, + + [Parameter(Mandatory)] + [hashtable]$FileSets, + + [Parameter(Mandatory)] + [object]$Context + ) + + LogGroup '🧭 Policy engine: enterprise controls first' { + if ($TargetScope.customProperties.enabled -eq $true) { + if ($TargetScope.customProperties.scope -ne 'enterprise') { + throw "Unsupported customProperties.scope '$($TargetScope.customProperties.scope)'." + } + if (-not $TargetScope.customProperties.enterprise) { + throw "customProperties.enterprise is required when customProperties.enabled is true." + } + + $typePropertyName = if ($TargetScope.customProperties.typePropertyName) { + $TargetScope.customProperties.typePropertyName + } else { + 'Type' + } + + $subscriptionPropertyName = if ($TargetScope.customProperties.subscriptionPropertyName) { + $TargetScope.customProperties.subscriptionPropertyName + } else { + 'SubscribeTo' + } + + Sync-EnterpriseCustomPropertySchema ` + -Enterprise $TargetScope.customProperties.enterprise ` + -FileSets $FileSets ` + -Context $Context ` + -TypePropertyName $typePropertyName ` + -SubscriptionPropertyName $subscriptionPropertyName + } else { + Write-Host 'ℹ️ customProperties sync disabled by config' + } + } +} + +function Get-AllSubscribingRepository { + <# + .SYNOPSIS + Resolves subscribing repositories after policy has been applied. + #> + [CmdletBinding()] + param( + [Parameter(Mandatory)] + [pscustomobject]$TargetScope, + + [Parameter(Mandatory)] + [object]$Context + ) + + $subscribingRepos = @() + + if ($TargetScope.scope -eq 'all-access') { + $reposByOrg = Get-SubscribingRepositoryByOrganizationFromAllAccess -Context $Context + + foreach ($org in @($reposByOrg.Keys | Sort-Object)) { + Write-Host "Found $($reposByOrg[$org].Count) subscribing repositories in $org" + $subscribingRepos += $reposByOrg[$org] + } + } else { + foreach ($org in $TargetScope.organizations) { + $orgRepos = Get-SubscribingRepository -Owner $org -Context $Context + Write-Host "Found $($orgRepos.Count) subscribing repositories in $org" + $subscribingRepos += $orgRepos + } + } + return $subscribingRepos } @@ -532,50 +635,11 @@ try { } } - if ($targetScope.customProperties.enabled -eq $true) { - LogGroup '🧭 Sync custom-property schema' { - if ($targetScope.customProperties.scope -ne 'enterprise') { - throw "Unsupported customProperties.scope '$($targetScope.customProperties.scope)'." - } - if (-not $targetScope.customProperties.enterprise) { - throw "customProperties.enterprise is required when customProperties.enabled is true." - } - - $typePropertyName = if ($targetScope.customProperties.typePropertyName) { - $targetScope.customProperties.typePropertyName - } else { - 'Type' - } - - $subscriptionPropertyName = if ($targetScope.customProperties.subscriptionPropertyName) { - $targetScope.customProperties.subscriptionPropertyName - } else { - 'SubscribeTo' - } - - Sync-EnterpriseCustomPropertySchema ` - -Enterprise $targetScope.customProperties.enterprise ` - -FileSets $fileSets ` - -Context $context ` - -TypePropertyName $typePropertyName ` - -SubscriptionPropertyName $subscriptionPropertyName - } - } else { - Write-Host 'ℹ️ customProperties sync disabled by config' - } + Invoke-PolicyEngine -TargetScope $targetScope -FileSets $fileSets -Context $context LogGroup '🔍 Find subscribing repositories' { - if ($targetScope.scope -eq 'all-access') { - $subscribingRepos = Get-SubscribingRepositoryFromAllAccess -Context $context - Write-Host "Found $($subscribingRepos.Count) subscribing repositories across all accessible repositories" - } else { - $subscribingRepos = @() - foreach ($org in $targetScope.organizations) { - $orgRepos = Get-SubscribingRepository -Owner $org -Context $context - Write-Host "Found $($orgRepos.Count) subscribing repositories in $org" - $subscribingRepos += $orgRepos - } - } + $subscribingRepos = Get-AllSubscribingRepository -TargetScope $targetScope -Context $context + Write-Host "Found $($subscribingRepos.Count) subscribing repositories in total" } if ($subscribingRepos.Count -eq 0) { From b7e3f06d1a45434f04a67f3549cf3d8d5d3e85bb Mon Sep 17 00:00:00 2001 From: Copilot <223556219+Copilot@users.noreply.github.com> Date: Sat, 25 Jul 2026 20:54:57 +0200 Subject: [PATCH 09/20] feat: support enterprise policy APIs via CUSTO_ENTERPRISE_PAT Add an enterprise-policy API helper that uses CUSTO_ENTERPRISE_PAT when present and falls back to GitHub App context otherwise. Wire enterprise custom-property sync to this helper, pass the optional secret into the workflow environment, and document the new secret. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- .github/workflows/sync-files.yml | 2 + AGENTS.md | 2 + README.md | 1 + scripts/Sync-Files.ps1 | 120 ++++++++++++++++++++++++------- 4 files changed, 100 insertions(+), 25 deletions(-) diff --git a/.github/workflows/sync-files.yml b/.github/workflows/sync-files.yml index a2a9955..c531a8f 100644 --- a/.github/workflows/sync-files.yml +++ b/.github/workflows/sync-files.yml @@ -18,6 +18,8 @@ jobs: - name: Sync managed files uses: PSModule/GitHub-Script@e3b0111c93df3686061cb2c65054f9216ed265e5 # main + env: + CUSTO_ENTERPRISE_PAT: ${{ secrets.CUSTO_ENTERPRISE_PAT }} with: Script: ./scripts/Sync-Files.ps1 ClientID: ${{ secrets.CUSTO_BOT_CLIENT_ID }} diff --git a/AGENTS.md b/AGENTS.md index 5629c8a..7e1bf39 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -26,6 +26,8 @@ cannot run against real repositories until an operator completes these steps: equivalent app with `contents:write`, `pull_requests:write`, and `repository_custom_properties:read` on target repositories, plus enterprise custom-properties write/admin permission if enterprise schema sync is enabled. + If enterprise rulesets/custom-properties must be managed via PAT, also add + `CUSTO_ENTERPRISE_PAT` and grant it enterprise-admin scope. 2. **Confirm app installation scope.** Verify the app is installed on the `PSModule` organization with access to the module repositories that should receive `AGENTS.md` (`gh api orgs/PSModule/installations`). diff --git a/README.md b/README.md index a1ddb07..d652cd5 100644 --- a/README.md +++ b/README.md @@ -77,6 +77,7 @@ Configure these repository secrets before enabling the scheduled sync: - `CUSTO_BOT_CLIENT_ID` - `CUSTO_BOT_PRIVATE_KEY` +- `CUSTO_ENTERPRISE_PAT` (optional, recommended for enterprise policy endpoints like enterprise rulesets/custom-properties when GitHub App enterprise permissions are unavailable) See [`AGENTS.md`](AGENTS.md) for operator runbook steps and current rollout blockers. diff --git a/scripts/Sync-Files.ps1 b/scripts/Sync-Files.ps1 index 113849f..ed37a37 100644 --- a/scripts/Sync-Files.ps1 +++ b/scripts/Sync-Files.ps1 @@ -129,6 +129,89 @@ function Get-FileSets { return $fileSets } +function Get-AllAccessibleRepository { + <# + .SYNOPSIS + Lists all repositories visible to the current installation token. + #> + [CmdletBinding()] + param( + [Parameter(Mandatory)] + [object]$Context + ) + + $allRepos = @() + $page = 1 + $perPage = 100 + + while ($true) { + $response = (Invoke-GitHubAPI -Method GET -ApiEndpoint '/installation/repositories' -Body @{ + per_page = $perPage + page = $page + } -Context $Context).Response + + if (-not $response.repositories -or $response.repositories.Count -eq 0) { + break + } + + $allRepos += $response.repositories + + if ($response.repositories.Count -lt $perPage) { + break + } + + $page++ + } + + return $allRepos +} + +function Invoke-EnterprisePolicyApi { + <# + .SYNOPSIS + Calls enterprise policy endpoints with PAT when available, otherwise with app context. + #> + [CmdletBinding()] + param( + [Parameter(Mandatory)] + [ValidateSet('GET', 'POST', 'PUT', 'PATCH', 'DELETE')] + [string]$Method, + + [Parameter(Mandatory)] + [string]$ApiEndpoint, + + [hashtable]$Body, + + [Parameter(Mandatory)] + [object]$Context + ) + + $enterprisePat = $env:CUSTO_ENTERPRISE_PAT + if (-not [string]::IsNullOrWhiteSpace($enterprisePat)) { + $headers = @{ + Authorization = "Bearer $enterprisePat" + Accept = 'application/vnd.github+json' + 'X-GitHub-Api-Version' = '2022-11-28' + } + + $uri = "https://api.github.com$ApiEndpoint" + $invokeArgs = @{ + Method = $Method + Uri = $uri + Headers = $headers + } + + if ($Body) { + $invokeArgs.ContentType = 'application/json' + $invokeArgs.Body = $Body | ConvertTo-Json -Depth 20 + } + + return Invoke-RestMethod @invokeArgs + } + + return (Invoke-GitHubAPI -Method $Method -ApiEndpoint $ApiEndpoint -Body $Body -Context $Context).Response +} + function Sync-EnterpriseCustomPropertySchema { <# .SYNOPSIS @@ -165,7 +248,7 @@ function Sync-EnterpriseCustomPropertySchema { throw 'Cannot sync custom-property schema: no file-set selections discovered in Repos/.' } - Invoke-GitHubAPI -Method PUT -ApiEndpoint "/enterprises/$Enterprise/properties/schema/$TypePropertyName" -Body @{ + Invoke-EnterprisePolicyApi -Method PUT -ApiEndpoint "/enterprises/$Enterprise/properties/schema/$TypePropertyName" -Body @{ value_type = 'single_select' required = $false default_value = $null @@ -173,7 +256,7 @@ function Sync-EnterpriseCustomPropertySchema { allowed_values = $typeValues } -Context $Context | Out-Null - Invoke-GitHubAPI -Method PUT -ApiEndpoint "/enterprises/$Enterprise/properties/schema/$SubscriptionPropertyName" -Body @{ + Invoke-EnterprisePolicyApi -Method PUT -ApiEndpoint "/enterprises/$Enterprise/properties/schema/$SubscriptionPropertyName" -Body @{ value_type = 'multi_select' required = $false default_value = $null @@ -257,31 +340,12 @@ function Get-SubscribingRepositoryByOrganizationFromAllAccess { [CmdletBinding()] param( [Parameter(Mandatory)] - [object]$Context - ) - - $allRepos = @() - $page = 1 - $perPage = 100 - - while ($true) { - $response = (Invoke-GitHubAPI -Method GET -ApiEndpoint '/installation/repositories' -Body @{ - per_page = $perPage - page = $page - } -Context $Context).Response - - if (-not $response.repositories -or $response.repositories.Count -eq 0) { - break - } - - $allRepos += $response.repositories + [object]$Context, - if ($response.repositories.Count -lt $perPage) { - break - } + [object[]]$Repositories + ) - $page++ - } + $allRepos = if ($Repositories) { $Repositories } else { Get-AllAccessibleRepository -Context $Context } $reposByOrg = @{} @@ -366,6 +430,12 @@ function Invoke-PolicyEngine { ) LogGroup '🧭 Policy engine: enterprise controls first' { + if (-not [string]::IsNullOrWhiteSpace($env:CUSTO_ENTERPRISE_PAT)) { + Write-Host 'Using CUSTO_ENTERPRISE_PAT for enterprise policy API calls' + } else { + Write-Host 'Using GitHub App token for enterprise policy API calls' + } + if ($TargetScope.customProperties.enabled -eq $true) { if ($TargetScope.customProperties.scope -ne 'enterprise') { throw "Unsupported customProperties.scope '$($TargetScope.customProperties.scope)'." From 471dc89a4d24ebf3f224a8785c4518761c188868 Mon Sep 17 00:00:00 2001 From: Copilot <223556219+Copilot@users.noreply.github.com> Date: Sat, 25 Jul 2026 21:02:10 +0200 Subject: [PATCH 10/20] feat: add layered policy documents and policy engine execution Refactor Sync-Files to load and execute policy documents by layer (enterprise -> organization -> repository), separating policy capabilities from deployment configuration. Add initial policy documents for enterprise custom properties, enterprise rulesets, organization placeholder, and repository file subscription service. Keep repository sync capability explicit in policy documents. Update enterprise API auth strategy to prefer GitHub App context and fall back to CUSTO_ENTERPRISE_PAT only when app calls fail. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- AGENTS.md | 17 +- ...nterprise-repo-custom-property.policy.json | 12 + Policies/enterprise-repo-rulesets.policy.json | 11 + Policies/organization-none.policy.json | 8 + ...tory-file-subscription-service.policy.json | 8 + README.md | 26 +- config/targets.json | 8 +- config/targets.schema.json | 60 +-- scripts/Sync-Files.ps1 | 480 +++++++++--------- 9 files changed, 333 insertions(+), 297 deletions(-) create mode 100644 Policies/enterprise-repo-custom-property.policy.json create mode 100644 Policies/enterprise-repo-rulesets.policy.json create mode 100644 Policies/organization-none.policy.json create mode 100644 Policies/repository-file-subscription-service.policy.json diff --git a/AGENTS.md b/AGENTS.md index 7e1bf39..ba6b743 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -33,10 +33,23 @@ cannot run against real repositories until an operator completes these steps: (`gh api orgs/PSModule/installations`). 3. **Set custom properties on target repositories.** Each subscribing module repository needs `Type = Module` and `SubscribeTo` including `AGENTS.md` set at the repository level. - Custo now maintains allowed values at enterprise level from `Repos/` file sets when - `customProperties.enabled=true` in `config/targets.json`. + Custo now maintains allowed values at enterprise level from `Repos/` file sets via policy + documents in `Policies/`. 4. **Dry-run via `workflow_dispatch`.** Trigger the `Sync Managed Files` workflow manually first and review the summary before relying on the daily schedule. +## Policy documents + +Policy configuration is declared in `Policies/*.policy.json` and executed in fixed layer order: +1. enterprise +2. organization +3. repository + +Current policies: +- `enterprise-repo-custom-property.policy.json` +- `enterprise-repo-rulesets.policy.json` +- `organization-none.policy.json` +- `repository-file-subscription-service.policy.json` + Until step 1 is complete, the workflow will fail at the authentication step (`Connect-GitHub App`) — this is the current, expected blocker for this MVP. diff --git a/Policies/enterprise-repo-custom-property.policy.json b/Policies/enterprise-repo-custom-property.policy.json new file mode 100644 index 0000000..7f41256 --- /dev/null +++ b/Policies/enterprise-repo-custom-property.policy.json @@ -0,0 +1,12 @@ +{ + "version": "1.0", + "name": "Enterprise repo custom property policy", + "layer": "enterprise", + "capability": "repo-custom-property", + "enabled": true, + "config": { + "enterprise": "MSXOrg", + "typePropertyName": "Type", + "subscriptionPropertyName": "SubscribeTo" + } +} diff --git a/Policies/enterprise-repo-rulesets.policy.json b/Policies/enterprise-repo-rulesets.policy.json new file mode 100644 index 0000000..0983d4c --- /dev/null +++ b/Policies/enterprise-repo-rulesets.policy.json @@ -0,0 +1,11 @@ +{ + "version": "1.0", + "name": "Enterprise repo ruleset policy", + "layer": "enterprise", + "capability": "repo-rulesets", + "enabled": true, + "config": { + "enterprise": "MSXOrg", + "rulesets": [] + } +} diff --git a/Policies/organization-none.policy.json b/Policies/organization-none.policy.json new file mode 100644 index 0000000..b7ffc7f --- /dev/null +++ b/Policies/organization-none.policy.json @@ -0,0 +1,8 @@ +{ + "version": "1.0", + "name": "Organization policy placeholder", + "layer": "organization", + "capability": "none", + "enabled": true, + "config": {} +} diff --git a/Policies/repository-file-subscription-service.policy.json b/Policies/repository-file-subscription-service.policy.json new file mode 100644 index 0000000..f569483 --- /dev/null +++ b/Policies/repository-file-subscription-service.policy.json @@ -0,0 +1,8 @@ +{ + "version": "1.0", + "name": "Repository file subscription service", + "layer": "repository", + "capability": "file-subscription-service", + "enabled": true, + "config": {} +} diff --git a/README.md b/README.md index d652cd5..07af137 100644 --- a/README.md +++ b/README.md @@ -40,17 +40,24 @@ in the sync script. The default scope is `all-access`, which scans all repositor current GitHub App installation token. This keeps Custo org-agnostic and lets one runtime process all organizations and repositories it can access. +Policy behavior is defined by JSON documents under [`Policies/`](Policies/), not embedded in +`targets.json`. This separates **capabilities** (what policy layers can do) from **policy +configuration** (which policies are enabled and with what settings). + The [`scripts/Sync-Files.ps1`](scripts/Sync-Files.ps1) script, run by the [`Sync Managed Files`](.github/workflows/sync-files.yml) workflow: 1. Reads repository discovery scope from `config/targets.json`. 2. Discovers file sets under `Repos/`. -3. Applies policy controls in order: **Enterprise** first, then **Organizations and their repositories**. -4. Syncs enterprise-level custom-property schema definitions (`Type` and `SubscribeTo`) and - allowed values from the discovered file-set tree when `customProperties.enabled=true`. -5. Discovers subscribing repositories from all accessible repositories (or explicit organizations +3. Loads policy documents from `Policies/*.policy.json`. +4. Applies policy controls in order: **Enterprise** first, then **Organization**, then **Repository**. +5. Runs enterprise policy capabilities such as: + - `repo-custom-property` (maintain `Type` and `SubscribeTo` enterprise property definitions) + - `repo-rulesets` (maintain enterprise repository rulesets) +6. Discovers subscribing repositories from all accessible repositories (or explicit organizations when configured) and reads their `Type` and `SubscribeTo` custom properties. -6. Clones subscribing repositories, copies the relevant files, and opens or updates a +7. Executes repository capability `file-subscription-service`, cloning repositories, syncing files, + and opening/updating `managed-files/update` pull request when changes are detected. ## MVP rollout scope @@ -65,8 +72,11 @@ Additional file sets and rollout targets are added incrementally after this MVP ## Required secrets -The sync workflow authenticates as a GitHub App. For repository sync plus enterprise custom -property maintenance, the app needs: +The sync workflow authenticates as a GitHub App. Enterprise policy API calls are attempted with +the GitHub App token first; when those calls are unavailable for GitHub Apps, Custo falls back to +`CUSTO_ENTERPRISE_PAT` if present. + +For repository sync plus enterprise policy maintenance, configure: - `contents:write` - `pull_requests:write` @@ -77,7 +87,7 @@ Configure these repository secrets before enabling the scheduled sync: - `CUSTO_BOT_CLIENT_ID` - `CUSTO_BOT_PRIVATE_KEY` -- `CUSTO_ENTERPRISE_PAT` (optional, recommended for enterprise policy endpoints like enterprise rulesets/custom-properties when GitHub App enterprise permissions are unavailable) +- `CUSTO_ENTERPRISE_PAT` (optional fallback PAT for enterprise policy endpoints, e.g. enterprise rulesets, with `admin:enterprise`) See [`AGENTS.md`](AGENTS.md) for operator runbook steps and current rollout blockers. diff --git a/config/targets.json b/config/targets.json index 0ff49bf..12ad32a 100644 --- a/config/targets.json +++ b/config/targets.json @@ -1,11 +1,5 @@ { "$schema": "./targets.schema.json", "scope": "all-access", - "customProperties": { - "enabled": true, - "scope": "enterprise", - "enterprise": "MSXOrg", - "typePropertyName": "Type", - "subscriptionPropertyName": "SubscribeTo" - } + "policyPath": "../Policies" } diff --git a/config/targets.schema.json b/config/targets.schema.json index 9d43b70..3ab46ce 100644 --- a/config/targets.schema.json +++ b/config/targets.schema.json @@ -1,7 +1,7 @@ { "$schema": "https://json-schema.org/draft/2020-12/schema", - "title": "Custo target organizations", - "description": "Repository discovery scope for the Sync Managed Files workflow.", + "title": "Custo target discovery", + "description": "Repository discovery scope and policy document location for Sync Managed Files.", "type": "object", "required": ["scope"], "properties": { @@ -19,38 +19,10 @@ }, "uniqueItems": true }, - "customProperties": { - "type": "object", - "description": "Controls automatic custom-property schema synchronization.", - "properties": { - "enabled": { - "type": "boolean", - "description": "When true, Custo synchronizes custom-property definitions before repository sync.", - "default": false - }, - "scope": { - "type": "string", - "enum": ["enterprise"], - "description": "Where to maintain custom-property schema definitions." - }, - "enterprise": { - "type": "string", - "minLength": 1, - "description": "Enterprise slug used for enterprise-level custom-property schema." - }, - "typePropertyName": { - "type": "string", - "minLength": 1, - "description": "Custom-property name for repository type. Defaults to Type." - }, - "subscriptionPropertyName": { - "type": "string", - "minLength": 1, - "description": "Custom-property name for subscribed file sets. Defaults to SubscribeTo." - } - }, - "required": ["enabled"], - "additionalProperties": false + "policyPath": { + "type": "string", + "minLength": 1, + "description": "Relative or absolute path to policy documents (*.policy.json). Defaults to ../Policies from scripts/." } }, "allOf": [ @@ -66,26 +38,6 @@ } } } - }, - { - "if": { - "properties": { - "customProperties": { - "properties": { "enabled": { "const": true } }, - "required": ["enabled"] - } - } - }, - "then": { - "properties": { - "customProperties": { - "required": ["scope", "enterprise"], - "properties": { - "scope": { "const": "enterprise" } - } - } - } - } } ], "additionalProperties": false diff --git a/scripts/Sync-Files.ps1 b/scripts/Sync-Files.ps1 index ed37a37..9975c81 100644 --- a/scripts/Sync-Files.ps1 +++ b/scripts/Sync-Files.ps1 @@ -1,30 +1,23 @@ #!/usr/bin/env pwsh <# .SYNOPSIS - Syncs managed files from this repository to subscribing repositories across configured - discovery scope. + Syncs managed files from this repository to subscribing repositories. .DESCRIPTION - This script: - 1. Authenticates as a GitHub App for repo-level operations. - 2. Discovers available file sets from the Repos/ directory structure. - 3. Reads target discovery mode from config/targets.json. - 4. Applies policy controls in order: - - enterprise custom-property schema - - organization and repository targeting - 5. Discovers subscribing repositories from either: - - all repositories visible to the current installation token, or - - explicit organizations from config. - 6. For each subscribing repository: - - Clones the repository - - Copies managed files from the appropriate file sets - - Detects changes using git - - Creates or updates a pull request if changes are detected - 7. Outputs a summary of actions taken. + This script runs a layered policy engine and then performs repository file sync: + 1. Authenticate as GitHub App for repository-level operations. + 2. Discover managed file sets from Repos/. + 3. Read discovery scope from config/targets.json. + 4. Load policy documents from Policies/*.policy.json. + 5. Apply policies in order: + - Enterprise layer + - Organization layer + - Repository layer + 6. If repository file-subscription policy is enabled, sync files to subscribing repositories. .NOTES - Requires the GitHub PowerShell module and GitHub App authentication via GitHub-Script action. - Target discovery scope is configuration, not code - see config/targets.json. + Enterprise policy API calls prefer GitHub App context first, then fall back to + CUSTO_ENTERPRISE_PAT only when App access fails and PAT is configured. #> [CmdletBinding()] @@ -45,10 +38,6 @@ $script:Summary = @{ #region Helper Functions function Get-TargetScope { - <# - .SYNOPSIS - Reads repository discovery scope from config/targets.json. - #> [CmdletBinding()] param( [Parameter(Mandatory)] @@ -60,7 +49,6 @@ function Get-TargetScope { } $config = Get-Content -Path $ConfigPath -Raw | ConvertFrom-Json - if (-not $config.scope) { throw "Missing required 'scope' in: $ConfigPath" } @@ -76,11 +64,58 @@ function Get-TargetScope { return $config } +function Get-PolicyDocuments { + [CmdletBinding()] + param( + [Parameter(Mandatory)] + [string]$PolicyPath + ) + + if (-not (Test-Path $PolicyPath)) { + throw "Policy path not found at: $PolicyPath" + } + + $policyFiles = Get-ChildItem -Path $PolicyPath -File -Filter '*.policy.json' | Sort-Object Name + if ($policyFiles.Count -eq 0) { + throw "No policy documents found in: $PolicyPath" + } + + $layerOrder = @{ + enterprise = 1 + organization = 2 + repository = 3 + } + + $documents = @() + foreach ($policyFile in $policyFiles) { + $doc = Get-Content -Path $policyFile.FullName -Raw | ConvertFrom-Json -AsHashtable + + if (-not $doc.layer) { + throw "Policy '$($policyFile.Name)' is missing required field 'layer'." + } + if (-not $doc.capability) { + throw "Policy '$($policyFile.Name)' is missing required field 'capability'." + } + if (-not $doc.ContainsKey('enabled')) { + throw "Policy '$($policyFile.Name)' is missing required field 'enabled'." + } + + $doc.layer = $doc.layer.ToLowerInvariant() + $doc.capability = $doc.capability.ToLowerInvariant() + $doc.sourceFile = $policyFile.FullName + + if (-not $layerOrder.ContainsKey($doc.layer)) { + throw "Policy '$($policyFile.Name)' has unsupported layer '$($doc.layer)'." + } + + $documents += $doc + } + + $documents | + Sort-Object @{ Expression = { $layerOrder[$_.layer] } }, @{ Expression = { $_.capability } }, @{ Expression = { $_.sourceFile } } +} + function Get-FileSets { - <# - .SYNOPSIS - Discovers available file sets from the Repos/ directory structure. - #> [CmdletBinding()] param( [Parameter(Mandatory)] @@ -125,52 +160,10 @@ function Get-FileSets { } $fileSetTable | Format-Table -AutoSize | Out-String - return $fileSets } -function Get-AllAccessibleRepository { - <# - .SYNOPSIS - Lists all repositories visible to the current installation token. - #> - [CmdletBinding()] - param( - [Parameter(Mandatory)] - [object]$Context - ) - - $allRepos = @() - $page = 1 - $perPage = 100 - - while ($true) { - $response = (Invoke-GitHubAPI -Method GET -ApiEndpoint '/installation/repositories' -Body @{ - per_page = $perPage - page = $page - } -Context $Context).Response - - if (-not $response.repositories -or $response.repositories.Count -eq 0) { - break - } - - $allRepos += $response.repositories - - if ($response.repositories.Count -lt $perPage) { - break - } - - $page++ - } - - return $allRepos -} - function Invoke-EnterprisePolicyApi { - <# - .SYNOPSIS - Calls enterprise policy endpoints with PAT when available, otherwise with app context. - #> [CmdletBinding()] param( [Parameter(Mandatory)] @@ -180,14 +173,22 @@ function Invoke-EnterprisePolicyApi { [Parameter(Mandatory)] [string]$ApiEndpoint, - [hashtable]$Body, + [object]$Body, [Parameter(Mandatory)] [object]$Context ) - $enterprisePat = $env:CUSTO_ENTERPRISE_PAT - if (-not [string]::IsNullOrWhiteSpace($enterprisePat)) { + try { + return (Invoke-GitHubAPI -Method $Method -ApiEndpoint $ApiEndpoint -Body $Body -Context $Context).Response + } catch { + $enterprisePat = $env:CUSTO_ENTERPRISE_PAT + if ([string]::IsNullOrWhiteSpace($enterprisePat)) { + throw + } + + Write-Host "ℹ️ App auth failed for $ApiEndpoint; retrying with CUSTO_ENTERPRISE_PAT" + $headers = @{ Authorization = "Bearer $enterprisePat" Accept = 'application/vnd.github+json' @@ -201,23 +202,49 @@ function Invoke-EnterprisePolicyApi { Headers = $headers } - if ($Body) { + if ($null -ne $Body) { $invokeArgs.ContentType = 'application/json' - $invokeArgs.Body = $Body | ConvertTo-Json -Depth 20 + $invokeArgs.Body = $Body | ConvertTo-Json -Depth 30 } return Invoke-RestMethod @invokeArgs } +} + +function Get-AllAccessibleRepository { + [CmdletBinding()] + param( + [Parameter(Mandatory)] + [object]$Context + ) + + $allRepos = @() + $page = 1 + $perPage = 100 + + while ($true) { + $response = (Invoke-GitHubAPI -Method GET -ApiEndpoint '/installation/repositories' -Body @{ + per_page = $perPage + page = $page + } -Context $Context).Response - return (Invoke-GitHubAPI -Method $Method -ApiEndpoint $ApiEndpoint -Body $Body -Context $Context).Response + if (-not $response.repositories -or $response.repositories.Count -eq 0) { + break + } + + $allRepos += $response.repositories + + if ($response.repositories.Count -lt $perPage) { + break + } + + $page++ + } + + return $allRepos } function Sync-EnterpriseCustomPropertySchema { - <# - .SYNOPSIS - Ensures enterprise-level custom property definitions exist with allowed values - derived from the Repos/ file-set tree. - #> [CmdletBinding()] param( [Parameter(Mandatory)] @@ -269,11 +296,59 @@ function Sync-EnterpriseCustomPropertySchema { Write-Host " - ${SubscriptionPropertyName}: $($subscriptionValues -join ', ')" } +function Sync-EnterpriseRulesets { + [CmdletBinding()] + param( + [Parameter(Mandatory)] + [string]$Enterprise, + + [Parameter(Mandatory)] + [object[]]$Rulesets, + + [Parameter(Mandatory)] + [object]$Context + ) + + if (-not $Rulesets -or $Rulesets.Count -eq 0) { + Write-Host "ℹ️ No enterprise rulesets declared for '$Enterprise'" + return + } + + $existingRulesets = @( + Invoke-EnterprisePolicyApi -Method GET -ApiEndpoint "/enterprises/$Enterprise/rulesets" -Context $Context + ) + + $existingByName = @{} + foreach ($existing in $existingRulesets) { + if ($existing.name) { + $existingByName[$existing.name] = $existing + } + } + + foreach ($ruleset in $Rulesets) { + if (-not $ruleset.name) { + throw "Enterprise ruleset entry is missing required field 'name'." + } + + $payload = @{} + foreach ($key in $ruleset.Keys) { + if ($key -ne 'id' -and $key -ne 'source' -and $key -ne 'source_type' -and $key -ne 'created_at' -and $key -ne 'updated_at') { + $payload[$key] = $ruleset[$key] + } + } + + if ($existingByName.ContainsKey($ruleset.name)) { + $rulesetId = $existingByName[$ruleset.name].id + Invoke-EnterprisePolicyApi -Method PUT -ApiEndpoint "/enterprises/$Enterprise/rulesets/$rulesetId" -Body $payload -Context $Context | Out-Null + Write-Host "✅ Updated enterprise ruleset '$($ruleset.name)'" + } else { + Invoke-EnterprisePolicyApi -Method POST -ApiEndpoint "/enterprises/$Enterprise/rulesets" -Body $payload -Context $Context | Out-Null + Write-Host "✅ Created enterprise ruleset '$($ruleset.name)'" + } + } +} + function Get-SubscribingRepository { - <# - .SYNOPSIS - Queries an organization's repositories for their Type and SubscribeTo custom properties. - #> [CmdletBinding()] param( [Parameter(Mandatory)] @@ -284,30 +359,18 @@ function Get-SubscribingRepository { ) $repos = Get-GitHubRepository -Owner $Owner -Context $Context - $subscribingRepos = @() foreach ($repo in $repos) { $customProps = $repo.CustomProperties - - if (-not $customProps) { - continue - } + if (-not $customProps) { continue } $type = ($customProps | Where-Object Name -EQ 'Type').Value $subscribeTo = ($customProps | Where-Object Name -EQ 'SubscribeTo').Value - if (-not $type -or -not $subscribeTo) { - continue - } - - if ($subscribeTo -is [string]) { - $subscribeTo = @($subscribeTo) - } - - if ($subscribeTo.Count -eq 0) { - continue - } + if (-not $type -or -not $subscribeTo) { continue } + if ($subscribeTo -is [string]) { $subscribeTo = @($subscribeTo) } + if ($subscribeTo.Count -eq 0) { continue } $subscribingRepos += @{ Name = $repo.Name @@ -319,63 +382,35 @@ function Get-SubscribingRepository { } } - $subscribingRepos | ForEach-Object { - [PSCustomObject]@{ - Owner = $_.Owner - Repo = $_.Name - Type = $_.Type - SubscribeTo = $_.SubscribeTo -join ', ' - } - } | Format-Table -AutoSize | Out-String - return $subscribingRepos } function Get-SubscribingRepositoryByOrganizationFromAllAccess { - <# - .SYNOPSIS - Discovers subscribing repositories from all repositories visible to the - current installation token, grouped by owning organization. - #> [CmdletBinding()] param( [Parameter(Mandatory)] - [object]$Context, - - [object[]]$Repositories + [object]$Context ) - $allRepos = if ($Repositories) { $Repositories } else { Get-AllAccessibleRepository -Context $Context } - + $allRepos = Get-AllAccessibleRepository -Context $Context $reposByOrg = @{} foreach ($repo in $allRepos) { - if ($repo.owner.type -ne 'Organization') { - continue - } + if ($repo.owner.type -ne 'Organization') { continue } $owner = $repo.owner.login $repoName = $repo.name $customProps = (Invoke-GitHubAPI -Method GET -ApiEndpoint "/repos/$owner/$repoName/properties/values" -Context $Context).Response - $typeProp = $customProps | Where-Object { $_.property_name -eq 'Type' } $subscribeToProp = $customProps | Where-Object { $_.property_name -eq 'SubscribeTo' } $type = $typeProp.value $subscribeTo = $subscribeToProp.value - if (-not $type -or -not $subscribeTo) { - continue - } - - if ($subscribeTo -is [string]) { - $subscribeTo = @($subscribeTo) - } - - if ($subscribeTo.Count -eq 0) { - continue - } + if (-not $type -or -not $subscribeTo) { continue } + if ($subscribeTo -is [string]) { $subscribeTo = @($subscribeTo) } + if ($subscribeTo.Count -eq 0) { continue } $repoEntry = @{ Name = $repoName @@ -389,92 +424,50 @@ function Get-SubscribingRepositoryByOrganizationFromAllAccess { if (-not $reposByOrg.ContainsKey($owner)) { $reposByOrg[$owner] = @() } - $reposByOrg[$owner] += $repoEntry } - $reposByOrg.GetEnumerator() | ForEach-Object { - $org = $_.Key - $_.Value | ForEach-Object { - [PSCustomObject]@{ - Owner = $org - Repo = $_.Name - Type = $_.Type - SubscribeTo = $_.SubscribeTo -join ', ' - } - } - } | Format-Table -AutoSize | Out-String - return $reposByOrg } -function Invoke-PolicyEngine { - <# - .SYNOPSIS - Applies top-level policy controls before repository sync. - .DESCRIPTION - Current policy order: - 1) Enterprise policy layer - 2) Organization and repository synchronization - #> +function Get-AllSubscribingRepository { [CmdletBinding()] param( [Parameter(Mandatory)] [pscustomobject]$TargetScope, - [Parameter(Mandatory)] - [hashtable]$FileSets, - [Parameter(Mandatory)] [object]$Context ) - LogGroup '🧭 Policy engine: enterprise controls first' { - if (-not [string]::IsNullOrWhiteSpace($env:CUSTO_ENTERPRISE_PAT)) { - Write-Host 'Using CUSTO_ENTERPRISE_PAT for enterprise policy API calls' - } else { - Write-Host 'Using GitHub App token for enterprise policy API calls' - } - - if ($TargetScope.customProperties.enabled -eq $true) { - if ($TargetScope.customProperties.scope -ne 'enterprise') { - throw "Unsupported customProperties.scope '$($TargetScope.customProperties.scope)'." - } - if (-not $TargetScope.customProperties.enterprise) { - throw "customProperties.enterprise is required when customProperties.enabled is true." - } - - $typePropertyName = if ($TargetScope.customProperties.typePropertyName) { - $TargetScope.customProperties.typePropertyName - } else { - 'Type' - } - - $subscriptionPropertyName = if ($TargetScope.customProperties.subscriptionPropertyName) { - $TargetScope.customProperties.subscriptionPropertyName - } else { - 'SubscribeTo' - } + $subscribingRepos = @() - Sync-EnterpriseCustomPropertySchema ` - -Enterprise $TargetScope.customProperties.enterprise ` - -FileSets $FileSets ` - -Context $Context ` - -TypePropertyName $typePropertyName ` - -SubscriptionPropertyName $subscriptionPropertyName - } else { - Write-Host 'ℹ️ customProperties sync disabled by config' + if ($TargetScope.scope -eq 'all-access') { + $reposByOrg = Get-SubscribingRepositoryByOrganizationFromAllAccess -Context $Context + foreach ($org in @($reposByOrg.Keys | Sort-Object)) { + Write-Host "Found $($reposByOrg[$org].Count) subscribing repositories in $org" + $subscribingRepos += $reposByOrg[$org] + } + } else { + foreach ($org in $TargetScope.organizations) { + $orgRepos = Get-SubscribingRepository -Owner $org -Context $Context + Write-Host "Found $($orgRepos.Count) subscribing repositories in $org" + $subscribingRepos += $orgRepos } } + + return $subscribingRepos } -function Get-AllSubscribingRepository { - <# - .SYNOPSIS - Resolves subscribing repositories after policy has been applied. - #> +function Invoke-PolicyEngine { [CmdletBinding()] param( + [Parameter(Mandatory)] + [object[]]$Policies, + + [Parameter(Mandatory)] + [hashtable]$FileSets, + [Parameter(Mandatory)] [pscustomobject]$TargetScope, @@ -482,31 +475,58 @@ function Get-AllSubscribingRepository { [object]$Context ) - $subscribingRepos = @() + $state = @{ + RepoFileSubscriptionEnabled = $false + } - if ($TargetScope.scope -eq 'all-access') { - $reposByOrg = Get-SubscribingRepositoryByOrganizationFromAllAccess -Context $Context + foreach ($policy in $Policies) { + $name = if ($policy.name) { $policy.name } else { [System.IO.Path]::GetFileName($policy.sourceFile) } + $capabilityId = "$($policy.layer).$($policy.capability)" - foreach ($org in @($reposByOrg.Keys | Sort-Object)) { - Write-Host "Found $($reposByOrg[$org].Count) subscribing repositories in $org" - $subscribingRepos += $reposByOrg[$org] + if ($policy.enabled -ne $true) { + Write-Host "⏭️ Policy disabled: $name ($capabilityId)" + continue } - } else { - foreach ($org in $TargetScope.organizations) { - $orgRepos = Get-SubscribingRepository -Owner $org -Context $Context - Write-Host "Found $($orgRepos.Count) subscribing repositories in $org" - $subscribingRepos += $orgRepos + + Write-Host "▶️ Applying policy: $name ($capabilityId)" + + switch ($capabilityId) { + 'enterprise.repo-custom-property' { + $enterprise = $policy.config.enterprise + if (-not $enterprise) { throw "Policy '$name' requires config.enterprise." } + + $typeName = if ($policy.config.typePropertyName) { $policy.config.typePropertyName } else { 'Type' } + $subscriptionName = if ($policy.config.subscriptionPropertyName) { $policy.config.subscriptionPropertyName } else { 'SubscribeTo' } + + Sync-EnterpriseCustomPropertySchema ` + -Enterprise $enterprise ` + -FileSets $FileSets ` + -Context $Context ` + -TypePropertyName $typeName ` + -SubscriptionPropertyName $subscriptionName + } + 'enterprise.repo-rulesets' { + $enterprise = $policy.config.enterprise + if (-not $enterprise) { throw "Policy '$name' requires config.enterprise." } + $rulesets = if ($policy.config.rulesets) { @($policy.config.rulesets) } else { @() } + Sync-EnterpriseRulesets -Enterprise $enterprise -Rulesets $rulesets -Context $Context + } + 'organization.none' { + Write-Host "ℹ️ Organization policy layer intentionally empty" + } + 'repository.file-subscription-service' { + $state.RepoFileSubscriptionEnabled = $true + } + default { + throw "Unsupported policy capability '$capabilityId' in '$name'." + } } } - return $subscribingRepos + return $state } function Sync-RepositoryFile { - <# - .SYNOPSIS - Syncs files to a single repository. - #> [Diagnostics.CodeAnalysis.SuppressMessageAttribute( 'PSAvoidUsingWriteHost', '', Scope = 'Function', Justification = 'Intended for logging in GitHub Actions runners.' @@ -549,7 +569,6 @@ function Sync-RepositoryFile { $script:Summary.TotalReposProcessed++ - # Validate before opening a log group - skipped repos stay quiet if (-not $FileSets.ContainsKey($type)) { Write-Host "⚠️ $repoFullName - Type folder '$type' not found, skipping" $script:Summary.ReposSkipped++ @@ -571,7 +590,6 @@ function Sync-RepositoryFile { return } - # All real work inside a log group LogGroup "📦 $repoFullName" { foreach ($selection in $subscribeTo) { if ($FileSets[$type].ContainsKey($selection)) { @@ -593,7 +611,6 @@ function Sync-RepositoryFile { try { Set-GitHubGitConfig -Context $Context - # Branch setup $remoteBranches = git branch -r 2>&1 if ($remoteBranches -match "origin/$BranchName") { git fetch origin $BranchName 2>&1 | Out-Null @@ -602,7 +619,6 @@ function Sync-RepositoryFile { git checkout -b $BranchName 2>&1 | Out-Null } - # Copy files foreach ($fileInfo in $filesToSync) { $targetPath = Join-Path $clonePath $fileInfo.RelativePath $targetDir = Split-Path $targetPath -Parent @@ -612,7 +628,6 @@ function Sync-RepositoryFile { Copy-Item -Path $fileInfo.SourcePath -Destination $targetPath -Force } - # Detect changes $status = git status --porcelain 2>&1 if ([string]::IsNullOrWhiteSpace($status)) { Write-Host '✅ Already in sync' @@ -622,7 +637,6 @@ function Sync-RepositoryFile { $status -split "`n" | ForEach-Object { Write-Host " $_" } - # Commit and push git add --all 2>&1 | Out-Null git commit -m $CommitMessage 2>&1 | Out-Null $pushResult = git push --force --set-upstream origin $BranchName 2>&1 @@ -630,7 +644,6 @@ function Sync-RepositoryFile { throw "Git push failed: $pushResult" } - # Create or update PR $existingPRs = (Invoke-GitHubAPI -Method GET -ApiEndpoint "/repos/$owner/$repoName/pulls" -Body @{ head = "${owner}:${BranchName}" state = 'open' @@ -705,7 +718,24 @@ try { } } - Invoke-PolicyEngine -TargetScope $targetScope -FileSets $fileSets -Context $context + LogGroup '📜 Load policy documents' { + $policyPath = if ($targetScope.policyPath) { $targetScope.policyPath } else { '../Policies' } + if (-not [System.IO.Path]::IsPathRooted($policyPath)) { + $policyPath = Join-Path $PSScriptRoot $policyPath + } + $policyPath = Resolve-Path $policyPath + $policies = Get-PolicyDocuments -PolicyPath $policyPath + Write-Host "Loaded $($policies.Count) policy documents from $policyPath" + } + + LogGroup '🧭 Execute policy engine' { + $policyState = Invoke-PolicyEngine -Policies $policies -FileSets $fileSets -TargetScope $targetScope -Context $context + } + + if ($policyState.RepoFileSubscriptionEnabled -ne $true) { + Write-Host 'ℹ️ Repository file-subscription policy is disabled; file sync skipped.' + exit 0 + } LogGroup '🔍 Find subscribing repositories' { $subscribingRepos = Get-AllSubscribingRepository -TargetScope $targetScope -Context $context @@ -717,7 +747,6 @@ try { exit 0 } - # Sync files to each repository $tempPath = Join-Path ([System.IO.Path]::GetTempPath()) "custo-sync-$(Get-Random)" New-Item -Path $tempPath -ItemType Directory -Force | Out-Null @@ -749,7 +778,6 @@ The files in this PR are centrally managed. Any local changes to these files wil } } - # Summary Write-Host '' Write-Host '📊 Summary' Write-Host " Processed: $($script:Summary.TotalReposProcessed)" From 0f5f5048609cbadf7180dbe54ca29c28e87e13a7 Mon Sep 17 00:00:00 2001 From: Copilot <223556219+Copilot@users.noreply.github.com> Date: Sat, 25 Jul 2026 21:06:10 +0200 Subject: [PATCH 11/20] feat: add enterprise-mapped policy engine folder hierarchy Introduce PolicyEngine/ with explicit separation between capabilities and policies: - Capabilities/{layer}/*.capability.json - Policies/{enterprise}/{layer}/*.policy.json Update sync runtime to load capabilities and validate policy docs against supported capabilities, and resolve policy sets by enterprise mapping from config/targets.json. Migrate existing policy docs into the new hierarchy and update documentation accordingly. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- AGENTS.md | 18 +++-- .../repo-custom-property.capability.json | 8 ++ .../enterprise/repo-rulesets.capability.json | 8 ++ .../organization/none.capability.json | 8 ++ .../file-subscription-service.capability.json | 8 ++ .../repo-custom-property.policy.json | 0 .../enterprise/repo-rulesets.policy.json | 0 .../MSXOrg/organization/none.policy.json | 0 .../file-subscription-service.policy.json | 0 README.md | 11 ++- config/targets.json | 5 +- config/targets.schema.json | 23 ++++-- scripts/Sync-Files.ps1 | 79 ++++++++++++++++--- 13 files changed, 142 insertions(+), 26 deletions(-) create mode 100644 PolicyEngine/Capabilities/enterprise/repo-custom-property.capability.json create mode 100644 PolicyEngine/Capabilities/enterprise/repo-rulesets.capability.json create mode 100644 PolicyEngine/Capabilities/organization/none.capability.json create mode 100644 PolicyEngine/Capabilities/repository/file-subscription-service.capability.json rename Policies/enterprise-repo-custom-property.policy.json => PolicyEngine/Policies/MSXOrg/enterprise/repo-custom-property.policy.json (100%) rename Policies/enterprise-repo-rulesets.policy.json => PolicyEngine/Policies/MSXOrg/enterprise/repo-rulesets.policy.json (100%) rename Policies/organization-none.policy.json => PolicyEngine/Policies/MSXOrg/organization/none.policy.json (100%) rename Policies/repository-file-subscription-service.policy.json => PolicyEngine/Policies/MSXOrg/repository/file-subscription-service.policy.json (100%) diff --git a/AGENTS.md b/AGENTS.md index ba6b743..bbd44c4 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -34,22 +34,26 @@ cannot run against real repositories until an operator completes these steps: 3. **Set custom properties on target repositories.** Each subscribing module repository needs `Type = Module` and `SubscribeTo` including `AGENTS.md` set at the repository level. Custo now maintains allowed values at enterprise level from `Repos/` file sets via policy - documents in `Policies/`. + documents in `PolicyEngine/Policies/{enterprise}/`. 4. **Dry-run via `workflow_dispatch`.** Trigger the `Sync Managed Files` workflow manually first and review the summary before relying on the daily schedule. ## Policy documents -Policy configuration is declared in `Policies/*.policy.json` and executed in fixed layer order: +Policy configuration is declared in `PolicyEngine/` and executed in fixed layer order: 1. enterprise 2. organization 3. repository -Current policies: -- `enterprise-repo-custom-property.policy.json` -- `enterprise-repo-rulesets.policy.json` -- `organization-none.policy.json` -- `repository-file-subscription-service.policy.json` +Current default enterprise mapping (`MSXOrg`): +- `PolicyEngine/Capabilities/enterprise/repo-custom-property.capability.json` +- `PolicyEngine/Capabilities/enterprise/repo-rulesets.capability.json` +- `PolicyEngine/Capabilities/organization/none.capability.json` +- `PolicyEngine/Capabilities/repository/file-subscription-service.capability.json` +- `PolicyEngine/Policies/MSXOrg/enterprise/repo-custom-property.policy.json` +- `PolicyEngine/Policies/MSXOrg/enterprise/repo-rulesets.policy.json` +- `PolicyEngine/Policies/MSXOrg/organization/none.policy.json` +- `PolicyEngine/Policies/MSXOrg/repository/file-subscription-service.policy.json` Until step 1 is complete, the workflow will fail at the authentication step (`Connect-GitHub App`) — this is the current, expected blocker for this MVP. diff --git a/PolicyEngine/Capabilities/enterprise/repo-custom-property.capability.json b/PolicyEngine/Capabilities/enterprise/repo-custom-property.capability.json new file mode 100644 index 0000000..7ebec90 --- /dev/null +++ b/PolicyEngine/Capabilities/enterprise/repo-custom-property.capability.json @@ -0,0 +1,8 @@ +{ + "version": "1.0", + "name": "Enterprise repository custom-property schema", + "layer": "enterprise", + "capability": "repo-custom-property", + "authPreference": "github-app-first-fallback-enterprise-pat", + "description": "Maintains enterprise-level repository custom-property definitions and allowed values." +} diff --git a/PolicyEngine/Capabilities/enterprise/repo-rulesets.capability.json b/PolicyEngine/Capabilities/enterprise/repo-rulesets.capability.json new file mode 100644 index 0000000..f0d3733 --- /dev/null +++ b/PolicyEngine/Capabilities/enterprise/repo-rulesets.capability.json @@ -0,0 +1,8 @@ +{ + "version": "1.0", + "name": "Enterprise repository rulesets", + "layer": "enterprise", + "capability": "repo-rulesets", + "authPreference": "github-app-first-fallback-enterprise-pat", + "description": "Maintains enterprise-level repository rulesets." +} diff --git a/PolicyEngine/Capabilities/organization/none.capability.json b/PolicyEngine/Capabilities/organization/none.capability.json new file mode 100644 index 0000000..114a949 --- /dev/null +++ b/PolicyEngine/Capabilities/organization/none.capability.json @@ -0,0 +1,8 @@ +{ + "version": "1.0", + "name": "Organization policy placeholder", + "layer": "organization", + "capability": "none", + "authPreference": "github-app", + "description": "Placeholder capability for future organization-level policies." +} diff --git a/PolicyEngine/Capabilities/repository/file-subscription-service.capability.json b/PolicyEngine/Capabilities/repository/file-subscription-service.capability.json new file mode 100644 index 0000000..b7ccde1 --- /dev/null +++ b/PolicyEngine/Capabilities/repository/file-subscription-service.capability.json @@ -0,0 +1,8 @@ +{ + "version": "1.0", + "name": "Repository file subscription service", + "layer": "repository", + "capability": "file-subscription-service", + "authPreference": "github-app", + "description": "Synchronizes managed files into subscribing repositories via pull requests." +} diff --git a/Policies/enterprise-repo-custom-property.policy.json b/PolicyEngine/Policies/MSXOrg/enterprise/repo-custom-property.policy.json similarity index 100% rename from Policies/enterprise-repo-custom-property.policy.json rename to PolicyEngine/Policies/MSXOrg/enterprise/repo-custom-property.policy.json diff --git a/Policies/enterprise-repo-rulesets.policy.json b/PolicyEngine/Policies/MSXOrg/enterprise/repo-rulesets.policy.json similarity index 100% rename from Policies/enterprise-repo-rulesets.policy.json rename to PolicyEngine/Policies/MSXOrg/enterprise/repo-rulesets.policy.json diff --git a/Policies/organization-none.policy.json b/PolicyEngine/Policies/MSXOrg/organization/none.policy.json similarity index 100% rename from Policies/organization-none.policy.json rename to PolicyEngine/Policies/MSXOrg/organization/none.policy.json diff --git a/Policies/repository-file-subscription-service.policy.json b/PolicyEngine/Policies/MSXOrg/repository/file-subscription-service.policy.json similarity index 100% rename from Policies/repository-file-subscription-service.policy.json rename to PolicyEngine/Policies/MSXOrg/repository/file-subscription-service.policy.json diff --git a/README.md b/README.md index 07af137..fbdd9d0 100644 --- a/README.md +++ b/README.md @@ -40,16 +40,19 @@ in the sync script. The default scope is `all-access`, which scans all repositor current GitHub App installation token. This keeps Custo org-agnostic and lets one runtime process all organizations and repositories it can access. -Policy behavior is defined by JSON documents under [`Policies/`](Policies/), not embedded in -`targets.json`. This separates **capabilities** (what policy layers can do) from **policy -configuration** (which policies are enabled and with what settings). +Policy behavior is defined by JSON documents under [`PolicyEngine/`](PolicyEngine/), not embedded +in `targets.json`. This separates **capabilities** from **policy configuration**: + +- `PolicyEngine/Capabilities/{layer}/*.capability.json` defines what each capability does. +- `PolicyEngine/Policies/{enterprise}/{layer}/*.policy.json` defines how that capability is + configured for a specific enterprise. The [`scripts/Sync-Files.ps1`](scripts/Sync-Files.ps1) script, run by the [`Sync Managed Files`](.github/workflows/sync-files.yml) workflow: 1. Reads repository discovery scope from `config/targets.json`. 2. Discovers file sets under `Repos/`. -3. Loads policy documents from `Policies/*.policy.json`. +3. Loads capability and policy documents from `PolicyEngine/`. 4. Applies policy controls in order: **Enterprise** first, then **Organization**, then **Repository**. 5. Runs enterprise policy capabilities such as: - `repo-custom-property` (maintain `Type` and `SubscribeTo` enterprise property definitions) diff --git a/config/targets.json b/config/targets.json index 12ad32a..1417be3 100644 --- a/config/targets.json +++ b/config/targets.json @@ -1,5 +1,8 @@ { "$schema": "./targets.schema.json", "scope": "all-access", - "policyPath": "../Policies" + "policy": { + "rootPath": "../PolicyEngine", + "enterprise": "MSXOrg" + } } diff --git a/config/targets.schema.json b/config/targets.schema.json index 3ab46ce..a234306 100644 --- a/config/targets.schema.json +++ b/config/targets.schema.json @@ -1,7 +1,7 @@ { "$schema": "https://json-schema.org/draft/2020-12/schema", "title": "Custo target discovery", - "description": "Repository discovery scope and policy document location for Sync Managed Files.", + "description": "Repository discovery scope and policy engine paths for Sync Managed Files.", "type": "object", "required": ["scope"], "properties": { @@ -19,10 +19,23 @@ }, "uniqueItems": true }, - "policyPath": { - "type": "string", - "minLength": 1, - "description": "Relative or absolute path to policy documents (*.policy.json). Defaults to ../Policies from scripts/." + "policy": { + "type": "object", + "description": "Policy engine folder mapping and enterprise selection.", + "properties": { + "rootPath": { + "type": "string", + "minLength": 1, + "description": "Relative or absolute root path that contains Capabilities/ and Policies/." + }, + "enterprise": { + "type": "string", + "minLength": 1, + "description": "Enterprise key used under Policies/{enterprise}/." + } + }, + "required": ["enterprise"], + "additionalProperties": false } }, "allOf": [ diff --git a/scripts/Sync-Files.ps1 b/scripts/Sync-Files.ps1 index 9975c81..bfb7fb8 100644 --- a/scripts/Sync-Files.ps1 +++ b/scripts/Sync-Files.ps1 @@ -8,7 +8,7 @@ 1. Authenticate as GitHub App for repository-level operations. 2. Discover managed file sets from Repos/. 3. Read discovery scope from config/targets.json. - 4. Load policy documents from Policies/*.policy.json. + 4. Load capability and policy documents from PolicyEngine/. 5. Apply policies in order: - Enterprise layer - Organization layer @@ -61,21 +61,71 @@ function Get-TargetScope { throw "Unsupported scope '$($config.scope)' in: $ConfigPath" } + if ($config.policy) { + if (-not $config.policy.enterprise) { + throw "Config policy section requires 'enterprise'." + } + } + return $config } +function Get-CapabilityCatalog { + [CmdletBinding()] + param( + [Parameter(Mandatory)] + [string]$CapabilityPath + ) + + if (-not (Test-Path $CapabilityPath)) { + throw "Capability path not found at: $CapabilityPath" + } + + $capabilityFiles = Get-ChildItem -Path $CapabilityPath -File -Recurse -Filter '*.capability.json' | Sort-Object FullName + if ($capabilityFiles.Count -eq 0) { + throw "No capability documents found in: $CapabilityPath" + } + + $catalog = @{} + foreach ($capabilityFile in $capabilityFiles) { + $doc = Get-Content -Path $capabilityFile.FullName -Raw | ConvertFrom-Json -AsHashtable + + if (-not $doc.layer -or -not $doc.capability) { + throw "Capability '$($capabilityFile.Name)' must define both layer and capability." + } + + $layer = $doc.layer.ToLowerInvariant() + $capability = $doc.capability.ToLowerInvariant() + $key = "$layer.$capability" + + if ($catalog.ContainsKey($key)) { + throw "Duplicate capability definition for '$key': $($capabilityFile.FullName)" + } + + $doc.layer = $layer + $doc.capability = $capability + $doc.sourceFile = $capabilityFile.FullName + $catalog[$key] = $doc + } + + return $catalog +} + function Get-PolicyDocuments { [CmdletBinding()] param( [Parameter(Mandatory)] - [string]$PolicyPath + [string]$PolicyPath, + + [Parameter(Mandatory)] + [hashtable]$CapabilityCatalog ) if (-not (Test-Path $PolicyPath)) { throw "Policy path not found at: $PolicyPath" } - $policyFiles = Get-ChildItem -Path $PolicyPath -File -Filter '*.policy.json' | Sort-Object Name + $policyFiles = Get-ChildItem -Path $PolicyPath -File -Recurse -Filter '*.policy.json' | Sort-Object FullName if ($policyFiles.Count -eq 0) { throw "No policy documents found in: $PolicyPath" } @@ -108,6 +158,11 @@ function Get-PolicyDocuments { throw "Policy '$($policyFile.Name)' has unsupported layer '$($doc.layer)'." } + $capabilityKey = "$($doc.layer).$($doc.capability)" + if (-not $CapabilityCatalog.ContainsKey($capabilityKey)) { + throw "Policy '$($policyFile.Name)' references unsupported capability '$capabilityKey'." + } + $documents += $doc } @@ -719,13 +774,19 @@ try { } LogGroup '📜 Load policy documents' { - $policyPath = if ($targetScope.policyPath) { $targetScope.policyPath } else { '../Policies' } - if (-not [System.IO.Path]::IsPathRooted($policyPath)) { - $policyPath = Join-Path $PSScriptRoot $policyPath + $policyRoot = if ($targetScope.policy.rootPath) { $targetScope.policy.rootPath } else { '../PolicyEngine' } + if (-not [System.IO.Path]::IsPathRooted($policyRoot)) { + $policyRoot = Join-Path $PSScriptRoot $policyRoot } - $policyPath = Resolve-Path $policyPath - $policies = Get-PolicyDocuments -PolicyPath $policyPath - Write-Host "Loaded $($policies.Count) policy documents from $policyPath" + $policyRoot = Resolve-Path $policyRoot + + $policyEnterprise = if ($targetScope.policy.enterprise) { $targetScope.policy.enterprise } else { 'default' } + $capabilityPath = Join-Path $policyRoot 'Capabilities' + $policyPath = Join-Path (Join-Path $policyRoot 'Policies') $policyEnterprise + + $capabilityCatalog = Get-CapabilityCatalog -CapabilityPath (Resolve-Path $capabilityPath) + $policies = Get-PolicyDocuments -PolicyPath (Resolve-Path $policyPath) -CapabilityCatalog $capabilityCatalog + Write-Host "Loaded $($policies.Count) policy documents for enterprise '$policyEnterprise'" } LogGroup '🧭 Execute policy engine' { From 79b2cf4e19ca925b486a55154123533712dda311 Mon Sep 17 00:00:00 2001 From: Copilot <223556219+Copilot@users.noreply.github.com> Date: Sat, 25 Jul 2026 21:12:12 +0200 Subject: [PATCH 12/20] feat: expand enterprise capability configs for properties and rulesets Build out explicit configuration for the two enterprise policy capabilities: - repo-custom-property: required/values-editability options - repo-rulesets: concrete starter ruleset payload config Update policy execution to consume the extended custom-property configuration fields and document the default enterprise policy files. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- .../repo-custom-property.capability.json | 37 +++++++++++- .../enterprise/repo-rulesets.capability.json | 15 ++++- .../repo-custom-property.policy.json | 5 +- .../enterprise/repo-rulesets.policy.json | 56 ++++++++++++++++++- README.md | 7 ++- scripts/Sync-Files.ps1 | 21 +++++-- 6 files changed, 132 insertions(+), 9 deletions(-) diff --git a/PolicyEngine/Capabilities/enterprise/repo-custom-property.capability.json b/PolicyEngine/Capabilities/enterprise/repo-custom-property.capability.json index 7ebec90..215ba91 100644 --- a/PolicyEngine/Capabilities/enterprise/repo-custom-property.capability.json +++ b/PolicyEngine/Capabilities/enterprise/repo-custom-property.capability.json @@ -4,5 +4,40 @@ "layer": "enterprise", "capability": "repo-custom-property", "authPreference": "github-app-first-fallback-enterprise-pat", - "description": "Maintains enterprise-level repository custom-property definitions and allowed values." + "description": "Maintains enterprise-level repository custom-property definitions and allowed values.", + "configSchema": { + "required": ["enterprise"], + "properties": { + "enterprise": { + "type": "string", + "description": "Enterprise slug to maintain property definitions on." + }, + "typePropertyName": { + "type": "string", + "default": "Type", + "description": "Repository type property name." + }, + "subscriptionPropertyName": { + "type": "string", + "default": "SubscribeTo", + "description": "Managed file-subscription property name." + }, + "required": { + "type": "boolean", + "default": false, + "description": "Whether repositories must explicitly set these property values." + }, + "valuesEditableBy": { + "type": "string", + "enum": ["org_actors", "org_and_repo_actors", null], + "default": "org_actors", + "description": "Who can edit property values." + }, + "requireExplicitValues": { + "type": "boolean", + "default": false, + "description": "Whether setting property values is mandatory." + } + } + } } diff --git a/PolicyEngine/Capabilities/enterprise/repo-rulesets.capability.json b/PolicyEngine/Capabilities/enterprise/repo-rulesets.capability.json index f0d3733..75c35fa 100644 --- a/PolicyEngine/Capabilities/enterprise/repo-rulesets.capability.json +++ b/PolicyEngine/Capabilities/enterprise/repo-rulesets.capability.json @@ -4,5 +4,18 @@ "layer": "enterprise", "capability": "repo-rulesets", "authPreference": "github-app-first-fallback-enterprise-pat", - "description": "Maintains enterprise-level repository rulesets." + "description": "Maintains enterprise-level repository rulesets.", + "configSchema": { + "required": ["enterprise", "rulesets"], + "properties": { + "enterprise": { + "type": "string", + "description": "Enterprise slug to maintain rulesets on." + }, + "rulesets": { + "type": "array", + "description": "Ruleset definitions mapped directly to GitHub enterprise ruleset payloads." + } + } + } } diff --git a/PolicyEngine/Policies/MSXOrg/enterprise/repo-custom-property.policy.json b/PolicyEngine/Policies/MSXOrg/enterprise/repo-custom-property.policy.json index 7f41256..07462ee 100644 --- a/PolicyEngine/Policies/MSXOrg/enterprise/repo-custom-property.policy.json +++ b/PolicyEngine/Policies/MSXOrg/enterprise/repo-custom-property.policy.json @@ -7,6 +7,9 @@ "config": { "enterprise": "MSXOrg", "typePropertyName": "Type", - "subscriptionPropertyName": "SubscribeTo" + "subscriptionPropertyName": "SubscribeTo", + "required": false, + "valuesEditableBy": "org_actors", + "requireExplicitValues": false } } diff --git a/PolicyEngine/Policies/MSXOrg/enterprise/repo-rulesets.policy.json b/PolicyEngine/Policies/MSXOrg/enterprise/repo-rulesets.policy.json index 0983d4c..32464d2 100644 --- a/PolicyEngine/Policies/MSXOrg/enterprise/repo-rulesets.policy.json +++ b/PolicyEngine/Policies/MSXOrg/enterprise/repo-rulesets.policy.json @@ -6,6 +6,60 @@ "enabled": true, "config": { "enterprise": "MSXOrg", - "rulesets": [] + "rulesets": [ + { + "name": "Custo default branch policy", + "target": "branch", + "enforcement": "disabled", + "conditions": { + "organization_name_and_repository_property": { + "organization_name": { + "include": [ + "~ALL" + ], + "exclude": [] + }, + "repository_property": { + "include": [ + { + "name": "Type", + "property_values": [ + "Module" + ], + "source": "custom" + } + ], + "exclude": [] + }, + "ref_name": { + "include": [ + "~DEFAULT_BRANCH" + ], + "exclude": [] + } + } + }, + "rules": [ + { + "type": "required_linear_history" + }, + { + "type": "pull_request", + "parameters": { + "dismiss_stale_reviews_on_push": true, + "require_code_owner_review": false, + "require_last_push_approval": false, + "required_approving_review_count": 1, + "required_review_thread_resolution": true, + "allowed_merge_methods": [ + "merge", + "squash", + "rebase" + ] + } + } + ] + } + ] } } diff --git a/README.md b/README.md index fbdd9d0..f3a79c5 100644 --- a/README.md +++ b/README.md @@ -56,7 +56,7 @@ The [`scripts/Sync-Files.ps1`](scripts/Sync-Files.ps1) script, run by the 4. Applies policy controls in order: **Enterprise** first, then **Organization**, then **Repository**. 5. Runs enterprise policy capabilities such as: - `repo-custom-property` (maintain `Type` and `SubscribeTo` enterprise property definitions) - - `repo-rulesets` (maintain enterprise repository rulesets) + - `repo-rulesets` (maintain enterprise repository rulesets; policy payload controls conditions/rules) 6. Discovers subscribing repositories from all accessible repositories (or explicit organizations when configured) and reads their `Type` and `SubscribeTo` custom properties. 7. Executes repository capability `file-subscription-service`, cloning repositories, syncing files, @@ -92,6 +92,11 @@ Configure these repository secrets before enabling the scheduled sync: - `CUSTO_BOT_PRIVATE_KEY` - `CUSTO_ENTERPRISE_PAT` (optional fallback PAT for enterprise policy endpoints, e.g. enterprise rulesets, with `admin:enterprise`) +Default enterprise policy configs live at: + +- `PolicyEngine/Policies/MSXOrg/enterprise/repo-custom-property.policy.json` +- `PolicyEngine/Policies/MSXOrg/enterprise/repo-rulesets.policy.json` + See [`AGENTS.md`](AGENTS.md) for operator runbook steps and current rollout blockers. ## License diff --git a/scripts/Sync-Files.ps1 b/scripts/Sync-Files.ps1 index bfb7fb8..ea69f9d 100644 --- a/scripts/Sync-Files.ps1 +++ b/scripts/Sync-Files.ps1 @@ -312,7 +312,10 @@ function Sync-EnterpriseCustomPropertySchema { [object]$Context, [string]$TypePropertyName = 'Type', - [string]$SubscriptionPropertyName = 'SubscribeTo' + [string]$SubscriptionPropertyName = 'SubscribeTo', + [bool]$Required = $false, + [string]$ValuesEditableBy = 'org_actors', + [bool]$RequireExplicitValues = $false ) $typeValues = @($FileSets.Keys | Sort-Object -Unique) @@ -332,18 +335,22 @@ function Sync-EnterpriseCustomPropertySchema { Invoke-EnterprisePolicyApi -Method PUT -ApiEndpoint "/enterprises/$Enterprise/properties/schema/$TypePropertyName" -Body @{ value_type = 'single_select' - required = $false + required = $Required default_value = $null description = 'Repository type used by Custo managed-file distribution.' allowed_values = $typeValues + values_editable_by = $ValuesEditableBy + require_explicit_values = $RequireExplicitValues } -Context $Context | Out-Null Invoke-EnterprisePolicyApi -Method PUT -ApiEndpoint "/enterprises/$Enterprise/properties/schema/$SubscriptionPropertyName" -Body @{ value_type = 'multi_select' - required = $false + required = $Required default_value = $null description = 'Managed file sets the repository subscribes to from Custo.' allowed_values = $subscriptionValues + values_editable_by = $ValuesEditableBy + require_explicit_values = $RequireExplicitValues } -Context $Context | Out-Null Write-Host "✅ Synced enterprise custom-property schema on '$Enterprise'" @@ -552,13 +559,19 @@ function Invoke-PolicyEngine { $typeName = if ($policy.config.typePropertyName) { $policy.config.typePropertyName } else { 'Type' } $subscriptionName = if ($policy.config.subscriptionPropertyName) { $policy.config.subscriptionPropertyName } else { 'SubscribeTo' } + $required = if ($policy.config.ContainsKey('required')) { [bool]$policy.config.required } else { $false } + $valuesEditableBy = if ($policy.config.valuesEditableBy) { [string]$policy.config.valuesEditableBy } else { 'org_actors' } + $requireExplicitValues = if ($policy.config.ContainsKey('requireExplicitValues')) { [bool]$policy.config.requireExplicitValues } else { $false } Sync-EnterpriseCustomPropertySchema ` -Enterprise $enterprise ` -FileSets $FileSets ` -Context $Context ` -TypePropertyName $typeName ` - -SubscriptionPropertyName $subscriptionName + -SubscriptionPropertyName $subscriptionName ` + -Required $required ` + -ValuesEditableBy $valuesEditableBy ` + -RequireExplicitValues $requireExplicitValues } 'enterprise.repo-rulesets' { $enterprise = $policy.config.enterprise From 8bba4c7a0088db91289462ca5ef463c3e3ef3384 Mon Sep 17 00:00:00 2001 From: Marius Storhaug Date: Sat, 25 Jul 2026 21:20:14 +0200 Subject: [PATCH 13/20] refactor: align layered policy paths Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- AGENTS.md | 4 +- .../repo-custom-property.policy.json | 1 + .../enterprise/repo-rulesets.policy.json | 1 + .../{ => _default}/none.policy.json | 1 + .../file-subscription-service.policy.json | 14 + .../file-subscription-service.policy.json | 8 - README.md | 7 +- scripts/Sync-Files.ps1 | 846 +++++++++--------- 8 files changed, 470 insertions(+), 412 deletions(-) rename PolicyEngine/Policies/MSXOrg/organization/{ => _default}/none.policy.json (91%) create mode 100644 PolicyEngine/Policies/MSXOrg/repository/_default/file-subscription-service.policy.json delete mode 100644 PolicyEngine/Policies/MSXOrg/repository/file-subscription-service.policy.json diff --git a/AGENTS.md b/AGENTS.md index bbd44c4..b34ccfd 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -52,8 +52,8 @@ Current default enterprise mapping (`MSXOrg`): - `PolicyEngine/Capabilities/repository/file-subscription-service.capability.json` - `PolicyEngine/Policies/MSXOrg/enterprise/repo-custom-property.policy.json` - `PolicyEngine/Policies/MSXOrg/enterprise/repo-rulesets.policy.json` -- `PolicyEngine/Policies/MSXOrg/organization/none.policy.json` -- `PolicyEngine/Policies/MSXOrg/repository/file-subscription-service.policy.json` +- `PolicyEngine/Policies/MSXOrg/organization/_default/none.policy.json` +- `PolicyEngine/Policies/MSXOrg/repository/_default/file-subscription-service.policy.json` Until step 1 is complete, the workflow will fail at the authentication step (`Connect-GitHub App`) — this is the current, expected blocker for this MVP. diff --git a/PolicyEngine/Policies/MSXOrg/enterprise/repo-custom-property.policy.json b/PolicyEngine/Policies/MSXOrg/enterprise/repo-custom-property.policy.json index 07462ee..be7bf37 100644 --- a/PolicyEngine/Policies/MSXOrg/enterprise/repo-custom-property.policy.json +++ b/PolicyEngine/Policies/MSXOrg/enterprise/repo-custom-property.policy.json @@ -3,6 +3,7 @@ "name": "Enterprise repo custom property policy", "layer": "enterprise", "capability": "repo-custom-property", + "order": 10, "enabled": true, "config": { "enterprise": "MSXOrg", diff --git a/PolicyEngine/Policies/MSXOrg/enterprise/repo-rulesets.policy.json b/PolicyEngine/Policies/MSXOrg/enterprise/repo-rulesets.policy.json index 32464d2..2f461b0 100644 --- a/PolicyEngine/Policies/MSXOrg/enterprise/repo-rulesets.policy.json +++ b/PolicyEngine/Policies/MSXOrg/enterprise/repo-rulesets.policy.json @@ -3,6 +3,7 @@ "name": "Enterprise repo ruleset policy", "layer": "enterprise", "capability": "repo-rulesets", + "order": 20, "enabled": true, "config": { "enterprise": "MSXOrg", diff --git a/PolicyEngine/Policies/MSXOrg/organization/none.policy.json b/PolicyEngine/Policies/MSXOrg/organization/_default/none.policy.json similarity index 91% rename from PolicyEngine/Policies/MSXOrg/organization/none.policy.json rename to PolicyEngine/Policies/MSXOrg/organization/_default/none.policy.json index b7ffc7f..7a06f89 100644 --- a/PolicyEngine/Policies/MSXOrg/organization/none.policy.json +++ b/PolicyEngine/Policies/MSXOrg/organization/_default/none.policy.json @@ -3,6 +3,7 @@ "name": "Organization policy placeholder", "layer": "organization", "capability": "none", + "order": 10, "enabled": true, "config": {} } diff --git a/PolicyEngine/Policies/MSXOrg/repository/_default/file-subscription-service.policy.json b/PolicyEngine/Policies/MSXOrg/repository/_default/file-subscription-service.policy.json new file mode 100644 index 0000000..a47a4cc --- /dev/null +++ b/PolicyEngine/Policies/MSXOrg/repository/_default/file-subscription-service.policy.json @@ -0,0 +1,14 @@ +{ + "version": "1.0", + "name": "Repository file subscription service", + "layer": "repository", + "capability": "file-subscription-service", + "order": 10, + "enabled": true, + "config": { + "branchName": "managed-files/update", + "commitMessage": "chore: sync managed files", + "prTitle": "⚙️ [Maintenance]: Sync managed files", + "prLabel": "NoRelease" + } +} diff --git a/PolicyEngine/Policies/MSXOrg/repository/file-subscription-service.policy.json b/PolicyEngine/Policies/MSXOrg/repository/file-subscription-service.policy.json deleted file mode 100644 index f569483..0000000 --- a/PolicyEngine/Policies/MSXOrg/repository/file-subscription-service.policy.json +++ /dev/null @@ -1,8 +0,0 @@ -{ - "version": "1.0", - "name": "Repository file subscription service", - "layer": "repository", - "capability": "file-subscription-service", - "enabled": true, - "config": {} -} diff --git a/README.md b/README.md index f3a79c5..bcc2ed2 100644 --- a/README.md +++ b/README.md @@ -44,8 +44,9 @@ Policy behavior is defined by JSON documents under [`PolicyEngine/`](PolicyEngin in `targets.json`. This separates **capabilities** from **policy configuration**: - `PolicyEngine/Capabilities/{layer}/*.capability.json` defines what each capability does. -- `PolicyEngine/Policies/{enterprise}/{layer}/*.policy.json` defines how that capability is - configured for a specific enterprise. +- `PolicyEngine/Policies/{enterprise}/{layer}/...` defines how that capability is configured for + a specific enterprise. Layer policies can be placed at `_default` plus increasingly specific + paths (`organization/{org}`, `repository/{org}/_default`, `repository/{org}/{repo}`). The [`scripts/Sync-Files.ps1`](scripts/Sync-Files.ps1) script, run by the [`Sync Managed Files`](.github/workflows/sync-files.yml) workflow: @@ -96,6 +97,8 @@ Default enterprise policy configs live at: - `PolicyEngine/Policies/MSXOrg/enterprise/repo-custom-property.policy.json` - `PolicyEngine/Policies/MSXOrg/enterprise/repo-rulesets.policy.json` +- `PolicyEngine/Policies/MSXOrg/organization/_default/none.policy.json` +- `PolicyEngine/Policies/MSXOrg/repository/_default/file-subscription-service.policy.json` See [`AGENTS.md`](AGENTS.md) for operator runbook steps and current rollout blockers. diff --git a/scripts/Sync-Files.ps1 b/scripts/Sync-Files.ps1 index ea69f9d..76900c0 100644 --- a/scripts/Sync-Files.ps1 +++ b/scripts/Sync-Files.ps1 @@ -1,31 +1,29 @@ #!/usr/bin/env pwsh <# .SYNOPSIS - Syncs managed files from this repository to subscribing repositories. + Policy-driven managed file and governance sync runtime. .DESCRIPTION - This script runs a layered policy engine and then performs repository file sync: - 1. Authenticate as GitHub App for repository-level operations. - 2. Discover managed file sets from Repos/. - 3. Read discovery scope from config/targets.json. - 4. Load capability and policy documents from PolicyEngine/. - 5. Apply policies in order: - - Enterprise layer - - Organization layer - - Repository layer - 6. If repository file-subscription policy is enabled, sync files to subscribing repositories. - -.NOTES - Enterprise policy API calls prefer GitHub App context first, then fall back to - CUSTO_ENTERPRISE_PAT only when App access fails and PAT is configured. + Execution model: + 0) GitHub App bootstrap auth (ClientID + PrivateKey) + 1) Enterprise layer: execute enterprise policies in order + 2) Organization layer: for each org installation, execute org policies in order + 3) Repository layer: for each repo in each org, execute repo policies in order + + Policies are loaded dynamically from PolicyEngine/Policies/{enterprise}/... + Capabilities are loaded dynamically from PolicyEngine/Capabilities/... + + Enterprise policy API calls prefer GitHub App and fall back to CUSTO_ENTERPRISE_PAT when + App access is unavailable. #> [CmdletBinding()] -param() +param( + [switch]$WhatIf +) $ErrorActionPreference = 'Stop' -# Track summary information $script:Summary = @{ TotalReposProcessed = 0 PRsCreated = 0 @@ -35,8 +33,6 @@ $script:Summary = @{ Errors = @() } -#region Helper Functions - function Get-TargetScope { [CmdletBinding()] param( @@ -61,15 +57,54 @@ function Get-TargetScope { throw "Unsupported scope '$($config.scope)' in: $ConfigPath" } - if ($config.policy) { - if (-not $config.policy.enterprise) { - throw "Config policy section requires 'enterprise'." - } + if (-not $config.policy) { + throw "Missing required 'policy' section in: $ConfigPath" + } + + if (-not $config.policy.enterprise) { + throw "Config policy section requires 'enterprise'." } return $config } +function Get-FileSets { + [CmdletBinding()] + param( + [Parameter(Mandatory)] + [string]$ReposPath + ) + + $fileSets = @{} + if (-not (Test-Path $ReposPath)) { + throw "Repos directory not found at: $ReposPath" + } + + $typeDirs = Get-ChildItem -Path $ReposPath -Directory + foreach ($typeDir in $typeDirs) { + $typeName = $typeDir.Name + $fileSets[$typeName] = @{} + $selectionDirs = Get-ChildItem -Path $typeDir.FullName -Directory + + foreach ($selectionDir in $selectionDirs) { + $selectionName = $selectionDir.Name + $files = Get-ChildItem -Path $selectionDir.FullName -File -Recurse + + $fileList = @() + foreach ($file in $files) { + $relativePath = $file.FullName.Substring($selectionDir.FullName.Length + 1) + $fileList += @{ + SourcePath = $file.FullName + RelativePath = $relativePath + } + } + $fileSets[$typeName][$selectionName] = $fileList + } + } + + return $fileSets +} + function Get-CapabilityCatalog { [CmdletBinding()] param( @@ -87,11 +122,10 @@ function Get-CapabilityCatalog { } $catalog = @{} - foreach ($capabilityFile in $capabilityFiles) { - $doc = Get-Content -Path $capabilityFile.FullName -Raw | ConvertFrom-Json -AsHashtable - + foreach ($file in $capabilityFiles) { + $doc = Get-Content -Path $file.FullName -Raw | ConvertFrom-Json -AsHashtable if (-not $doc.layer -or -not $doc.capability) { - throw "Capability '$($capabilityFile.Name)' must define both layer and capability." + throw "Capability '$($file.Name)' must define both layer and capability." } $layer = $doc.layer.ToLowerInvariant() @@ -99,36 +133,30 @@ function Get-CapabilityCatalog { $key = "$layer.$capability" if ($catalog.ContainsKey($key)) { - throw "Duplicate capability definition for '$key': $($capabilityFile.FullName)" + throw "Duplicate capability definition for '$key': $($file.FullName)" } $doc.layer = $layer $doc.capability = $capability - $doc.sourceFile = $capabilityFile.FullName + $doc.sourceFile = $file.FullName $catalog[$key] = $doc } return $catalog } -function Get-PolicyDocuments { +function Get-PolicyDocumentsFromPaths { [CmdletBinding()] param( [Parameter(Mandatory)] - [string]$PolicyPath, + [string[]]$Paths, [Parameter(Mandatory)] - [hashtable]$CapabilityCatalog - ) + [hashtable]$CapabilityCatalog, - if (-not (Test-Path $PolicyPath)) { - throw "Policy path not found at: $PolicyPath" - } - - $policyFiles = Get-ChildItem -Path $PolicyPath -File -Recurse -Filter '*.policy.json' | Sort-Object FullName - if ($policyFiles.Count -eq 0) { - throw "No policy documents found in: $PolicyPath" - } + [Parameter(Mandatory)] + [string]$Layer + ) $layerOrder = @{ enterprise = 1 @@ -137,85 +165,120 @@ function Get-PolicyDocuments { } $documents = @() - foreach ($policyFile in $policyFiles) { - $doc = Get-Content -Path $policyFile.FullName -Raw | ConvertFrom-Json -AsHashtable - - if (-not $doc.layer) { - throw "Policy '$($policyFile.Name)' is missing required field 'layer'." - } - if (-not $doc.capability) { - throw "Policy '$($policyFile.Name)' is missing required field 'capability'." - } - if (-not $doc.ContainsKey('enabled')) { - throw "Policy '$($policyFile.Name)' is missing required field 'enabled'." + foreach ($path in $Paths) { + if (-not (Test-Path $path)) { + continue } - $doc.layer = $doc.layer.ToLowerInvariant() - $doc.capability = $doc.capability.ToLowerInvariant() - $doc.sourceFile = $policyFile.FullName + $files = Get-ChildItem -Path $path -File -Filter '*.policy.json' | Sort-Object FullName + foreach ($file in $files) { + $doc = Get-Content -Path $file.FullName -Raw | ConvertFrom-Json -AsHashtable + if (-not $doc.layer -or -not $doc.capability -or -not $doc.ContainsKey('enabled')) { + throw "Policy '$($file.Name)' must include layer, capability, and enabled." + } - if (-not $layerOrder.ContainsKey($doc.layer)) { - throw "Policy '$($policyFile.Name)' has unsupported layer '$($doc.layer)'." - } + $doc.layer = $doc.layer.ToLowerInvariant() + $doc.capability = $doc.capability.ToLowerInvariant() + $doc.sourceFile = $file.FullName + if ($doc.layer -ne $Layer) { + continue + } - $capabilityKey = "$($doc.layer).$($doc.capability)" - if (-not $CapabilityCatalog.ContainsKey($capabilityKey)) { - throw "Policy '$($policyFile.Name)' references unsupported capability '$capabilityKey'." - } + $capabilityKey = "$($doc.layer).$($doc.capability)" + if (-not $CapabilityCatalog.ContainsKey($capabilityKey)) { + throw "Policy '$($file.Name)' references unsupported capability '$capabilityKey'." + } - $documents += $doc + $documents += $doc + } } - $documents | - Sort-Object @{ Expression = { $layerOrder[$_.layer] } }, @{ Expression = { $_.capability } }, @{ Expression = { $_.sourceFile } } + $documents | Sort-Object @{ Expression = { if ($_.ContainsKey('order')) { [int]$_.order } else { 1000 } } }, @{ Expression = { $layerOrder[$_.layer] } }, @{ Expression = { $_.capability } }, @{ Expression = { $_.sourceFile } } } -function Get-FileSets { +function Resolve-PolicyLayerPaths { [CmdletBinding()] param( [Parameter(Mandatory)] - [string]$ReposPath + [string]$PolicyRoot, + + [Parameter(Mandatory)] + [string]$Enterprise, + + [Parameter(Mandatory)] + [ValidateSet('enterprise', 'organization', 'repository')] + [string]$Layer, + + [string]$Organization, + [string]$Repository ) - $fileSets = @{} + $base = Join-Path (Join-Path $PolicyRoot 'Policies') $Enterprise + $paths = @() - if (-not (Test-Path $ReposPath)) { - throw "Repos directory not found at: $ReposPath" + switch ($Layer) { + 'enterprise' { + $paths += (Join-Path $base 'enterprise') + } + 'organization' { + $paths += (Join-Path $base 'organization') + $paths += (Join-Path (Join-Path $base 'organization') '_default') + if ($Organization) { + $paths += (Join-Path (Join-Path $base 'organization') $Organization) + } + } + 'repository' { + $paths += (Join-Path $base 'repository') + $paths += (Join-Path (Join-Path $base 'repository') '_default') + if ($Organization) { + $orgBase = Join-Path (Join-Path $base 'repository') $Organization + $paths += (Join-Path $orgBase '_default') + if ($Repository) { + $paths += (Join-Path $orgBase $Repository) + } + } + } } - $typeDirs = Get-ChildItem -Path $ReposPath -Directory - $fileSetTable = @() + return $paths +} - foreach ($typeDir in $typeDirs) { - $typeName = $typeDir.Name - $fileSets[$typeName] = @{} +function ConvertTo-NormalizedJson { + [CmdletBinding()] + param( + [Parameter(Mandatory)] + $InputObject + ) - $selectionDirs = Get-ChildItem -Path $typeDir.FullName -Directory + return ($InputObject | ConvertTo-Json -Depth 50) +} - foreach ($selectionDir in $selectionDirs) { - $selectionName = $selectionDir.Name - $files = Get-ChildItem -Path $selectionDir.FullName -File -Recurse +function Write-ConfigDiff { + [CmdletBinding()] + param( + [Parameter(Mandatory)] + [string]$Title, - $fileList = @() - foreach ($file in $files) { - $relativePath = $file.FullName.Substring($selectionDir.FullName.Length + 1) - $fileList += @{ - SourcePath = $file.FullName - RelativePath = $relativePath - } - } + $Current, + $Desired + ) - $fileSets[$typeName][$selectionName] = $fileList - $fileSetTable += [PSCustomObject]@{ - Type = $typeName - FileSet = $selectionName - Files = $fileList.Count - } - } + $currentJson = ConvertTo-NormalizedJson -InputObject $Current + $desiredJson = ConvertTo-NormalizedJson -InputObject $Desired + + if ($currentJson -eq $desiredJson) { + Write-Host "✅ $Title already matches desired state" + return $false } - $fileSetTable | Format-Table -AutoSize | Out-String - return $fileSets + Write-Host "🔎 Diff for $Title" + $currentLines = $currentJson -split "`r?`n" + $desiredLines = $desiredJson -split "`r?`n" + $diff = Compare-Object -ReferenceObject $currentLines -DifferenceObject $desiredLines + foreach ($line in $diff) { + Write-Host " $($line.SideIndicator) $($line.InputObject)" + } + return $true } function Invoke-EnterprisePolicyApi { @@ -259,44 +322,60 @@ function Invoke-EnterprisePolicyApi { if ($null -ne $Body) { $invokeArgs.ContentType = 'application/json' - $invokeArgs.Body = $Body | ConvertTo-Json -Depth 30 + $invokeArgs.Body = $Body | ConvertTo-Json -Depth 50 } return Invoke-RestMethod @invokeArgs } } -function Get-AllAccessibleRepository { +function Get-OrganizationContext { [CmdletBinding()] param( [Parameter(Mandatory)] - [object]$Context + [string]$Organization + ) + + return (Connect-GitHubApp -Owner $Organization -PassThru) +} + +function Get-TargetOrganizations { + [CmdletBinding()] + param( + [Parameter(Mandatory)] + [pscustomobject]$TargetScope, + + [Parameter(Mandatory)] + [object]$RootContext ) - $allRepos = @() - $page = 1 - $perPage = 100 + if ($TargetScope.scope -eq 'organizations') { + return @($TargetScope.organizations | Sort-Object -Unique) + } - while ($true) { - $response = (Invoke-GitHubAPI -Method GET -ApiEndpoint '/installation/repositories' -Body @{ - per_page = $perPage - page = $page - } -Context $Context).Response + $installations = @((Invoke-GitHubAPI -Method GET -ApiEndpoint '/app/installations' -Context $RootContext).Response) + $orgs = $installations | + Where-Object { $_.account.type -eq 'Organization' } | + ForEach-Object { $_.account.login } | + Sort-Object -Unique - if (-not $response.repositories -or $response.repositories.Count -eq 0) { - break - } + return @($orgs) +} - $allRepos += $response.repositories +function Get-RepositoryCustomProperties { + [CmdletBinding()] + param( + [Parameter(Mandatory)] + [string]$Owner, - if ($response.repositories.Count -lt $perPage) { - break - } + [Parameter(Mandatory)] + [string]$Repo, - $page++ - } + [Parameter(Mandatory)] + [object]$Context + ) - return $allRepos + return @((Invoke-GitHubAPI -Method GET -ApiEndpoint "/repos/$Owner/$Repo/properties/values" -Context $Context).Response) } function Sync-EnterpriseCustomPropertySchema { @@ -314,8 +393,9 @@ function Sync-EnterpriseCustomPropertySchema { [string]$TypePropertyName = 'Type', [string]$SubscriptionPropertyName = 'SubscribeTo', [bool]$Required = $false, - [string]$ValuesEditableBy = 'org_actors', - [bool]$RequireExplicitValues = $false + [object]$ValuesEditableBy = 'org_actors', + [bool]$RequireExplicitValues = $false, + [switch]$WhatIf ) $typeValues = @($FileSets.Keys | Sort-Object -Unique) @@ -323,39 +403,53 @@ function Sync-EnterpriseCustomPropertySchema { throw 'Cannot sync custom-property schema: no repository types discovered in Repos/.' } - $subscriptionValues = @( - $FileSets.Values | - ForEach-Object { $_.Keys } | - Sort-Object -Unique - ) - + $subscriptionValues = @($FileSets.Values | ForEach-Object { $_.Keys } | Sort-Object -Unique) if ($subscriptionValues.Count -eq 0) { throw 'Cannot sync custom-property schema: no file-set selections discovered in Repos/.' } - Invoke-EnterprisePolicyApi -Method PUT -ApiEndpoint "/enterprises/$Enterprise/properties/schema/$TypePropertyName" -Body @{ - value_type = 'single_select' - required = $Required - default_value = $null - description = 'Repository type used by Custo managed-file distribution.' - allowed_values = $typeValues - values_editable_by = $ValuesEditableBy + $desiredType = @{ + value_type = 'single_select' + required = $Required + default_value = $null + description = 'Repository type used by Custo managed-file distribution.' + allowed_values = $typeValues + values_editable_by = $ValuesEditableBy require_explicit_values = $RequireExplicitValues - } -Context $Context | Out-Null - - Invoke-EnterprisePolicyApi -Method PUT -ApiEndpoint "/enterprises/$Enterprise/properties/schema/$SubscriptionPropertyName" -Body @{ - value_type = 'multi_select' - required = $Required - default_value = $null - description = 'Managed file sets the repository subscribes to from Custo.' - allowed_values = $subscriptionValues - values_editable_by = $ValuesEditableBy + } + + $desiredSubscription = @{ + value_type = 'multi_select' + required = $Required + default_value = $null + description = 'Managed file sets the repository subscribes to from Custo.' + allowed_values = $subscriptionValues + values_editable_by = $ValuesEditableBy require_explicit_values = $RequireExplicitValues - } -Context $Context | Out-Null + } + + $currentType = $null + $currentSubscription = $null + try { $currentType = Invoke-EnterprisePolicyApi -Method GET -ApiEndpoint "/enterprises/$Enterprise/properties/schema/$TypePropertyName" -Context $Context } catch {} + try { $currentSubscription = Invoke-EnterprisePolicyApi -Method GET -ApiEndpoint "/enterprises/$Enterprise/properties/schema/$SubscriptionPropertyName" -Context $Context } catch {} + + $typeChanged = Write-ConfigDiff -Title "enterprise property '$TypePropertyName'" -Current $currentType -Desired $desiredType + $subscriptionChanged = Write-ConfigDiff -Title "enterprise property '$SubscriptionPropertyName'" -Current $currentSubscription -Desired $desiredSubscription + + if ($WhatIf) { + Write-Host "WhatIf: skipping enterprise property updates" + return + } + + if ($typeChanged) { + Invoke-EnterprisePolicyApi -Method PUT -ApiEndpoint "/enterprises/$Enterprise/properties/schema/$TypePropertyName" -Body $desiredType -Context $Context | Out-Null + Write-Host "✅ Updated enterprise property '$TypePropertyName'" + } - Write-Host "✅ Synced enterprise custom-property schema on '$Enterprise'" - Write-Host " - ${TypePropertyName}: $($typeValues -join ', ')" - Write-Host " - ${SubscriptionPropertyName}: $($subscriptionValues -join ', ')" + if ($subscriptionChanged) { + Invoke-EnterprisePolicyApi -Method PUT -ApiEndpoint "/enterprises/$Enterprise/properties/schema/$SubscriptionPropertyName" -Body $desiredSubscription -Context $Context | Out-Null + Write-Host "✅ Updated enterprise property '$SubscriptionPropertyName'" + } } function Sync-EnterpriseRulesets { @@ -368,7 +462,9 @@ function Sync-EnterpriseRulesets { [object[]]$Rulesets, [Parameter(Mandatory)] - [object]$Context + [object]$Context, + + [switch]$WhatIf ) if (-not $Rulesets -or $Rulesets.Count -eq 0) { @@ -392,209 +488,57 @@ function Sync-EnterpriseRulesets { throw "Enterprise ruleset entry is missing required field 'name'." } - $payload = @{} + $desired = @{} foreach ($key in $ruleset.Keys) { - if ($key -ne 'id' -and $key -ne 'source' -and $key -ne 'source_type' -and $key -ne 'created_at' -and $key -ne 'updated_at') { - $payload[$key] = $ruleset[$key] + if ($key -notin @('id', 'source', 'source_type', 'created_at', 'updated_at', 'node_id', '_links', 'current_user_can_bypass')) { + $desired[$key] = $ruleset[$key] } } + $currentComparable = $null + $endpoint = "/enterprises/$Enterprise/rulesets" + $method = 'POST' if ($existingByName.ContainsKey($ruleset.name)) { - $rulesetId = $existingByName[$ruleset.name].id - Invoke-EnterprisePolicyApi -Method PUT -ApiEndpoint "/enterprises/$Enterprise/rulesets/$rulesetId" -Body $payload -Context $Context | Out-Null - Write-Host "✅ Updated enterprise ruleset '$($ruleset.name)'" - } else { - Invoke-EnterprisePolicyApi -Method POST -ApiEndpoint "/enterprises/$Enterprise/rulesets" -Body $payload -Context $Context | Out-Null - Write-Host "✅ Created enterprise ruleset '$($ruleset.name)'" - } - } -} - -function Get-SubscribingRepository { - [CmdletBinding()] - param( - [Parameter(Mandatory)] - [string]$Owner, - - [Parameter(Mandatory)] - [object]$Context - ) - - $repos = Get-GitHubRepository -Owner $Owner -Context $Context - $subscribingRepos = @() - - foreach ($repo in $repos) { - $customProps = $repo.CustomProperties - if (-not $customProps) { continue } - - $type = ($customProps | Where-Object Name -EQ 'Type').Value - $subscribeTo = ($customProps | Where-Object Name -EQ 'SubscribeTo').Value - - if (-not $type -or -not $subscribeTo) { continue } - if ($subscribeTo -is [string]) { $subscribeTo = @($subscribeTo) } - if ($subscribeTo.Count -eq 0) { continue } - - $subscribingRepos += @{ - Name = $repo.Name - Owner = $repo.Owner.Login - FullName = $repo.FullName - Type = $type - SubscribeTo = $subscribeTo - DefaultBranch = $repo.DefaultBranch + $existing = $existingByName[$ruleset.name] + $currentComparable = @{} + foreach ($key in $desired.Keys) { + if ($existing.PSObject.Properties.Name -contains $key) { + $currentComparable[$key] = $existing.$key + } + } + $endpoint = "/enterprises/$Enterprise/rulesets/$($existing.id)" + $method = 'PUT' } - } - - return $subscribingRepos -} - -function Get-SubscribingRepositoryByOrganizationFromAllAccess { - [CmdletBinding()] - param( - [Parameter(Mandatory)] - [object]$Context - ) - $allRepos = Get-AllAccessibleRepository -Context $Context - $reposByOrg = @{} - - foreach ($repo in $allRepos) { - if ($repo.owner.type -ne 'Organization') { continue } - - $owner = $repo.owner.login - $repoName = $repo.name - - $customProps = (Invoke-GitHubAPI -Method GET -ApiEndpoint "/repos/$owner/$repoName/properties/values" -Context $Context).Response - $typeProp = $customProps | Where-Object { $_.property_name -eq 'Type' } - $subscribeToProp = $customProps | Where-Object { $_.property_name -eq 'SubscribeTo' } - - $type = $typeProp.value - $subscribeTo = $subscribeToProp.value - - if (-not $type -or -not $subscribeTo) { continue } - if ($subscribeTo -is [string]) { $subscribeTo = @($subscribeTo) } - if ($subscribeTo.Count -eq 0) { continue } - - $repoEntry = @{ - Name = $repoName - Owner = $owner - FullName = $repo.full_name - Type = $type - SubscribeTo = $subscribeTo - DefaultBranch = $repo.default_branch + $changed = Write-ConfigDiff -Title "enterprise ruleset '$($ruleset.name)'" -Current $currentComparable -Desired $desired + if (-not $changed) { + continue } - if (-not $reposByOrg.ContainsKey($owner)) { - $reposByOrg[$owner] = @() + if ($WhatIf) { + Write-Host "WhatIf: skipping ruleset write for '$($ruleset.name)'" + continue } - $reposByOrg[$owner] += $repoEntry - } - return $reposByOrg -} - -function Get-AllSubscribingRepository { - [CmdletBinding()] - param( - [Parameter(Mandatory)] - [pscustomobject]$TargetScope, - - [Parameter(Mandatory)] - [object]$Context - ) - - $subscribingRepos = @() - - if ($TargetScope.scope -eq 'all-access') { - $reposByOrg = Get-SubscribingRepositoryByOrganizationFromAllAccess -Context $Context - foreach ($org in @($reposByOrg.Keys | Sort-Object)) { - Write-Host "Found $($reposByOrg[$org].Count) subscribing repositories in $org" - $subscribingRepos += $reposByOrg[$org] - } - } else { - foreach ($org in $TargetScope.organizations) { - $orgRepos = Get-SubscribingRepository -Owner $org -Context $Context - Write-Host "Found $($orgRepos.Count) subscribing repositories in $org" - $subscribingRepos += $orgRepos - } + Invoke-EnterprisePolicyApi -Method $method -ApiEndpoint $endpoint -Body $desired -Context $Context | Out-Null + Write-Host "✅ Upserted enterprise ruleset '$($ruleset.name)'" } - - return $subscribingRepos } -function Invoke-PolicyEngine { +function Get-RepositoriesForOrganization { [CmdletBinding()] param( [Parameter(Mandatory)] - [object[]]$Policies, - - [Parameter(Mandatory)] - [hashtable]$FileSets, - - [Parameter(Mandatory)] - [pscustomobject]$TargetScope, + [string]$Organization, [Parameter(Mandatory)] [object]$Context ) - $state = @{ - RepoFileSubscriptionEnabled = $false - } - - foreach ($policy in $Policies) { - $name = if ($policy.name) { $policy.name } else { [System.IO.Path]::GetFileName($policy.sourceFile) } - $capabilityId = "$($policy.layer).$($policy.capability)" - - if ($policy.enabled -ne $true) { - Write-Host "⏭️ Policy disabled: $name ($capabilityId)" - continue - } - - Write-Host "▶️ Applying policy: $name ($capabilityId)" - - switch ($capabilityId) { - 'enterprise.repo-custom-property' { - $enterprise = $policy.config.enterprise - if (-not $enterprise) { throw "Policy '$name' requires config.enterprise." } - - $typeName = if ($policy.config.typePropertyName) { $policy.config.typePropertyName } else { 'Type' } - $subscriptionName = if ($policy.config.subscriptionPropertyName) { $policy.config.subscriptionPropertyName } else { 'SubscribeTo' } - $required = if ($policy.config.ContainsKey('required')) { [bool]$policy.config.required } else { $false } - $valuesEditableBy = if ($policy.config.valuesEditableBy) { [string]$policy.config.valuesEditableBy } else { 'org_actors' } - $requireExplicitValues = if ($policy.config.ContainsKey('requireExplicitValues')) { [bool]$policy.config.requireExplicitValues } else { $false } - - Sync-EnterpriseCustomPropertySchema ` - -Enterprise $enterprise ` - -FileSets $FileSets ` - -Context $Context ` - -TypePropertyName $typeName ` - -SubscriptionPropertyName $subscriptionName ` - -Required $required ` - -ValuesEditableBy $valuesEditableBy ` - -RequireExplicitValues $requireExplicitValues - } - 'enterprise.repo-rulesets' { - $enterprise = $policy.config.enterprise - if (-not $enterprise) { throw "Policy '$name' requires config.enterprise." } - $rulesets = if ($policy.config.rulesets) { @($policy.config.rulesets) } else { @() } - Sync-EnterpriseRulesets -Enterprise $enterprise -Rulesets $rulesets -Context $Context - } - 'organization.none' { - Write-Host "ℹ️ Organization policy layer intentionally empty" - } - 'repository.file-subscription-service' { - $state.RepoFileSubscriptionEnabled = $true - } - default { - throw "Unsupported policy capability '$capabilityId' in '$name'." - } - } - } - - return $state + return @((Get-GitHubRepository -Owner $Organization -Context $Context)) } -function Sync-RepositoryFile { +function Sync-RepositoryFileSubscription { [Diagnostics.CodeAnalysis.SuppressMessageAttribute( 'PSAvoidUsingWriteHost', '', Scope = 'Function', Justification = 'Intended for logging in GitHub Actions runners.' @@ -626,7 +570,9 @@ function Sync-RepositoryFile { [string]$PRLabel, [Parameter(Mandatory)] - [object]$Context + [object]$Context, + + [switch]$WhatIf ) $repoFullName = $Repository.FullName @@ -659,12 +605,6 @@ function Sync-RepositoryFile { } LogGroup "📦 $repoFullName" { - foreach ($selection in $subscribeTo) { - if ($FileSets[$type].ContainsKey($selection)) { - Write-Host " + $type/$selection ($($FileSets[$type][$selection].Count) files)" - } - } - $clonePath = Join-Path $TempPath "clone-$repoName-$(Get-Random)" New-Item -Path $clonePath -ItemType Directory -Force | Out-Null @@ -703,8 +643,14 @@ function Sync-RepositoryFile { return } + Write-Host '🔎 Repository file diff' $status -split "`n" | ForEach-Object { Write-Host " $_" } + if ($WhatIf) { + Write-Host "WhatIf: skipping commit/push/PR for $repoFullName" + return + } + git add --all 2>&1 | Out-Null git commit -m $CommitMessage 2>&1 | Out-Null $pushResult = git push --force --set-upstream origin $BranchName 2>&1 @@ -739,11 +685,9 @@ function Sync-RepositoryFile { Write-Host "✅ Created PR #$($pr.number) - $($pr.html_url)" $script:Summary.PRsCreated++ } - } finally { Pop-Location } - } catch { Write-Host "❌ $_" $script:Summary.Errors += "$repoFullName : $_" @@ -755,96 +699,200 @@ function Sync-RepositoryFile { } } -#endregion +function Invoke-PoliciesForScope { + [CmdletBinding()] + param( + [Parameter(Mandatory)] + [object[]]$Policies, -#region Main Script + [Parameter(Mandatory)] + [hashtable]$FileSets, -try { - LogGroup '🔑 Authenticate' { - $context = Connect-GitHubApp -PassThru - $context | Format-List | Out-String - } + [Parameter(Mandatory)] + [object]$Context, - LogGroup '📂 Discover file sets' { - $reposPath = Join-Path $PSScriptRoot '../Repos' - $reposPath = Resolve-Path $reposPath - $fileSets = Get-FileSets -ReposPath $reposPath + [string]$Enterprise, + [string]$Organization, + [object]$Repository, + [Parameter(Mandatory)] + [string]$TempPath, + [switch]$WhatIf + ) + + foreach ($policy in $Policies) { + $name = if ($policy.name) { $policy.name } else { [System.IO.Path]::GetFileName($policy.sourceFile) } + $capabilityId = "$($policy.layer).$($policy.capability)" + + if ($policy.enabled -ne $true) { + Write-Host "⏭️ Policy disabled: $name ($capabilityId)" + continue + } + + Write-Host "▶️ Applying policy: $name ($capabilityId)" + + switch ($capabilityId) { + 'enterprise.repo-custom-property' { + $typeName = if ($policy.config.typePropertyName) { [string]$policy.config.typePropertyName } else { 'Type' } + $subscriptionName = if ($policy.config.subscriptionPropertyName) { [string]$policy.config.subscriptionPropertyName } else { 'SubscribeTo' } + $required = if ($policy.config.ContainsKey('required')) { [bool]$policy.config.required } else { $false } + $valuesEditableBy = if ($policy.config.ContainsKey('valuesEditableBy')) { $policy.config.valuesEditableBy } else { 'org_actors' } + $requireExplicitValues = if ($policy.config.ContainsKey('requireExplicitValues')) { [bool]$policy.config.requireExplicitValues } else { $false } + + Sync-EnterpriseCustomPropertySchema ` + -Enterprise $policy.config.enterprise ` + -FileSets $FileSets ` + -Context $Context ` + -TypePropertyName $typeName ` + -SubscriptionPropertyName $subscriptionName ` + -Required $required ` + -ValuesEditableBy $valuesEditableBy ` + -RequireExplicitValues $requireExplicitValues ` + -WhatIf:$WhatIf + } + 'enterprise.repo-rulesets' { + $rulesets = if ($policy.config.rulesets) { @($policy.config.rulesets) } else { @() } + Sync-EnterpriseRulesets -Enterprise $policy.config.enterprise -Rulesets $rulesets -Context $Context -WhatIf:$WhatIf + } + 'organization.none' { + Write-Host "ℹ️ Organization policy layer placeholder for $Organization" + } + 'repository.file-subscription-service' { + if (-not $Repository) { + throw "repository.file-subscription-service requires a repository scope." + } + + $owner = $Repository.Owner.Login + $repoName = $Repository.Name + $customProps = Get-RepositoryCustomProperties -Owner $owner -Repo $repoName -Context $Context + $typeProp = $customProps | Where-Object { $_.property_name -eq 'Type' } + $subscribeToProp = $customProps | Where-Object { $_.property_name -eq 'SubscribeTo' } + + $type = $typeProp.value + $subscribeTo = $subscribeToProp.value + + if (-not $type -or -not $subscribeTo) { + Write-Host "ℹ️ $owner/$repoName has no Type/SubscribeTo subscription, skipping." + $script:Summary.ReposSkipped++ + continue + } + if ($subscribeTo -is [string]) { $subscribeTo = @($subscribeTo) } + + $repoSpec = @{ + Name = $Repository.Name + Owner = $owner + FullName = $Repository.FullName + Type = $type + SubscribeTo = $subscribeTo + DefaultBranch = $Repository.DefaultBranch + } + + $branchName = if ($policy.config.branchName) { [string]$policy.config.branchName } else { 'managed-files/update' } + $commitMessage = if ($policy.config.commitMessage) { [string]$policy.config.commitMessage } else { 'chore: sync managed files' } + $prTitle = if ($policy.config.prTitle) { [string]$policy.config.prTitle } else { '⚙️ [Maintenance]: Sync managed files' } + $prLabel = if ($policy.config.prLabel) { [string]$policy.config.prLabel } else { 'NoRelease' } + $prBody = if ($policy.config.prBody) { + [string]$policy.config.prBody + } else { +@' +This pull request was automatically created by the [Custo](https://github.com/MSXOrg/Custo) workflow that keeps shared files in sync across the organization's repositories. + +The files in this PR are centrally managed. Any local changes to these files will be overwritten on the next sync. To propose changes, update the source files in the Custo repo instead. +'@ + } + + Sync-RepositoryFileSubscription ` + -Repository $repoSpec ` + -FileSets $FileSets ` + -TempPath $TempPath ` + -BranchName $branchName ` + -CommitMessage $commitMessage ` + -PRTitle $prTitle ` + -PRBody $prBody ` + -PRLabel $prLabel ` + -Context $Context ` + -WhatIf:$WhatIf + } + default { + throw "Unsupported policy capability '$capabilityId' in '$name'." + } + } } +} - if ($fileSets.Count -eq 0) { - Write-Host '⚠️ No file sets found - nothing to do' - exit 0 +try { + LogGroup '🔑 Authenticate bootstrap context' { + $rootContext = Connect-GitHubApp -PassThru + $rootContext | Format-List | Out-String } - LogGroup '🎯 Read target discovery scope' { - $targetsPath = Join-Path $PSScriptRoot '../config/targets.json' - $targetsPath = Resolve-Path $targetsPath + LogGroup '🎯 Read target and policy scope' { + $targetsPath = Resolve-Path (Join-Path $PSScriptRoot '../config/targets.json') $targetScope = Get-TargetScope -ConfigPath $targetsPath Write-Host "Target scope: $($targetScope.scope)" - if ($targetScope.scope -eq 'organizations') { - Write-Host "Target organizations: $($targetScope.organizations -join ', ')" + } + + LogGroup '📂 Discover managed file sets' { + $reposPath = Resolve-Path (Join-Path $PSScriptRoot '../Repos') + $fileSets = Get-FileSets -ReposPath $reposPath + if ($fileSets.Count -eq 0) { + throw 'No managed file sets discovered under Repos/.' } } - LogGroup '📜 Load policy documents' { + LogGroup '🧩 Load capabilities and policy documents' { $policyRoot = if ($targetScope.policy.rootPath) { $targetScope.policy.rootPath } else { '../PolicyEngine' } if (-not [System.IO.Path]::IsPathRooted($policyRoot)) { $policyRoot = Join-Path $PSScriptRoot $policyRoot } $policyRoot = Resolve-Path $policyRoot - $policyEnterprise = if ($targetScope.policy.enterprise) { $targetScope.policy.enterprise } else { 'default' } - $capabilityPath = Join-Path $policyRoot 'Capabilities' - $policyPath = Join-Path (Join-Path $policyRoot 'Policies') $policyEnterprise - - $capabilityCatalog = Get-CapabilityCatalog -CapabilityPath (Resolve-Path $capabilityPath) - $policies = Get-PolicyDocuments -PolicyPath (Resolve-Path $policyPath) -CapabilityCatalog $capabilityCatalog - Write-Host "Loaded $($policies.Count) policy documents for enterprise '$policyEnterprise'" + $policyEnterprise = [string]$targetScope.policy.enterprise + $capabilityCatalog = Get-CapabilityCatalog -CapabilityPath (Resolve-Path (Join-Path $policyRoot 'Capabilities')) + Write-Host "Loaded $($capabilityCatalog.Count) capability definitions" } - LogGroup '🧭 Execute policy engine' { - $policyState = Invoke-PolicyEngine -Policies $policies -FileSets $fileSets -TargetScope $targetScope -Context $context - } + $tempPath = Join-Path ([System.IO.Path]::GetTempPath()) "custo-sync-$(Get-Random)" + New-Item -Path $tempPath -ItemType Directory -Force | Out-Null - if ($policyState.RepoFileSubscriptionEnabled -ne $true) { - Write-Host 'ℹ️ Repository file-subscription policy is disabled; file sync skipped.' - exit 0 - } + try { + LogGroup '🏢 Enterprise layer' { + $enterprisePolicyPaths = Resolve-PolicyLayerPaths -PolicyRoot $policyRoot -Enterprise $policyEnterprise -Layer 'enterprise' + $enterprisePolicies = Get-PolicyDocumentsFromPaths -Paths $enterprisePolicyPaths -CapabilityCatalog $capabilityCatalog -Layer 'enterprise' + Write-Host "Enterprise policies loaded: $($enterprisePolicies.Count)" - LogGroup '🔍 Find subscribing repositories' { - $subscribingRepos = Get-AllSubscribingRepository -TargetScope $targetScope -Context $context - Write-Host "Found $($subscribingRepos.Count) subscribing repositories in total" - } + Invoke-PoliciesForScope -Policies $enterprisePolicies -FileSets $fileSets -Context $rootContext -Enterprise $policyEnterprise -TempPath $tempPath -WhatIf:$WhatIf + } - if ($subscribingRepos.Count -eq 0) { - Write-Host '⚠️ No subscribing repositories found - nothing to do' - exit 0 - } + LogGroup '🏢➡️🏬 Organization and repository layers' { + $organizations = Get-TargetOrganizations -TargetScope $targetScope -RootContext $rootContext + Write-Host "Organizations discovered: $($organizations -join ', ')" - $tempPath = Join-Path ([System.IO.Path]::GetTempPath()) "custo-sync-$(Get-Random)" - New-Item -Path $tempPath -ItemType Directory -Force | Out-Null + foreach ($org in $organizations) { + LogGroup "🔐 Connect org installation: $org" { + $orgContext = Get-OrganizationContext -Organization $org + } - $branchName = 'managed-files/update' - $commitMessage = 'chore: sync managed files' - $prTitle = '⚙️ [Maintenance]: Sync managed files' - $prLabel = 'NoRelease' - $prBody = @' -This pull request was automatically created by the [Custo](https://github.com/MSXOrg/Custo) workflow that keeps shared files in sync across the organization's repositories. + LogGroup "🏬 Organization policies: $org" { + $orgPolicyPaths = Resolve-PolicyLayerPaths -PolicyRoot $policyRoot -Enterprise $policyEnterprise -Layer 'organization' -Organization $org + $orgPolicies = Get-PolicyDocumentsFromPaths -Paths $orgPolicyPaths -CapabilityCatalog $capabilityCatalog -Layer 'organization' + Write-Host "Organization policies loaded: $($orgPolicies.Count)" + Invoke-PoliciesForScope -Policies $orgPolicies -FileSets $fileSets -Context $orgContext -Enterprise $policyEnterprise -Organization $org -TempPath $tempPath -WhatIf:$WhatIf + } -The files in this PR are centrally managed. Any local changes to these files will be overwritten on the next sync. To propose changes, update the source files in the Custo repo instead. -'@ + LogGroup "📦 Repositories in $org" { + $repos = Get-RepositoriesForOrganization -Organization $org -Context $orgContext + Write-Host "Repositories discovered in ${org}: $($repos.Count)" + } - try { - foreach ($repo in $subscribingRepos) { - Sync-RepositoryFile -Repository $repo ` - -FileSets $fileSets ` - -TempPath $tempPath ` - -BranchName $branchName ` - -CommitMessage $commitMessage ` - -PRTitle $prTitle ` - -PRBody $prBody ` - -PRLabel $prLabel ` - -Context $context + foreach ($repo in $repos) { + LogGroup "🏷️ Repository policies: $($repo.FullName)" { + $repoPolicyPaths = Resolve-PolicyLayerPaths -PolicyRoot $policyRoot -Enterprise $policyEnterprise -Layer 'repository' -Organization $org -Repository $repo.Name + $repoPolicies = Get-PolicyDocumentsFromPaths -Paths $repoPolicyPaths -CapabilityCatalog $capabilityCatalog -Layer 'repository' + Write-Host "Repository policies loaded: $($repoPolicies.Count)" + Invoke-PoliciesForScope -Policies $repoPolicies -FileSets $fileSets -Context $orgContext -Enterprise $policyEnterprise -Organization $org -Repository $repo -TempPath $tempPath -WhatIf:$WhatIf + } + } + } } } finally { if (Test-Path $tempPath) { @@ -859,12 +907,12 @@ The files in this PR are centrally managed. Any local changes to these files wil Write-Host " Updated: $($script:Summary.PRsUpdated)" Write-Host " In sync: $($script:Summary.ReposAlreadyInSync)" Write-Host " Skipped: $($script:Summary.ReposSkipped)" - if ($script:Summary.Errors.Count -gt 0) { Write-Host " Errors: $($script:Summary.Errors.Count)" foreach ($err in $script:Summary.Errors) { Write-Host " ❌ $err" } + exit 1 } } catch { @@ -872,5 +920,3 @@ The files in this PR are centrally managed. Any local changes to these files wil Write-Host $_.ScriptStackTrace exit 1 } - -#endregion From 9d74edbae5d04ff568939f8ec785bd2deb50fcf4 Mon Sep 17 00:00:00 2001 From: Marius Storhaug Date: Sat, 25 Jul 2026 21:25:32 +0200 Subject: [PATCH 14/20] refactor: enforce explicit enterprise auth modes Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- AGENTS.md | 2 +- .../repo-custom-property.capability.json | 2 +- .../enterprise/repo-rulesets.capability.json | 2 +- .../organization/none.capability.json | 2 +- .../file-subscription-service.capability.json | 2 +- README.md | 9 +- scripts/Sync-Files.ps1 | 89 +++++++++++-------- 7 files changed, 64 insertions(+), 44 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index b34ccfd..71559e2 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -26,7 +26,7 @@ cannot run against real repositories until an operator completes these steps: equivalent app with `contents:write`, `pull_requests:write`, and `repository_custom_properties:read` on target repositories, plus enterprise custom-properties write/admin permission if enterprise schema sync is enabled. - If enterprise rulesets/custom-properties must be managed via PAT, also add + For capabilities configured with `authMode: enterprise-pat`, also add `CUSTO_ENTERPRISE_PAT` and grant it enterprise-admin scope. 2. **Confirm app installation scope.** Verify the app is installed on the `PSModule` organization with access to the module repositories that should receive `AGENTS.md` diff --git a/PolicyEngine/Capabilities/enterprise/repo-custom-property.capability.json b/PolicyEngine/Capabilities/enterprise/repo-custom-property.capability.json index 215ba91..f711426 100644 --- a/PolicyEngine/Capabilities/enterprise/repo-custom-property.capability.json +++ b/PolicyEngine/Capabilities/enterprise/repo-custom-property.capability.json @@ -3,7 +3,7 @@ "name": "Enterprise repository custom-property schema", "layer": "enterprise", "capability": "repo-custom-property", - "authPreference": "github-app-first-fallback-enterprise-pat", + "authMode": "github-app", "description": "Maintains enterprise-level repository custom-property definitions and allowed values.", "configSchema": { "required": ["enterprise"], diff --git a/PolicyEngine/Capabilities/enterprise/repo-rulesets.capability.json b/PolicyEngine/Capabilities/enterprise/repo-rulesets.capability.json index 75c35fa..c4e5d22 100644 --- a/PolicyEngine/Capabilities/enterprise/repo-rulesets.capability.json +++ b/PolicyEngine/Capabilities/enterprise/repo-rulesets.capability.json @@ -3,7 +3,7 @@ "name": "Enterprise repository rulesets", "layer": "enterprise", "capability": "repo-rulesets", - "authPreference": "github-app-first-fallback-enterprise-pat", + "authMode": "enterprise-pat", "description": "Maintains enterprise-level repository rulesets.", "configSchema": { "required": ["enterprise", "rulesets"], diff --git a/PolicyEngine/Capabilities/organization/none.capability.json b/PolicyEngine/Capabilities/organization/none.capability.json index 114a949..2a6a3af 100644 --- a/PolicyEngine/Capabilities/organization/none.capability.json +++ b/PolicyEngine/Capabilities/organization/none.capability.json @@ -3,6 +3,6 @@ "name": "Organization policy placeholder", "layer": "organization", "capability": "none", - "authPreference": "github-app", + "authMode": "github-app", "description": "Placeholder capability for future organization-level policies." } diff --git a/PolicyEngine/Capabilities/repository/file-subscription-service.capability.json b/PolicyEngine/Capabilities/repository/file-subscription-service.capability.json index b7ccde1..5dcfb95 100644 --- a/PolicyEngine/Capabilities/repository/file-subscription-service.capability.json +++ b/PolicyEngine/Capabilities/repository/file-subscription-service.capability.json @@ -3,6 +3,6 @@ "name": "Repository file subscription service", "layer": "repository", "capability": "file-subscription-service", - "authPreference": "github-app", + "authMode": "github-app", "description": "Synchronizes managed files into subscribing repositories via pull requests." } diff --git a/README.md b/README.md index bcc2ed2..4d914f9 100644 --- a/README.md +++ b/README.md @@ -76,9 +76,10 @@ Additional file sets and rollout targets are added incrementally after this MVP ## Required secrets -The sync workflow authenticates as a GitHub App. Enterprise policy API calls are attempted with -the GitHub App token first; when those calls are unavailable for GitHub Apps, Custo falls back to -`CUSTO_ENTERPRISE_PAT` if present. +The sync workflow authenticates as a GitHub App. Enterprise policy capabilities choose auth +explicitly via `PolicyEngine/Capabilities`: +- `authMode: github-app` uses the GitHub App installation token. +- `authMode: enterprise-pat` uses `CUSTO_ENTERPRISE_PAT` for endpoints requiring enterprise PAT scope. For repository sync plus enterprise policy maintenance, configure: @@ -91,7 +92,7 @@ Configure these repository secrets before enabling the scheduled sync: - `CUSTO_BOT_CLIENT_ID` - `CUSTO_BOT_PRIVATE_KEY` -- `CUSTO_ENTERPRISE_PAT` (optional fallback PAT for enterprise policy endpoints, e.g. enterprise rulesets, with `admin:enterprise`) +- `CUSTO_ENTERPRISE_PAT` (required for any capability configured with `authMode: enterprise-pat`, e.g. enterprise rulesets, with `admin:enterprise`) Default enterprise policy configs live at: diff --git a/scripts/Sync-Files.ps1 b/scripts/Sync-Files.ps1 index 76900c0..4c97c09 100644 --- a/scripts/Sync-Files.ps1 +++ b/scripts/Sync-Files.ps1 @@ -13,8 +13,9 @@ Policies are loaded dynamically from PolicyEngine/Policies/{enterprise}/... Capabilities are loaded dynamically from PolicyEngine/Capabilities/... - Enterprise policy API calls prefer GitHub App and fall back to CUSTO_ENTERPRISE_PAT when - App access is unavailable. + Enterprise policy capabilities declare auth mode explicitly: + - github-app: use GitHub App installation context + - enterprise-pat: use CUSTO_ENTERPRISE_PAT for enterprise-admin endpoints #> [CmdletBinding()] @@ -293,39 +294,44 @@ function Invoke-EnterprisePolicyApi { [object]$Body, + [Parameter(Mandatory)] + [ValidateSet('github-app', 'enterprise-pat')] + [string]$AuthMode, + [Parameter(Mandatory)] [object]$Context ) - try { - return (Invoke-GitHubAPI -Method $Method -ApiEndpoint $ApiEndpoint -Body $Body -Context $Context).Response - } catch { - $enterprisePat = $env:CUSTO_ENTERPRISE_PAT - if ([string]::IsNullOrWhiteSpace($enterprisePat)) { - throw + switch ($AuthMode) { + 'github-app' { + return (Invoke-GitHubAPI -Method $Method -ApiEndpoint $ApiEndpoint -Body $Body -Context $Context).Response } + 'enterprise-pat' { + $enterprisePat = $env:CUSTO_ENTERPRISE_PAT + if ([string]::IsNullOrWhiteSpace($enterprisePat)) { + throw "CUSTO_ENTERPRISE_PAT is required for auth mode 'enterprise-pat' ($ApiEndpoint)." + } - Write-Host "ℹ️ App auth failed for $ApiEndpoint; retrying with CUSTO_ENTERPRISE_PAT" + $headers = @{ + Authorization = "Bearer $enterprisePat" + Accept = 'application/vnd.github+json' + 'X-GitHub-Api-Version' = '2022-11-28' + } - $headers = @{ - Authorization = "Bearer $enterprisePat" - Accept = 'application/vnd.github+json' - 'X-GitHub-Api-Version' = '2022-11-28' - } + $uri = "https://api.github.com$ApiEndpoint" + $invokeArgs = @{ + Method = $Method + Uri = $uri + Headers = $headers + } - $uri = "https://api.github.com$ApiEndpoint" - $invokeArgs = @{ - Method = $Method - Uri = $uri - Headers = $headers - } + if ($null -ne $Body) { + $invokeArgs.ContentType = 'application/json' + $invokeArgs.Body = $Body | ConvertTo-Json -Depth 50 + } - if ($null -ne $Body) { - $invokeArgs.ContentType = 'application/json' - $invokeArgs.Body = $Body | ConvertTo-Json -Depth 50 + return Invoke-RestMethod @invokeArgs } - - return Invoke-RestMethod @invokeArgs } } @@ -387,6 +393,9 @@ function Sync-EnterpriseCustomPropertySchema { [Parameter(Mandatory)] [hashtable]$FileSets, + [Parameter(Mandatory)] + [hashtable]$CapabilityCatalog, + [Parameter(Mandatory)] [object]$Context, @@ -395,6 +404,9 @@ function Sync-EnterpriseCustomPropertySchema { [bool]$Required = $false, [object]$ValuesEditableBy = 'org_actors', [bool]$RequireExplicitValues = $false, + [Parameter(Mandatory)] + [ValidateSet('github-app', 'enterprise-pat')] + [string]$AuthMode, [switch]$WhatIf ) @@ -430,8 +442,8 @@ function Sync-EnterpriseCustomPropertySchema { $currentType = $null $currentSubscription = $null - try { $currentType = Invoke-EnterprisePolicyApi -Method GET -ApiEndpoint "/enterprises/$Enterprise/properties/schema/$TypePropertyName" -Context $Context } catch {} - try { $currentSubscription = Invoke-EnterprisePolicyApi -Method GET -ApiEndpoint "/enterprises/$Enterprise/properties/schema/$SubscriptionPropertyName" -Context $Context } catch {} + try { $currentType = Invoke-EnterprisePolicyApi -Method GET -ApiEndpoint "/enterprises/$Enterprise/properties/schema/$TypePropertyName" -AuthMode $AuthMode -Context $Context } catch {} + try { $currentSubscription = Invoke-EnterprisePolicyApi -Method GET -ApiEndpoint "/enterprises/$Enterprise/properties/schema/$SubscriptionPropertyName" -AuthMode $AuthMode -Context $Context } catch {} $typeChanged = Write-ConfigDiff -Title "enterprise property '$TypePropertyName'" -Current $currentType -Desired $desiredType $subscriptionChanged = Write-ConfigDiff -Title "enterprise property '$SubscriptionPropertyName'" -Current $currentSubscription -Desired $desiredSubscription @@ -442,12 +454,12 @@ function Sync-EnterpriseCustomPropertySchema { } if ($typeChanged) { - Invoke-EnterprisePolicyApi -Method PUT -ApiEndpoint "/enterprises/$Enterprise/properties/schema/$TypePropertyName" -Body $desiredType -Context $Context | Out-Null + Invoke-EnterprisePolicyApi -Method PUT -ApiEndpoint "/enterprises/$Enterprise/properties/schema/$TypePropertyName" -Body $desiredType -AuthMode $AuthMode -Context $Context | Out-Null Write-Host "✅ Updated enterprise property '$TypePropertyName'" } if ($subscriptionChanged) { - Invoke-EnterprisePolicyApi -Method PUT -ApiEndpoint "/enterprises/$Enterprise/properties/schema/$SubscriptionPropertyName" -Body $desiredSubscription -Context $Context | Out-Null + Invoke-EnterprisePolicyApi -Method PUT -ApiEndpoint "/enterprises/$Enterprise/properties/schema/$SubscriptionPropertyName" -Body $desiredSubscription -AuthMode $AuthMode -Context $Context | Out-Null Write-Host "✅ Updated enterprise property '$SubscriptionPropertyName'" } } @@ -464,6 +476,10 @@ function Sync-EnterpriseRulesets { [Parameter(Mandatory)] [object]$Context, + [Parameter(Mandatory)] + [ValidateSet('github-app', 'enterprise-pat')] + [string]$AuthMode, + [switch]$WhatIf ) @@ -473,7 +489,7 @@ function Sync-EnterpriseRulesets { } $existingRulesets = @( - Invoke-EnterprisePolicyApi -Method GET -ApiEndpoint "/enterprises/$Enterprise/rulesets" -Context $Context + Invoke-EnterprisePolicyApi -Method GET -ApiEndpoint "/enterprises/$Enterprise/rulesets" -AuthMode $AuthMode -Context $Context ) $existingByName = @{} @@ -520,7 +536,7 @@ function Sync-EnterpriseRulesets { continue } - Invoke-EnterprisePolicyApi -Method $method -ApiEndpoint $endpoint -Body $desired -Context $Context | Out-Null + Invoke-EnterprisePolicyApi -Method $method -ApiEndpoint $endpoint -Body $desired -AuthMode $AuthMode -Context $Context | Out-Null Write-Host "✅ Upserted enterprise ruleset '$($ruleset.name)'" } } @@ -722,6 +738,8 @@ function Invoke-PoliciesForScope { foreach ($policy in $Policies) { $name = if ($policy.name) { $policy.name } else { [System.IO.Path]::GetFileName($policy.sourceFile) } $capabilityId = "$($policy.layer).$($policy.capability)" + $capabilityDefinition = $CapabilityCatalog[$capabilityId] + $authMode = if ($capabilityDefinition.authMode) { [string]$capabilityDefinition.authMode } else { 'github-app' } if ($policy.enabled -ne $true) { Write-Host "⏭️ Policy disabled: $name ($capabilityId)" @@ -747,11 +765,12 @@ function Invoke-PoliciesForScope { -Required $required ` -ValuesEditableBy $valuesEditableBy ` -RequireExplicitValues $requireExplicitValues ` + -AuthMode $authMode ` -WhatIf:$WhatIf } 'enterprise.repo-rulesets' { $rulesets = if ($policy.config.rulesets) { @($policy.config.rulesets) } else { @() } - Sync-EnterpriseRulesets -Enterprise $policy.config.enterprise -Rulesets $rulesets -Context $Context -WhatIf:$WhatIf + Sync-EnterpriseRulesets -Enterprise $policy.config.enterprise -Rulesets $rulesets -Context $Context -AuthMode $authMode -WhatIf:$WhatIf } 'organization.none' { Write-Host "ℹ️ Organization policy layer placeholder for $Organization" @@ -860,7 +879,7 @@ try { $enterprisePolicies = Get-PolicyDocumentsFromPaths -Paths $enterprisePolicyPaths -CapabilityCatalog $capabilityCatalog -Layer 'enterprise' Write-Host "Enterprise policies loaded: $($enterprisePolicies.Count)" - Invoke-PoliciesForScope -Policies $enterprisePolicies -FileSets $fileSets -Context $rootContext -Enterprise $policyEnterprise -TempPath $tempPath -WhatIf:$WhatIf + Invoke-PoliciesForScope -Policies $enterprisePolicies -FileSets $fileSets -CapabilityCatalog $capabilityCatalog -Context $rootContext -Enterprise $policyEnterprise -TempPath $tempPath -WhatIf:$WhatIf } LogGroup '🏢➡️🏬 Organization and repository layers' { @@ -876,7 +895,7 @@ try { $orgPolicyPaths = Resolve-PolicyLayerPaths -PolicyRoot $policyRoot -Enterprise $policyEnterprise -Layer 'organization' -Organization $org $orgPolicies = Get-PolicyDocumentsFromPaths -Paths $orgPolicyPaths -CapabilityCatalog $capabilityCatalog -Layer 'organization' Write-Host "Organization policies loaded: $($orgPolicies.Count)" - Invoke-PoliciesForScope -Policies $orgPolicies -FileSets $fileSets -Context $orgContext -Enterprise $policyEnterprise -Organization $org -TempPath $tempPath -WhatIf:$WhatIf + Invoke-PoliciesForScope -Policies $orgPolicies -FileSets $fileSets -CapabilityCatalog $capabilityCatalog -Context $orgContext -Enterprise $policyEnterprise -Organization $org -TempPath $tempPath -WhatIf:$WhatIf } LogGroup "📦 Repositories in $org" { @@ -889,7 +908,7 @@ try { $repoPolicyPaths = Resolve-PolicyLayerPaths -PolicyRoot $policyRoot -Enterprise $policyEnterprise -Layer 'repository' -Organization $org -Repository $repo.Name $repoPolicies = Get-PolicyDocumentsFromPaths -Paths $repoPolicyPaths -CapabilityCatalog $capabilityCatalog -Layer 'repository' Write-Host "Repository policies loaded: $($repoPolicies.Count)" - Invoke-PoliciesForScope -Policies $repoPolicies -FileSets $fileSets -Context $orgContext -Enterprise $policyEnterprise -Organization $org -Repository $repo -TempPath $tempPath -WhatIf:$WhatIf + Invoke-PoliciesForScope -Policies $repoPolicies -FileSets $fileSets -CapabilityCatalog $capabilityCatalog -Context $orgContext -Enterprise $policyEnterprise -Organization $org -Repository $repo -TempPath $tempPath -WhatIf:$WhatIf } } } From b9f9c0c860b16920aec75a43cbc598e37e40b3ac Mon Sep 17 00:00:00 2001 From: Marius Storhaug Date: Sat, 25 Jul 2026 21:40:29 +0200 Subject: [PATCH 15/20] fix: correct CapabilityCatalog param wiring in policy engine The auth-mode refactor misplaced the CapabilityCatalog parameter onto Sync-EnterpriseCustomPropertySchema (mandatory, never passed) while Invoke-PoliciesForScope referenced it without declaring it. Callers passed -CapabilityCatalog to a CmdletBinding function that rejected it, so the runtime failed on the first policy scope; even absent that, authMode was always null and fell back to github-app, so the enterprise-pat routing never executed. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- scripts/Sync-Files.ps1 | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/scripts/Sync-Files.ps1 b/scripts/Sync-Files.ps1 index 4c97c09..db31214 100644 --- a/scripts/Sync-Files.ps1 +++ b/scripts/Sync-Files.ps1 @@ -393,9 +393,6 @@ function Sync-EnterpriseCustomPropertySchema { [Parameter(Mandatory)] [hashtable]$FileSets, - [Parameter(Mandatory)] - [hashtable]$CapabilityCatalog, - [Parameter(Mandatory)] [object]$Context, @@ -724,6 +721,9 @@ function Invoke-PoliciesForScope { [Parameter(Mandatory)] [hashtable]$FileSets, + [Parameter(Mandatory)] + [hashtable]$CapabilityCatalog, + [Parameter(Mandatory)] [object]$Context, From 511c5539a925481e4ecbd72d7acc08fc307d10d2 Mon Sep 17 00:00:00 2001 From: Marius Storhaug Date: Sat, 25 Jul 2026 21:42:20 +0200 Subject: [PATCH 16/20] fix: make policy diffs order-insensitive and drift-accurate Replace line-based JSON comparison with recursive canonicalization (sorted keys) and a deep-subset check. Server-added fields and key ordering no longer register as drift, so enterprise property/ruleset PUTs only fire on real changes; array order remains significant. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- scripts/Sync-Files.ps1 | 97 ++++++++++++++++++++++++++++++++++++++---- 1 file changed, 89 insertions(+), 8 deletions(-) diff --git a/scripts/Sync-Files.ps1 b/scripts/Sync-Files.ps1 index db31214..36a4402 100644 --- a/scripts/Sync-Files.ps1 +++ b/scripts/Sync-Files.ps1 @@ -244,14 +244,95 @@ function Resolve-PolicyLayerPaths { return $paths } +function ConvertTo-CanonicalObject { + [CmdletBinding()] + param( + $InputObject + ) + + if ($null -eq $InputObject) { return $null } + + if ($InputObject -is [string] -or $InputObject -is [ValueType]) { + return $InputObject + } + + if ($InputObject -is [System.Collections.IDictionary]) { + $ordered = [ordered]@{} + foreach ($key in ($InputObject.Keys | Sort-Object)) { + $ordered[[string]$key] = ConvertTo-CanonicalObject -InputObject $InputObject[$key] + } + return $ordered + } + + if ($InputObject -is [System.Management.Automation.PSCustomObject]) { + $ordered = [ordered]@{} + foreach ($prop in ($InputObject.PSObject.Properties.Name | Sort-Object)) { + $ordered[$prop] = ConvertTo-CanonicalObject -InputObject $InputObject.$prop + } + return $ordered + } + + if ($InputObject -is [System.Collections.IEnumerable]) { + $list = @() + foreach ($item in $InputObject) { + $list += , (ConvertTo-CanonicalObject -InputObject $item) + } + return , $list + } + + return $InputObject +} + +function Test-CanonicalSubset { + <# + Returns $true when every value declared in $Subset is present and equal in + $Superset. Both inputs must already be canonicalized. A $null desired value is + satisfied when the key is absent or already null, so server-added fields never + register as drift. + #> + [CmdletBinding()] + param( + $Superset, + $Subset + ) + + if ($Subset -is [System.Collections.IDictionary]) { + if (-not ($Superset -is [System.Collections.IDictionary])) { return $false } + foreach ($key in $Subset.Keys) { + $desiredValue = $Subset[$key] + $hasKey = $Superset.Contains($key) + $currentValue = if ($hasKey) { $Superset[$key] } else { $null } + + if ($null -eq $desiredValue) { + if ($hasKey -and $null -ne $currentValue) { return $false } + continue + } + + if (-not $hasKey) { return $false } + if (-not (Test-CanonicalSubset -Superset $currentValue -Subset $desiredValue)) { return $false } + } + return $true + } + + if ($Subset -is [array]) { + if (-not ($Superset -is [array])) { return $false } + if ($Superset.Count -ne $Subset.Count) { return $false } + for ($i = 0; $i -lt $Subset.Count; $i++) { + if (-not (Test-CanonicalSubset -Superset $Superset[$i] -Subset $Subset[$i])) { return $false } + } + return $true + } + + return ($Superset -eq $Subset) +} + function ConvertTo-NormalizedJson { [CmdletBinding()] param( - [Parameter(Mandatory)] $InputObject ) - return ($InputObject | ConvertTo-Json -Depth 50) + return (ConvertTo-CanonicalObject -InputObject $InputObject | ConvertTo-Json -Depth 50) } function Write-ConfigDiff { @@ -264,17 +345,17 @@ function Write-ConfigDiff { $Desired ) - $currentJson = ConvertTo-NormalizedJson -InputObject $Current - $desiredJson = ConvertTo-NormalizedJson -InputObject $Desired + $canonicalCurrent = ConvertTo-CanonicalObject -InputObject $Current + $canonicalDesired = ConvertTo-CanonicalObject -InputObject $Desired - if ($currentJson -eq $desiredJson) { + if (Test-CanonicalSubset -Superset $canonicalCurrent -Subset $canonicalDesired) { Write-Host "✅ $Title already matches desired state" return $false } - Write-Host "🔎 Diff for $Title" - $currentLines = $currentJson -split "`r?`n" - $desiredLines = $desiredJson -split "`r?`n" + Write-Host "🔎 Diff for $Title (< current / > desired)" + $currentLines = ($canonicalCurrent | ConvertTo-Json -Depth 50) -split "`r?`n" + $desiredLines = ($canonicalDesired | ConvertTo-Json -Depth 50) -split "`r?`n" $diff = Compare-Object -ReferenceObject $currentLines -DifferenceObject $desiredLines foreach ($line in $diff) { Write-Host " $($line.SideIndicator) $($line.InputObject)" From 670a04353ec3a7c25d67e148067159ab18753d9a Mon Sep 17 00:00:00 2001 From: Marius Storhaug Date: Sat, 25 Jul 2026 21:44:19 +0200 Subject: [PATCH 17/20] feat: isolate per-org failures and add run report Wrap the enterprise layer and each organization in try/catch so a single connect or traversal failure no longer aborts the whole fleet run; failures are recorded and counted (OrgsProcessed/OrgsFailed) and still fail the job at the end. Add Write-RunReport to emit a Markdown summary to GITHUB_STEP_SUMMARY (falls back to stdout locally). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- scripts/Sync-Files.ps1 | 114 ++++++++++++++++++++++++++++++++--------- 1 file changed, 90 insertions(+), 24 deletions(-) diff --git a/scripts/Sync-Files.ps1 b/scripts/Sync-Files.ps1 index 36a4402..5443b0f 100644 --- a/scripts/Sync-Files.ps1 +++ b/scripts/Sync-Files.ps1 @@ -26,6 +26,8 @@ param( $ErrorActionPreference = 'Stop' $script:Summary = @{ + OrgsProcessed = 0 + OrgsFailed = 0 TotalReposProcessed = 0 PRsCreated = 0 PRsUpdated = 0 @@ -363,6 +365,52 @@ function Write-ConfigDiff { return $true } +function Write-RunReport { + [Diagnostics.CodeAnalysis.SuppressMessageAttribute( + 'PSAvoidUsingWriteHost', '', Scope = 'Function', + Justification = 'Intended for logging in GitHub Actions runners.' + )] + [CmdletBinding()] + param( + [Parameter(Mandatory)] + [hashtable]$Summary, + + [switch]$WhatIf + ) + + $mode = if ($WhatIf) { 'plan (WhatIf — no writes)' } else { 'apply' } + $lines = @( + "## Custo sync report — $mode", + '', + '| Metric | Count |', + '| --- | --- |', + "| Orgs processed | $($Summary.OrgsProcessed) |", + "| Orgs failed | $($Summary.OrgsFailed) |", + "| Repos processed | $($Summary.TotalReposProcessed) |", + "| PRs created | $($Summary.PRsCreated) |", + "| PRs updated | $($Summary.PRsUpdated) |", + "| Already in sync | $($Summary.ReposAlreadyInSync) |", + "| Repos skipped | $($Summary.ReposSkipped) |", + "| Errors | $($Summary.Errors.Count) |" + ) + + if ($Summary.Errors.Count -gt 0) { + $lines += '' + $lines += '### Errors' + foreach ($err in $Summary.Errors) { + $lines += "- $err" + } + } + + $report = $lines -join "`n" + + if ($env:GITHUB_STEP_SUMMARY) { + $report | Out-File -FilePath $env:GITHUB_STEP_SUMMARY -Append -Encoding utf8 + } else { + Write-Host $report + } +} + function Invoke-EnterprisePolicyApi { [CmdletBinding()] param( @@ -956,11 +1004,16 @@ try { try { LogGroup '🏢 Enterprise layer' { - $enterprisePolicyPaths = Resolve-PolicyLayerPaths -PolicyRoot $policyRoot -Enterprise $policyEnterprise -Layer 'enterprise' - $enterprisePolicies = Get-PolicyDocumentsFromPaths -Paths $enterprisePolicyPaths -CapabilityCatalog $capabilityCatalog -Layer 'enterprise' - Write-Host "Enterprise policies loaded: $($enterprisePolicies.Count)" - - Invoke-PoliciesForScope -Policies $enterprisePolicies -FileSets $fileSets -CapabilityCatalog $capabilityCatalog -Context $rootContext -Enterprise $policyEnterprise -TempPath $tempPath -WhatIf:$WhatIf + try { + $enterprisePolicyPaths = Resolve-PolicyLayerPaths -PolicyRoot $policyRoot -Enterprise $policyEnterprise -Layer 'enterprise' + $enterprisePolicies = Get-PolicyDocumentsFromPaths -Paths $enterprisePolicyPaths -CapabilityCatalog $capabilityCatalog -Layer 'enterprise' + Write-Host "Enterprise policies loaded: $($enterprisePolicies.Count)" + + Invoke-PoliciesForScope -Policies $enterprisePolicies -FileSets $fileSets -CapabilityCatalog $capabilityCatalog -Context $rootContext -Enterprise $policyEnterprise -TempPath $tempPath -WhatIf:$WhatIf + } catch { + Write-Host "❌ Enterprise layer failed: $_" + $script:Summary.Errors += "enterprise:$policyEnterprise : $_" + } } LogGroup '🏢➡️🏬 Organization and repository layers' { @@ -968,29 +1021,38 @@ try { Write-Host "Organizations discovered: $($organizations -join ', ')" foreach ($org in $organizations) { - LogGroup "🔐 Connect org installation: $org" { - $orgContext = Get-OrganizationContext -Organization $org - } + try { + $orgContext = $null + LogGroup "🔐 Connect org installation: $org" { + $orgContext = Get-OrganizationContext -Organization $org + } - LogGroup "🏬 Organization policies: $org" { - $orgPolicyPaths = Resolve-PolicyLayerPaths -PolicyRoot $policyRoot -Enterprise $policyEnterprise -Layer 'organization' -Organization $org - $orgPolicies = Get-PolicyDocumentsFromPaths -Paths $orgPolicyPaths -CapabilityCatalog $capabilityCatalog -Layer 'organization' - Write-Host "Organization policies loaded: $($orgPolicies.Count)" - Invoke-PoliciesForScope -Policies $orgPolicies -FileSets $fileSets -CapabilityCatalog $capabilityCatalog -Context $orgContext -Enterprise $policyEnterprise -Organization $org -TempPath $tempPath -WhatIf:$WhatIf - } + LogGroup "🏬 Organization policies: $org" { + $orgPolicyPaths = Resolve-PolicyLayerPaths -PolicyRoot $policyRoot -Enterprise $policyEnterprise -Layer 'organization' -Organization $org + $orgPolicies = Get-PolicyDocumentsFromPaths -Paths $orgPolicyPaths -CapabilityCatalog $capabilityCatalog -Layer 'organization' + Write-Host "Organization policies loaded: $($orgPolicies.Count)" + Invoke-PoliciesForScope -Policies $orgPolicies -FileSets $fileSets -CapabilityCatalog $capabilityCatalog -Context $orgContext -Enterprise $policyEnterprise -Organization $org -TempPath $tempPath -WhatIf:$WhatIf + } - LogGroup "📦 Repositories in $org" { - $repos = Get-RepositoriesForOrganization -Organization $org -Context $orgContext - Write-Host "Repositories discovered in ${org}: $($repos.Count)" - } + LogGroup "📦 Repositories in $org" { + $repos = Get-RepositoriesForOrganization -Organization $org -Context $orgContext + Write-Host "Repositories discovered in ${org}: $($repos.Count)" + } - foreach ($repo in $repos) { - LogGroup "🏷️ Repository policies: $($repo.FullName)" { - $repoPolicyPaths = Resolve-PolicyLayerPaths -PolicyRoot $policyRoot -Enterprise $policyEnterprise -Layer 'repository' -Organization $org -Repository $repo.Name - $repoPolicies = Get-PolicyDocumentsFromPaths -Paths $repoPolicyPaths -CapabilityCatalog $capabilityCatalog -Layer 'repository' - Write-Host "Repository policies loaded: $($repoPolicies.Count)" - Invoke-PoliciesForScope -Policies $repoPolicies -FileSets $fileSets -CapabilityCatalog $capabilityCatalog -Context $orgContext -Enterprise $policyEnterprise -Organization $org -Repository $repo -TempPath $tempPath -WhatIf:$WhatIf + foreach ($repo in $repos) { + LogGroup "🏷️ Repository policies: $($repo.FullName)" { + $repoPolicyPaths = Resolve-PolicyLayerPaths -PolicyRoot $policyRoot -Enterprise $policyEnterprise -Layer 'repository' -Organization $org -Repository $repo.Name + $repoPolicies = Get-PolicyDocumentsFromPaths -Paths $repoPolicyPaths -CapabilityCatalog $capabilityCatalog -Layer 'repository' + Write-Host "Repository policies loaded: $($repoPolicies.Count)" + Invoke-PoliciesForScope -Policies $repoPolicies -FileSets $fileSets -CapabilityCatalog $capabilityCatalog -Context $orgContext -Enterprise $policyEnterprise -Organization $org -Repository $repo -TempPath $tempPath -WhatIf:$WhatIf + } } + + $script:Summary.OrgsProcessed++ + } catch { + Write-Host "❌ Organization '$org' failed: $_" + $script:Summary.Errors += "org:$org : $_" + $script:Summary.OrgsFailed++ } } } @@ -1000,8 +1062,12 @@ try { } } + Write-RunReport -Summary $script:Summary -WhatIf:$WhatIf + Write-Host '' Write-Host '📊 Summary' + Write-Host " Orgs OK: $($script:Summary.OrgsProcessed)" + Write-Host " Orgs fail: $($script:Summary.OrgsFailed)" Write-Host " Processed: $($script:Summary.TotalReposProcessed)" Write-Host " Created: $($script:Summary.PRsCreated)" Write-Host " Updated: $($script:Summary.PRsUpdated)" From 3655b9d541ac9548e3f5c04d24c95769dd070813 Mon Sep 17 00:00:00 2001 From: Marius Storhaug Date: Sat, 25 Jul 2026 21:49:57 +0200 Subject: [PATCH 18/20] test: add policy schemas, Pester tests, and CI gate Add JSON Schemas for capability/policy documents; add Pester suites covering canonical diffing, subset drift detection, policy-path resolution, authMode routing, and conformance of every shipped capability/policy doc. Add a CI workflow running Pester + PSScriptAnalyzer (error severity) on push/PR. Also make the enterprise property GET catch blocks explicit rather than silently swallowing errors. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- .github/workflows/ci.yml | 39 ++++++ PolicyEngine/schemas/capability.schema.json | 43 +++++++ PolicyEngine/schemas/policy.schema.json | 44 +++++++ scripts/Sync-Files.ps1 | 12 +- tests/PolicyDocuments.Tests.ps1 | 67 ++++++++++ tests/Sync-Files.Tests.ps1 | 132 ++++++++++++++++++++ 6 files changed, 335 insertions(+), 2 deletions(-) create mode 100644 .github/workflows/ci.yml create mode 100644 PolicyEngine/schemas/capability.schema.json create mode 100644 PolicyEngine/schemas/policy.schema.json create mode 100644 tests/PolicyDocuments.Tests.ps1 create mode 100644 tests/Sync-Files.Tests.ps1 diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml new file mode 100644 index 0000000..ced3460 --- /dev/null +++ b/.github/workflows/ci.yml @@ -0,0 +1,39 @@ +name: CI + +on: + push: + branches: [main] + pull_request: + +permissions: + contents: read + +jobs: + test: + name: Lint and test + runs-on: ubuntu-latest + steps: + - name: Checkout repository + uses: actions/checkout@eef61447b9ff4aafe5dcd4e0bbf5d482be7e7871 # v4.2.1 + + - name: Run Pester tests + shell: pwsh + run: | + Set-PSRepository -Name PSGallery -InstallationPolicy Trusted + Install-Module Pester -MinimumVersion 5.5.0 -Force -SkipPublisherCheck + $config = New-PesterConfiguration + $config.Run.Path = './tests' + $config.Run.Exit = $true + $config.Output.Verbosity = 'Detailed' + Invoke-Pester -Configuration $config + + - name: Run PSScriptAnalyzer + shell: pwsh + run: | + Install-Module PSScriptAnalyzer -Force -SkipPublisherCheck + $results = Invoke-ScriptAnalyzer -Path ./scripts -Recurse -Severity Error + if ($results) { + $results | Format-Table -AutoSize + throw "PSScriptAnalyzer reported $($results.Count) error-level finding(s)." + } + Write-Host 'PSScriptAnalyzer: no error-level findings.' diff --git a/PolicyEngine/schemas/capability.schema.json b/PolicyEngine/schemas/capability.schema.json new file mode 100644 index 0000000..2ab289c --- /dev/null +++ b/PolicyEngine/schemas/capability.schema.json @@ -0,0 +1,43 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://msxorg.github.io/custo/schemas/capability.schema.json", + "title": "Custo capability document", + "description": "Defines what a policy capability does and how it authenticates. One document per Layer.Capability.", + "type": "object", + "required": ["version", "name", "layer", "capability", "authMode"], + "properties": { + "version": { + "type": "string", + "description": "Document schema version." + }, + "name": { + "type": "string", + "minLength": 1, + "description": "Human-readable capability name." + }, + "layer": { + "type": "string", + "enum": ["enterprise", "organization", "repository"], + "description": "Policy layer this capability runs at." + }, + "capability": { + "type": "string", + "minLength": 1, + "pattern": "^[a-z0-9]+(-[a-z0-9]+)*$", + "description": "Capability identifier (kebab-case) unique within its layer." + }, + "authMode": { + "type": "string", + "enum": ["github-app", "enterprise-pat"], + "description": "Explicit authentication mode. github-app uses the installation token; enterprise-pat uses CUSTO_ENTERPRISE_PAT for enterprise-admin endpoints the app cannot reach." + }, + "description": { + "type": "string" + }, + "configSchema": { + "type": "object", + "description": "Optional inline description of the config object accepted by matching policy documents." + } + }, + "additionalProperties": true +} diff --git a/PolicyEngine/schemas/policy.schema.json b/PolicyEngine/schemas/policy.schema.json new file mode 100644 index 0000000..4b12ed9 --- /dev/null +++ b/PolicyEngine/schemas/policy.schema.json @@ -0,0 +1,44 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://msxorg.github.io/custo/schemas/policy.schema.json", + "title": "Custo policy document", + "description": "Configures a capability for a specific enterprise/layer. Placed under PolicyEngine/Policies/{enterprise}/{layer}/...", + "type": "object", + "required": ["version", "name", "layer", "capability", "enabled"], + "properties": { + "version": { + "type": "string", + "description": "Document schema version." + }, + "name": { + "type": "string", + "minLength": 1, + "description": "Human-readable policy name." + }, + "layer": { + "type": "string", + "enum": ["enterprise", "organization", "repository"], + "description": "Policy layer. Must match the capability's layer and the folder it lives in." + }, + "capability": { + "type": "string", + "minLength": 1, + "pattern": "^[a-z0-9]+(-[a-z0-9]+)*$", + "description": "Capability this policy configures. Must exist in the capability catalog for this layer." + }, + "order": { + "type": "integer", + "minimum": 0, + "description": "Execution order within the layer (ascending). Defaults to 1000 when omitted." + }, + "enabled": { + "type": "boolean", + "description": "Whether this policy is applied. Disabled policies are loaded and skipped." + }, + "config": { + "type": "object", + "description": "Capability-specific configuration payload." + } + }, + "additionalProperties": true +} diff --git a/scripts/Sync-Files.ps1 b/scripts/Sync-Files.ps1 index 5443b0f..eb244a8 100644 --- a/scripts/Sync-Files.ps1 +++ b/scripts/Sync-Files.ps1 @@ -568,8 +568,16 @@ function Sync-EnterpriseCustomPropertySchema { $currentType = $null $currentSubscription = $null - try { $currentType = Invoke-EnterprisePolicyApi -Method GET -ApiEndpoint "/enterprises/$Enterprise/properties/schema/$TypePropertyName" -AuthMode $AuthMode -Context $Context } catch {} - try { $currentSubscription = Invoke-EnterprisePolicyApi -Method GET -ApiEndpoint "/enterprises/$Enterprise/properties/schema/$SubscriptionPropertyName" -AuthMode $AuthMode -Context $Context } catch {} + try { + $currentType = Invoke-EnterprisePolicyApi -Method GET -ApiEndpoint "/enterprises/$Enterprise/properties/schema/$TypePropertyName" -AuthMode $AuthMode -Context $Context + } catch { + Write-Host "ℹ️ Could not read current '$TypePropertyName' definition (treating as absent): $_" + } + try { + $currentSubscription = Invoke-EnterprisePolicyApi -Method GET -ApiEndpoint "/enterprises/$Enterprise/properties/schema/$SubscriptionPropertyName" -AuthMode $AuthMode -Context $Context + } catch { + Write-Host "ℹ️ Could not read current '$SubscriptionPropertyName' definition (treating as absent): $_" + } $typeChanged = Write-ConfigDiff -Title "enterprise property '$TypePropertyName'" -Current $currentType -Desired $desiredType $subscriptionChanged = Write-ConfigDiff -Title "enterprise property '$SubscriptionPropertyName'" -Current $currentSubscription -Desired $desiredSubscription diff --git a/tests/PolicyDocuments.Tests.ps1 b/tests/PolicyDocuments.Tests.ps1 new file mode 100644 index 0000000..f88efc8 --- /dev/null +++ b/tests/PolicyDocuments.Tests.ps1 @@ -0,0 +1,67 @@ +#requires -Modules Pester + +BeforeAll { + $engineRoot = Resolve-Path (Join-Path $PSScriptRoot '..' 'PolicyEngine') + $script:CapabilityFiles = @(Get-ChildItem -Path (Join-Path $engineRoot 'Capabilities') -Recurse -Filter '*.capability.json') + $script:PolicyFiles = @(Get-ChildItem -Path (Join-Path $engineRoot 'Policies') -Recurse -Filter '*.policy.json') + + # Build a catalog keyed by "layer.capability" for cross-checking policies. + $script:Catalog = @{} + foreach ($file in $script:CapabilityFiles) { + $doc = Get-Content -Path $file.FullName -Raw | ConvertFrom-Json + $script:Catalog["$($doc.layer).$($doc.capability)"] = $doc + } +} + +Describe 'Capability documents' { + It 'ships at least one capability' { + $script:CapabilityFiles.Count | Should -BeGreaterThan 0 + } + + It ' is valid JSON with required fields and enums' -ForEach ( + Get-ChildItem -Path (Join-Path $PSScriptRoot '..' 'PolicyEngine' 'Capabilities') -Recurse -Filter '*.capability.json' | + ForEach-Object { @{ Name = $_.Name; Path = $_.FullName } } + ) { + $doc = Get-Content -Path $Path -Raw | ConvertFrom-Json + $doc.version | Should -Not -BeNullOrEmpty + $doc.name | Should -Not -BeNullOrEmpty + $doc.layer | Should -BeIn @('enterprise', 'organization', 'repository') + $doc.capability | Should -Match '^[a-z0-9]+(-[a-z0-9]+)*$' + $doc.authMode | Should -BeIn @('github-app', 'enterprise-pat') + } +} + +Describe 'Policy documents' { + It 'ships at least one policy' { + $script:PolicyFiles.Count | Should -BeGreaterThan 0 + } + + It ' is valid JSON with required fields' -ForEach ( + Get-ChildItem -Path (Join-Path $PSScriptRoot '..' 'PolicyEngine' 'Policies') -Recurse -Filter '*.policy.json' | + ForEach-Object { @{ Name = $_.Name; Path = $_.FullName } } + ) { + $doc = Get-Content -Path $Path -Raw | ConvertFrom-Json + $doc.version | Should -Not -BeNullOrEmpty + $doc.name | Should -Not -BeNullOrEmpty + $doc.layer | Should -BeIn @('enterprise', 'organization', 'repository') + $doc.capability | Should -Match '^[a-z0-9]+(-[a-z0-9]+)*$' + ($doc.PSObject.Properties.Name -contains 'enabled') | Should -BeTrue + } + + It ' references a known capability for its layer' -ForEach ( + Get-ChildItem -Path (Join-Path $PSScriptRoot '..' 'PolicyEngine' 'Policies') -Recurse -Filter '*.policy.json' | + ForEach-Object { @{ Name = $_.Name; Path = $_.FullName } } + ) { + $doc = Get-Content -Path $Path -Raw | ConvertFrom-Json + $key = "$($doc.layer).$($doc.capability)" + $script:Catalog.ContainsKey($key) | Should -BeTrue -Because "capability '$key' must exist in PolicyEngine/Capabilities" + } + + It ' lives under a folder matching its declared layer' -ForEach ( + Get-ChildItem -Path (Join-Path $PSScriptRoot '..' 'PolicyEngine' 'Policies') -Recurse -Filter '*.policy.json' | + ForEach-Object { @{ Name = $_.Name; Path = $_.FullName; Layer = ($_ | Get-Content -Raw | ConvertFrom-Json).layer } } + ) { + $separator = [System.IO.Path]::DirectorySeparatorChar + $Path | Should -Match ([regex]::Escape("$separator$Layer$separator")) + } +} diff --git a/tests/Sync-Files.Tests.ps1 b/tests/Sync-Files.Tests.ps1 new file mode 100644 index 0000000..30a89e9 --- /dev/null +++ b/tests/Sync-Files.Tests.ps1 @@ -0,0 +1,132 @@ +#requires -Modules Pester + +BeforeAll { + # Load only the function definitions from the runtime script, without executing the + # top-level pipeline (which requires GitHub authentication). + $scriptPath = Resolve-Path (Join-Path $PSScriptRoot '..' 'scripts' 'Sync-Files.ps1') + $tokens = $null + $parseErrors = $null + $ast = [System.Management.Automation.Language.Parser]::ParseFile($scriptPath, [ref]$tokens, [ref]$parseErrors) + if ($parseErrors.Count -gt 0) { + throw "Sync-Files.ps1 has parse errors: $($parseErrors -join '; ')" + } + $funcs = $ast.FindAll({ param($n) $n -is [System.Management.Automation.Language.FunctionDefinitionAst] }, $false) + $definitions = ($funcs | ForEach-Object { $_.Extent.Text }) -join "`n`n" + . ([scriptblock]::Create($definitions)) + + # Minimal stub so functions that log through LogGroup still run in tests. + function LogGroup { + param($Title, [scriptblock]$ScriptBlock) + & $ScriptBlock + } +} + +Describe 'ConvertTo-CanonicalObject' { + It 'sorts dictionary keys deterministically' { + $result = ConvertTo-CanonicalObject -InputObject @{ b = 1; a = 2; c = 3 } + @($result.Keys) | Should -Be @('a', 'b', 'c') + } + + It 'preserves array order' { + $result = ConvertTo-CanonicalObject -InputObject @('z', 'a', 'm') + $result | Should -Be @('z', 'a', 'm') + } + + It 'normalizes PSCustomObject the same as an equivalent hashtable' { + $fromJson = '{"b":1,"a":2}' | ConvertFrom-Json + $fromHash = @{ a = 2; b = 1 } + (ConvertTo-CanonicalObject $fromJson | ConvertTo-Json) | + Should -Be (ConvertTo-CanonicalObject $fromHash | ConvertTo-Json) + } +} + +Describe 'Test-CanonicalSubset' { + It 'treats server-added fields as satisfied' { + $current = ConvertTo-CanonicalObject ('{"a":1,"b":2,"extra":"x"}' | ConvertFrom-Json) + $desired = ConvertTo-CanonicalObject @{ a = 1; b = 2 } + Test-CanonicalSubset -Superset $current -Subset $desired | Should -BeTrue + } + + It 'detects real value drift' { + $current = ConvertTo-CanonicalObject ('{"a":1,"b":9}' | ConvertFrom-Json) + $desired = ConvertTo-CanonicalObject @{ a = 1; b = 2 } + Test-CanonicalSubset -Superset $current -Subset $desired | Should -BeFalse + } + + It 'treats array order as significant' { + $current = ConvertTo-CanonicalObject @{ values = @('Module', 'Action') } + $desired = ConvertTo-CanonicalObject @{ values = @('Action', 'Module') } + Test-CanonicalSubset -Superset $current -Subset $desired | Should -BeFalse + } + + It 'treats a null desired value as satisfied when the key is absent' { + $current = ConvertTo-CanonicalObject @{ a = 1 } + $desired = ConvertTo-CanonicalObject @{ a = 1; default_value = $null } + Test-CanonicalSubset -Superset $current -Subset $desired | Should -BeTrue + } +} + +Describe 'Write-ConfigDiff' { + It 'returns $false when the desired state is already satisfied' { + $current = '{"value_type":"single_select","required":false,"extra":"x"}' | ConvertFrom-Json + $desired = @{ value_type = 'single_select'; required = $false } + Write-ConfigDiff -Title 'test' -Current $current -Desired $desired | Should -BeFalse + } + + It 'returns $true when drift exists' { + $current = '{"value_type":"multi_select"}' | ConvertFrom-Json + $desired = @{ value_type = 'single_select' } + Write-ConfigDiff -Title 'test' -Current $current -Desired $desired | Should -BeTrue + } + + It 'returns $true when current state is missing' { + Write-ConfigDiff -Title 'test' -Current $null -Desired @{ a = 1 } | Should -BeTrue + } +} + +Describe 'Resolve-PolicyLayerPaths' { + It 'returns the enterprise folder for the enterprise layer' { + $paths = Resolve-PolicyLayerPaths -PolicyRoot 'C:\root\PolicyEngine' -Enterprise 'MSXOrg' -Layer 'enterprise' + $paths | Should -Contain (Join-Path 'C:\root\PolicyEngine' 'Policies' 'MSXOrg' 'enterprise') + } + + It 'includes _default and org-specific folders for the repository layer' { + $paths = Resolve-PolicyLayerPaths -PolicyRoot 'C:\root\PolicyEngine' -Enterprise 'MSXOrg' -Layer 'repository' -Organization 'PSModule' -Repository 'GitHub' + $repoBase = Join-Path (Join-Path (Join-Path 'C:\root\PolicyEngine' 'Policies') 'MSXOrg') 'repository' + $paths | Should -Contain (Join-Path $repoBase '_default') + $paths | Should -Contain (Join-Path (Join-Path $repoBase 'PSModule') '_default') + $paths | Should -Contain (Join-Path (Join-Path $repoBase 'PSModule') 'GitHub') + } +} + +Describe 'Invoke-PoliciesForScope authMode routing' { + It 'routes the capability authMode to the enterprise ruleset handler' { + $script:CapturedAuthMode = $null + function Sync-EnterpriseRulesets { + param($Enterprise, $Rulesets, $Context, $AuthMode, [switch]$WhatIf) + $script:CapturedAuthMode = $AuthMode + } + + $catalog = @{ 'enterprise.repo-rulesets' = @{ authMode = 'enterprise-pat' } } + $policy = @{ + layer = 'enterprise' + capability = 'repo-rulesets' + enabled = $true + name = 'rs' + sourceFile = 'x' + config = @{ enterprise = 'MSXOrg'; rulesets = @(@{ name = 'r' }) } + } + + Invoke-PoliciesForScope -Policies @($policy) -FileSets @{} -CapabilityCatalog $catalog ` + -Context ([pscustomobject]@{}) -Enterprise 'MSXOrg' -TempPath $env:TEMP -WhatIf + + $script:CapturedAuthMode | Should -Be 'enterprise-pat' + } + + It 'skips disabled policies' { + $catalog = @{ 'enterprise.repo-rulesets' = @{ authMode = 'enterprise-pat' } } + $policy = @{ layer = 'enterprise'; capability = 'repo-rulesets'; enabled = $false; name = 'rs'; sourceFile = 'x' } + { Invoke-PoliciesForScope -Policies @($policy) -FileSets @{} -CapabilityCatalog $catalog ` + -Context ([pscustomobject]@{}) -Enterprise 'MSXOrg' -TempPath $env:TEMP -WhatIf } | Should -Not -Throw + } +} From 7a184a680d2edaede6c963612bf555afacc9b49f Mon Sep 17 00:00:00 2001 From: Marius Storhaug Date: Sat, 25 Jul 2026 21:50:38 +0200 Subject: [PATCH 19/20] docs: reconcile README with implementation and PAT risk Correct the file-set layout description (no org dimension) and clarify all-access discovery (org installations, org repos only). Document the CUSTO_ENTERPRISE_PAT blast radius and mitigations, and the WhatIf/report/test-gate safety model. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- README.md | 30 ++++++++++++++++++++++++++---- 1 file changed, 26 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 4d914f9..21fdba8 100644 --- a/README.md +++ b/README.md @@ -25,7 +25,7 @@ building their own distributor from scratch. ## How it works -File sets are organized by target organization, repository type, and selection: +File sets are organized by repository type and selection: ```text Repos/{Type}/{Selection}/ @@ -36,9 +36,10 @@ Repos/{Type}/{Selection}/ property. Each selection folder mirrors the root of a target repository. Target discovery scope is declared in [`config/targets.json`](config/targets.json), not hardcoded -in the sync script. The default scope is `all-access`, which scans all repositories visible to the -current GitHub App installation token. This keeps Custo org-agnostic and lets one runtime process -all organizations and repositories it can access. +in the sync script. The default scope is `all-access`, which enumerates the organizations the +current GitHub App installation can see (`GET /app/installations`, organization accounts only) and +then the repositories within each. User-owned repositories are not targeted. This keeps Custo +org-agnostic and lets one runtime process every organization and repository it can access. Policy behavior is defined by JSON documents under [`PolicyEngine/`](PolicyEngine/), not embedded in `targets.json`. This separates **capabilities** from **policy configuration**: @@ -101,6 +102,27 @@ Default enterprise policy configs live at: - `PolicyEngine/Policies/MSXOrg/organization/_default/none.policy.json` - `PolicyEngine/Policies/MSXOrg/repository/_default/file-subscription-service.policy.json` +### `CUSTO_ENTERPRISE_PAT` blast radius + +`admin:enterprise` is a high-privilege scope. Treat the PAT as a sensitive credential and reduce +its blast radius: + +- Scope the workflow to a protected [environment](https://docs.github.com/actions/deployment/targeting-different-environments) + with required reviewers, so the PAT is only exposed on approved runs. +- Restrict who can trigger `workflow_dispatch` and limit branches that can run the workflow. +- Prefer a dedicated bot identity and rotate the token regularly. +- Only enable capabilities with `authMode: enterprise-pat` when their enterprise API genuinely + cannot be reached by the GitHub App. + +## Safety and validation + +- Run the workflow (or `scripts/Sync-Files.ps1`) with `-WhatIf` first: every capability evaluates + current vs desired state and prints a diff without writing. +- A Markdown run report (orgs/repos processed, PRs, errors) is written to the job summary. +- `tests/` holds Pester suites for the diff, policy-loading and routing logic, plus conformance + checks for every capability/policy document. The [`CI`](.github/workflows/ci.yml) workflow runs + them with PSScriptAnalyzer on every push and pull request. + See [`AGENTS.md`](AGENTS.md) for operator runbook steps and current rollout blockers. ## License From 49772591d179849cd016b9857ad6cdb1d8c28a9d Mon Sep 17 00:00:00 2001 From: Marius Storhaug Date: Sat, 25 Jul 2026 21:52:28 +0200 Subject: [PATCH 20/20] test: make path and temp fixtures cross-platform The Resolve-PolicyLayerPaths and authMode routing tests hardcoded a C: drive and \C:\Users\marst\AppData\Local\Temp, which fail on the Linux CI runner (drive qualifier resolution and empty TEMP). Use System.IO.Path temp helpers so the suite passes on Windows and Linux. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- tests/Sync-Files.Tests.ps1 | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-) diff --git a/tests/Sync-Files.Tests.ps1 b/tests/Sync-Files.Tests.ps1 index 30a89e9..76024a8 100644 --- a/tests/Sync-Files.Tests.ps1 +++ b/tests/Sync-Files.Tests.ps1 @@ -85,14 +85,18 @@ Describe 'Write-ConfigDiff' { } Describe 'Resolve-PolicyLayerPaths' { + BeforeAll { + $script:PolicyRootFixture = Join-Path ([System.IO.Path]::GetTempPath()) 'PolicyEngine' + } + It 'returns the enterprise folder for the enterprise layer' { - $paths = Resolve-PolicyLayerPaths -PolicyRoot 'C:\root\PolicyEngine' -Enterprise 'MSXOrg' -Layer 'enterprise' - $paths | Should -Contain (Join-Path 'C:\root\PolicyEngine' 'Policies' 'MSXOrg' 'enterprise') + $paths = Resolve-PolicyLayerPaths -PolicyRoot $script:PolicyRootFixture -Enterprise 'MSXOrg' -Layer 'enterprise' + $paths | Should -Contain (Join-Path $script:PolicyRootFixture 'Policies' 'MSXOrg' 'enterprise') } It 'includes _default and org-specific folders for the repository layer' { - $paths = Resolve-PolicyLayerPaths -PolicyRoot 'C:\root\PolicyEngine' -Enterprise 'MSXOrg' -Layer 'repository' -Organization 'PSModule' -Repository 'GitHub' - $repoBase = Join-Path (Join-Path (Join-Path 'C:\root\PolicyEngine' 'Policies') 'MSXOrg') 'repository' + $paths = Resolve-PolicyLayerPaths -PolicyRoot $script:PolicyRootFixture -Enterprise 'MSXOrg' -Layer 'repository' -Organization 'PSModule' -Repository 'GitHub' + $repoBase = Join-Path (Join-Path (Join-Path $script:PolicyRootFixture 'Policies') 'MSXOrg') 'repository' $paths | Should -Contain (Join-Path $repoBase '_default') $paths | Should -Contain (Join-Path (Join-Path $repoBase 'PSModule') '_default') $paths | Should -Contain (Join-Path (Join-Path $repoBase 'PSModule') 'GitHub') @@ -100,6 +104,10 @@ Describe 'Resolve-PolicyLayerPaths' { } Describe 'Invoke-PoliciesForScope authMode routing' { + BeforeAll { + $script:TempFixture = [System.IO.Path]::GetTempPath() + } + It 'routes the capability authMode to the enterprise ruleset handler' { $script:CapturedAuthMode = $null function Sync-EnterpriseRulesets { @@ -118,7 +126,7 @@ Describe 'Invoke-PoliciesForScope authMode routing' { } Invoke-PoliciesForScope -Policies @($policy) -FileSets @{} -CapabilityCatalog $catalog ` - -Context ([pscustomobject]@{}) -Enterprise 'MSXOrg' -TempPath $env:TEMP -WhatIf + -Context ([pscustomobject]@{}) -Enterprise 'MSXOrg' -TempPath $script:TempFixture -WhatIf $script:CapturedAuthMode | Should -Be 'enterprise-pat' } @@ -127,6 +135,6 @@ Describe 'Invoke-PoliciesForScope authMode routing' { $catalog = @{ 'enterprise.repo-rulesets' = @{ authMode = 'enterprise-pat' } } $policy = @{ layer = 'enterprise'; capability = 'repo-rulesets'; enabled = $false; name = 'rs'; sourceFile = 'x' } { Invoke-PoliciesForScope -Policies @($policy) -FileSets @{} -CapabilityCatalog $catalog ` - -Context ([pscustomobject]@{}) -Enterprise 'MSXOrg' -TempPath $env:TEMP -WhatIf } | Should -Not -Throw + -Context ([pscustomobject]@{}) -Enterprise 'MSXOrg' -TempPath $script:TempFixture -WhatIf } | Should -Not -Throw } }