Clarify PAT requirements for user-owned vs org-owned Projects v2 (#7041)

Copilot · web-flow · commit dd5fbd98d15a · 2025-12-20T06:31:51.000-08:00
diff --git a/docs/src/content/docs/examples/issue-pr-events/projectops.md b/docs/src/content/docs/examples/issue-pr-events/projectops.md
@@ -13,6 +13,46 @@ By default, `update-project` is update-only: create the Project once in the GitH
 
 **Important**: GitHub Projects v2 requires a PAT or GitHub App token - the default `GITHUB_TOKEN` cannot access Projects v2. Configure [`GH_AW_PROJECT_GITHUB_TOKEN`](/gh-aw/reference/tokens/#gh_aw_project_github_token-github-projects-v2) before using `update-project`.
 
+## Token Requirements for Projects v2
+
+The type of Personal Access Token (PAT) you need depends on whether you're working with **user-owned** or **organization-owned** Projects:
+
+### User-owned Projects (v2)
+
+**Must use a classic PAT** - Fine-grained PATs do **not** work with user-owned Projects.
+
+Required scopes:
+- `project` (required for user Projects)
+- `repo` (required if accessing private repositories)
+
+Create at: [https://github.com/settings/tokens/new](https://github.com/settings/tokens/new)
+
+### Organization-owned Projects (v2)
+
+**Can use either classic or fine-grained PAT**:
+
+**Option 1: Classic PAT**
+- `project` (required)
+- `read:org` (required for org Projects)
+- `repo` (required if accessing private repositories)
+
+**Option 2: Fine-grained PAT**
+- Repository access: Select specific repos or "All repositories"
+- **Organization permissions** (must be explicitly granted):
+  - **Organization access**: Must be granted to the target organization
+  - **Projects**: Read+Write
+- **Important**: Fine-grained PATs work by default only for public org resources. You must explicitly grant organization access and Projects permissions.
+
+Create at: [https://github.com/settings/personal-access-tokens/new](https://github.com/settings/personal-access-tokens/new)
+
+**Setup**: After creating your PAT, add it to your repository:
+
+```bash
+gh aw secrets set GH_AW_PROJECT_GITHUB_TOKEN --value "YOUR_PROJECT_PAT"
+```
+
+See the [GitHub Projects v2 token reference](/gh-aw/reference/tokens/#gh_aw_project_github_token-github-projects-v2) for complete details.
+
 ## When to Use ProjectOps
 
 ProjectOps complements [GitHub's built-in Projects automation](https://docs.github.com/en/issues/planning-and-tracking-with-projects/automating-your-project/using-the-built-in-automations) with AI-powered intelligence:
diff --git a/docs/src/content/docs/reference/safe-outputs.md b/docs/src/content/docs/reference/safe-outputs.md
@@ -360,7 +360,11 @@ safe-outputs:
 
 Agent output **must include a full GitHub project URL** in the `project` field (e.g., `https://github.com/orgs/myorg/projects/42` or `https://github.com/users/username/projects/5`). Project names or numbers alone are not accepted. Can also supply `content_number`, `content_type`, `fields`, and `campaign_id`. The job adds the issue or PR to the board, updates custom fields, applies `campaign:<id>` labels, and exposes `project-id`, `project-number`, `project-url`, `campaign-id`, and `item-id` outputs. Cross-repository targeting not supported.
 
-To opt in to creating missing project boards, include `create_if_missing: true` in the `update_project` output. Your token must have sufficient permissions to create projects in the organization (classic PAT with `project` + `repo` scopes, or fine-grained PAT with Projects: Read+Write, or GitHub App with Projects permissions).
+To opt in to creating missing project boards, include `create_if_missing: true` in the `update_project` output. Your token must have sufficient permissions:
+- **User-owned Projects**: Classic PAT with `project` + `repo` scopes (fine-grained PATs don't work)
+- **Organization-owned Projects**: Classic PAT with `project` + `read:org` scopes, or fine-grained PAT with explicit Organization access and Projects: Read+Write, or GitHub App with Projects permissions
+
+See [GitHub Projects v2 token requirements](/gh-aw/reference/tokens/#gh_aw_project_github_token-github-projects-v2) for detailed setup instructions.
 
 ### Pull Request Creation (`create-pull-request:`)
 
diff --git a/docs/src/content/docs/reference/tokens.md b/docs/src/content/docs/reference/tokens.md
@@ -197,13 +197,35 @@ A specialized token for GitHub Projects v2 operations used by the [`update-proje
 
 **Setup**:
 
-1. Create a **classic PAT** with `project` and `repo` scopes, **OR** a [fine-grained PAT](https://github.com/settings/personal-access-tokens/new) with:
+The required token type depends on whether you're working with **user-owned** or **organization-owned** Projects:
+
+**For User-owned Projects (v2)**:
+
+You **must** use a **classic PAT** with the `project` scope. Fine-grained PATs do **not** work with user-owned Projects.
+
+1. Create a [classic PAT](https://github.com/settings/tokens/new) with scopes:
+   - `project` (required for user Projects)
+   - `repo` (required if accessing private repositories)
+
+**For Organization-owned Projects (v2)**:
+
+You can use either a classic PAT or a fine-grained PAT:
+
+1. **Option A**: Create a **classic PAT** with `project` and `read:org` scopes:
+   - `project` (required)
+   - `read:org` (required for org Projects)
+   - `repo` (required if accessing private repositories)
+
+2. **Option B**: Create a [fine-grained PAT](https://github.com/settings/personal-access-tokens/new) with:
    - Repository access: Select specific repos or "All repositories"
-   - Organization permissions:
-     - Projects: Read+Write (for creating and managing org Projects)
-   - **OR** use a GitHub App with Projects: Read+Write permission
+   - **Organization permissions** (must be explicitly granted):
+     - **Organization access**: Must be granted to the target organization
+     - **Projects**: Read+Write (for creating and managing org Projects)
+   - **Important**: Fine-grained PATs work by default only for public org resources. You must explicitly grant organization access and Projects permissions.
 
-2. Add to repository secrets:
+3. **Option C**: Use a GitHub App with Projects: Read+Write permission
+
+After creating your token, add it to repository secrets:
 
 ```bash wrap
 gh aw secrets set GH_AW_PROJECT_GITHUB_TOKEN --value "YOUR_PROJECT_PAT"
@@ -235,7 +257,12 @@ safe-outputs:
 :::note[Default behavior]
 By default, `update-project` is **update-only**: it will not create projects. If a project doesn't exist, the job fails with instructions to create it manually.
 
-**Important**: The default `GITHUB_TOKEN` **cannot** be used for Projects v2 operations. You **must** configure `GH_AW_PROJECT_GITHUB_TOKEN` or provide a custom token via `safe-outputs.update-project.github-token`. GitHub Projects v2 requires GraphQL API access with a PAT (classic with `project` + `repo` scopes, or fine-grained with Projects permissions) or a GitHub App.
+**Important**: The default `GITHUB_TOKEN` **cannot** be used for Projects v2 operations. You **must** configure `GH_AW_PROJECT_GITHUB_TOKEN` or provide a custom token via `safe-outputs.update-project.github-token`. 
+
+**GitHub Projects v2 PAT Requirements**:
+- **User-owned Projects**: Require a **classic PAT** with the `project` scope (plus `repo` if accessing private repos). Fine-grained PATs do **not** work with user-owned Projects.
+- **Organization-owned Projects**: Can use either a classic PAT with `project` + `read:org` scopes, **or** a fine-grained PAT with explicit Organization access granted plus Projects: Read+Write permission. Fine-grained PATs work by default only for public org resources and must be explicitly granted organization access.
+- **GitHub App**: Works for both user and org Projects with Projects: Read+Write permission.
 
 To opt-in to creating projects, the agent must include `create_if_missing: true` in its output, and the token must have sufficient permissions to create projects in the organization.
 :::
diff --git a/pkg/cli/workflows/test-projectops-pat-requirements.md b/pkg/cli/workflows/test-projectops-pat-requirements.md
@@ -0,0 +1,184 @@
+---
+description: Test ProjectOps PAT requirements with actual trial repository testing
+on:
+  workflow_dispatch:
+    inputs:
+      test_user_projects:
+        description: "Test user-owned Projects (requires classic PAT in GH_AW_PROJECT_GITHUB_TOKEN)"
+        type: boolean
+        default: "true"
+      test_org_projects:
+        description: "Test org-owned Projects (requires PAT with org access)"
+        type: boolean
+        default: "true"
+      cleanup_trial_repos:
+        description: "Delete trial repositories after testing"
+        type: boolean
+        default: "true"
+  schedule: weekly on monday
+permissions:
+  contents: read
+  actions: read
+  issues: read
+  pull-requests: read
+name: Test ProjectOps PAT Requirements
+engine: copilot
+timeout-minutes: 45
+tools:
+  bash:
+    - "*"
+  github:
+    mode: remote
+    toolsets: [default]
+safe-outputs:
+  create-issue:
+    max: 1
+    expires: 7d
+    labels: [documentation, projectops, testing, trialops]
+---
+
+# Test ProjectOps PAT Requirements with TrialOps
+
+This workflow performs actual integration testing of ProjectOps PAT requirements by:
+1. Creating trial repositories
+2. Testing different PAT configurations with actual GitHub Projects v2 API calls
+3. Verifying documented requirements match real behavior
+4. Cleaning up trial repositories after testing
+
+## Prerequisites
+
+This test requires PATs to be configured:
+- `GH_AW_PROJECT_GITHUB_TOKEN`: PAT for testing (should have appropriate project scopes)
+- The workflow will test if the configured PAT works as documented
+
+## Test Execution
+
+Use `gh aw trial` to test ProjectOps workflows with different PAT configurations in isolated trial repositories:
+
+### Test 1: User-owned Projects with Classic PAT
+
+Test if a classic PAT with `project` scope can successfully manage user-owned Projects v2.
+
+**Steps**:
+1. Create a trial repository: `gh-aw-trial-projectops-user`
+2. Create a test workflow that uses `update-project` to:
+   - Add an issue to a user-owned Project
+   - Update project fields
+   - Verify the operations succeed
+3. Run the workflow with the configured PAT
+4. Check if operations succeed as documented
+
+**Expected Result**: Operations should succeed if using classic PAT with `project` scope
+
+### Test 2: Organization-owned Projects with Classic PAT
+
+Test if a classic PAT with `project` + `read:org` can manage org-owned Projects v2.
+
+**Steps**:
+1. Create a trial repository in an organization context
+2. Create a test workflow that uses `update-project` to:
+   - Add an issue to an org-owned Project
+   - Update project fields
+   - Verify the operations succeed
+3. Run with classic PAT having `project` + `read:org` scopes
+
+**Expected Result**: Operations should succeed with proper org permissions
+
+### Test 3: Organization-owned Projects with Fine-grained PAT
+
+Test if a fine-grained PAT with explicit org access + Projects: Read+Write can manage org-owned Projects.
+
+**Steps**:
+1. Use the same trial repository
+2. Test with fine-grained PAT (if configured separately)
+3. Verify org-owned project operations work
+
+**Expected Result**: Should work if org access was explicitly granted
+
+### Test 4: Document Fine-grained PAT Limitation for User Projects
+
+Test and document that fine-grained PATs do NOT work with user-owned Projects.
+
+**Note**: This test documents the limitation but may not be executable if we don't have a fine-grained PAT configured for comparison.
+
+## Implementation
+
+Execute the following tests using bash commands:
+
+**Step 1**: Create a minimal test workflow that attempts ProjectOps operations
+
+**Step 2**: Use `gh aw trial` command to run the workflow in an isolated trial repository
+
+**Step 3**: Analyze the results to determine if the configured PAT works as documented
+
+**Step 4**: Report findings in an issue
+
+The AI agent will:
+1. Generate a test workflow dynamically
+2. Execute it using the gh-aw trial command with appropriate flags
+3. Parse the results to determine PAT permission compatibility
+4. Create a summary issue with findings
+
+This provides real integration testing of the documented PAT requirements using actual GitHub Projects v2 API operations.
+
+## Test Results Report
+
+Create an issue summarizing the test results:
+
+**Title**: "ProjectOps PAT TrialOps Test Results - $(date +%Y-%m-%d)"
+
+**Body**:
+
+### Test Summary
+
+**Test Date**: $(date -u +"%Y-%m-%d %H:%M:%S UTC")
+**Trial Repository**: [Link to trial repo if not deleted]
+**Workflow Run**: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
+
+### Test Configuration
+
+- User-owned Projects Test: Configured via workflow dispatch input
+- Org-owned Projects Test: Configured via workflow dispatch input
+- Cleanup Trial Repos: Configured via workflow dispatch input
+
+### Results
+
+#### User-owned Projects
+- **Classic PAT with project scope**: [PASS/FAIL/NOT_TESTED]
+  - Details: [Error messages or success confirmation]
+  
+- **Fine-grained PAT**: [EXPECTED_TO_FAIL/NOT_TESTED]
+  - Details: [Confirmation of documented limitation]
+
+#### Organization-owned Projects
+- **Classic PAT with project + read:org**: [PASS/FAIL/NOT_TESTED]
+  - Details: [Error messages or success confirmation]
+  
+- **Fine-grained PAT with org access**: [PASS/FAIL/NOT_TESTED]
+  - Details: [Error messages or success confirmation]
+
+### Documentation Validation
+
+- **Documentation Accuracy**: [CONFIRMED/ISSUES_FOUND]
+- **Issues Found**: [List any discrepancies between docs and actual behavior]
+
+### Recommendations
+
+[Any updates needed to documentation based on test results]
+
+### Trial Artifacts
+
+- Trial repository: [URL or "Deleted after test"]
+- Trial results JSON: [Path to local results file]
+- GitHub Actions logs: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
+
+---
+
+**Note**: These tests use actual GitHub Projects v2 API calls with configured PATs. Results may vary based on:
+- PAT type and scopes configured
+- Organization settings and policies
+- Project ownership (user vs org)
+- GitHub API rate limits
+
+For manual testing, see the [TrialOps Guide](/gh-aw/guides/trialops/) and [ProjectOps Documentation](/gh-aw/examples/issue-pr-events/projectops/).
+
