diff --git a/docs/src/content/docs/examples/issue-pr-events/projectops.md b/docs/src/content/docs/examples/issue-pr-events/projectops.md index d2e3fb7c349..003cbfe8c57 100644 --- a/docs/src/content/docs/examples/issue-pr-events/projectops.md +++ b/docs/src/content/docs/examples/issue-pr-events/projectops.md @@ -13,6 +13,46 @@ By default, `update-project` is update-only: create the Project once in the GitH **Important**: GitHub Projects v2 requires a PAT or GitHub App token - the default `GITHUB_TOKEN` cannot access Projects v2. Configure [`GH_AW_PROJECT_GITHUB_TOKEN`](/gh-aw/reference/tokens/#gh_aw_project_github_token-github-projects-v2) before using `update-project`. +## Token Requirements for Projects v2 + +The type of Personal Access Token (PAT) you need depends on whether you're working with **user-owned** or **organization-owned** Projects: + +### User-owned Projects (v2) + +**Must use a classic PAT** - Fine-grained PATs do **not** work with user-owned Projects. + +Required scopes: +- `project` (required for user Projects) +- `repo` (required if accessing private repositories) + +Create at: [https://github.com/settings/tokens/new](https://github.com/settings/tokens/new) + +### Organization-owned Projects (v2) + +**Can use either classic or fine-grained PAT**: + +**Option 1: Classic PAT** +- `project` (required) +- `read:org` (required for org Projects) +- `repo` (required if accessing private repositories) + +**Option 2: Fine-grained PAT** +- Repository access: Select specific repos or "All repositories" +- **Organization permissions** (must be explicitly granted): + - **Organization access**: Must be granted to the target organization + - **Projects**: Read+Write +- **Important**: Fine-grained PATs work by default only for public org resources. You must explicitly grant organization access and Projects permissions. + +Create at: [https://github.com/settings/personal-access-tokens/new](https://github.com/settings/personal-access-tokens/new) + +**Setup**: After creating your PAT, add it to your repository: + +```bash +gh aw secrets set GH_AW_PROJECT_GITHUB_TOKEN --value "YOUR_PROJECT_PAT" +``` + +See the [GitHub Projects v2 token reference](/gh-aw/reference/tokens/#gh_aw_project_github_token-github-projects-v2) for complete details. + ## When to Use ProjectOps ProjectOps complements [GitHub's built-in Projects automation](https://docs.github.com/en/issues/planning-and-tracking-with-projects/automating-your-project/using-the-built-in-automations) with AI-powered intelligence: diff --git a/docs/src/content/docs/reference/safe-outputs.md b/docs/src/content/docs/reference/safe-outputs.md index 980b483a040..3b7f3224968 100644 --- a/docs/src/content/docs/reference/safe-outputs.md +++ b/docs/src/content/docs/reference/safe-outputs.md @@ -360,7 +360,11 @@ safe-outputs: Agent output **must include a full GitHub project URL** in the `project` field (e.g., `https://github.com/orgs/myorg/projects/42` or `https://github.com/users/username/projects/5`). Project names or numbers alone are not accepted. Can also supply `content_number`, `content_type`, `fields`, and `campaign_id`. The job adds the issue or PR to the board, updates custom fields, applies `campaign:` labels, and exposes `project-id`, `project-number`, `project-url`, `campaign-id`, and `item-id` outputs. Cross-repository targeting not supported. -To opt in to creating missing project boards, include `create_if_missing: true` in the `update_project` output. Your token must have sufficient permissions to create projects in the organization (classic PAT with `project` + `repo` scopes, or fine-grained PAT with Projects: Read+Write, or GitHub App with Projects permissions). +To opt in to creating missing project boards, include `create_if_missing: true` in the `update_project` output. Your token must have sufficient permissions: +- **User-owned Projects**: Classic PAT with `project` + `repo` scopes (fine-grained PATs don't work) +- **Organization-owned Projects**: Classic PAT with `project` + `read:org` scopes, or fine-grained PAT with explicit Organization access and Projects: Read+Write, or GitHub App with Projects permissions + +See [GitHub Projects v2 token requirements](/gh-aw/reference/tokens/#gh_aw_project_github_token-github-projects-v2) for detailed setup instructions. ### Pull Request Creation (`create-pull-request:`) diff --git a/docs/src/content/docs/reference/tokens.md b/docs/src/content/docs/reference/tokens.md index 34c7eb1b45f..721eff47cdb 100644 --- a/docs/src/content/docs/reference/tokens.md +++ b/docs/src/content/docs/reference/tokens.md @@ -197,13 +197,35 @@ A specialized token for GitHub Projects v2 operations used by the [`update-proje **Setup**: -1. Create a **classic PAT** with `project` and `repo` scopes, **OR** a [fine-grained PAT](https://github.com/settings/personal-access-tokens/new) with: +The required token type depends on whether you're working with **user-owned** or **organization-owned** Projects: + +**For User-owned Projects (v2)**: + +You **must** use a **classic PAT** with the `project` scope. Fine-grained PATs do **not** work with user-owned Projects. + +1. Create a [classic PAT](https://github.com/settings/tokens/new) with scopes: + - `project` (required for user Projects) + - `repo` (required if accessing private repositories) + +**For Organization-owned Projects (v2)**: + +You can use either a classic PAT or a fine-grained PAT: + +1. **Option A**: Create a **classic PAT** with `project` and `read:org` scopes: + - `project` (required) + - `read:org` (required for org Projects) + - `repo` (required if accessing private repositories) + +2. **Option B**: Create a [fine-grained PAT](https://github.com/settings/personal-access-tokens/new) with: - Repository access: Select specific repos or "All repositories" - - Organization permissions: - - Projects: Read+Write (for creating and managing org Projects) - - **OR** use a GitHub App with Projects: Read+Write permission + - **Organization permissions** (must be explicitly granted): + - **Organization access**: Must be granted to the target organization + - **Projects**: Read+Write (for creating and managing org Projects) + - **Important**: Fine-grained PATs work by default only for public org resources. You must explicitly grant organization access and Projects permissions. -2. Add to repository secrets: +3. **Option C**: Use a GitHub App with Projects: Read+Write permission + +After creating your token, add it to repository secrets: ```bash wrap gh aw secrets set GH_AW_PROJECT_GITHUB_TOKEN --value "YOUR_PROJECT_PAT" @@ -235,7 +257,12 @@ safe-outputs: :::note[Default behavior] By default, `update-project` is **update-only**: it will not create projects. If a project doesn't exist, the job fails with instructions to create it manually. -**Important**: The default `GITHUB_TOKEN` **cannot** be used for Projects v2 operations. You **must** configure `GH_AW_PROJECT_GITHUB_TOKEN` or provide a custom token via `safe-outputs.update-project.github-token`. GitHub Projects v2 requires GraphQL API access with a PAT (classic with `project` + `repo` scopes, or fine-grained with Projects permissions) or a GitHub App. +**Important**: The default `GITHUB_TOKEN` **cannot** be used for Projects v2 operations. You **must** configure `GH_AW_PROJECT_GITHUB_TOKEN` or provide a custom token via `safe-outputs.update-project.github-token`. + +**GitHub Projects v2 PAT Requirements**: +- **User-owned Projects**: Require a **classic PAT** with the `project` scope (plus `repo` if accessing private repos). Fine-grained PATs do **not** work with user-owned Projects. +- **Organization-owned Projects**: Can use either a classic PAT with `project` + `read:org` scopes, **or** a fine-grained PAT with explicit Organization access granted plus Projects: Read+Write permission. Fine-grained PATs work by default only for public org resources and must be explicitly granted organization access. +- **GitHub App**: Works for both user and org Projects with Projects: Read+Write permission. To opt-in to creating projects, the agent must include `create_if_missing: true` in its output, and the token must have sufficient permissions to create projects in the organization. ::: diff --git a/pkg/cli/workflows/test-projectops-pat-requirements.md b/pkg/cli/workflows/test-projectops-pat-requirements.md new file mode 100644 index 00000000000..e7d98edd1d9 --- /dev/null +++ b/pkg/cli/workflows/test-projectops-pat-requirements.md @@ -0,0 +1,184 @@ +--- +description: Test ProjectOps PAT requirements with actual trial repository testing +on: + workflow_dispatch: + inputs: + test_user_projects: + description: "Test user-owned Projects (requires classic PAT in GH_AW_PROJECT_GITHUB_TOKEN)" + type: boolean + default: "true" + test_org_projects: + description: "Test org-owned Projects (requires PAT with org access)" + type: boolean + default: "true" + cleanup_trial_repos: + description: "Delete trial repositories after testing" + type: boolean + default: "true" + schedule: weekly on monday +permissions: + contents: read + actions: read + issues: read + pull-requests: read +name: Test ProjectOps PAT Requirements +engine: copilot +timeout-minutes: 45 +tools: + bash: + - "*" + github: + mode: remote + toolsets: [default] +safe-outputs: + create-issue: + max: 1 + expires: 7d + labels: [documentation, projectops, testing, trialops] +--- + +# Test ProjectOps PAT Requirements with TrialOps + +This workflow performs actual integration testing of ProjectOps PAT requirements by: +1. Creating trial repositories +2. Testing different PAT configurations with actual GitHub Projects v2 API calls +3. Verifying documented requirements match real behavior +4. Cleaning up trial repositories after testing + +## Prerequisites + +This test requires PATs to be configured: +- `GH_AW_PROJECT_GITHUB_TOKEN`: PAT for testing (should have appropriate project scopes) +- The workflow will test if the configured PAT works as documented + +## Test Execution + +Use `gh aw trial` to test ProjectOps workflows with different PAT configurations in isolated trial repositories: + +### Test 1: User-owned Projects with Classic PAT + +Test if a classic PAT with `project` scope can successfully manage user-owned Projects v2. + +**Steps**: +1. Create a trial repository: `gh-aw-trial-projectops-user` +2. Create a test workflow that uses `update-project` to: + - Add an issue to a user-owned Project + - Update project fields + - Verify the operations succeed +3. Run the workflow with the configured PAT +4. Check if operations succeed as documented + +**Expected Result**: Operations should succeed if using classic PAT with `project` scope + +### Test 2: Organization-owned Projects with Classic PAT + +Test if a classic PAT with `project` + `read:org` can manage org-owned Projects v2. + +**Steps**: +1. Create a trial repository in an organization context +2. Create a test workflow that uses `update-project` to: + - Add an issue to an org-owned Project + - Update project fields + - Verify the operations succeed +3. Run with classic PAT having `project` + `read:org` scopes + +**Expected Result**: Operations should succeed with proper org permissions + +### Test 3: Organization-owned Projects with Fine-grained PAT + +Test if a fine-grained PAT with explicit org access + Projects: Read+Write can manage org-owned Projects. + +**Steps**: +1. Use the same trial repository +2. Test with fine-grained PAT (if configured separately) +3. Verify org-owned project operations work + +**Expected Result**: Should work if org access was explicitly granted + +### Test 4: Document Fine-grained PAT Limitation for User Projects + +Test and document that fine-grained PATs do NOT work with user-owned Projects. + +**Note**: This test documents the limitation but may not be executable if we don't have a fine-grained PAT configured for comparison. + +## Implementation + +Execute the following tests using bash commands: + +**Step 1**: Create a minimal test workflow that attempts ProjectOps operations + +**Step 2**: Use `gh aw trial` command to run the workflow in an isolated trial repository + +**Step 3**: Analyze the results to determine if the configured PAT works as documented + +**Step 4**: Report findings in an issue + +The AI agent will: +1. Generate a test workflow dynamically +2. Execute it using the gh-aw trial command with appropriate flags +3. Parse the results to determine PAT permission compatibility +4. Create a summary issue with findings + +This provides real integration testing of the documented PAT requirements using actual GitHub Projects v2 API operations. + +## Test Results Report + +Create an issue summarizing the test results: + +**Title**: "ProjectOps PAT TrialOps Test Results - $(date +%Y-%m-%d)" + +**Body**: + +### Test Summary + +**Test Date**: $(date -u +"%Y-%m-%d %H:%M:%S UTC") +**Trial Repository**: [Link to trial repo if not deleted] +**Workflow Run**: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + +### Test Configuration + +- User-owned Projects Test: Configured via workflow dispatch input +- Org-owned Projects Test: Configured via workflow dispatch input +- Cleanup Trial Repos: Configured via workflow dispatch input + +### Results + +#### User-owned Projects +- **Classic PAT with project scope**: [PASS/FAIL/NOT_TESTED] + - Details: [Error messages or success confirmation] + +- **Fine-grained PAT**: [EXPECTED_TO_FAIL/NOT_TESTED] + - Details: [Confirmation of documented limitation] + +#### Organization-owned Projects +- **Classic PAT with project + read:org**: [PASS/FAIL/NOT_TESTED] + - Details: [Error messages or success confirmation] + +- **Fine-grained PAT with org access**: [PASS/FAIL/NOT_TESTED] + - Details: [Error messages or success confirmation] + +### Documentation Validation + +- **Documentation Accuracy**: [CONFIRMED/ISSUES_FOUND] +- **Issues Found**: [List any discrepancies between docs and actual behavior] + +### Recommendations + +[Any updates needed to documentation based on test results] + +### Trial Artifacts + +- Trial repository: [URL or "Deleted after test"] +- Trial results JSON: [Path to local results file] +- GitHub Actions logs: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + +--- + +**Note**: These tests use actual GitHub Projects v2 API calls with configured PATs. Results may vary based on: +- PAT type and scopes configured +- Organization settings and policies +- Project ownership (user vs org) +- GitHub API rate limits + +For manual testing, see the [TrialOps Guide](/gh-aw/guides/trialops/) and [ProjectOps Documentation](/gh-aw/examples/issue-pr-events/projectops/). +