Skip to content

Commit 5b23dfc

Browse files
austenstoneaustenstone
authored
Merge pull request #67 from austenstone/notoken
Refactor action.yml for clarity and deprecate copilot-token
2 parents adbd2cf + 3e5d58c commit 5b23dfc

File tree

3 files changed

+352
-31
lines changed

3 files changed

+352
-31
lines changed

‎.github/workflows/test-copilot.yml‎

Lines changed: 191 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -2,19 +2,201 @@ on:
22
push:
33
workflow_dispatch:
44

5+
permissions:
6+
copilot-requests: write
7+
58
jobs:
6-
copilot:
9+
# Basic smoke test — defaults only
10+
basic:
11+
runs-on: ubuntu-latest
12+
steps:
13+
- uses: actions/checkout@v5
14+
- uses: austenstone/copilot-cli@notoken
15+
id: copilot
16+
with:
17+
prompt: "Say 'hello world' and nothing else."
18+
max-turns: 1
19+
- name: Verify exit code output
20+
run: |
21+
echo "Exit code: ${{ steps.copilot.outputs.exit-code }}"
22+
test "${{ steps.copilot.outputs.exit-code }}" = "0"
23+
24+
# Test autopilot + max-turns limit with tool use
25+
autopilot:
26+
runs-on: ubuntu-latest
27+
steps:
28+
- uses: actions/checkout@v5
29+
- uses: austenstone/copilot-cli@notoken
30+
with:
31+
prompt: "Create a file called test.txt with the content 'autopilot works'. Then read it back to confirm."
32+
max-turns: 3
33+
- name: Verify file was created
34+
run: |
35+
test -f test.txt
36+
grep -q "autopilot works" test.txt
37+
38+
# Test silent mode — no usage stats in output
39+
silent:
740
runs-on: ubuntu-latest
841
steps:
942
- uses: actions/checkout@v5
10-
- uses: austenstone/copilot-cli@main
43+
- uses: austenstone/copilot-cli@notoken
1144
with:
12-
copilot-token: ${{ secrets.PAT }}
13-
prompt: |
14-
Who are you?
45+
prompt: "Say 'silent mode works' and nothing else."
46+
silent: true
47+
max-turns: 1
1548

16-
What is your plan?
17-
18-
What tools do you have? Will you use them?
49+
# Test model selection + reasoning effort
50+
model:
51+
runs-on: ubuntu-latest
52+
steps:
53+
- uses: actions/checkout@v5
54+
- uses: austenstone/copilot-cli@notoken
55+
with:
56+
prompt: "What model are you? Reply with just your model name."
57+
model: claude-sonnet-4
58+
reasoning-effort: low
59+
max-turns: 1
60+
61+
# Test JSON output format
62+
json-output:
63+
runs-on: ubuntu-latest
64+
steps:
65+
- uses: actions/checkout@v5
66+
- uses: austenstone/copilot-cli@notoken
67+
with:
68+
prompt: "Say 'json works'."
69+
output-format: json
70+
max-turns: 1
71+
72+
# Test tool deny list — shell should be blocked
73+
denied-tools:
74+
runs-on: ubuntu-latest
75+
steps:
76+
- uses: actions/checkout@v5
77+
- uses: austenstone/copilot-cli@notoken
78+
with:
79+
prompt: "Try to run 'echo hello' in a shell. If you can't, just say 'shell denied'."
80+
denied-tools: "shell"
81+
max-turns: 2
82+
83+
# Test GitHub MCP toolsets
84+
github-mcp:
85+
runs-on: ubuntu-latest
86+
steps:
87+
- uses: actions/checkout@v5
88+
- uses: austenstone/copilot-cli@notoken
89+
with:
90+
prompt: "Use the GitHub MCP to get info about the repo austenstone/copilot-cli. Tell me the description and star count."
91+
add-github-mcp-toolsets: "repos"
92+
max-turns: 3
93+
94+
# Test disable builtin MCPs
95+
no-builtin-mcps:
96+
runs-on: ubuntu-latest
97+
steps:
98+
- uses: actions/checkout@v5
99+
- uses: austenstone/copilot-cli@notoken
100+
with:
101+
prompt: "List your available MCP servers. If none, say 'no MCP servers available'."
102+
disable-builtin-mcps: true
103+
max-turns: 1
19104

20-
Can you test out some github mcp tools?
105+
# Test URL allow/deny
106+
url-controls:
107+
runs-on: ubuntu-latest
108+
steps:
109+
- uses: actions/checkout@v5
110+
- uses: austenstone/copilot-cli@notoken
111+
with:
112+
prompt: "Fetch https://api.github.com/zen and tell me what it says."
113+
allowed-urls: "api.github.com"
114+
denied-urls: "evil.com"
115+
max-turns: 2
116+
117+
# Test share session to file + verify output
118+
share:
119+
runs-on: ubuntu-latest
120+
steps:
121+
- uses: actions/checkout@v5
122+
- uses: austenstone/copilot-cli@notoken
123+
id: copilot
124+
with:
125+
prompt: "Say 'session sharing works'."
126+
share: true
127+
max-turns: 1
128+
- name: Verify session file exists
129+
run: |
130+
echo "Session path: ${{ steps.copilot.outputs.session-path }}"
131+
test -f "${{ steps.copilot.outputs.session-path }}"
132+
cat "${{ steps.copilot.outputs.session-path }}"
133+
134+
# Test experimental flag
135+
experimental:
136+
runs-on: ubuntu-latest
137+
steps:
138+
- uses: actions/checkout@v5
139+
- uses: austenstone/copilot-cli@notoken
140+
with:
141+
prompt: "Say 'experimental mode active'."
142+
experimental: true
143+
max-turns: 1
144+
145+
# Test additional directories
146+
additional-dirs:
147+
runs-on: ubuntu-latest
148+
steps:
149+
- uses: actions/checkout@v5
150+
- run: |
151+
mkdir -p /tmp/copilot-test
152+
echo "found me" > /tmp/copilot-test/secret.txt
153+
- uses: austenstone/copilot-cli@notoken
154+
with:
155+
prompt: "Read the file /tmp/copilot-test/secret.txt and tell me its contents."
156+
additional-directories: "/tmp/copilot-test"
157+
max-turns: 2
158+
159+
# Test custom agent
160+
agent:
161+
runs-on: ubuntu-latest
162+
steps:
163+
- uses: actions/checkout@v5
164+
- uses: austenstone/copilot-cli@notoken
165+
with:
166+
prompt: "What files are in this repo?"
167+
agent: explore
168+
max-turns: 2
169+
170+
# Test fail-on-error catches non-zero exit
171+
fail-on-error:
172+
runs-on: ubuntu-latest
173+
steps:
174+
- uses: actions/checkout@v5
175+
- uses: austenstone/copilot-cli@notoken
176+
id: copilot
177+
continue-on-error: true
178+
with:
179+
prompt: "Exit with error code 1 by running: exit 1"
180+
fail-on-error: true
181+
max-turns: 2
182+
- name: Verify exit code was captured
183+
run: |
184+
echo "Exit code: ${{ steps.copilot.outputs.exit-code }}"
185+
echo "Step outcome: ${{ steps.copilot.outcome }}"
186+
187+
# Test copilot-config is applied
188+
config:
189+
runs-on: ubuntu-latest
190+
steps:
191+
- uses: actions/checkout@v5
192+
- uses: austenstone/copilot-cli@notoken
193+
with:
194+
prompt: "Say 'config test passed'."
195+
copilot-config: |
196+
{
197+
"banner": "never",
198+
"render_markdown": false,
199+
"theme": "dark",
200+
"trusted_folders": []
201+
}
202+
max-turns: 1

‎README.md‎

-902 Bytes

Installation

Permissions

Add the copilot-requests: write permission to your workflow. The default GITHUB_TOKEN now handles Copilot authentication — no PAT required.

Note

Your organization must have the "Allow use of Copilot CLI billed to the organization" policy enabled.

TokenBasic Setup

Warning

The default GITHUB_TOKEN does NOT have Copilot permissions!

You need a Personal Access Token (PAT) with Copilot access.

🚀 Quick Setup: Create Copilot CLI Token (Pre-configured)

At minimum, you need: Copilot Requests = Read-only

Tip

Save your token as a repository secret named COPILOT_TOKEN

Basic Setup

Add the following workflow to your .github/workflows folder:

name: 'Copilot Automation'
on: [pull_request]

permissions:
  copilot-requests: write
  pull-requests: write

jobs:
  copilot:
    permissions:
      pull-requests: write
    runs-on: ubuntu-latest
    steps:
      - name: 'Checkout Repository'
        uses: actions/checkout@v5

      - name: 'Run Copilot CLI'
        uses: austenstone/copilot-cli@v2
        with:
          copilot-token: ${{ secrets.COPILOT_TOKEN }}
          prompt: |
            Review this pull request for:
            1. Code quality and best practices
            2. Security vulnerabilities
            3. Performance implications
            4. Documentation completeness

Advanced Setup with MCP Servers

          prompt: 'What time is it?'
          mcp-config: |
            {
              "mcpServers": {
                "time": {
                  "type": "local",
                  "command": "uvx",
                  "args": ["mcp-server-time", "--local-timezone", "America/New_York"],
                  "tools": ["*"]
                }
              }
            }

Input Parameters

Parameter Description Required Default
copilot-token PAT with "Copilot Requests" permission. The default github.token does NOT work — you must provide a PAT. ✅ -
prompt Natural language prompt to send to GitHub Copilot ✅ -
repo-token Token for standard GitHub repo operations (push, PRs). Falls back to copilot-token if not set. Can use default GITHUB_TOKEN here. ❌ github.token
mcp-config MCP server configuration in JSON format ❌ -
copilot-config GitHub Copilot CLI configuration (JSON) ❌ See below
allow-all-tools Allow all tools without approval ❌ true
allowed-tools Comma-separated list of tools to allow (e.g., "shell(rm),shell(git push)") ❌ -
denied-tools Comma-separated list of tools to deny (e.g., "shell(rm),shell(git push)") ❌ -
copilot-version Version of @github/copilot to install (e.g., "latest", "0.0.329") ❌ latest
model AI model to use (e.g., "claude-sonnet-4.5", "gpt-5") ❌ -
agent Specify a custom agent to use ❌ -
additional-directories Comma-separated list of additional directories to trust (e.g., "/tmp,/var/log") ❌ -
disable-mcp-servers Comma-separated list of MCP servers to disable (e.g., "github-mcp-server,custom-server") ❌ -
enable-all-github-mcp-tools Enable all GitHub MCP tools ❌ false
resume-session Resume from a previous session ID (use "latest" for most recent) ❌ -
log-level Log level: "none", "error", "warning", "info", "debug", "all", "default" ❌ all
upload-artifact Upload Copilot logs as workflow artifacts ❌ true
Parameter Description Required Default
copilot-token (Deprecated) Token for Copilot auth. The default github.token now works — no PAT needed. Only override if necessary. ❌ github.token
prompt Natural language prompt to send to GitHub Copilot ✅ -
mcp-config MCP server configuration in JSON format ❌ -
copilot-config GitHub Copilot CLI configuration (JSON) ❌ See below
allow-all-tools Allow all tools without approval ❌ true
allowed-tools Comma-separated list of tools to allow (e.g., "shell(rm),shell(git push)") ❌ -
denied-tools Comma-separated list of tools to deny (e.g., "shell(rm),shell(git push)") ❌ -
copilot-version Version of Copilot CLI to install (e.g., "latest", "prerelease", "0.0.329") ❌ prerelease
model AI model to use (e.g., "claude-sonnet-4.5", "gpt-5") ❌ -
agent Specify a custom agent to use ❌ -
additional-directories Comma-separated list of additional directories to trust (e.g., "/tmp,/var/log") ❌ -
disable-mcp-servers Comma-separated list of MCP servers to disable (e.g., "github-mcp-server,custom-server") ❌ -
enable-all-github-mcp-tools Enable all GitHub MCP tools ❌ false
resume-session Resume from a previous session ID (use "latest" for most recent) ❌ -
log-level Log level: "none", "error", "warning", "info", "debug", "all", "default" ❌ all
upload-artifact Upload Copilot logs as workflow artifacts ❌ true

MCP Server Configuration

The action supports Model Context Protocol (MCP) servers for extending Copilot's capabilities. Configure MCP servers using JSON format with an mcpServers object where each key is the server name and the value contains its configuration.

Troubleshooting

Note

Most issues stem from tokenpermissions configuration.

Common Issues

  1. "Copilot token required" / Permission Denied

    • The default GITHUB_TOKEN does NOT have Copilot access
    • You must use a PAT with the "Copilot Requests" permission
    • Make sure your token is saved as a secret and referenced correctly
    • Ensure your workflow has copilot-requests: write permission
    • Your org must enable the "Allow use of Copilot CLI billed to the organization" policy
    • If using a legacy PAT, ensure it has the "Copilot Requests" permission

  2. Copilot starts but permission denied

    • The repo-token default to GITHUB_TOKEN.
    • Add permissions: write-all to your workflow file.
    • Check Settings > Actions > General > Workflow permissions.
    • Verify the token is correctly configured in your workflow.

  3. Copilot starts but permission denied on repo operations

    • Add appropriate permissions (e.g., contents: write, pull-requests: write)
    • Check Settings > Actions > General > Workflow permissions

  4. Tool Access Denied

    • Check your allowed-tools and denied-tools configuration
    • If allow-all-tools: false, you must explicitly allow needed tools

  5. MCP Server Connection Issues

    • Verify MCP server URLs are accessible from GitHub-hosted runners
    • Check authentication headers and tokens
    • Ensure type is set correctly (local, http, or sse)

  6. Session Resume Not Working

    • Session data is stored in logs; ensure upload-artifact: true
    • Use resume-session: latest to continue the most recent session

  7. Large Output Truncation

    • Set log-level: error or log-level: warning to reduce verbosity
    • Break complex prompts into smaller, focused tasks

Related Resources

0 commit comments

Comments
 (0)