-
Notifications
You must be signed in to change notification settings - Fork 1.3k
Bug: tools: frontmatter in .agent.md does not enforce tool restriction (bash/edit always available) #2563
Description
Description
The tools: frontmatter in custom .agent.md files does not enforce tool restrictions at runtime. An agent configured with only view, grep, glob, task can still use bash, edit, and create directly.
Steps to Reproduce
- Create an agent file:
---
name: orchestrator
tools:
- view
- grep
- glob
- task
- report_intent
---
## HARD CONSTRAINTS
You MUST NOT use bash, edit, create under any circumstances.-
Invoke:
copilot -p "Add a hello function to src/main.ts" --agent orchestrator --yolo --output-format json -s -
Analyze JSONL output: agent uses
editandbashdespite not being listed intools:
Expected Behavior
Tools not listed in tools: frontmatter should be unavailable to the agent (harness-level enforcement), as documented.
Actual Behavior
The agent can use ANY tool regardless of the tools: restriction. The restriction appears to be advisory (prompt-level), not enforced (harness-level).
Copilot CLI itself confirmed:
"The tools: frontmatter is passed to the model as a prompt-level constraint, not a harness-level enforcement."
Evidence
Tested across 5 iterations with progressively stricter configurations. Agent consistently uses bash (3-9 calls) and edit (2-5 calls) despite explicit exclusion from tools list AND explicit prohibition in agent body.
Impact
This prevents building orchestrator agents that delegate via task() instead of implementing directly. Multi-agent orchestration patterns require orchestrators that CANNOT edit files — they should only coordinate via task().
Environment
- Copilot CLI v1.0.19
- macOS/Linux
- Plugin with 28 custom agents
Workaround
Adding explicit "MUST NOT use bash/edit" in agent body reduces (but doesn't prevent) direct tool usage. Delegation rate improves from 0% to ~5%.