Skip to content

Commit e59994b

Browse files
labudisCopilotdsymereneexeener
authored
Improve issue triage template with structured triage flow (#332)
* Improve issue triage template with structured triage flow - Add set-issue-type and set-issue-field safe outputs - Restructure flow: gather context, quality check, triage, apply - Add quality gate for incomplete issues (ask for missing info) - Add issue type assignment (skip if already set) - Add issue field assignment (skip if already set) - Add duplicate detection with confidence levels (duplicate vs related) - Add structured triage report format with assessment table - Add label safety warning (labels can trigger automation) - Add explicit guardrails: don't invent context, don't overwrite metadata - Clarify comment audience is maintainers, not issue authors - Keep web search, debugging strategies, sub-task breakdown - Update docs to reflect new capabilities Inspired by patterns from the GitHub Issues triage agent. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Remove set-issue-field safe output (not yet supported by compiler) Keep field-related instructions in the prompt body so the agent can still recommend field values in the triage report. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add duplicate label action, remove unsupported set_issue_field - Instruct agent to apply duplicate label when high-confidence duplicate is detected and the label exists in the repo - Remove set_issue_field from Step 4 (compiler does not support it yet) - Keep field guidance in Step 3c for comment-only recommendations Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add auto-close spam, coding agent suitability, labels toolset - Add close-issue safe output to auto-close spam as not_planned - Apply invalid/spam label to spam issues - Apply needs-info/question label to incomplete issues - Add coding agent suitability assessment (Suitable/Needs more info/Not suitable) - Replace gh label list bash with labels toolset for richer metadata - Update Step 4 with close_issue action Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Remove issue fields references (not supported yet) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Remove field name row from assessment table Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix spam double-comment: single close comment, no triage report Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Apply suggestion from @reneexeener Co-authored-by: Renee Xu <reneexeener@github.com> --------- Co-authored-by: Tadas Labudis <labudis@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Don Syme <dsyme@users.noreply.github.com> Co-authored-by: Renee Xu <reneexeener@github.com>
1 parent c7d030c commit e59994b

2 files changed

Lines changed: 113 additions & 53 deletions

File tree

docs/issue-triage.md

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44
55
**Automatically triage issues when they are created or reopened**
66

7-
The [Issue Triage workflow](../workflows/issue-triage.md?plain=1) runs when issues are created or reopened to analyze content, check related items, categorize, add labels, and post triage comments.
7+
The [Issue Triage workflow](../workflows/issue-triage.md?plain=1) runs when issues are created or reopened to analyze content, check related items, set issue type, add labels, detect duplicates, and post structured triage reports.
88

99
## Installation
1010

@@ -22,11 +22,11 @@ This walks you through adding the workflow to your repository.
2222

2323
```mermaid
2424
graph LR
25-
A[Issue Created/Reopened] --> B[Analyze Content]
26-
B --> C[Check Related Items]
27-
C --> D[Categorize Issue]
28-
D --> E[Add Labels]
29-
E --> F[Post Triage Comment]
25+
A[Issue Created/Reopened] --> B[Gather Context]
26+
B --> C[Spam & Quality Check]
27+
C --> D[Triage: Type, Labels]
28+
D --> E[Detect Duplicates]
29+
E --> F[Post Triage Report]
3030
```
3131

3232
The workflow may search for relevant documentation, error messages, or similar issues online to assist with triage.
@@ -41,8 +41,8 @@ This workflow requires no configuration and works out of the box. You can custom
4141

4242
After editing run `gh aw compile` to update the workflow and commit all changes to the default branch.
4343

44-
### Human in the Loop
44+
### Human in the loop
4545

46-
- Review triage comments for accuracy
47-
- Validate label assignments and priority assessments
46+
- Review triage reports for accuracy
47+
- Validate label, type, and field assignments
4848
- Override or adjust triage decisions when needed

workflows/issue-triage.md

Lines changed: 104 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
11
---
22
description: |
33
Intelligent issue triage assistant that processes new and reopened issues.
4-
Analyzes issue content, selects appropriate labels, detects spam, gathers context
5-
from similar issues, and provides analysis notes including debugging strategies,
6-
reproduction steps, and resource links. Helps maintainers quickly understand and
7-
prioritize incoming issues.
4+
Analyzes issue content, detects spam and incomplete reports, selects appropriate
5+
labels, sets issue type, detects duplicates, and provides structured
6+
triage reports with debugging strategies and resource links. Helps maintainers
7+
quickly understand and prioritize incoming issues.
88
99
on:
1010
issues:
@@ -23,11 +23,17 @@ safe-outputs:
2323
add-labels:
2424
max: 5
2525
add-comment:
26+
set-issue-type:
27+
max: 1
28+
close-issue:
29+
target: "triggering"
30+
state-reason: "not_planned"
31+
max: 1
2632

2733
tools:
2834
web-fetch:
2935
github:
30-
toolsets: [issues]
36+
toolsets: [issues, labels]
3137
min-integrity: none # This workflow is allowed to examine and comment on any issues
3238

3339
timeout-minutes: 10
@@ -37,54 +43,108 @@ timeout-minutes: 10
3743

3844
<!-- Note - this file can be customized to your needs. Replace this section directly, or add further instructions here. After editing run 'gh aw compile' -->
3945

40-
You're a triage assistant for GitHub issues. Your task is to analyze issue #${{ github.event.issue.number }} and perform some initial triage tasks related to that issue.
46+
You are a triage assistant for GitHub issues. Your task is to analyze issue #${{ github.event.issue.number }}, categorize it with the right metadata, and help maintainers act quickly. Your triage comments are written for maintainers reviewing the triage, not for the issue author.
4147

42-
1. Select appropriate labels for the issue from the provided list.
48+
Do not make assumptions beyond what the issue content supports. Do not invent missing context.
4349

44-
2. Retrieve the issue content using the `get_issue` tool. If the issue is obviously spam, or generated by bot, or something else that is not an actual issue to be worked on, then add an issue comment to the issue with a one-sentence analysis and exit the workflow.
50+
## Step 1: Gather context
4551

46-
3. Next, use the GitHub tools to gather additional context about the issue:
52+
1. Retrieve the issue content using the `get_issue` tool.
53+
2. Fetch any comments on the issue using the `get_issue_comments` tool.
54+
3. Fetch the list of labels available in this repository using the `list_label` tool.
55+
4. Search for similar issues using the `search_issues` tool.
4756

48-
- Fetch the list of labels available in this repository. Use 'gh label list' bash command to fetch the labels. This will give you the labels you can use for triaging issues.
49-
- Fetch any comments on the issue using the `get_issue_comments` tool
50-
- Find similar issues if needed using the `search_issues` tool
51-
- List the issues to see other open issues in the repository using the `list_issues` tool
57+
## Step 2: Spam and quality check
5258

53-
4. Analyze the issue content, considering:
59+
**Spam and invalid issues:** If the issue is obviously spam, bot-generated, gibberish, or a test issue:
60+
- Apply the `invalid` or `spam` label if one exists in the repository.
61+
- Close the issue as "not planned" with a one-sentence reason (e.g., "Closing as spam."). No triage report, no assessment table.
62+
- Do not apply any other metadata. **Stop here; do not continue to Steps 3 or 4.**
5463

55-
- The issue title and description
56-
- The type of issue (bug report, feature request, question, etc.)
57-
- Technical areas mentioned
58-
- Severity or priority indicators
59-
- User impact
60-
- Components affected
64+
**Incomplete issues:** If the issue lacks enough detail for meaningful triage, add a comment that politely asks the author to provide the missing information:
65+
- For bugs: steps to reproduce, expected vs actual behavior, logs/errors, environment details.
66+
- For other issue types: equivalent details that would make the report actionable.
67+
- Apply a `needs-info` or `question` label if one exists in the repository.
6168

62-
5. Write notes, ideas, nudges, resource links, debugging strategies and/or reproduction steps for the team to consider relevant to the issue.
69+
Be specific about what is missing and why it is needed. Do not attempt to apply type or other labels to incomplete issues.
6370

64-
6. Select appropriate labels from the available labels list provided above:
71+
If the issue has sufficient detail, proceed to Step 3.
6572

66-
- Choose labels that accurately reflect the issue's nature
67-
- Be specific but comprehensive
68-
- Select priority labels if you can determine urgency (high-priority, med-priority, or low-priority)
69-
- Consider platform labels (android, ios) if applicable
70-
- Search for similar issues, and if you find similar issues consider using a "duplicate" label if appropriate. Only do so if the issue is a duplicate of another OPEN issue.
71-
- Only select labels from the provided list above
72-
- It's okay to not add any labels if none are clearly applicable
73+
## Step 3: Triage
7374

74-
7. Apply the selected labels:
75+
### 3a: Set issue type
7576

76-
- Use the `update_issue` tool to apply the labels to the issue
77-
- DO NOT communicate directly with users
78-
- If no labels are clearly applicable, do not apply any labels
77+
- If the issue already has a type set, do not change it.
78+
- Otherwise, determine the single best issue type (e.g., Bug, Feature, Task).
79+
- If no type is clearly supported by the issue content, leave it unset and note what is missing.
7980

80-
8. If very confident, add an issue comment to the issue with your analysis:
81-
- Start with "🎯 Agentic Issue Triage"
82-
- Provide a brief summary of the issue
83-
- Mention any relevant details that might help the team understand the issue better
84-
- Include any debugging strategies or reproduction steps if applicable
85-
- Suggest resources or links that might be helpful for resolving the issue or learning skills related to the issue or the particular area of the codebase affected by it
86-
- Mention any nudges or ideas that could help the team in addressing the issue
87-
- If you have possible reproduction steps, include them in the comment
88-
- If you have any debugging strategies, include them in the comment
89-
- If appropriate break the issue down to sub-tasks and write a checklist of things to do.
90-
- Use collapsed-by-default sections in the GitHub markdown to keep the comment tidy. Collapse all sections except the short main summary at the top.
81+
### 3b: Select labels
82+
83+
- Be cautious with labels; they can trigger automation in many repositories.
84+
- Choose labels that accurately reflect the issue's nature from the repository's available labels.
85+
- Select priority labels if you can determine urgency (high-priority, med-priority, low-priority).
86+
- Consider platform labels (android, ios) if applicable.
87+
- Do not apply labels that do not exist in the repository.
88+
- If no labels are clearly applicable, do not apply any.
89+
- It is better to under-label than to speculatively add labels.
90+
91+
### 3c: Detect duplicates and related issues
92+
93+
- Review the similar issues found in Step 1.
94+
- Classify matches as:
95+
- **Duplicate** (high confidence): the issue describes the same problem as an existing open issue. Include up to 3.
96+
- **Related**: similar domain or adjacent problem, but not a duplicate. Include up to 3.
97+
- If a high-confidence duplicate is found and the repository has a `duplicate` label, apply it.
98+
- If no similar issues are found, state that explicitly in your report.
99+
100+
### 3e: Assess coding agent suitability
101+
102+
Assess whether the issue is suitable for automated coding agent assignment:
103+
- **Suitable**: clear requirements, sufficient context, well-defined success criteria, self-contained scope.
104+
- **Needs more info**: potentially suitable but missing details needed to start.
105+
- **Not suitable**: requires investigation, design decisions, extensive coordination, or policy/architectural choices.
106+
107+
### 3f: Additional analysis
108+
109+
- Write notes, debugging strategies, and/or reproduction steps relevant to the issue.
110+
- Search the web for relevant documentation, error messages, or known solutions if applicable.
111+
- Suggest resources or links that might help resolve the issue.
112+
- If appropriate, break the issue down into sub-tasks with a checklist.
113+
114+
## Step 4: Apply results
115+
116+
Apply all triage results:
117+
- Use `set_issue_type` to set the issue type (if determined).
118+
- Use `update_issue` to apply labels.
119+
- Use `close_issue` to close the issue if it is spam (state reason: "not planned").
120+
- Add an issue comment with your triage report using the format below.
121+
122+
## Comment format
123+
124+
Use this structure for the triage comment. Use collapsed sections to keep it tidy.
125+
126+
```markdown
127+
## 🎯 Triage report
128+
129+
{2-3 sentence summary to help a maintainer quickly grasp the issue.}
130+
131+
### 📊 Assessment
132+
133+
| Dimension | Value | Reasoning |
134+
|---|---|---|
135+
| **Type** | [value or "unchanged"] | [brief] |
136+
| **Labels** | [values or "none"] | [brief] |
137+
| **Coding agent** | [Suitable / Needs more info / Not suitable] | [brief] |
138+
139+
### 🔗 Similar issues
140+
141+
- issue-url (duplicate/related) — [brief explanation]
142+
143+
<details><summary>💡 Notes and suggestions</summary>
144+
145+
{Debugging strategies, reproduction steps, resource links, sub-task checklists, nudges for the team.}
146+
147+
</details>
148+
```
149+
150+
If no similar issues were found, omit the "Similar issues" section. If there are no notes to add, omit the collapsed section.

0 commit comments

Comments
 (0)