Skip to content

Commit fb2bc78

Browse files
authored
chore(skills): rework /issue into a confidence-scored Q&A flow (tldraw#9034)
In order to make `/issue` useful for quick, low-context reports without sending downstream agents past missing intent, this PR creates issues immediately and then matures them through a confidence-scored follow-up flow. The old flow front-loaded codebase research before creating the issue. The new flow inverts that: create the issue from the user's initial description, read back the problem in product terms, score how complete the captured intent is, and ask the questions that would fill the gaps — updating the issue in place as answers arrive. ### What changes - **Create first, mature later.** The skill creates the issue right away so the user can track it, rather than researching the codebase to death first. - **Standard body shape.** The body starts with the user's verbatim description, then a short product-facing **readback** of the problem, a numbered list of **Open questions**, and a plain-text **Confidence** status line (e.g. `Confidence: 84%, ready to get started.`). - **Light investigation.** Codebase searching is kept to just enough to ground the confidence score and ask sharp questions. Anything answerable from the code should not become an open question, and implementation breadcrumbs (file paths, function names, fix recipes) stay out of the body unless the user asks for them. - **Q&A loop.** As the user answers, the issue is updated via `gh issue edit`: answers are written beneath each question, the readback is revised, the confidence line is recomputed, and the issue framing (title, type, labels) can be adjusted if an answer changes what the issue is. - **Critical vs. deferred questions.** Questions prefixed `Critical:` block readiness and cannot be skipped. Non-critical questions the user chooses not to answer are marked `_Deferred by user; not blocking implementation._` and do not block. - **Downstream guardrails.** The `take` skill now treats unresolved `Critical:` questions, `_Awaiting answer._` placeholders, and the `More Info Needed` label as blockers before implementation. - **Issue type helper.** `skills/issue/scripts/set-issue-type.sh` sets GitHub issue types via GraphQL because `gh issue create --type` is unreliable across versions. ### Change type - [x] `other` ### Test plan 1. Read the `/issue` workflow and confirm follow-up answers retrigger the skill and update the issue body in place. 2. Read the `write-issue` standards and confirm the readback, open questions, and deferred-question conventions are documented. 3. Read the `take` workflow and confirm unresolved agent-drafted questions block implementation. 4. Run `git diff --check`. ### Code changes | Section | LOC change | | -------------- | ---------- | | Config/tooling | +110 / -23 |
1 parent d9412f4 commit fb2bc78

4 files changed

Lines changed: 110 additions & 23 deletions

File tree

skills/issue/SKILL.md

Lines changed: 61 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -1,62 +1,101 @@
11
---
22
name: issue
3-
description: Create and research a GitHub issue in the tldraw repository from a user description. Use when the user invokes issue, asks to create an issue, report a bug, file a feature request, or add research to a new issue.
3+
description: Create a GitHub issue in the tldraw repository from a user description, then mature it through follow-up questions. Use when the user invokes issue, asks to create an issue, report a bug, file a feature request, or answers follow-up questions for an issue created by this skill.
44
---
55

66
# Issue
77

8-
Create a GitHub issue on `tldraw/tldraw` from a user description, then research it.
8+
Create a GitHub issue on `tldraw/tldraw` from a user description, then interrogate the user to capture enough of their intent for the issue to be worked on.
99

1010
Use `../write-issue/SKILL.md` as the standards reference for issue titles, bodies, types, labels, and triage conventions.
1111

12-
At the top of the issue, include the user's original description verbatim. Annotate it with the human emoji, separated from your work by a horizontal rule. Example:
12+
The goal is not to research the codebase to death. It is to capture the user's full intent and context in the issue, so that whoever (or whatever) picks it up later has what they need. You do this by creating the issue immediately, writing a brief readback of the problem, and then asking the user the questions that would sharpen that understanding.
13+
14+
## Issue body shape
15+
16+
The body always starts with the user's original description, verbatim, annotated with the human emoji and separated from the rest by a horizontal rule:
1317

1418
```
15-
🧑: {user_description}
19+
🗣️: {user_description}
1620
1721
---
22+
23+
{two to five sentences that read back your understanding of the problem, expected behavior, and scope}
24+
25+
## Open questions
26+
27+
1. **{question}**
28+
_Awaiting answer._
29+
2. **{question}**
30+
_Awaiting answer._
31+
32+
Confidence: {n}%, {ready_status}.
1833
```
1934

35+
- The readback paragraph is the agent's understanding of the problem. It should be short enough for the user to quickly correct, but specific enough to make the intended behavior clear. Keep it product-facing: no code blocks, no long implementation analysis, and no line-by-line diagnosis.
36+
- Do not include implementation breadcrumbs in the issue body by default. Avoid file paths, function names, line numbers, code snippets, likely causes, and fix recipes unless the user explicitly asks to include them. Whoever picks up the work can rediscover that technical context.
37+
- Keep product intent ahead of code diagnosis. The issue should record what the user wants, what they observed, and what scope or behavior they confirmed. Do not present a root cause or fix direction as fact unless the user has confirmed that framing.
38+
- Each open question targets a specific gap in intent or context. Avoid questions you could answer yourself by looking at the code.
39+
- Keep questions atomic. Do not bundle user-intent questions with implementer-verifiable technical checks. If the user answers only part of a compound question, keep the remaining part open only when it still needs the user's intent or context.
40+
- Prefer omitting implementer-verifiable unknowns over asking the user to verify them. Ask only when the user's own context or intent matters.
41+
- Prefix a question with `Critical:` inside the bold, e.g. `**Critical: Which package should we rename?**`, only when it is genuinely blocking — the issue cannot be worked on at all until it is answered. Most issues have none; do not inflate ordinary gaps into critical ones.
42+
- As the user answers, replace `_Awaiting answer._` with their answer (lightly cleaned up) directly beneath the question, revise the readback, and add or drop unanswered questions as needed.
43+
- If the user chooses not to answer a non-critical question, replace `_Awaiting answer._` with `_Deferred by user; not blocking implementation._`. Do not use this for critical questions.
44+
- The confidence line is a plain-text status line, not a section. It reflects whether the issue contains enough of the user's context and intent to be worked on, not confidence in the eventual fix. Use a short status phrase such as `ready to get started`, `still need more information`, or `blocked on a critical question`.
45+
2046
## Workflow
2147

2248
1. Gather context:
2349
- User's issue description.
2450
- Current branch: `git branch --show-current`.
2551
- Recent issues: `gh issue list --repo tldraw/tldraw --limit 5 --json number,title --jq '.[] | "#\(.number) \(.title)"'`.
26-
2. Do a quick codebase investigation:
27-
- Search for relevant files, functions, or patterns mentioned in the description.
28-
- Identify likely affected packages, apps, or examples.
29-
- Note obvious causes, related issues, or existing code paths.
52+
2. Do a lightweight triage check, not implementation research:
53+
- Look for duplicate or closely related issues.
54+
- Use repo context only to choose issue type, labels, milestone, and broad affected area.
55+
- If a shallow code search is needed to avoid asking a bad question, do it, but do not add file paths, function names, line numbers, code snippets, likely causes, or fix recipes to the issue body.
56+
- Anything you can answer yourself from repo context should not become an open question.
3057
3. For visual bugs, identify a reproduction target when possible:
3158
- Examples app: `localhost:5420` from `yarn dev`.
3259
- tldraw.com app: `localhost:3000` from `yarn dev-app`.
3360
- Docs site: `localhost:3001` from `yarn dev-docs`.
34-
- If screenshots are useful but not feasible locally, ask the user for screenshots and specific reproduction details.
35-
4. Write the issue title and body using `../write-issue/SKILL.md`.
61+
- If the user provided an image and you have a path or URL for it, embed or attach it in the GitHub issue.
62+
- If the image is visible only in the chat and cannot be attached, describe it as visual context. Do not write "screenshot attached" unless the issue actually contains the image.
63+
- If screenshots are useful but not feasible locally and the user has not provided one, make a screenshot request one of your open questions.
64+
4. Write the issue title and body using `../write-issue/SKILL.md`. The body follows the shape above: verbatim description, readback paragraph, open questions, and the confidence status line.
3665
5. Create the issue:
3766

3867
```bash
3968
gh issue create --repo tldraw/tldraw --title "..." --body "..."
4069
```
4170

42-
6. Set the issue type through GitHub GraphQL when possible, since `gh issue create --type` is not reliable across versions.
71+
6. Set the issue type (`Bug`, `Feature`, `Example`, or `Task`) — `gh issue create --type` is unreliable, so use the script:
72+
73+
```bash
74+
skills/issue/scripts/set-issue-type.sh <issue-number> <type-name>
75+
```
4376
7. Assign a milestone only when there is a clear fit:
4477
- `Improve developer resources` for examples, documentation, comments, starter kits, and `npm create tldraw`.
4578
- `Improve automations` for GitHub Actions, review bots, CI/CD, and automation work.
46-
8. Share the issue URL with the user immediately after creation.
47-
9. Do deeper research after creation:
48-
- Identify relevant files and line numbers.
49-
- Explain the root cause for bugs.
50-
- Summarize architecture context and related code.
51-
- Note edge cases, testing needs, and likely implementation risks.
52-
10. Add the research as an issue comment:
79+
8. Manage the `More Info Needed` label consistently:
80+
- Add it only when a critical question is unanswered or the confidence line says the issue still needs more information.
81+
- Remove it when the issue is ready to get started, even if non-blocking considerations remain.
82+
9. Respond to the user with the issue URL, your readback, and the list of open questions. Ask them directly, leading with any critical question.
83+
10. Interrogate the user. After each reply:
84+
- Update the issue body: write the user's answer beneath the relevant question, revise the readback, recompute the confidence status line, and add or drop questions as their answers reveal new gaps or close old ones.
85+
- Reconsider the issue's framing in light of the new context. If an answer changes what the issue actually is, update the title (`gh issue edit --title`), the issue type (the script from step 6), or its labels to match — for example, when a reported bug turns out to be a feature request, or the real problem is narrower than the title suggests.
86+
- If the user corrects your framing, especially with phrases like "actually" or "well actually," treat it as a signal to rewrite the readback around the corrected distinction. Remove or soften obsolete assumptions before asking more questions.
5387

5488
```bash
55-
gh issue comment <issue-number> --repo tldraw/tldraw --body "..."
89+
gh issue edit <issue-number> --repo tldraw/tldraw --body "..."
5690
```
5791

92+
- Keep going, one round at a time, until the readback and question answers hold enough of the user's intent and context to be worked on.
93+
11. The issue is ready when no critical question is unanswered and it holds enough of the user's intent to be worked on. Never declare it ready while a critical question is open, regardless of how complete the rest is.
94+
- Low-priority questions the user has chosen to defer may stay open; readiness does not require answering them. Mark them as `_Deferred by user; not blocking implementation._` rather than leaving `_Awaiting answer._` placeholders.
95+
- When ready, remove the `More Info Needed` label, tell the user the issue can be picked up, and set the confidence line accordingly, e.g. `Confidence: 84%, ready to get started.` Leave the readback, every question, and the confidence line in the issue as a record of the discussion. Do not delete them.
96+
5897
## Rules
5998

60-
- Always create the issue before doing deep research so the user can track it.
61-
- Follow `../write-issue/SKILL.md` for all issue content standards.
62-
- Do not include AI attribution in issue titles, bodies, comments, or metadata.
99+
- Always create the issue before interrogating the user, so they can track it from the first reply.
100+
- Ask only questions that capture the user's intent or context. Do not offload work you could do yourself.
101+
- Update the issue after every reply rather than batching answers at the end.
Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
#!/usr/bin/env bash
2+
# Set a GitHub issue's type (Bug, Feature, Example, Task, ...) via GraphQL.
3+
# The `gh issue create --type` flag is not reliable across versions, so we
4+
# resolve the issue and type node IDs and run the updateIssueIssueType mutation.
5+
#
6+
# Usage: set-issue-type.sh <issue-number> <type-name> [owner/repo]
7+
# set-issue-type.sh 1234 Bug
8+
# set-issue-type.sh 1234 Feature tldraw/tldraw
9+
set -euo pipefail
10+
11+
issue_number="${1:?usage: set-issue-type.sh <issue-number> <type-name> [owner/repo]}"
12+
type_name="${2:?usage: set-issue-type.sh <issue-number> <type-name> [owner/repo]}"
13+
repo="${3:-tldraw/tldraw}"
14+
owner="${repo%/*}"
15+
name="${repo#*/}"
16+
17+
issue_id=$(gh issue view "$issue_number" --repo "$repo" --json id --jq '.id')
18+
19+
type_id=$(gh api graphql \
20+
-f owner="$owner" -f name="$name" \
21+
-f query='query($owner:String!,$name:String!){repository(owner:$owner,name:$name){issueTypes(first:20){nodes{id name}}}}' \
22+
--jq ".data.repository.issueTypes.nodes[] | select(.name == \"$type_name\") | .id")
23+
24+
if [ -z "$type_id" ]; then
25+
echo "error: no issue type named '$type_name' in $repo" >&2
26+
echo "available types:" >&2
27+
gh api graphql -f owner="$owner" -f name="$name" \
28+
-f query='query($owner:String!,$name:String!){repository(owner:$owner,name:$name){issueTypes(first:20){nodes{name}}}}' \
29+
--jq '.data.repository.issueTypes.nodes[].name' >&2
30+
exit 1
31+
fi
32+
33+
gh api graphql \
34+
-f issueId="$issue_id" -f issueTypeId="$type_id" \
35+
-f query='mutation($issueId:ID!,$issueTypeId:ID!){updateIssueIssueType(input:{issueId:$issueId,issueTypeId:$issueTypeId}){issue{number issueType{name}}}}' \
36+
--jq '"set #\(.data.updateIssueIssueType.issue.number) type to \(.data.updateIssueIssueType.issue.issueType.name)"'

skills/take/SKILL.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,8 @@ Read the full issue and comments. Identify:
4242
- Technical notes and affected files.
4343
- Relevant discussion or clarifications.
4444

45+
Agent-drafted issues may include `## Confidence` and `## Open questions` sections. Treat any unanswered `Critical:` question, `_Awaiting answer._` entry, or `More Info Needed` label as a blocker unless codebase exploration proves the intended behavior is unambiguous. Resolve blockers with the user before implementing; non-critical questions marked `_Deferred by user; not blocking implementation._` may remain open.
46+
4547
If the issue lacks detail, explore the codebase before deciding whether implementation is safe.
4648

4749
### 3. Assign the issue
@@ -103,7 +105,7 @@ End with:
103105

104106
## Rules
105107

106-
- Ask the user when requirements are unclear.
108+
- Ask the user when requirements are unclear. Do not implement past unanswered critical questions or unresolved `_Awaiting answer._` placeholders.
107109
- Do not guess at unspecified product behavior.
108110
- Keep the implementation scoped to the issue.
109111
- Never include AI attribution in commits, issues, or PRs.

skills/write-issue/SKILL.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -90,6 +90,16 @@ Use sparingly (1-2 per issue) for metadata, not categorization.
9090
3. Suggested approach
9191
4. Which example category it belongs to
9292

93+
### Agent-drafted issues (problem readback and open questions)
94+
95+
Issues created by the `/issue` skill capture the user's intent over a short interrogation, so they carry an unheaded readback paragraph beneath the verbatim description, followed by open questions and a confidence status line at the bottom:
96+
97+
- **Problem readback** - A brief, unheaded paragraph that states the agent's interpretation of the problem, expected behavior, and scope. It should be easy for the user to correct. Keep it product-facing: no code blocks, no long implementation analysis, no file paths, no function names, no line numbers, and no fix recipes unless the user explicitly asks for them.
98+
- **Open questions** - A numbered list of the specific gaps in intent or context still worth asking the user about. Answers are written beneath each question as the user replies, and the readback is revised each round. A question prefixed with `Critical:` is genuinely blocking — the issue cannot be worked on until it is answered. Most issues have none. Non-critical questions the user chooses not to answer are marked `_Deferred by user; not blocking implementation._`.
99+
- **Confidence line** - A plain-text line at the bottom, not a section heading, such as `Confidence: 84%, ready to get started.` or `Confidence: 42%, still need more information.` It reflects whether the issue has enough of the user's intent and context to work on, not confidence in the eventual fix.
100+
101+
Leave the readback, open questions, and confidence line in the issue once interrogation is complete. The answered questions are a record of the discussion that produced the issue, so keep them rather than deleting them — mark questions resolved in place and keep the final readback as the issue's concise problem statement.
102+
93103
## Triage workflow
94104

95105
### New issues

0 commit comments

Comments
 (0)