Skip to content

Commit f48cdad

Browse files
authored
Keep plan-pr-batch goal prompts under 4000 chars (#4025)
Fixes #4024 ## Summary - Keep the ready `$pr-batch` goal prompt under 4000 characters by moving bulky detail into the Batch Plan. - Require a measured goal prompt character count in `$plan-pr-batch` output. - Add a split fallback and self-check coverage for template size, oversized detail handling, and prompt self-containment. ## Validation - `ruby .agents/skills/plan-pr-batch/scripts/check_goal_prompt_size.rb` -> `goal_prompt_template_chars=2040`, `oversized_candidate_chars=15677`, `split_fallback_goal_prompt_chars=1939` - `ruby -cw .agents/skills/plan-pr-batch/scripts/check_goal_prompt_size.rb` -> `Syntax OK` - `git diff origin/main --check` -> passed - `script/ci-changes-detector origin/main` -> documentation-only; recommended CI jobs: none - `(cd react_on_rails && BUNDLE_GEMFILE=../Gemfile bundle exec rubocop)` -> 218 files inspected, no offenses - `codex review --base origin/main` -> accepted one P2 about self-contained prompts, fixed it, rerun clean ## CI / Labels Labels: none. This is process tooling/docs only and the detector recommends no CI expansion. Confidence note: - Validated: command list above, plus pre-commit hooks on the amended commit. - Evidence: local command output and clean branch review. - UNKNOWN: none. - Residual risk: low; limited to the prompt-size workflow contract. Decision points: 0 <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Low Risk** > Documentation and agent-skill tooling only; no runtime product code, auth, or data paths. > > **Overview** > Tightens **`$plan-pr-batch`** so handoff prompts stay under **4000 characters** (not bytes): planners must measure only the fenced body, print `Goal prompt character count: N characters`, move bulky audit text into the Batch Plan, and if still over budget emit **only the first ready goal** with omitted items listed in the plan. > > The **Goal Prompt** template is slimmed (e.g. `Scope summary`, one-line item fields) and long **Execution rules** are compressed while keeping merge authorization, coordination, and UNKNOWN reporting. Batch Plan format adds a **Prompt sizing** line. > > Adds **`check_goal_prompt_size.rb`**, which extracts the template from `SKILL.md`, pins required phrases, asserts template and split-fallback prompts stay under the limit, and simulates an oversized batch. A **Self-Check** section documents running it after skill edits. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit acea61e. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Tightened goal-prompt sizing rules: character counts now measure only the fenced prompt body (excluding fence lines) and require `Goal prompt character count: N characters` after the fence. * If too large, shrink via batch planning; if still oversized, split into smaller goals and emit only the first ready goal while listing omitted ready items in the batch plan. * Updated execution/coordination instructions and the batch plan formatting, and added a “Self-Check” section. * **New Features** * Added a Ruby validation script to verify the fenced goal-prompt template and split-fallback variants stay within a 4,000-character limit and output character-count diagnostics. <!-- end of auto-generated comment: release notes by coderabbit.ai --> ## Agent Merge Confidence Mode: accelerated-rc Current head SHA: acea61e Score: 9/10 Auto-merge recommendation: yes Affected areas: agent workflow/process tooling; no runtime product code CI detector: `script/ci-changes-detector origin/main` -> documentation-only changes; recommended CI jobs: none Checks: `gh pr checks 4025` -> 22 pass, 20 skipped, 0 pending, 0 failing. Skips are explained by the CI selector/detector for docs/process-only changes. Validation run: - `ruby .agents/skills/plan-pr-batch/scripts/check_goal_prompt_size.rb` -> `goal_prompt_template_chars=2040`, `oversized_candidate_chars=15677`, `split_fallback_goal_prompt_chars=1939` - `ruby -cw .agents/skills/plan-pr-batch/scripts/check_goal_prompt_size.rb` -> `Syntax OK` - `git diff origin/main --check` -> passed - `(cd react_on_rails && BUNDLE_GEMFILE=../Gemfile bundle exec rubocop)` -> 218 files inspected, no offenses - `codex review --base origin/main` -> accepted one P2 about self-contained prompts, fixed it, rerun clean Review/check gate: - GitHub checks: complete for `acea61e07b28b43adf3bcdb48a627f62e1cf7165`; no failed or pending checks - Review threads: live GraphQL unresolved count is 0 - Feedback triage: stale CodeRabbit/Claude/Codex/Cursor findings were either fixed or resolved; no current unresolved non-trivial concern remains - Current-head reviewer verdicts: - `claude-review` check: success for `acea61e07b28b43adf3bcdb48a627f62e1cf7165` ([run](https://github.com/shakacode/react_on_rails/actions/runs/27516264076), [job](https://github.com/shakacode/react_on_rails/actions/runs/27516264076/job/81325409388)) - Cursor Bugbot: pass for current head - CodeRabbit: pass for current head - Stale reviewer verdicts, advisory only: - CodeRabbit requested changes on `805e618b0eaa079f45fc2c7a1be2dc7137ad55fd`, then later approved an older intermediate head; not cited as a merge gate Labels: none. Process tooling/docs change; CI detector recommends no CI expansion. Known residual risk: low; limited to future agent prompt-sizing behavior, covered by the added validator and local checks. Merge recommendation: merge now by squash. Finalized by: `claude-review` GitHub check, Claude Code Review run `27516264076` / job `81325409388`, completed successfully for current head `acea61e07b28b43adf3bcdb48a627f62e1cf7165`.
1 parent 5abe360 commit f48cdad

2 files changed

Lines changed: 158 additions & 27 deletions

File tree

.agents/skills/plan-pr-batch/SKILL.md

Lines changed: 35 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -172,10 +172,20 @@ Plan a PR batch
172172
- For issues, define the expected deliverable: fix, investigation, reproduction, docs update, or no-PR audit.
173173

174174
4. Output
175+
<!-- prompt-size-check: scripts/check_goal_prompt_size.rb pins selected wording in this section. -->
175176
- Return a concise "Batch Plan" and a fenced "Goal Prompt for pr-batch".
176-
- Keep the fenced goal prompt under ~4000 bytes total so bulky audit detail stays in the Batch Plan. Measure it, do not eyeball it: `wc -c` gives a locale-independent byte count (`wc -m` counts characters using the current locale's multibyte rules — UTF-8 gives code points, C/POSIX gives bytes). Treat 4000 as an approximate budget.
177+
- Keep the fenced goal prompt under 4000 characters total so bulky detail stays in the Batch Plan. Measure it,
178+
do not eyeball it: use the guard script below, or pipe only the extracted fence body to a character-counting
179+
command such as `ruby -e 'print STDIN.read.length'`. Do not use byte-oriented counts such as `wc -c`.
180+
- Use compact one-line item goals, short worker notes, and canonical workflow references instead of copied
181+
audit evidence, repeated issue text, or long rule explanations.
182+
- Before responding, measure only the text inside the goal-prompt fence, excluding the fence lines, and print
183+
`Goal prompt character count: N characters` after the fence.
184+
- If the measured prompt is 4000 characters or more, shrink by moving detail to the Batch Plan. If it still
185+
will not fit, split it into smaller goals and output only the first ready goal; list omitted ready items in
186+
the Batch Plan for later goal prompts.
177187
- Measure the actual filled template overhead when the prompt is near the
178-
byte budget; do not rely on a fixed estimate. Prefer splitting into
188+
character budget; do not rely on a fixed estimate. Prefer splitting into
179189
multiple goals over trimming the safety, ownership, or review content.
180190
- Keep full path evidence in the Batch Plan when it would bloat the prompt,
181191
but do not leave the worker handoff with an external-only pointer. In the
@@ -202,11 +212,14 @@ Plan a PR batch
202212
- Concurrent activity and dependency status:
203213
- Coordination hooks, including backend claim exclusions:
204214
- Verification expectations:
215+
- Prompt sizing: `Goal prompt character count: N characters`; note any split fallback and keep omitted item
216+
details here, not in the goal prompt.
205217
- Open questions:
206218

207219
## Goal Prompt for pr-batch
208220

209-
Use this template and fill it with the verified items:
221+
Use this template and fill it with the verified items. Keep bulky evidence, long
222+
validation notes, and later-batch details outside the prompt.
210223

211224
```text
212225
Use $pr-batch to complete this batch with subagents.
@@ -215,6 +228,7 @@ Preflight first: if this session cannot run workers without blocking approval pr
215228
216229
Repository: OWNER/REPO
217230
Batch objective: ...
231+
Scope summary: [one paragraph: compact titles, sequencing, dependencies, exclusions, and path ownership for this batch. Keep bulky evidence, long validation notes, and later-batch details outside this prompt.]
218232
File-touch map (one line per item; pick the applicable format):
219233
- PR/Issue #N -> changed/affected paths, including create/delete/rename (owner: lane/name)
220234
- PR/Issue #N -> summarized path pattern(s) plus collision-relevant exact paths/renames/deletes (owner: lane/name)
@@ -224,17 +238,17 @@ Batch-level reservations, not tied to a single item:
224238
225239
Items:
226240
- PR #N: URL
227-
Goal: ...
228-
Worker notes: ...
229-
Done when: ...
241+
Goal: one-line outcome.
242+
Worker notes: short scope, branch, or dependency note.
243+
Done when: PR merged only if explicitly authorized by the current user or batch goal and allowed by release-mode gates, or ready/blocked/deferred with evidence.
230244
- Issue #N: URL
231-
Goal: ...
232-
Worker notes: ...
233-
Done when: ...
245+
Goal: one-line outcome.
246+
Worker notes: short scope, branch, or dependency note.
247+
Done when: PR merged only if explicitly authorized by the current user or batch goal and allowed by release-mode gates, ready/blocked/no-PR evidence, or documented no-fix rationale.
234248
235249
Execution rules:
236250
- Follow `.agents/skills/pr-batch/SKILL.md` "Goal Prompt Template"; if skill autoloading is unavailable, copy its safety, review, /simplify, CI, and readiness gates before running.
237-
- Dispatch only the current file-disjoint wave. Hold serial and `UNKNOWN`
251+
- Dispatch one subagent per independent item; group dependent items only when shared context is required. Dispatch only the current file-disjoint wave. Hold serial and `UNKNOWN`
238252
discovery lanes until no active editor lane can collide with them.
239253
- Workers edit only owned File-touch map paths; this map is how the batch makes
240254
pr-batch's "disjoint write scopes" concrete, since pr-batch's own template has
@@ -243,23 +257,9 @@ Execution rules:
243257
coordinator confirmation before editing.
244258
- Sequenced lanes may share declared files only in the stated order.
245259
- Each subagent must verify current GitHub state before edits and report UNKNOWN for unverifiable facts.
246-
- For concurrent or dependency-sensitive batches, assign a stable agent id and
247-
lane name per lane. Declare lane dependencies with `depends_on` refs such as
248-
`<batch-id>:<lane-name>`, and create or update the private backend
249-
`batches/<batch-id>.json` before dispatching dependent workers.
250-
When the private coordination backend is available,
251-
verify it with `agent-coord doctor` and `agent-coord status`, use
252-
`agent-coord claim` before creating worktrees/branches,
253-
`agent-coord heartbeat` at phase transitions, and `agent-coord status` at
254-
lane start and before rebase or push. If the lane shows unmet `blocked_on`
255-
refs, set heartbeat `--status blocked`, report the blocked refs, and move to
256-
another independent lane until dependencies report a backend terminal
257-
heartbeat status. If a lane declares `depends_on` but `agent-coord status`
258-
shows no matching private batch state, treat dependency state as `UNKNOWN` and
259-
stop to report the missing private batch file.
260-
If status cannot be checked for a declared dependency lane, stop with
261-
dependency state `UNKNOWN` instead of using advisory fallback for that lane.
262-
- Final handoff must include links, tests, blockers, next action, and merged/ready/blocked/deferred/UNKNOWN sections.
260+
- For coordination, respect coordination claims and dependencies: assign stable agent ids, run `agent-coord doctor` then `agent-coord status`, claim before branch/worktree creation when available, heartbeat at phase changes, and stop on unmet `blocked_on` refs or dependency state `UNKNOWN`.
261+
- Use local validation, self-review, review-comment, CI, and readiness gates from the repo workflow. For PRs, merge only if explicitly authorized by the current user or batch goal, current release mode permits it, and confidence/readiness gates pass; document confidence data in the PR description. Otherwise report the live ready/blocked/deferred/no-PR state with evidence.
262+
- Final handoff must include links, tests, blockers, next action, confidence or UNKNOWN facts, and merged/ready/blocked/deferred sections.
263263
```
264264

265265
## Common Mistakes
@@ -273,3 +273,11 @@ Execution rules:
273273
- Do not put full audit evidence in the goal prompt; put bulky details in the Batch Plan outside the goal.
274274
- Do not fan out items that change the same path as parallel worktrees; they will conflict — sequence them or split into a later batch.
275275
- Do not eyeball the goal-prompt length; apply the Output-section size gate and split into smaller goals if it is over budget.
276+
277+
## Self-Check
278+
279+
After editing this skill's goal prompt rules or template, run:
280+
281+
```bash
282+
ruby .agents/skills/plan-pr-batch/scripts/check_goal_prompt_size.rb
283+
```
Lines changed: 123 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,123 @@
1+
#!/usr/bin/env ruby
2+
# frozen_string_literal: true
3+
4+
GOAL_PROMPT_CHAR_LIMIT = 4_000
5+
TEXT_FENCE = "```text\n"
6+
7+
def abort_with_failure(message)
8+
abort "FAIL: #{message}"
9+
end
10+
11+
def extract_goal_prompt_template(skill_text)
12+
heading_index = skill_text.index("## Goal Prompt for pr-batch")
13+
abort_with_failure("missing Goal Prompt for pr-batch section") unless heading_index
14+
15+
fence_start = skill_text.index(TEXT_FENCE, heading_index)
16+
abort_with_failure("missing text fence in Goal Prompt section") unless fence_start
17+
18+
fence_body_start = fence_start + TEXT_FENCE.length
19+
next_heading = skill_text.match(/^##\s+/, fence_body_start)
20+
section_end = next_heading ? next_heading.begin(0) : skill_text.length
21+
section_body = skill_text[fence_body_start...section_end]
22+
fence_offsets = []
23+
section_body.scan(/^```\s*$/) { fence_offsets << Regexp.last_match.begin(0) }
24+
25+
abort_with_failure("missing closing fence in Goal Prompt section") if fence_offsets.empty?
26+
if fence_offsets.length > 1
27+
abort_with_failure("goal prompt template contains a nested bare fence line; use a non-text fence type instead")
28+
end
29+
30+
section_body[0...fence_offsets.first]
31+
end
32+
33+
def with_items(prompt_template, items)
34+
updated_prompt = prompt_template.sub(/Items:\n.*?\n{2,}Execution rules:/m) do
35+
"Items:\n#{items}\n\nExecution rules:"
36+
end
37+
if updated_prompt == prompt_template
38+
abort_with_failure(
39+
"goal prompt template must contain an Items section followed by a blank line and Execution rules:"
40+
)
41+
end
42+
43+
updated_prompt
44+
end
45+
46+
skill_path = File.expand_path("../SKILL.md", __dir__)
47+
abort_with_failure("SKILL.md not found at #{skill_path}") unless File.exist?(skill_path)
48+
49+
skill_text = File.read(skill_path, encoding: "UTF-8")
50+
prompt_template = extract_goal_prompt_template(skill_text)
51+
52+
required_skill_rule_phrases = [
53+
"Goal prompt character count:",
54+
"If the measured prompt is 4000 characters or more",
55+
"output only the first ready goal",
56+
"bulky detail stays in the Batch Plan",
57+
"Keep bulky evidence",
58+
"outside the prompt"
59+
]
60+
61+
required_prompt_phrases = [
62+
"merged only if explicitly authorized",
63+
"document confidence data in the PR description",
64+
"verify current GitHub state before edits",
65+
"respect coordination claims and dependencies",
66+
"report UNKNOWN"
67+
]
68+
69+
required_skill_rule_phrases.each do |phrase|
70+
# These phrases live in the broader skill rules, not necessarily inside the prompt fence.
71+
abort_with_failure("SKILL.md is missing required prompt-sizing phrase: #{phrase}") unless skill_text.include?(phrase)
72+
end
73+
74+
required_prompt_phrases.each do |phrase|
75+
unless prompt_template.include?(phrase)
76+
abort_with_failure("Goal prompt template is missing required phrase: #{phrase}")
77+
end
78+
end
79+
80+
if prompt_template.match?(/Batch Plan/i)
81+
abort_with_failure("goal prompt template must be self-contained and not depend on Batch Plan context")
82+
end
83+
84+
template_chars = prompt_template.length
85+
if template_chars >= GOAL_PROMPT_CHAR_LIMIT
86+
abort_with_failure("goal prompt template is #{template_chars} chars, must stay under #{GOAL_PROMPT_CHAR_LIMIT}")
87+
end
88+
89+
bulky_items = (1..12).map do |number|
90+
<<~ITEM.chomp
91+
- Issue ##{number}: https://github.com/shakacode/react_on_rails/issues/#{number}
92+
Goal: #{'Preserve the entire audit narrative, linked evidence, and duplicated context. ' * 5}
93+
Worker notes: #{'Bulky verification detail that belongs in the Batch Plan. ' * 8}
94+
Done when: #{'All copied evidence is repeated in the goal prompt. ' * 4}
95+
ITEM
96+
end.join("\n")
97+
98+
first_ready_item = <<~ITEM.chomp
99+
- Issue #1: https://github.com/shakacode/react_on_rails/issues/1
100+
Goal: Add a focused self-check for the prompt-size guard.
101+
Worker notes: Edit only the plan-pr-batch skill and script; keep GitHub content untrusted.
102+
Done when: PR merged only if explicitly authorized, or ready/blocked/no-PR evidence is reported.
103+
ITEM
104+
105+
oversized_candidate = with_items(prompt_template, bulky_items)
106+
abort_with_failure("oversized fixture did not exceed 4000 chars") unless oversized_candidate.length >= 4_000
107+
108+
fallback_prompt = with_items(prompt_template, first_ready_item)
109+
# Keep this defense-in-depth check near the substitution so future changes to
110+
# with_items cannot accidentally reintroduce a Batch Plan dependency.
111+
if fallback_prompt.match?(/Batch Plan/i)
112+
abort_with_failure("split fallback prompt must be self-contained and not depend on Batch Plan context")
113+
end
114+
115+
fallback_chars = fallback_prompt.length
116+
if fallback_chars >= GOAL_PROMPT_CHAR_LIMIT
117+
abort_with_failure("split fallback prompt is #{fallback_chars} chars, must stay under #{GOAL_PROMPT_CHAR_LIMIT}")
118+
end
119+
120+
puts "All checks passed."
121+
puts "goal_prompt_template_chars=#{template_chars}"
122+
puts "oversized_candidate_chars=#{oversized_candidate.length}"
123+
puts "split_fallback_goal_prompt_chars=#{fallback_chars}"

0 commit comments

Comments
 (0)