Skip to content

Commit 094dcd2

Browse files
committed
feat(skills): strengthen enforcement mechanisms inspired by superpowers
Incorporate anti-gaslighting, anti-rationalization, and context-survival patterns from obra/superpowers into pb-spec skills. These changes add teeth to existing enforcement without altering the Generator/Evaluator architecture or BDD-first workflow. pb-build/SKILL.md: - Add Durable Progress Ledger (Step 2a): writes task completion/failure to specs/<spec-dir>/progress.md so context compaction does not cause re-dispatching of completed tasks (the single most expensive failure) - Update invariants, stopping conditions, key principles to reference the ledger as the authoritative progress record - Mark Auto Mode to use the ledger for decision documentation pb-build/references/evaluator_prompt.md: - Add "Do Not Trust the Report" section: treat Generator claims as unverified — design rationales are self-grading, not evidence - Add Calibration rubric: Critical/Important/Minor severity table so the evaluator categorizes findings by actual impact, not vibes - Add "No performative feedback" constraint: no praise before completing all checks; earned praise belongs in PASS verdict Strengths section pb-build/references/implementer_prompt.md: - Add anti-patterns: self-assessment prohibition (Generator reports facts, not opinions) and pre-emptive justification ban - Strengthen "Generator, not the Judge" constraint with explicit prohibition on quality assessments pb-test-driven-development/SKILL.md: - Add 3 Red Flag internal signals: "I will just check if it works first", "The test would be trivial anyway", "I know what the code does" - Strengthen delete-enforcement: remove files, revert changes, clean state - Add explicit "spirit" rationalization shutdown: violating the letter IS violating the spirit — there is no "spirit" that permits skipping RED pb-verification-before-completion/SKILL.md: - Add 4 rationalizations: "test was passing before my change", "trivial change", "can see by reading", "build pipeline will catch it" 5 files changed, 62 insertions(+), 6 deletions(-)
1 parent 84cc955 commit 094dcd2

5 files changed

Lines changed: 62 additions & 6 deletions

File tree

skills/pb-build/SKILL.md

Lines changed: 27 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ The build operates in one of two modes, set by the user or inferred from context
5151
### Auto Mode
5252

5353
- Execute all actions autonomously without user confirmation
54-
- Document all decisions, assumptions, and reasoning in `specs/<spec-dir>/progress.md`
54+
- Document all decisions, assumptions, and reasoning in the progress ledger (`specs/<spec-dir>/progress.md`)
5555
- When multiple approaches exist, select the most appropriate and document why
5656
- Provide comprehensive summaries at completion
5757
- Only pause for: DCR filing, Escalation results, or hard stop conditions
@@ -129,6 +129,28 @@ If all tasks are already checked (`- [x]`), report:
129129
✅ All tasks in specs/<spec-dir>/tasks.md are already completed.
130130
```
131131

132+
### Step 2a: Durable Progress Ledger (Mandatory)
133+
134+
Context does not survive compaction. In long-running builds, controllers that lose their place have re-dispatched entire completed task sequences — the single most expensive failure observed. Track progress in a ledger file, not only in memory.
135+
136+
1. **At build start**, check for an existing ledger:
137+
```bash
138+
cat "specs/<spec-dir>/progress.md" 2>/dev/null || echo "No ledger found"
139+
```
140+
2. **If a ledger exists**, tasks listed as complete are DONE — do not re-dispatch them. Resume at the first task not marked complete. Cross-check the ledger against `tasks.md` and `git log` — trust files over memory.
141+
3. **When a task passes evaluation**, append one line to the ledger:
142+
```
143+
Task X.Y: complete (commits <base7>..<head7>, eval PASS)
144+
```
145+
4. **When a task fails or triggers DCR**, append:
146+
```
147+
Task X.Y: FAILED (attempt N) — [one-line reason]
148+
Task X.Y: DCR — [one-line reason]
149+
```
150+
5. **After compaction or session resume**, read the ledger first. The commits it names exist in git even when your context no longer remembers creating them.
151+
152+
> **ponytail: global ledger file, per-task commit refs. Upgrade to structured JSON only if multi-user concurrent builds ever materialize.**
153+
132154
### Step 3: Execute Tasks Sequentially
133155

134156
For each unfinished task, in order:
@@ -379,7 +401,7 @@ After updating `tasks.md`, **verify the mark actually took effect** before proce
379401
- All `- [ ]` evidence checkboxes are now `- [x]`.
380402
3. **If verification fails** — apply the mark immediately and re-verify.
381403

382-
> **⚠️ Context Reset:** After completing all tasks (or when context grows large), output: "Recommend starting a fresh session. Run `/pb-build <feature-name>` again to continue from where you left off."
404+
> **⚠️ Context Reset:** After completing all tasks (or when context grows large), output: "Recommend starting a fresh session. Run `/pb-build <feature-name>` again to continue from where you left off." The progress ledger ensures no completed work is lost.
383405
384406
### Step 4: Handle Failures (The Recovery Loop with Escalation)
385407

@@ -571,14 +593,15 @@ These are absolute rules. Everything else is a guideline derived from the workfl
571593
3. Run incremental tests after each task; run full suite only after ALL tasks complete.
572594
4. Apply the ponytail ladder before writing code: YAGNI → stdlib → native → existing dep → one-liner → minimum code.
573595
5. File a DCR if the design is infeasible; suspend after 3 consecutive failures.
596+
6. Update the progress ledger after each task — it survives compaction; your memory does not.
574597

575598
## Stopping Conditions
576599

577600
After each task evaluation, ask: "Did the Evaluator independently confirm the tests pass and the code matches the spec?" If yes, proceed to the next task. If no, follow the failure loop.
578601

579602
When the failure loop reaches 3 consecutive failures, stop the build — do not thrash. File a DCR with root-cause analysis and hand off to `/pb-refine`.
580603

581-
If context grows large after many tasks, recommend starting a fresh session and re-running `/pb-build` to continue from where you left off.
604+
If context grows large after many tasks, recommend starting a fresh session and re-running `/pb-build` to continue from where you left off. The progress ledger (`specs/<spec-dir>/progress.md`) is your recovery map — trust it over memory.
582605

583606
---
584607

@@ -654,4 +677,4 @@ The Evaluator should watch for these common agent mistakes:
654677
7. **Architecture decisions are binding.** Execute the approved design — do not improvise a different pattern.
655678
8. **ponytail ladder.** Before writing code: YAGNI → stdlib → native → existing dep → one-liner → minimum code.
656679
9. **Context hygiene.** Pass minimal, relevant context to subagents. Extract only the design sections that bind this task.
657-
10. **State lives on disk.** `tasks.md` checkboxes and committed code are the only persistent state.
680+
10. **State lives on disk.** `tasks.md` checkboxes, the progress ledger, and committed code are the only persistent state.

skills/pb-build/references/evaluator_prompt.md

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,22 @@ You are the **Evaluator** — an independent adversarial critic. You did NOT bui
5252

5353
Execute these three checks in order. Be **harsh, skeptical, and specific.** Do not give the benefit of the doubt. If something is ambiguous, mark it as a failure.
5454

55+
## Do Not Trust the Report
56+
57+
Treat the Generator's report as unverified claims about the code. It may be incomplete, inaccurate, or optimistic. Verify every claim against the diff and live behavior. Design rationales in the report are claims too: "left it per YAGNI," "kept it simple deliberately," or any other justification is the Generator grading its own work. Judge the code on its merits — a stated rationale never downgrades a finding's severity.
58+
59+
## Calibration
60+
61+
Categorize issues by actual severity. Not everything is Critical.
62+
63+
| Severity | Meaning | Examples |
64+
|----------|---------|---------|
65+
| **Critical** | Blocks trust — code must not ship | Security hole, data loss path, spec requirement completely missing, broken error handling |
66+
| **Important** | This task cannot be trusted until fixed | Incorrect or fragile behavior, missed requirement, maintainability damage (verbatim duplication, swallowed errors, tests that assert nothing) |
67+
| **Minor** | Polish, not blockers | "Coverage could be broader," naming improvements, style consistency |
68+
69+
If the task spec explicitly mandates something this rubric calls a defect, that IS a finding — report it as Important, labeled spec-mandated. The spec's authorship does not grade its own work.
70+
5571
### Check A — Diff Audit
5672

5773
Analyze the git diff against the task contract.
@@ -256,3 +272,4 @@ Impact: [Which other tasks are affected]
256272
- **Do not be lenient.** Anthropic's research shows that "agents tend to respond by confidently praising the work — even when, to a human observer, the quality is obviously mediocre." Override this tendency. Score harshly.
257273
- **Live verification is mandatory for BDD+TDD tasks.** Do not pass a task based on test logs alone. You must interact with the running application.
258274
- **Check MCP tool availability first.** If Playwright MCP or HTTP MCP tools are unavailable, document the limitation and fall back to CLI-based verification (curl, wget, etc.), but note which checks were skipped.
275+
- **No performative feedback.** Never output "Looks good overall!", "Great implementation!", or any praise before completing all checks. Start with the checks, end with the verdict. Praise belongs in the Strengths section of a PASS verdict, earned by evidence.

skills/pb-build/references/implementer_prompt.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,8 @@ You are the **Generator**. Your sole objective is to make tests pass. You do NOT
3131
1. **Fake Implementations**: You MUST NOT use hardcoded mock returns, `pass`, `TODO`, or `FIXME` to bypass logic.
3232
2. **Tautological Tests**: Tests must test behavior, not just assert `mock_function() == mock_function()`.
3333
3. **Sleep-based Waiting**: NEVER use `time.sleep()` or equivalent in tests. Use `condition-based-waiting` (e.g., polling, wait-for-selectors).
34+
4. **Self-Assessment**: Do NOT judge your own work quality. Do NOT output "This looks good", "Quality is high", "Clean implementation" — the Evaluator decides, not you. Report facts (tests pass, diff is scoped), not opinions.
35+
5. **Pre-emptive Justification**: Do NOT explain why potential issues are acceptable ("I left this simple per YAGNI", "This is deliberately minimal"). Report what you did; the Evaluator judges whether it's sufficient.
3436

3537
If your code contains `Mock`, `TODO`, or hardcoded dummy data for business logic, the Evaluator will REJECT your work immediately.
3638

@@ -275,7 +277,7 @@ READY_FOR_EVAL: Task {{TASK_NUMBER}}
275277
- **Runtime evidence is mandatory when applicable.** Do not claim completion without runtime logs/probe evidence for runtime-facing tasks.
276278
- **File a Design Change Request** if the design is infeasible rather than forcing a broken approach.
277279
- **No unverified claims.** Do not report success without command output evidence.
278-
- **You are the Generator, not the Judge.** Do not evaluate your own work's quality. Signal `READY_FOR_EVAL` and let an independent Evaluator determine if the task is done.
280+
- **You are the Generator, not the Judge.** Do not evaluate your own work's quality. Signal `READY_FOR_EVAL` and let an independent Evaluator determine if the task is done. Never output quality assessments — report facts (command output, test counts, diff scope) and let the Evaluator interpret them.
279281
- **Do not claim completion.** Only output `READY_FOR_EVAL: Task X.Y` — the orchestrator will run an adversarial evaluation before marking anything as done.
280282

281283
## Harness Rules (Strict)

skills/pb-test-driven-development/SKILL.md

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,8 @@ Write code before the test? Delete it. Start over.
2424
- Don't keep it as "reference"
2525
- Don't "adapt" it while writing tests
2626
- Don't look at it
27-
- Delete means delete
27+
- Don't skim it for ideas
28+
- Delete means delete — remove the files, revert the changes, start from a clean state
2829

2930
## When to Use
3031

@@ -249,6 +250,10 @@ Tests-after are biased by your implementation. You test what you built, not what
249250

250251
30 minutes of tests after ≠ TDD. You get coverage, lose proof tests work.
251252

253+
**"I'm following the spirit of TDD"**
254+
255+
Violating the letter of the rules is violating the spirit of the rules. There is no "spirit" that permits skipping RED. The entire point is watching the test fail — that's not a ritual, it's the mechanism that proves your test detects the missing behavior. Skip RED, and you have zero evidence your test works.
256+
252257
## Common Rationalizations
253258

254259
| Excuse | Reality |
@@ -267,6 +272,8 @@ Tests-after are biased by your implementation. You test what you built, not what
267272

268273
## Red Flags - STOP and Start Over
269274

275+
These are specific thoughts that mean you are about to violate the Iron Law. Recognize them as danger signals:
276+
270277
- Code before test
271278
- Test after implementation
272279
- Test passes immediately
@@ -280,6 +287,9 @@ Tests-after are biased by your implementation. You test what you built, not what
280287
- "Already spent X hours, deleting is wasteful"
281288
- "TDD is dogmatic, I'm being pragmatic"
282289
- "This is different because..."
290+
- "I'll just check if it works first"
291+
- "The test would be trivial anyway"
292+
- "I know what the code does, I wrote it"
283293

284294
**All of these mean: Delete code. Start over with TDD.**
285295

skills/pb-verification-before-completion/SKILL.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,10 @@ Skip any step = lying, not verifying
7070
| "I'm tired" | Exhaustion ≠ excuse |
7171
| "Partial check is enough" | Partial proves nothing |
7272
| "Different words so rule doesn't apply" | Spirit over letter |
73+
| "The test was passing before my change" | Run it NOW, after your change |
74+
| "It's a trivial change" | Trivial changes break things trivially |
75+
| "I can see it's correct by reading" | Reading ≠ running |
76+
| "The build pipeline will catch it" | Fix it before you push, not after CI fails |
7377

7478
## Key Patterns
7579

0 commit comments

Comments
 (0)