Skip to content

fix(systematic-debugging): defuse "ultrathink" keyword scanner trigger (fixes #1283)#1558

Open
ngalatis wants to merge 1 commit into
obra:devfrom
ngalatis:fix/escape-ultrathink-trigger-in-systematic-debugging
Open

fix(systematic-debugging): defuse "ultrathink" keyword scanner trigger (fixes #1283)#1558
ngalatis wants to merge 1 commit into
obra:devfrom
ngalatis:fix/escape-ultrathink-trigger-in-systematic-debugging

Conversation

@ngalatis
Copy link
Copy Markdown

@ngalatis ngalatis commented May 16, 2026

Fixes #1283.

What problem are you trying to solve?

skills/systematic-debugging/SKILL.md line 240 contains the literal token that Claude Code's runtime scans for in tool result bodies. Every time the Skill tool returns this skill's content, the harness injects:

<system-reminder>The user included the keyword "ultrathink", requesting deeper reasoning on this turn. Reason as thoroughly as the task warrants.</system-reminder>

even when the user never typed it. This silently bumps every session that loads systematic-debugging into extended thinking, raising token cost and shifting model behavior without consent. This is exactly the issue filed in #1283 by @sk3d; I encountered it in my daily workflow.

Reproduction (Claude Code 2.1.143, Opus 4.7):

  1. Start a clean session. Send a debugging-flavored message that does NOT contain the keyword (e.g. "investigate this test failure").
  2. The model invokes superpowers:systematic-debugging via the Skill tool.
  3. Immediately after the skill content lands in context, the harness emits the spurious reminder.

I verified this with two independent clean subagent contexts whose own prompts did not contain the keyword. Both saw the reminder fire. Same behavior was reported by @sk3d on superpowers 5.0.7 / CC 2.1.119.

What does this PR change?

One bullet on one line: UltrathinkUltra-think. Breaks the contiguous letter sequence the harness keyword scanner matches; preserves the signal's meaning for the agent and the documented action.

Is this change appropriate for the core library?

Yes. The bug affects every Claude Code user of this core skill. The fix benefits all users on Claude Code regardless of project. No third-party dependency, no domain-specific content, no new harness.

What alternatives did you consider?

  • \Ultrathink this — rejected, substring still present, scanner still matches (this was my initial instinct in the issue comments; I was wrong).
  • `Ultrathink this` (backticks) — same problem.
  • U!ltrathink this (sk3d's local workaround) — works but reads as a typo.
  • Ultra<wbr>think this — works in raw text but adds HTML to markdown.
  • Drop the bullet entirely — loses a useful documented signal.
  • Ultra-think this (chosen) — natural English compounding, breaks the scanner, signal still readable as the same phrase by an LLM.

Does this PR contain multiple unrelated changes?

No. One line, one file, one purpose.

Existing PRs

Environment tested

Harness (e.g. Claude Code, Cursor) Harness version Model Model version/ID
Claude Code 2.1.143 Claude Opus 4.7 (1M ctx) claude-opus-4-7

New harness support (required if this PR adds a new harness)

Not applicable — no new harness.

Evaluation

  • Initial prompt: continuation of my daily workflow where I'd hit the false-positive from systematic-debugging skill body triggers Claude Code "ultrathink" auto-reminder via tool-result keyword scan #1283 repeatedly. I asked my agent to investigate and fix.
  • "Before" runs: two clean subagent contexts each invoked the Skill tool on the unmodified skill content. In both, the spurious system-reminder fired immediately after the skill body loaded, even though neither subagent's own prompt contained the keyword. Sterile reproduction.
  • "After" runs: blocked in the session that issued the edit, because Claude Code caches loaded skill content in-process and does not reload skills mid-session. A fresh CC session is required to load the patched file from disk. The patch removes the only ultrathink substring in the file (grep -i 'ultrathink' SKILL.md returns zero hits after). Since the scanner triggers on substring match in tool result bodies, no substring → no trigger. Reviewers can confirm by running the same Skill-tool repro in a fresh session against this branch.
  • The signal text remains semantically identical for an LLM reading the bullet, and the documented action is byte-identical.

Rigor

  • If this is a skills change: I used superpowers:writing-skills and completed adversarial pressure testing — not applicable for this change. This is a defusal of a harness side effect at the character level, not a behavior-shaping content rewrite. No bullet was added, removed, or re-worded; only the spelling of one word was broken with a hyphen. The agent's behavioral response to the signal is unchanged.
  • This change was tested adversarially, not just on the happy path — the "before" reproduction used two independent subagent contexts whose own prompts did not contain the keyword, ruling out user-message-triggered reminders as the source and isolating the trigger to the skill body.
  • I did not modify carefully-tuned content (Red Flags table, rationalizations, "human partner" language) without extensive evals showing the change is an improvement. The change is one hyphen on one bullet in the "Signals" list; the surrounding structure and language are byte-identical.

Human review

  • A human has reviewed the COMPLETE proposed diff before submission

…ner trigger

The "Signals You're Doing It Wrong" bullet in systematic-debugging/SKILL.md
contains the literal token Claude Code's runtime scans for in tool result
bodies. Every Skill-tool invocation of this skill caused the harness to
inject a spurious system-reminder claiming the user requested deeper
reasoning, silently bumping every session into extended thinking.

Replace the bullet's spelling so the contiguous letter sequence the scanner
matches is broken with a hyphen. The signal text remains recognizable to
the agent and the documented action ("Question fundamentals, not just
symptoms") is unchanged.

Fixes obra#1283
@ngalatis ngalatis changed the base branch from main to dev May 18, 2026 21:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

systematic-debugging skill body triggers Claude Code "ultrathink" auto-reminder via tool-result keyword scan

1 participant