running-in-ci: verify external-tool behavior by run or clone, not just hand-test

[tend-agent]

Problem

The bundled running-in-ci skill's Grounded Analysis section covers behavioral claims by telling the bot to "run the command yourself" or hedge — but that fallback doesn't obviously apply to external CLIs and APIs that aren't installed in CI and aren't exercised by automated tests. In worktrunk#1907 the bot read upstream mintlify docs describing a cmux list-workspaces --json flag, believed it, and committed a recipe that broke for every reader — the installed cmux had no such flag.

Solution

Adds a new ### Verifying external-tool behavior subsection under Grounded Analysis. It points at two concrete verification paths in order of preference: install and run the tool, or clone its public repo and grep the source. Deferring to a human with the tool installed is the fallback only when both paths fail. The "don't make overconfident claims" framing already lives in the preceding ### User-facing comments require source evidence subsection.

Bad/good example is drawn from the cmux incident: the good path now shows cloning the upstream repo and checking the CLI parser, not asking a human to confirm.

Testing

Skill text only. pre-commit run on the modified file passes (typos, trim-whitespace, bang-backtick, end-of-files).

---

Closes #326 — automated triage

[max-sixty]

overall decent, but:

• the focus should be on verifying claims, if possible by running code
• it is possible to read a tool's source! tend can clone the repo!
• fine to retain the "don't make overconfident claims" but we should have that elsewhere

[tend-agent]

Good points, rewrote in 6437170:

• Section now leads with verifying by running the tool (install-and-run) or reading source (gh repo clone then grep). Deferring to a human is the fallback only when both fail.
• Retitled to ### Verifying external-tool behavior — positive framing, since the primary action is verification rather than flagging a gap.
• Dropped the "do not post a confident claim" language; that rule already lives in the preceding ### User-facing comments require source evidence subsection, so no new home needed.
• Updated the good example to show cloning cmux's source and checking the CLI parser rather than asking a human to confirm.

[max-sixty]

was this the actual problem in this specific case — that the cmux docs were wrong? if so, this is fine. if not, and actually we hallucinated something, then we should adjust

[tend-agent]

Checked both sources. The mintlify page at manaflow-ai-cmux.mintlify.app/cli/workspaces does describe a --json flag for list-workspaces, with the exact JSON shape I posted (a ParamField path="--json" block plus a sample {"workspaces": [{"ref": ..., "id": ..., "index": ..., "title": ..., "selected": ...}]} payload). So I wasn't synthesizing the schema — I quoted what the docs said.

The cmux source tells the other half of the story. In CLI/cmux.swift the list-workspaces help block is just Usage: cmux list-workspaces with no flags; --json is only wired up on tree (and a few others). So the published docs are ahead of (or diverged from) the shipped CLI parser.

Net: docs were wrong, not a hallucination — but the new section's "clone the repo and grep the CLI parser" step would have caught it, since the discrepancy is visible in one grep --json CLI/cmux.swift. So the framing in this PR holds. Happy to file a docs issue with manaflow-ai/cmux if you'd like, separately.

[tend-agent]

Small correction to my earlier comment for the record: --json is in fact wired into list-workspaces at CLI/cmux.swift:2003-2005 (even at tag v0.63.2), via the shared if jsonOutput block.

The real reason the published recipe broke: cmux parses --json as a global flag in a loop that breaks at the first non-global token (see CLI/cmux.swift:1665-1710). So cmux list-workspaces --json silently drops --json and returns text; the form that works is cmux --json list-workspaces. The mintlify docs show the broken flag ordering, which is what I copied.

Still "docs wrong, not hallucination" — but a naïve grep -- --json CLI/cmux.swift would have confirmed --json works for list-workspaces and shipped the same broken recipe. Path 1 (run the tool) is what would have caught this; Path 2 alone misses flag-placement bugs unless you also read the arg-parser structure. Worth keeping that caveat in mind when this guidance gets applied to other fast-moving CLIs.

[max-sixty]

ok, can you add a comment on the original issue that the --json flag is available if expressed differently?

[tend-agent]

Done — worktrunk#1907 comment explains that --json does work for list-workspaces but only as a global flag (cmux --json list-workspaces), with source pointers at CLI/cmux.swift:1635-1690 (parser) and :2003-2006 (handler).