feat(autoresearch): reduce bash output token spend

[MattPua]

Results

Scope	Before	After	Saved
Eligible noisy pnpm test	17766	96	17670 (99.46%)
Full seven-case corpus	214969	197299	17670 (8.22%)
Semantic fidelity	7/7	7/7	No detected loss

Before  ████████████████████████████████████████  214969
After   █████████████████████████████████████     197299
Saved   ███                                         17670

The measured saving comes from a representative noisy pnpm test fixture. Runtime validation also covered npm and Python test execution.

Created with Autoresearch.

Problem

Test output can spend substantial agent context on repetitive passing lines, hiding the failure or summary that matters.

The safety contract is strict:

flowchart LR
    A[Original command] --> B{Verified test form?}
    B -->|No| C[Run unchanged]
    B -->|Yes| D{Compatible RTK?}
    D -->|No| C
    D -->|Yes| E[Filter repetitive test output]
    E --> F[Preserve summary, failure, and exit code]

Loading

Methodology

Only test ecosystems exercised locally are eligible:

Ecosystem	Eligible forms	Runtime validation
JavaScript	pnpm and npm test commands	Real passing and failing processes
Python	pytest, unittest, uv, and Poetry forms	Real Python processes
Everything else	Unchanged	Explicit policy tests

flowchart TD
    C[Candidate command] --> S{Single bare command?}
    S -->|No| RAW[Run unchanged]
    S -->|Yes| T{Supported test form?}
    T -->|No| RAW
    T -->|Yes| M{Machine-readable flags?}
    M -->|Yes| RAW
    M -->|No| R{Executable RTK 0.43.0+?}
    R -->|No| RAW
    R -->|Yes| O[Use RTK test filtering]

Loading

Changes

Area	Change	Why it helps
Shared policy	One provider-neutral eligibility policy	Prevents Claude and Codex policy drift
Claude	Deterministic hook uses the shared policy	Compresses verified test forms automatically
Codex	Guidance generated from the shared policy	Applies the same narrow safety boundary
RTK detection	Validate executable, identity, and minimum version	Avoids breaking commands with unusable binaries
Exact output	Reject shell operators and machine-readable modes	Preserves command semantics and structured output

No PostHog-specific runtime commands or package names are included.

Verification

Check	Result
Focused rewrite and guidance tests	92 passed
Provider/server RTK integration tests	7 passed
Agent typecheck	Passed
Repository typecheck	24/24 tasks passed
Agent production build	Passed
Real pnpm/npm/Python execution	Exit codes and useful output preserved
Formatting and diff validation	Passed

The research benchmark was run locally and is intentionally excluded from this PR.

Areas to watch

Risk	Mitigation
RTK output behavior changes	Require RTK 0.43.0+ and keep the allowlist narrow
Structured test output	Commands with machine-readable flags remain raw
Unsupported ecosystems	Remain raw until validated with real projects
Provider behavior differs	Shared policy, plus provider-specific integration tests
Token estimate differs from a model tokenizer	Use the estimate only for deterministic relative comparison

Automatic notifications

• Publish to changelog?
• Alert Sales and Marketing teams?

---

Created with PostHog Code

[trunk-io]

Merging to main in this repository is managed by Trunk.

• To merge this pull request, check the box to the left or comment /trunk merge below.

After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here

[github-actions]

React Doctor found no issues in the changed files. 🎉

_{Reviewed by React Doctor for commit 6961434.}