Skip to content

PR auto-heal should not escalate narrow fix PRs into unrelated test_extend coverage churn #1403

@Serhan-Asad

Description

@Serhan-Asad

Problem

PR auto-heal is currently optimizing for "make the whole module synchronized" instead of "keep this PR narrowly scoped".

That is causing narrow fix PRs to get re-bloated by unrelated test_extend output.

Concrete example:

  • PR: promptdriven/pdd#1390
  • Core fix commit: 9fbf9e041 (fix(agentic): fast-fail interactive Claude on subscription/usage caps)
  • Auto-heal head: 4ca11f357 (chore: auto-heal prompt/example drift for agentic_common)

At PR head, the actual bug fix is still fine, but auto-heal has appended a large unrelated test block to tests/test_agentic_common.py and changed .pdd/meta/agentic_common_python.json to "command": "test_extend".

Greg's review on June 4, 2026 is correct: the PR is non-mergeable because auto-heal reintroduced unrelated generated tests.

Why this happens

PR auto-heal runs the installed pdd-cli against changed modules:

  • scripts/ci/run_pdd_cli_auto_heal_pr.sh calls python -I -m pdd.ci_drift_heal --diff-base ... --modules ...

Inside pdd.ci_drift_heal:

  • detect_drift() calls sync_determine_operation(..., target_coverage=90.0, log_mode=True)
  • heal_module() sends several operations through plain pdd sync <module>

Inside sync_determine_operation():

  • for Python modules, if tests pass but coverage is below target, the decision is intentionally test_extend

That behavior is valid for full module sync, but it is wrong for PR auto-heal. In PR mode, it turns a narrow production fix into an unrelated coverage-expansion PR.

Expected behavior

When ci_drift_heal is running in PR auto-heal mode (diff_base set / changed-modules mode), it should preserve PR scope.

Allowed in PR auto-heal:

  • narrow prompt/code/example/meta drift fixes
  • update
  • example
  • possibly auto-deps when clearly required by the changed prompt/code pair

Disallowed in PR auto-heal:

  • coverage-driven test_extend
  • broad pdd sync fan-out that appends unrelated tests to satisfy module-level coverage targets
  • any auto-generated test block unrelated to the PR's changed behavior

Required fix

Add a PR-scope guard to the auto-heal path so test_extend is never auto-committed during PR heal runs.

Important: detection-only is not enough.

  • Even if detect_drift() suppresses test_extend, heal_module() can still route other ops through pdd sync, and sync can escalate into test_extend internally.
  • The guard needs to exist at execution time too, not just at initial drift classification.

Acceptable fixes include either:

  1. A dedicated PR-heal mode in ci_drift_heal that only permits narrow operations and never calls module-wide pdd sync for review-scope heals, or
  2. A scoped pdd sync mode / flag that forbids coverage-expanding test_extend and returns a no-op / manual-follow-up decision instead.

Regression coverage wanted

Add tests proving that in PR auto-heal mode:

  • a narrow PR does not get reclassified into coverage growth
  • test_extend is not dispatched
  • review-only / scope-preserving example drift still works
  • the resulting commit does not append unrelated generated tests

Temporary workaround

For affected PRs today:

  • keep the PR in draft before force-pushing cleanup, so synchronize on a draft PR does not re-run auto-heal
  • do not mark it ready for review again until this bug is fixed, because ready_for_review retriggers auto-heal
  • if the change must land immediately, merge via a path that temporarily bypasses PR auto-heal rather than letting the bot re-bloat the branch

Metadata

Metadata

Assignees

Labels

pdd-codexPDD: use OpenAI Codex model

Type

No type
No fields configured for issues without a type.

Projects

Status
Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions