feat(pipeline): escalate a degenerate no-op worker instead of re-revising it to STUCK (INT-2521)#265
Merged
Merged
Conversation
…sing it to STUCK (INT-2521)
A worker that reports success but changed 0 files, ran 0 commands, and produced no
substantive report (trivial summary AND output) did NOTHING. calculateConfidence
scored it ~55% and the pipeline re-revised it identically every iteration until
STUCK — and a worker that self-reported high confidencePercent bypassed even that.
- isDegenerateWorkerResult (agentPair) detects the pure no-op; requiring an empty
summary too spares legitimate no-edit outcomes ('no change needed' with a real
summary) — only a pure empty no-op is flagged.
- The HALT gate checks it INDEPENDENTLY of the confidence threshold (a 90%-self-
reported no-op no longer slips past), escalates the next attempt (buildRepeat-
Escalation → stronger effort/model) and injects a clear 'you changed nothing —
implement now' feedback instead of the misleading 'low confidence' loop.
Found by the INT-2521 harness audit (finding ②/P1). Tests: helper matrix incl. the
substantive-summary no-edit case; pipeline test that a degenerate + 90% confidence
run is not success AND never reaches the reviewer. tsc clean + full vitest 1714.
openswarm review: round 1 caught the confidence-bypass gap (fixed: check is now
independent of the threshold) + false-positive risk (fixed: summary requirement +
test). Round 2's final was truncated at its turn limit with no new articulated
finding; its thinking re-explored the already-addressed no-edit edge.
|
Bugbot is not enabled for your account, so this pull request was not reviewed. Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A worker that reports success but changed 0 files, ran 0 commands, and produced no substantive report (trivial summary AND output) did nothing.
calculateConfidencescored it ~55% and the pipeline re-revised it identically until STUCK — and a worker self-reporting highconfidencePercentbypassed even that.isDegenerateWorkerResultdetects the pure no-op; requiring an empty summary too spares legitimate no-edit outcomes ('no change needed' with a real summary).buildRepeatEscalation→ stronger effort/model) + injects a clear 'you changed nothing — implement now' feedback.Found by the INT-2521 audit (②/P1). Tests: helper matrix incl. substantive-summary no-edit case; pipeline test that a degenerate + 90% confidence run is not success AND never reaches the reviewer. tsc clean + full vitest 1714. Review round 1 caught the bypass gap + false-positive risk (both fixed); round 2's final truncated at turn limit with no new finding.
🤖 Generated with Claude Code