Skip to content

Commit df32c59

Browse files
committed
Fix #6: resolve contradiction in verification rules
defaultSystem had conflicting instructions: - 'do NOT run tests or verify with shell commands' (line 127) - 'Verify — Do NOT skip verification on complex changes' (line 96, scaffold step 4) The model cannot follow both. Removed the anti-verification instruction and softened 'Do NOT write, then test, then rewrite, then retest' to 'Avoid unnecessary iteration — verify in one shot', aligning with the Reasoning Scaffold. Verified compile + run tests on all 19 packages.
1 parent 931f06a commit df32c59

2 files changed

Lines changed: 22 additions & 3 deletions

File tree

cmd/odek/main.go

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -120,12 +120,10 @@ Code generation discipline — critical for writing files:
120120
not "X.py" or "output/X.py".
121121
- NEVER modify existing source files when the task asks you to CREATE new output files.
122122
Read-only operations on source files are fine; just don't write to them.
123-
- Write correct code on the FIRST attempt. Do NOT write, then test, then rewrite, then retest.
123+
- Write correct code on the FIRST attempt. Avoid unnecessary iteration — verify in one shot.
124124
If you need to verify, read the file back once, then move on.
125125
- Follow the specified function signature, class structure, and design EXACTLY.
126126
Do not invent a different architecture or framework than what was asked for.
127-
- After writing a file, do NOT run tests or verify with shell commands unless
128-
the task explicitly asks you to. One write is enough.
129127
130128
Tool conventions — use these dedicated tools, NOT shell commands:
131129
- Do NOT use cat/head/tail to read files — use read_file instead (line numbers, pagination).

cmd/odek/main_test.go

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2139,6 +2139,27 @@ func TestDefaultSystem_AllowsSkillExploration(t *testing.T) {
21392139
}
21402140
}
21412141

2142+
// TestDefaultSystem_AllowsVerification verifies that defaultSystem does NOT
2143+
// contain the "do NOT test after write" instruction that contradicts the
2144+
// Reasoning Scaffold's step 4 ("Verify — Do NOT skip verification on complex
2145+
// changes"). Both instructions exist at the same time: one says "never test
2146+
// after writing", the other says "always verify complex changes". The model
2147+
// cannot follow both — resolve by removing the anti-verification instruction.
2148+
func TestDefaultSystem_AllowsVerification(t *testing.T) {
2149+
if strings.Contains(defaultSystem, "do NOT run tests or verify") {
2150+
t.Error("defaultSystem says 'After writing a file, do NOT run tests or verify with shell commands' " +
2151+
"but the Reasoning Scaffold (lines 95-97) says 'Verify — Do NOT skip verification on complex changes'. " +
2152+
"These directly contradict. The model cannot follow both instructions. " +
2153+
"Remove the anti-verification instruction so the Verify step works as intended.")
2154+
}
2155+
if strings.Contains(defaultSystem, "Do NOT write, then test, then rewrite") {
2156+
t.Error("defaultSystem says 'Do NOT write, then test, then rewrite, then retest' which discourages " +
2157+
"verification. Combined with the 'Verify' step in the Reasoning Scaffold, the model gets " +
2158+
"conflicting instructions. Reword to 'Avoid unnecessary iteration — verify in one shot' " +
2159+
"to remove the contradiction.")
2160+
}
2161+
}
2162+
21422163
// ── --deliver flag tests ──────────────────────────────────────────────────
21432164

21442165
func TestParseRunFlags_Deliver(t *testing.T) {

0 commit comments

Comments
 (0)