Skip to content

Commit f1c4a89

Browse files
committed
fix(loop): unify durable resume head
1 parent feea8f9 commit f1c4a89

9 files changed

Lines changed: 231 additions & 33 deletions

File tree

loops/issue-dev-loop/SKILL.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ Run exactly one bounded issue cycle. Treat [`LOOP.md`](./LOOP.md) as the constit
77
## Start safely
88

99
1. Read `LOOP.md`, `state.md`, and `dependencies.md` completely.
10-
2. Require absolute `ECHO_UI_LOOP_CONTROL_PLANE` and `ECHO_UI_LOOP_TARGET_ROOT` values from the scheduler. Run activation through `"$ECHO_UI_LOOP_CONTROL_PLANE/scripts/with-github-identity" --loop-root "$ECHO_UI_LOOP_TARGET_ROOT" automation -- node "$ECHO_UI_LOOP_CONTROL_PLANE/scripts/loopctl.mjs" validate --activation --loop-root "$ECHO_UI_LOOP_TARGET_ROOT"`. The installed launcher verifies its hash manifest, probes both configured profiles, and attempts full validation first. It may fall back for an older target only after matching the local run to an automation-authored remote durable checkpoint, proving the clean exact branch and head without index concealment, and proving the issue diff did not modify the protected control or verification plane. A one-use in-memory router capability then checks stable target state, owner channel, JSON history, and a conservatively parsed low-privilege evidence workflow before rechecking the exact clean worktree. There is no standalone reduced validator, and callers and the public validation API cannot request this mode. `restore-checkpoint` still independently requires the durable exact branch, head, and clean worktree.
10+
2. Require absolute `ECHO_UI_LOOP_CONTROL_PLANE` and `ECHO_UI_LOOP_TARGET_ROOT` values from the scheduler. Run activation through `"$ECHO_UI_LOOP_CONTROL_PLANE/scripts/with-github-identity" --loop-root "$ECHO_UI_LOOP_TARGET_ROOT" automation -- node "$ECHO_UI_LOOP_CONTROL_PLANE/scripts/loopctl.mjs" validate --activation --loop-root "$ECHO_UI_LOOP_TARGET_ROOT"`. The installed launcher verifies its hash manifest, probes both configured profiles, and attempts full validation first. It may fall back for an older target only after excluding durably finalized runs, matching the worktree to an automation-authored remote active checkpoint, proving the clean event-derived branch and head without index concealment, and proving the issue diff did not modify the protected control or verification plane. A one-use in-memory router capability then checks stable target state, owner channel, JSON history, and a conservatively parsed low-privilege evidence workflow before rechecking the exact clean worktree. Activation, detection, and restore share the same event-derived head resolver. There is no standalone reduced validator, and callers and the public validation API cannot request this mode.
1111
3. Read [`references/github-operations.md`](./references/github-operations.md). Run every operational `loopctl`, executor GitHub command, remote Git command, trigger, and reviewer publication through the installed control plane with the explicit target root. Never use the credential-refusing repository launcher, invoke the `.mjs` router directly, install control code from an issue branch, or alter global `gh` or Git credential configuration.
1212
4. Run `loopctl.mjs reconcile` through the automation wrapper to rebuild verified terminal history, pending/completed evolve state, and active runs from the append-only GitHub state journal. It tombstones local terminal-cache rows with no durable counterpart before recomputing metrics. For returned `workType: resume`, fetch the recorded branch through the wrapper, create a clean isolated worktree at the returned exact head, and run `restore-checkpoint --run-id <id>` inside that worktree. The restore command rejects the wrong branch, a dirty checkout, or any head other than the durable head. Resume it before selecting a new issue.
1313
5. Run `loopctl.mjs evolve-status`. If `evolveDue` is true, start `echo_ui_loop_evolver` with fresh context; do not silently replace it with product work.

loops/issue-dev-loop/references/github-operations.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ Run every executor GitHub command through:
1010

1111
`ECHO_UI_LOOP_CONTROL_PLANE` must name the versioned installation created from a clean owner-merged `dev`; `ECHO_UI_LOOP_TARGET_ROOT` names the active worktree's `loops/issue-dev-loop`. Operational `loopctl` and trigger commands must use the scripts inside the installed root and pass `--loop-root "$ECHO_UI_LOOP_TARGET_ROOT"`. The repository launcher intentionally refuses credentials.
1212

13-
For a durable active run whose target predates newer trusted runtime files, the installed activation router attempts full validation first. Only if that fails may it select historical-target compatibility, and only after an automation-authored remote durable checkpoint exactly matches the local active run, the worktree is on its clean exact branch and head with no index concealment flags, and the issue diff is proven not to modify the protected control or verification plane. A one-use in-memory router capability then checks stable target state, owner channel, JSON history, and conservatively rejects unrecognized YAML, triggers, or job-level/write permissions before rechecking the worktree. No standalone reduced validator exists, and caller-supplied flags and the public validation API cannot select the reduced mode; the subsequent restore independently enforces the journal's exact branch and head on a clean worktree.
13+
For a durable active run whose target predates newer trusted runtime files, the installed activation router attempts full validation first. Only if that fails may it select historical-target compatibility, and only after excluding durably finalized runs, matching an automation-authored remote active checkpoint, proving its event-derived clean exact branch and head with no index concealment flags, and proving the issue diff does not modify the protected control or verification plane. A one-use in-memory router capability then checks stable target state, owner channel, JSON history, and conservatively rejects unrecognized YAML, triggers, or job-level/write permissions before rechecking the worktree. Activation, detection, and restore use the same checkpoint-head resolver. No standalone reduced validator exists, and caller-supplied flags and the public validation API cannot select the reduced mode.
1414

1515
Run every reviewer publication command through the installed wrapper with role `reviewer`. Before reading either profile, it verifies every installed file, pins absolute Node/Git/`gh` executables, compares the target's security-critical owner-channel values to its trusted copy, removes token environment overrides, runs `gh api user`, and refuses an unexpected or owner identity. For Git, it clears global credential helpers and injects `gh auth git-credential` for the entire trusted child tree. Descendant `git` and `gh` processes pass through a role gate; arbitrary `sh`, `env`, Node scripts, caller PATH shims, and issue-worktree router changes are not authenticated. Never use owner credentials for executor or reviewer actions, never run raw remote `gh`/`git push` commands, and never call `gh auth setup-git`.
1616

loops/issue-dev-loop/scripts/lib/active-journal.mjs

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ import {
1919
checkpointJournalConfiguration,
2020
checkpointPublicationBody,
2121
checkpointRecordDigest,
22+
checkpointWorktreeHead,
2223
parseCheckpointRecord,
2324
validateCheckpointRecord,
2425
verifyPublishedCheckpoint,
@@ -227,7 +228,7 @@ async function defaultWorkspaceValidator({ loopRoot, record }) {
227228
if (branch.stdout.trim() !== record.run.branch) {
228229
throw new Error(`restore requires isolated worktree branch ${record.run.branch}`)
229230
}
230-
const expectedHead = record.run.headSha ?? record.run.implementationCommit ?? record.run.baseSha
231+
const expectedHead = checkpointWorktreeHead(record)
231232
if (head.stdout.trim() !== expectedHead) {
232233
throw new Error(`restore requires exact durable head ${expectedHead}`)
233234
}

loops/issue-dev-loop/scripts/lib/checkpoint-proof.mjs

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,20 @@ import {
1313

1414
const ACTIVE_STATUSES = new Set(['running', 'waiting_for_owner', 'awaiting_owner_review'])
1515

16+
export function checkpointWorktreeHead(record) {
17+
let expectedHead = record?.run?.baseSha
18+
for (const event of record?.events ?? []) {
19+
if (event.type === 'implementation_completed' && event.status === 'passed') {
20+
expectedHead = event.payload?.commitSha
21+
}
22+
if (event.type === 'pr_published') expectedHead = event.payload?.headSha
23+
}
24+
if (!/^[0-9a-f]{40}$/i.test(expectedHead ?? '')) {
25+
throw new Error('durable active checkpoint has no valid working head')
26+
}
27+
return expectedHead
28+
}
29+
1630
export async function checkpointJournalConfiguration(loopRoot) {
1731
const channel = await readJson(
1832
path.resolve(loopRoot, '..', '_shared', 'owner-channel', 'channel.json'),

loops/issue-dev-loop/scripts/lib/finalization-journal.mjs

Lines changed: 17 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -320,9 +320,8 @@ export async function recordFinalizationPublication({
320320
return { record, digest, commentUrl }
321321
}
322322

323-
export async function reconcileFinalizationJournal({
323+
export async function loadDurableFinalizationRecords({
324324
loopRoot = DEFAULT_LOOP_ROOT,
325-
now = new Date(),
326325
githubPaginatedApi = defaultGitHubPaginatedApi,
327326
githubApi = defaultGitHubApi,
328327
latestActiveCheckpoints = null,
@@ -380,6 +379,22 @@ export async function reconcileFinalizationJournal({
380379
effectiveRecords.sort(
381380
(left, right) => Date.parse(left.finishedAt) - Date.parse(right.finishedAt),
382381
)
382+
return effectiveRecords
383+
}
384+
385+
export async function reconcileFinalizationJournal({
386+
loopRoot = DEFAULT_LOOP_ROOT,
387+
now = new Date(),
388+
githubPaginatedApi = defaultGitHubPaginatedApi,
389+
githubApi = defaultGitHubApi,
390+
latestActiveCheckpoints = null,
391+
} = {}) {
392+
const effectiveRecords = await loadDurableFinalizationRecords({
393+
loopRoot,
394+
githubPaginatedApi,
395+
githubApi,
396+
latestActiveCheckpoints,
397+
})
383398

384399
const indexPath = path.join(loopRoot, 'logs', 'index.jsonl')
385400
const existing = (await readFile(indexPath, 'utf8'))

loops/issue-dev-loop/scripts/lib/github-identity.mjs

Lines changed: 18 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -17,11 +17,13 @@ import {
1717
import { reconcileActiveJournal } from './active-journal.mjs'
1818
import {
1919
checkpointRecordDigest,
20+
checkpointWorktreeHead,
2021
parseCheckpointRecord,
2122
validateCheckpointRecord,
2223
verifyLatestDurableCheckpoint,
2324
} from './checkpoint-proof.mjs'
2425
import { verifyPublishedEvolveRequest } from './evolve.mjs'
26+
import { loadDurableFinalizationRecords } from './finalization-journal.mjs'
2527
import {
2628
parseReviewPublisherArguments,
2729
reviewPublisherSyntheticGitHubArguments,
@@ -1355,20 +1357,6 @@ export function consumeHistoricalValidationCapability(capability) {
13551357
}
13561358
}
13571359

1358-
export function durableCheckpointWorktreeHead(record) {
1359-
let expectedHead = record?.run?.baseSha
1360-
for (const event of record?.events ?? []) {
1361-
if (event.type === 'implementation_completed' && event.status === 'passed') {
1362-
expectedHead = event.payload?.commitSha
1363-
}
1364-
if (event.type === 'pr_published') expectedHead = event.payload?.headSha
1365-
}
1366-
if (!/^[0-9a-f]{40}$/i.test(expectedHead ?? '')) {
1367-
throw new Error('durable active checkpoint has no valid working head')
1368-
}
1369-
return expectedHead
1370-
}
1371-
13721360
async function assertCleanExactDurableWorktree({
13731361
realGit,
13741362
repositoryRoot,
@@ -1471,19 +1459,30 @@ async function authorizeHistoricalTargetValidation({
14711459
})
14721460
return JSON.parse(stdout)
14731461
}
1474-
const { activeCheckpoints } = await reconcileActiveJournal({
1462+
const paginatedApi = (endpoint) =>
1463+
paginateGitHubApi(githubApi, endpoint.replace(/[?&]per_page=100$/, ''))
1464+
const { activeCheckpoints: allActiveCheckpoints } = await reconcileActiveJournal({
14751465
loopRoot,
1476-
githubPaginatedApi: (endpoint) =>
1477-
paginateGitHubApi(githubApi, endpoint.replace(/[?&]per_page=100$/, '')),
1466+
githubPaginatedApi: paginatedApi,
14781467
})
1468+
const finalizations = await loadDurableFinalizationRecords({
1469+
loopRoot,
1470+
githubPaginatedApi: paginatedApi,
1471+
githubApi,
1472+
latestActiveCheckpoints: allActiveCheckpoints,
1473+
})
1474+
const terminalRunIds = new Set(finalizations.map((record) => record.runId))
1475+
const activeCheckpoints = allActiveCheckpoints.filter(
1476+
(checkpoint) => !terminalRunIds.has(checkpoint.record.run.runId),
1477+
)
14791478
const durableMatches = activeCheckpoints.filter(
14801479
(checkpoint) =>
14811480
checkpoint.record.run.branch === checkedOutBranch.stdout.trim() &&
1482-
durableCheckpointWorktreeHead(checkpoint.record) === checkedOutHead.stdout.trim(),
1481+
checkpointWorktreeHead(checkpoint.record) === checkedOutHead.stdout.trim(),
14831482
)
14841483
const durable = durableMatches.length === 1 ? durableMatches[0] : null
14851484
const run = durable?.record.run
1486-
const expectedHead = durable ? durableCheckpointWorktreeHead(durable.record) : null
1485+
const expectedHead = durable ? checkpointWorktreeHead(durable.record) : null
14871486
if (
14881487
!run ||
14891488
run.finishedAt !== null ||

loops/issue-dev-loop/scripts/lib/github.mjs

Lines changed: 2 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ import { reconcileActiveJournal } from './active-journal.mjs'
2626
import { reconcileEvolveJournal } from './evolve.mjs'
2727
import { defaultReleaseIssueClaim } from './issue-claim.mjs'
2828
import { appendValidatedEvent, finalizeRun, readEvents, readRun } from './run-store.mjs'
29-
import { verifyLatestDurableCheckpoint } from './checkpoint-proof.mjs'
29+
import { checkpointWorktreeHead, verifyLatestDurableCheckpoint } from './checkpoint-proof.mjs'
3030
import { validateFinalizationHistory } from './validation.mjs'
3131

3232
const PRIORITY = new Map([
@@ -168,10 +168,7 @@ export async function detectWork({
168168
workType: 'resume',
169169
runId: resumable.record.run.runId,
170170
branch: resumable.record.run.branch,
171-
expectedHeadSha:
172-
resumable.record.run.headSha ??
173-
resumable.record.run.implementationCommit ??
174-
resumable.record.run.baseSha,
171+
expectedHeadSha: checkpointWorktreeHead(resumable.record),
175172
issue: {
176173
number: resumable.record.run.issueNumber,
177174
title: resumable.record.run.issueTitle,

loops/issue-dev-loop/tests/github-identity-routing.test.mjs

Lines changed: 88 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,10 @@ import {
2828
prepareEvolveRequestPublication,
2929
recordEvolveRequestPublication,
3030
} from '../scripts/lib/evolve.mjs'
31+
import {
32+
canonicalFinalizationRecord,
33+
finalizationRecordDigest,
34+
} from '../scripts/lib/finalization-proof.mjs'
3135
import { resolveExecutable } from '../scripts/lib/github-identity.mjs'
3236

3337
const execFileAsync = promisify(execFile)
@@ -67,6 +71,7 @@ async function createFixture({
6771
historicalActivation = false,
6872
remoteCheckpoint = true,
6973
removeLocalRun = false,
74+
terminalFinalization = false,
7075
} = {}) {
7176
const parent = await mkdtemp(path.join(os.tmpdir(), 'echo-ui-identity-routing-'))
7277
const loopRoot = path.join(parent, 'issue-dev-loop')
@@ -305,10 +310,60 @@ async function createFixture({
305310
'utf8',
306311
)
307312
}
313+
if (terminalFinalization) {
314+
const finalization = {
315+
schemaVersion: 1,
316+
runId: run.runId,
317+
issueNumber: run.issueNumber,
318+
status: 'cancelled',
319+
startedAt: run.startedAt,
320+
finishedAt: '2026-07-23T00:06:00.000Z',
321+
prUrl: run.prUrl,
322+
headSha,
323+
mergeSha: null,
324+
failureFingerprint: null,
325+
notificationUrl: null,
326+
readyNotificationUrl: null,
327+
readyNotifiedAt: null,
328+
completionNotifiedAt: null,
329+
notificationWebhookStatus: null,
330+
predecessorCheckpointUrl: null,
331+
predecessorCheckpointDigest: null,
332+
pauseStartedAt: null,
333+
notificationNotifiedAt: null,
334+
}
335+
const finalizationBody = [
336+
`<!-- issue-dev-loop:finalization:${run.runId}:sha256:${finalizationRecordDigest(finalization)} -->`,
337+
'```json',
338+
canonicalFinalizationRecord(finalization),
339+
'```',
340+
].join('\n')
341+
const finalizationComment = {
342+
id: 3,
343+
user: { login: 'executor-user' },
344+
body: finalizationBody,
345+
html_url: 'https://github.com/example/repo/issues/999#issuecomment-3',
346+
created_at: finalization.finishedAt,
347+
}
348+
const journal = JSON.parse(
349+
await readFile(path.join(parent, 'checkpoint-journal.json'), 'utf8'),
350+
)
351+
await writeFile(
352+
path.join(parent, 'checkpoint-journal.json'),
353+
`${JSON.stringify([...journal, finalizationComment])}\n`,
354+
'utf8',
355+
)
356+
await writeFile(
357+
path.join(parent, 'finalization-comment.json'),
358+
`${JSON.stringify(finalizationComment)}\n`,
359+
'utf8',
360+
)
361+
}
308362
await writeFile(
309363
path.join(parent, 'live-pr.json'),
310364
`${JSON.stringify({
311-
state: 'open',
365+
state: terminalFinalization ? 'closed' : 'open',
366+
merged: false,
312367
draft: liveDraft,
313368
user: { login: 'executor-user' },
314369
base: { ref: 'dev', repo: { full_name: 'example/repo' } },
@@ -454,6 +509,7 @@ if (commandArguments[0] === 'validate' && ${JSON.stringify(historicalActivation)
454509
`export async function validateLoop() {
455510
return { valid: true, historicalTargetCompatibility: true }
456511
}
512+
export function validateFinalizationHistory() {}
457513
`,
458514
'utf8',
459515
)
@@ -507,6 +563,10 @@ if [ "$1 $2 $3 $4" != "api user --jq .login" ]; then
507563
first_line "$parent_dir/evolve-comment.json"
508564
exit 0
509565
fi
566+
if [ "$1" = "api" ] && [ "$2" = "repos/example/repo/issues/comments/3" ]; then
567+
first_line "$parent_dir/finalization-comment.json"
568+
exit 0
569+
fi
510570
if [ "$1 $2" = "pr review" ]; then
511571
echo "comment review published"
512572
exit 0
@@ -917,6 +977,33 @@ test('wrapped activation can select an exact durable checkpoint without local ru
917977
})
918978
})
919979

980+
test('wrapped activation excludes checkpoints superseded by durable finalization', async () => {
981+
const fixture = await createFixture({
982+
historicalActivation: true,
983+
removeLocalRun: true,
984+
terminalFinalization: true,
985+
})
986+
await assert.rejects(
987+
execFileAsync(
988+
routerLauncherPath,
989+
[
990+
'--loop-root',
991+
fixture.loopRoot,
992+
'automation',
993+
'--',
994+
process.execPath,
995+
fixture.loopctlPath,
996+
'validate',
997+
'--activation',
998+
'--loop-root',
999+
fixture.loopRoot,
1000+
],
1001+
{ env: fixture.env },
1002+
),
1003+
/historical target validation requires the exact remote durable active checkpoint/,
1004+
)
1005+
})
1006+
9201007
test('wrapped activation rejects local active state that differs from its durable checkpoint', async () => {
9211008
const fixture = await createFixture({ historicalActivation: true })
9221009
const runPath = path.join(fixture.loopRoot, 'logs', 'runs', 'fixture-run', 'run.json')

0 commit comments

Comments
 (0)