Skip to content

Commit 2a89273

Browse files
sgwannabeclaude
andcommitted
test(e2e): apply PR #92 review feedback (gemini + codex)
- Reject empty --out-dir / --out-dir= values during arg parsing so evidence-capture callers do not silently fall back to the auto-cleaned tmp dir and lose artifacts (codex P2). - Clear known artifact patterns (idea*, previews*, P[0-9][0-9].json, spec-anchor-audit.json, .h1-helper.out, mockups/, etc.) when --out-dir is supplied, so reusing a prior dir for a different profile cannot leave stale evidence that masks regressions (codex P2). - Sanitize .h1-helper.out under --out-dir to strip the machine-local REPO_ROOT prefix, leaving a repo-relative URL so the committed evidence is portable across machines (gemini medium). - Use a per-profile fixed timestamp_utc in trace.log when --out-dir is set so re-running the ASSESSMENT.md reproduce command produces byte-stable evidence; live (tmp) runs still emit wall-clock time for triage (gemini medium). - Regenerate runs/r-clean-20260425-{standard,pro,max}/.h1-helper.out with the sanitized URL form. Out of scope for this PR (file-scope: harness + runs/r-clean-* + ASSESSMENT.md + .gitignore): scripts/generate-gallery.sh hardcoded lang="ko" — flagged for a separate fix in the gallery generator. Refs PR #92 review comments Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 976d9e0 commit 2a89273

4 files changed

Lines changed: 49 additions & 4 deletions

File tree

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
{"mode":"browser","url":"/Users/sgwannabe/2026/PreviewForgeForClaudeCode/.claude/worktrees/agent-ae4f1bf9f868d71e4/runs/r-clean-20260425-max/mockups/gallery.html"}
1+
{"mode":"browser","url":"runs/r-clean-20260425-max/mockups/gallery.html"}
Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
{"mode":"browser","url":"/Users/sgwannabe/2026/PreviewForgeForClaudeCode/.claude/worktrees/agent-ae4f1bf9f868d71e4/runs/r-clean-20260425-pro/mockups/gallery.html"}
1+
{"mode":"browser","url":"runs/r-clean-20260425-pro/mockups/gallery.html"}
Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
{"mode":"browser","url":"/Users/sgwannabe/2026/PreviewForgeForClaudeCode/.claude/worktrees/agent-ae4f1bf9f868d71e4/runs/r-clean-20260425-standard/mockups/gallery.html"}
1+
{"mode":"browser","url":"runs/r-clean-20260425-standard/mockups/gallery.html"}

tests/e2e/mock-bootstrap.sh

Lines changed: 46 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -65,10 +65,14 @@ while [ $# -gt 0 ]; do
6565
--out-dir)
6666
[ $# -ge 2 ] || { echo "usage: $0 <profile> [--out-dir <path>]" >&2; exit 2; }
6767
OUT_DIR="$2"
68+
[ -n "$OUT_DIR" ] || { echo "$0: --out-dir requires a non-empty path" >&2; exit 2; }
6869
shift 2
6970
;;
7071
--out-dir=*)
7172
OUT_DIR="${1#--out-dir=}"
73+
# Reject empty --out-dir= (would silently fall back to auto-cleaned tmp,
74+
# surprising evidence-capture callers — codex review PR #92).
75+
[ -n "$OUT_DIR" ] || { echo "$0: --out-dir= requires a non-empty path" >&2; exit 2; }
7276
shift
7377
;;
7478
-h|--help)
@@ -145,6 +149,17 @@ if [ -n "$OUT_DIR" ]; then
145149
mkdir -p "$OUT_DIR"
146150
RUN_DIR=$(cd "$OUT_DIR" && pwd)
147151
RUN_ID=$(basename "$RUN_DIR")
152+
# Clear known artifact patterns to prevent stale evidence from masking
153+
# regressions (e.g. an old `max` dir reused for a `standard` run would
154+
# leave P10..P26 + spec-anchor-audit.json behind, and step 9 only
155+
# checks required-file presence — codex review PR #92).
156+
rm -f "$RUN_DIR"/idea.json "$RUN_DIR"/idea.spec.json \
157+
"$RUN_DIR"/previews.json "$RUN_DIR"/chosen_preview.json \
158+
"$RUN_DIR"/chosen_preview.json.lock "$RUN_DIR"/spec-anchor-audit.json \
159+
"$RUN_DIR"/trace.log "$RUN_DIR"/.filled-ratio-gate.out \
160+
"$RUN_DIR"/.h1-helper.out "$RUN_DIR"/.convergence-lint.out
161+
rm -f "$RUN_DIR"/P[0-9][0-9].json
162+
rm -rf "$RUN_DIR"/mockups
148163
else
149164
RUN_ID="r-e2e-$PROFILE-$(date -u +%Y%m%d%H%M%S)"
150165
RUN_DIR="$TMP_PF_HOME/runs/$RUN_ID"
@@ -350,6 +365,23 @@ grep -q '"mode":"browser"' "$RUN_DIR/.h1-helper.out" \
350365
grep -qE '^(open|xdg-open) .*gallery\.html' "$OPEN_BROWSER_TRACE" \
351366
|| fail "step 5b: stub did not record opener invocation hitting gallery.html"
352367

368+
# Sanitize the absolute machine path out of .h1-helper.out under --out-dir
369+
# so committed evidence is portable across machines (gemini review PR #92).
370+
# h1-modal-helper.sh resolves the gallery to an absolute path; we rewrite
371+
# it to a repo-relative form using REPO_ROOT as the prefix to strip.
372+
if [ -n "$OUT_DIR" ]; then
373+
python3 - "$RUN_DIR/.h1-helper.out" "$REPO_ROOT" <<'PY' || fail "step 5b: .h1-helper.out sanitization failed"
374+
import json, pathlib, sys
375+
out = pathlib.Path(sys.argv[1])
376+
repo_root = sys.argv[2].rstrip("/")
377+
data = json.loads(out.read_text())
378+
url = data.get("url", "")
379+
if url.startswith(repo_root + "/"):
380+
data["url"] = url[len(repo_root) + 1:]
381+
out.write_text(json.dumps(data, separators=(",", ":")) + "\n", encoding="utf-8")
382+
PY
383+
fi
384+
353385
# ---------- step 6: chosen_preview lock (canned H1 pick) ----------
354386

355387
python3 - "$CANNED" "$RUN_DIR" <<'PY' || fail "step 6: chosen_preview lock"
@@ -480,7 +512,20 @@ TRACE_LOG="$RUN_DIR/trace.log"
480512
echo "iframe_count=$IFRAME_COUNT"
481513
echo "framework_lint_rc=$LINT_RC"
482514
echo "h1_pick=$PF_H1_PICK"
483-
echo "timestamp_utc=$(date -u +%Y-%m-%dT%H:%M:%SZ)"
515+
# Byte-reproducible timestamp under --out-dir so committed evidence
516+
# diffs against re-runs cleanly (gemini review PR #92). Live runs keep
517+
# the real wall-clock for triage.
518+
if [ -n "$OUT_DIR" ]; then
519+
# Per-profile fixed timestamp — keeps committed evidence stable (and
520+
# distinguishable per profile in the historical record) across re-runs.
521+
case "$PROFILE" in
522+
standard) echo "timestamp_utc=2026-04-25T10:01:05Z" ;;
523+
pro) echo "timestamp_utc=2026-04-25T10:01:07Z" ;;
524+
max) echo "timestamp_utc=2026-04-25T10:01:10Z" ;;
525+
esac
526+
else
527+
echo "timestamp_utc=$(date -u +%Y-%m-%dT%H:%M:%SZ)"
528+
fi
484529
echo "run_id=$RUN_ID"
485530
echo "out_dir_mode=$([ -n "$OUT_DIR" ] && echo committed || echo tmp)"
486531
echo

0 commit comments

Comments
 (0)