Skip to content

Commit 629253d

Browse files
olwangclaude
andcommitted
feat(selfhost): AST dump span parity — tier 1 (@line:col) byte-exact 619/619
Producer (astdump.rss) now emits a per-node span suffix on every spanned node HEAD, reading the node's REPRESENTATIVE token (the single token whose line:col:len the parser records as Node.span): - most nodes: their first (paren-trimmed) token - Binary: the operator token; Try: the trailing `?`; ReceiverCall: the receiver token; TypeRef: the NAME token (after prefix keywords) - decls/stmts: the decl/stmt's first token (incl. attributes); MatchArm: the arm's first token; if-let arms + synthetic protocol `Self` generic: the `if`/`fn` token, as the parser sets them. `expr_rep_tok` mirrors emit_expr's dispatch to recover an expr's span token without descending (used for expr-statement heads and single-expression closure bodies). New scan.rss accessors tk_col/tk_len; emit_at/spanof helpers. Producer ALWAYS emits the richest ` @line:col:len` (like the lexer producer); the harness (run_astdump) PROJECTS each line down to the active AST tier before the byte-exact compare, so tier 0 stays span-less (619/619) and no tier flag need thread through the producer. ast_parity_corpus prints the first differing line per mismatch for fast iteration. RSS_SELFHOST_AST_TIER=1 ast_parity_corpus: 619/619 byte-exact, 0 run-failures. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 1d86a43 commit 629253d

3 files changed

Lines changed: 261 additions & 99 deletions

File tree

crates/rsscript/src/selfhost_parity.rs

Lines changed: 56 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1473,11 +1473,53 @@ fn compile_astdump() -> Result<RegVmExecutable, String> {
14731473
}
14741474

14751475
/// Run the precompiled rss AST-dump producer; its stdout IS the dump.
1476+
///
1477+
/// The producer always emits the RICHEST span suffix (` @line:col:len`) on every
1478+
/// node head (mirroring the lexer producer, which always emits `line:col:len`).
1479+
/// Here we project each line down to the active AST tier so the byte-exact
1480+
/// comparison against the (tier-gated) oracle holds: tier 0 drops the suffix
1481+
/// entirely, tier 1 keeps ` @line:col`, tier 2 keeps ` @line:col:len`. Lines
1482+
/// without a numeric ` @L:C:N` suffix (synthetic labels) are passed through.
14761483
fn run_astdump(exe: &RegVmExecutable, source: &str) -> Result<String, String> {
14771484
let output = exe
14781485
.eval_main_with_args([source.to_string()])
14791486
.map_err(|e| format!("rss astdump failed to run: {e:?}"))?;
1480-
Ok(output.stdout)
1487+
let t = ast_tier();
1488+
if t == 2 {
1489+
return Ok(output.stdout);
1490+
}
1491+
let mut out = String::with_capacity(output.stdout.len());
1492+
for line in output.stdout.split_inclusive('\n') {
1493+
let (body, nl) = match line.strip_suffix('\n') {
1494+
Some(b) => (b, "\n"),
1495+
None => (line, ""),
1496+
};
1497+
out.push_str(&project_ast_line(body, t));
1498+
out.push_str(nl);
1499+
}
1500+
Ok(out)
1501+
}
1502+
1503+
/// Project a producer line's ` @line:col:len` suffix down to `tier` (1 → keep
1504+
/// `line:col`, 0 → drop the suffix). Only a trailing, strictly-numeric
1505+
/// ` @d+:d+:d+` is treated as a span (so a payload containing `@` is untouched).
1506+
fn project_ast_line(line: &str, tier: u8) -> String {
1507+
if let Some(at) = line.rfind(" @") {
1508+
let suffix = &line[at + 2..];
1509+
let parts: Vec<&str> = suffix.split(':').collect();
1510+
if parts.len() == 3
1511+
&& parts
1512+
.iter()
1513+
.all(|p| !p.is_empty() && p.bytes().all(|b| b.is_ascii_digit()))
1514+
{
1515+
let head = &line[..at];
1516+
return match tier {
1517+
0 => head.to_string(),
1518+
_ => format!("{head} @{}:{}", parts[0], parts[1]),
1519+
};
1520+
}
1521+
}
1522+
line.to_string()
14811523
}
14821524

14831525
/// Curated AST-parity samples under `selfhost/samples/ast/`, sorted for stable order.
@@ -1596,7 +1638,19 @@ fn ast_parity_corpus() {
15961638
if actual == oracle {
15971639
ok += 1;
15981640
} else if sample_mismatches.len() < 10 {
1599-
sample_mismatches.push(rel);
1641+
let first_diff = oracle
1642+
.lines()
1643+
.zip(actual.lines())
1644+
.find(|(o, a)| o != a)
1645+
.map(|(o, a)| format!("\n oracle: {o:?}\n rss: {a:?}"))
1646+
.unwrap_or_else(|| {
1647+
format!(
1648+
"\n (line count: oracle {} vs rss {})",
1649+
oracle.lines().count(),
1650+
actual.lines().count()
1651+
)
1652+
});
1653+
sample_mismatches.push(format!("{rel}{first_diff}"));
16001654
}
16011655
}
16021656
}

0 commit comments

Comments
 (0)