Skip to content

Commit 12dc993

Browse files
gHashTaggHashTag
andauthored
fix: reject stray token after bare expression statement (#1118)
A stray token that still lexes as an identifier (e.g. `frobnicate x`) used to be accepted as a bare no-op statement and leak into codegen as `frobnicate;` (the #1110 known-gap characterization). parse_body_stmt now requires a valid statement boundary after a bare expression; any other token is a syntax error, so the malformed statement is dropped by recover_to_stmt_boundary. Promotes the characterization test to a rejection assertion. Full suite green, 0 regressions. Closes #1115 Co-authored-by: gHashTag <admin@t27.ai>
1 parent b14d4df commit 12dc993

2 files changed

Lines changed: 44 additions & 23 deletions

File tree

bootstrap/src/compiler.rs

Lines changed: 36 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -1745,9 +1745,31 @@ impl Parser {
17451745
return Ok(assign);
17461746
}
17471747

1748-
// Bare expression statement
1749-
if self.current.kind == TokenKind::Semicolon {
1750-
self.advance();
1748+
// Bare expression statement. A well-formed bare expression is
1749+
// terminated by a semicolon, or sits as the final expression directly
1750+
// before the block's closing brace (or EOF). Any OTHER token here means
1751+
// the statement did not end where it should -- e.g. `frobnicate x`,
1752+
// where `frobnicate` parses as a bare ident expression and the stray
1753+
// `x` is left dangling. Previously such a leftover token was silently
1754+
// ignored and the bare expression accepted, letting malformed input
1755+
// leak into codegen (the #1110 known-gap characterization). Treat the
1756+
// dangling token as a syntax error so the caller's statement-level
1757+
// recovery (`recover_to_stmt_boundary`) drops the malformed statement,
1758+
// consistent with how other malformed statements are handled.
1759+
match self.current.kind {
1760+
TokenKind::Semicolon => {
1761+
self.advance();
1762+
}
1763+
TokenKind::RBrace | TokenKind::Eof => {
1764+
// Final expression of a block, or end of input: no terminator
1765+
// required.
1766+
}
1767+
_ => {
1768+
return Err(format!(
1769+
"unexpected token after expression statement: {:?}",
1770+
self.current.kind
1771+
));
1772+
}
17511773
}
17521774
let mut stmt = Node::new(NodeKind::StmtExpr);
17531775
stmt.children.push(expr);
@@ -19837,14 +19859,16 @@ mod tests_compiler_rejects {
1983719859
);
1983819860
}
1983919861

19840-
// (c) KNOWN GAP characterization: a stray token that still lexes as an
19841-
// identifier (`frobnicate`) is currently accepted as a bare no-op statement
19842-
// and LEAKS into codegen as `frobnicate;`, and the following `return` still
19843-
// lowers. This is NOT the desired contract -- it is pinned so the leak is
19844-
// visible and any future hardening of the parser is detected by this test
19845-
// failing (at which point it should be converted to an `assert_dropped`).
19862+
// (c) GAP CLOSED (#1110 -> #1115): a stray token that still lexes as an
19863+
// identifier (`frobnicate x`) used to be accepted as a bare no-op statement
19864+
// and LEAK into codegen as `frobnicate;`. The bare-expression branch in
19865+
// `parse_body_stmt` now rejects a dangling token after an expression, so the
19866+
// malformed statement triggers statement-level recovery and the body is
19867+
// dropped to `// TODO: implement` -- the bogus `frobnicate` never reaches
19868+
// codegen. This was a characterization test for the known gap; with the gap
19869+
// closed it is now a rejection assertion (renamed accordingly).
1984619870
#[test]
19847-
fn characterizes_stray_ident_leak_known_gap() {
19871+
fn rejects_stray_ident() {
1984819872
let v = emit(
1984919873
r#"module GapStrayIdent {
1985019874
pub fn k(x: u8) -> u8 {
@@ -19853,19 +19877,8 @@ mod tests_compiler_rejects {
1985319877
}
1985419878
}"#,
1985519879
);
19856-
// Current (undesired) behavior: the stray identifier leaks as a no-op
19857-
// statement and the body is NOT dropped.
19858-
assert!(
19859-
v.contains("frobnicate;"),
19860-
"KNOWN GAP changed: stray ident no longer leaks as `frobnicate;`. \
19861-
If the parser now rejects it, convert this test to assert_dropped. Got:\n{}",
19862-
v
19863-
);
19864-
assert!(
19865-
!v.contains("// TODO: implement"),
19866-
"KNOWN GAP changed: body is now dropped; update this characterization. Got:\n{}",
19867-
v
19868-
);
19880+
// The malformed statement is dropped; the bogus ident never leaks.
19881+
assert_dropped(&v, "k", &["frobnicate"]);
1986919882
}
1987019883
}
1987119884

docs/NOW.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,14 @@
22

33
Last updated: 2026-06-14
44

5+
## fix-stray-ident-leak -- close the stray-identifier parser leak (#1110 known-gap -> fix) (Closes #1115)
6+
7+
- **WHERE**: `bootstrap/src/compiler.rs` (`parse_body_stmt` bare-expression branch; the #1110 characterization test renamed `characterizes_stray_ident_leak_known_gap` -> `rejects_stray_ident` and switched to `assert_dropped`).
8+
- **WHAT**: #1110 pinned a known gap -- a stray token that still lexes as an identifier (`frobnicate x`) was accepted as a bare no-op statement and LEAKED into codegen as `frobnicate;`, body NOT dropped. Root cause: after parsing a bare expression, `parse_body_stmt` silently ignored whatever token followed, leaving the dangling `x` for the next loop iteration instead of rejecting the malformed statement. Fix: after a bare expression the next token must be a valid statement boundary -- `;` (consumed), or `}` / EOF (final expression of a block); any other token is now a syntax error, so the malformed statement triggers the existing statement-level recovery (`recover_to_stmt_boundary`) and the body is dropped to `// TODO: implement`, consistent with how unclosed parens and malformed binops are already handled. The bogus ident never reaches codegen. The characterization test is promoted to a rejection assertion.
9+
- **Why** this is a real production-parser change (stricter), so it was validated against the full suite: 1452 passed / 0 failed / 2 ignored, 0 regressions -- confirming the previous lenient behavior was a genuine gap, not relied upon by any test or fixture. The honest #1110 framing pays off: the documented gap is now closed and the test converted exactly as that entry promised. L6 gf16 SSOT untouched; catalog stays 83; no gen/ edits; ASCII-only added lines; no quality claim added. Closes #1115.
10+
- **Anchor**: phi^2 + phi^-2 = 3
11+
12+
513
## deadcode-annotate-host-mod -- #969 next layer: annotate the host/mod.rs re-export facade (Closes #1111)
614

715
- **WHERE**: `bootstrap/src/host/mod.rs` (one module-scoped inner attribute + explanatory comment, after the header comment, before the `pub mod` declarations).

0 commit comments

Comments
 (0)