From f8d17c6fa8653e6a50f792d85c8c57a12252b70e Mon Sep 17 00:00:00 2001 From: Dylan Piercey Date: Thu, 30 Jul 2026 21:34:36 -0700 Subject: [PATCH 1/2] docs(agent-feedback): take over two tokenizer defects from the marko backlog --- agent-feedback/bugs.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/agent-feedback/bugs.md b/agent-feedback/bugs.md index b95fa68..81289ba 100644 --- a/agent-feedback/bugs.md +++ b/agent-feedback/bugs.md @@ -1,3 +1,15 @@ # Suspected Bugs Out-of-scope defects noticed while working on something else. Format and rules: [README.md](README.md). + +## Emit an error when a `>=` truncates an unquoted attribute-value expression + +`src/states/ATTRIBUTE.ts` › `shouldTerminateHtmlAttrValue` | 2026-07-30 | impact:med | effort:low + +`shouldTerminateHtmlAttrValue` terminates on the first unenclosed `CODE.CLOSE_ANGLE_BRACKET` with a single look-behind exception for `=>`, and nothing covers the sibling `>=`, so `= 0/>` tokenizes as `attrValue "input.delta"`, `openTagEnd`, then text `"= 0/>"` and raises no `onError` at all — marko turns that into `const positive = input.delta` plus a literal `= 0/>` text node, a silent miscompile of an ordinary comparison. Unlike a lone `>`, which is genuinely ambiguous with idiomatic HTML (`
text
`), a tag close immediately followed by `=` is almost never intended, so this form is decidable here rather than in a consumer's heuristic. Add the look-ahead to the `CLOSE_ANGLE_BRACKET` case: keep terminating, but when `pos !== this.start && data.charCodeAt(pos + 1) === CODE.EQUAL` record it on the `ExpressionMeta` so `ATTRIBUTE.return`'s `ATTR_STAGE.VALUE` case — which already calls `this.emitError` for `INVALID_ATTRIBUTE_VALUE` — can emit `ErrorCode.INVALID_EXPRESSION`; the emit has to land there because `shouldTerminate` is handed no `Parser`. The one false positive to weigh is `
=1
`, whose text legitimately begins with `=`. Re-verify with `node --input-type=module -e 'import{createParser,TagType}from"./src/index.ts";const p=createParser({onError:e=>console.log("ERR",e.message),onText:r=>console.log("text",JSON.stringify(p.read(r))),onAttrValue:r=>console.log("value",JSON.stringify(p.read(r.value))),onOpenTagName:()=>TagType.html});p.parse("= 0>yes")'` — it prints `value "input.delta"` and `text "= 0>yes"` with no `ERR` line today, and `pnpm test` must stay green after the change. + +## Treat a backslash-escaped quote as text in parsed-text bodies and parsed strings + +`src/states/PARSED_TEXT_CONTENT.ts` › `PARSED_TEXT_CONTENT` | 2026-07-30 | impact:med | effort:low + +Inside a `TagType.text` body the only backslash handling is `STATE.checkForPlaceholder`, which declines unless the run of backslashes leads to `${`, so in `PARSED_TEXT_CONTENT.parse` a `\` falls through to the eager text run and the `"`/`'` after it still enters `STATE.PARSED_STRING`; `src/states/PARSED_STRING.ts` › `PARSED_STRING` has the mirror hole, where the quote of `\"` matches `str.quoteCharCode` and closes the string. An odd number of escaped quotes therefore runs to EOF and `PARSED_STRING.parse` emits `INVALID_TEMPLATE_STRING` "EOF reached while parsing string expression" — marko, which maps `` and `content: 'it\'s'` are both unparseable today; `src/__tests__/fixtures/parsed-text-style-tag` only passes because its escaped quotes happen to come in pairs, and `\2014` parses fine, so the defect is quote-specific rather than backslash-general. The fix is symmetric in the two files: when `checkForPlaceholder` declines a `CODE.BACK_SLASH` and the next char is a quote (the active `quoteCharCode` in `PARSED_STRING`), consume both chars as text instead of letting the quote change state. Re-verify with `node --input-type=module -e 'import{createParser,TagType}from"./src/index.ts";const p=createParser({onError:e=>console.log("ERR",e.message),onText:r=>console.log("text",JSON.stringify(p.read(r))),onOpenTagName:()=>TagType.text});p.parse("")'` — it prints `ERR EOF reached while parsing string expression` today and should print a single `text` range; lock it in with a new `src/__tests__/fixtures/` dir plus `pnpm test:update`. From 8080e578a37311481e7d2e736e7c80ae5fef6c54 Mon Sep 17 00:00:00 2001 From: Dylan Piercey Date: Fri, 31 Jul 2026 08:05:41 -0700 Subject: [PATCH 2/2] style(agent-feedback): satisfy prettier --- agent-feedback/bugs.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/agent-feedback/bugs.md b/agent-feedback/bugs.md index 81289ba..b516434 100644 --- a/agent-feedback/bugs.md +++ b/agent-feedback/bugs.md @@ -12,4 +12,4 @@ Out-of-scope defects noticed while working on something else. Format and rules: `src/states/PARSED_TEXT_CONTENT.ts` › `PARSED_TEXT_CONTENT` | 2026-07-30 | impact:med | effort:low -Inside a `TagType.text` body the only backslash handling is `STATE.checkForPlaceholder`, which declines unless the run of backslashes leads to `${`, so in `PARSED_TEXT_CONTENT.parse` a `\` falls through to the eager text run and the `"`/`'` after it still enters `STATE.PARSED_STRING`; `src/states/PARSED_STRING.ts` › `PARSED_STRING` has the mirror hole, where the quote of `\"` matches `str.quoteCharCode` and closes the string. An odd number of escaped quotes therefore runs to EOF and `PARSED_STRING.parse` emits `INVALID_TEMPLATE_STRING` "EOF reached while parsing string expression" — marko, which maps `` and `content: 'it\'s'` are both unparseable today; `src/__tests__/fixtures/parsed-text-style-tag` only passes because its escaped quotes happen to come in pairs, and `\2014` parses fine, so the defect is quote-specific rather than backslash-general. The fix is symmetric in the two files: when `checkForPlaceholder` declines a `CODE.BACK_SLASH` and the next char is a quote (the active `quoteCharCode` in `PARSED_STRING`), consume both chars as text instead of letting the quote change state. Re-verify with `node --input-type=module -e 'import{createParser,TagType}from"./src/index.ts";const p=createParser({onError:e=>console.log("ERR",e.message),onText:r=>console.log("text",JSON.stringify(p.read(r))),onOpenTagName:()=>TagType.text});p.parse("")'` — it prints `ERR EOF reached while parsing string expression` today and should print a single `text` range; lock it in with a new `src/__tests__/fixtures/` dir plus `pnpm test:update`. +Inside a `TagType.text` body the only backslash handling is `STATE.checkForPlaceholder`, which declines unless the run of backslashes leads to `${`, so in `PARSED_TEXT_CONTENT.parse` a `\` falls through to the eager text run and the `"`/`'` after it still enters `STATE.PARSED_STRING`; `src/states/PARSED_STRING.ts` › `PARSED_STRING` has the mirror hole, where the quote of `\"` matches `str.quoteCharCode` and closes the string. An odd number of escaped quotes therefore runs to EOF and `PARSED_STRING.parse` emits `INVALID_TEMPLATE_STRING` "EOF reached while parsing string expression" — marko, which maps `` and `content: 'it\'s'` are both unparseable today; `src/__tests__/fixtures/parsed-text-style-tag` only passes because its escaped quotes happen to come in pairs, and `\2014` parses fine, so the defect is quote-specific rather than backslash-general. The fix is symmetric in the two files: when `checkForPlaceholder` declines a `CODE.BACK_SLASH` and the next char is a quote (the active `quoteCharCode` in `PARSED_STRING`), consume both chars as text instead of letting the quote change state. Re-verify with `node --input-type=module -e 'import{createParser,TagType}from"./src/index.ts";const p=createParser({onError:e=>console.log("ERR",e.message),onText:r=>console.log("text",JSON.stringify(p.read(r))),onOpenTagName:()=>TagType.text});p.parse("")'` — it prints `ERR EOF reached while parsing string expression` today and should print a single `text` range; lock it in with a new `src/__tests__/fixtures/` dir plus `pnpm test:update`.