Skip to content

fix: improve handling of grapheme in OptimizedBuffer and GraphemeTracker#1006

Open
viralcodex wants to merge 9 commits into
anomalyco:mainfrom
viralcodex:panic-fix
Open

fix: improve handling of grapheme in OptimizedBuffer and GraphemeTracker#1006
viralcodex wants to merge 9 commits into
anomalyco:mainfrom
viralcodex:panic-fix

Conversation

@viralcodex
Copy link
Copy Markdown
Contributor

Hi @kommander @simonklee ,

I have made some changes for the regression here (fixes: #971) based on the findings on my end.

So an overview is that now the grapheme.zig add() doesn't panic (throws) when it gets a WrongGeneration or InvalidId error, it deletes that grapheme id from the pool, only on OutOfMemory it throws (which i think makes sense)

also to handle clipping of characters outside the terminal area, some modifications are made to return early when such condition arises, specially in wide characters because the code assigns the grapheme id to them but if they clip it's never cleared or used.

This are the proposed changes based on my findings and overall understanding, I have also run some tests and the problem seems to not occur anymore.

please let me know if anything is missing or needs change from my side, it's my first time on such a bug so could be wrong on my end.

Thanks

@simonklee
Copy link
Copy Markdown
Member

gpt found these. i have my head focused on some other things so i didn't think deeply about it but its worth to check out:

  • packages/core/src/zig/buffer.zig:1473: when drawTextBuffer skips a wide grapheme that overflows the row, it only breaks the current chunk loop and does not advance currentX or column_in_line. If the same visual line has another VirtualChunk after that grapheme, the next chunk can render at the stale edge column even though it should be off-screen. Example: a line built from "aaa界" plus an appended "Z" in a width-4 buffer would skip at x=3, then draw Z at x=3. The overflow path should advance to/past the buffer edge or stop processing remaining chunks for the line.
  • packages/core/src/zig/grapheme.zig:540: GraphemeTracker.add now ignores stale/invalid IDs, but callers like OptimizedBuffer.setInternal write cell.char before calling add. If this path is hit, the buffer can contain an invalid packed grapheme with no tracker entry, which breaks the invariant used by fast paths gated on grapheme_tracker.hasAny(). If stale IDs are expected to be recoverable, the write path needs a way to clear/skip the cell rather than only removing the ID from used_ids.

Co-authored-by: Copilot <copilot@github.com>
@viralcodex
Copy link
Copy Markdown
Contributor Author

@simonklee , added a small utility that checks for grapheme id before adding to buffer and if its stale, returns early before adding anything to buffer.
Also fixed the character positioning for the first issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(core): bufferDrawBox panics (SIGTRAP brk 1) after #920 refactor — regression in 0.1.103 vs 0.1.92

2 participants