Skip to content

fix: render sixel and iTerm2 inline images (imgcat) again#1703

Open
cantona wants to merge 3 commits into
raphamorim:mainfrom
cantona:fix/iterm2-sixel-rendering
Open

fix: render sixel and iTerm2 inline images (imgcat) again#1703
cantona wants to merge 3 commits into
raphamorim:mainfrom
cantona:fix/iterm2-sixel-rendering

Conversation

@cantona

@cantona cantona commented Jul 8, 2026

Copy link
Copy Markdown

Fixes #1591imgcat, icat, viu, chafa -f iterm and sixel tools parsed fine but nothing ever rendered.

Root cause

Since the kitty overlay rewrite, atlas graphics (sixel / iTerm2 OSC 1337) are parsed and uploaded into sugarloaf.graphics — a store no draw path reads (Renderer::prepare receives it as _graphics), and the per-cell GraphicCell references aren't consumed anywhere either. Only kitty images render, via image_data + GraphicOverlays.

Fix (3 commits)

  1. crosswords: make per-cell graphics state survive large inline images — every covered cell used to allocate its own slot in the u16-indexed extras table, so ~a dozen screen-sized images exhausted the id space (images then rendered partially or not at all until restart). One shared slot per image row + a mark-and-sweep of unreferenced slots (which also reclaims images dropped off scrollback), plus two snapshot-freshness fixes.
  2. deps: enable TIFF decode — chafa's iTerm2 renderer transmits TIFF; same change as Enable TIFF decode for iTerm2 images #1428.
  3. rioterm: render sixel and iTerm2 graphics through the image overlay pipeline — upload atlas pixels into the per-image texture store under a namespaced key (high bit keeps sequential GraphicIds out of the kitty protocol id space) and build overlays per frame by scanning visible rows for graphic cells, mirroring the kitty Unicode-placeholder pass. Row-runs coalesce into one overlay with a normalized source-rect slice, so partial overwrites hide exactly the erased region and images scroll with the grid. Also marks the panel dirty when UpdateGraphics arrives so the pixels aren't invisible until the next unrelated repaint.

Tested (Linux/Wayland, Vulkan)

  • imgcat (PNG), chafa -f iterm (TIFF), chafa -f sixels, viu — all render; chafa -f kitty unaffected.
  • Repeated large (1600×900) images well past the old ~14-image exhaustion point; scrollback, clear, and partial overwrite behave.
  • cargo fmt, clippy, and the full test suite pass; new behavior covered by updated graphics tests.

@cantona

cantona commented Jul 9, 2026

Copy link
Copy Markdown
Author

Hi @raphamorim 👋

First off — thanks for rio, it's become my daily driver. I've been scratching my own itches over the past weeks and have accumulated a batch of PRs. No rush at all, but when you find time, here's a suggested order to make reviewing painless:

Bug fix for a known issue:

Small, low-risk fixes (each a focused diff, quick to review):

Small features:

Bigger features (understand if these need more discussion — happy to adjust scope, split them up, or rework anything to fit your vision for rio):

All are green on fmt/clippy/tests and rebased on current main. If any conflict by the time you get to them, ping me and I'll rebase same-day. And if some don't fit the project direction, no hard feelings — just close them.

Thanks for all the work you put into this project! 🙏

cantona added 3 commits July 9, 2026 17:41
Every cell covered by a sixel/iTerm2 graphic allocated its own slot in
the u16-indexed extras table, so a dozen screen-sized images (~4,500
cells each) exhausted the id space: later images rendered partially or
not at all, and slots referenced by scrollback rows were never
reclaimed.

Share one extras slot per covered image row: cells derive their pixel
offset positionally from an anchor column and the insert-time cell size
recorded on the texture. When the table still nears the cap,
mark-and-sweep the slots against every live row (visible + history) and
the cursor templates; swept graphic slots drop their TextureRef, which
already queues the image removal downstream.

Keep row snapshots fresh when a frame's damage resolves to
Noop/CursorOnly while rows still carry dirty bits (a graphics insert
racing a redraw), and stop reading extras ids off bg-only cells when
snapshotting extras (those bits hold the bg color).
chafa's iTerm2 renderer transmits TIFF payloads over OSC 1337; image-rs
needs the tiff feature to decode them. Same change as raphamorim#1428.
…ipeline

Since the kitty overlay rewrite, atlas graphics (sixel and iTerm2
OSC 1337) were parsed and uploaded into a store no draw path reads:
Renderer::prepare receives it as _graphics, and nothing consumes the
GraphicCell grid references either — the pixels never reached the
screen.

Bridge them into the pipeline that already works: upload the pixels
into the per-image texture store under a namespaced key (high bit set
so sequential GraphicIds stay out of the kitty protocol id space), and
build GraphicOverlays each frame by scanning visible rows for graphic
cells — mirroring the kitty Unicode-placeholder pass. Runs of covered
cells coalesce into one overlay per image row with a normalized
source-rect slice, so partial overwrites hide exactly the erased region
and images scroll with the grid.

The UpdateGraphics handler also marks the panel dirty before requesting
a redraw: the renderer skips non-dirty panels, so pixels arriving after
the last damage-driven frame stayed invisible until unrelated output
forced a repaint.

Fixes raphamorim#1591.
@cantona cantona force-pushed the fix/iterm2-sixel-rendering branch from 94df3a4 to 50f5b33 Compare July 9, 2026 09:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

iTerm2 image protocol broken on 0.4.2 & 0.4.3

1 participant