Skip to content

Commit 69bd0d6

Browse files
anandgupta42anandgupta42
authored
feat: smooth streaming mode for TUI response rendering (#281)
* feat: [AI-280] add `ALTIMATE_SMOOTH_STREAMING` flag for smoother TUI response rendering During LLM streaming, the `<markdown>` element re-lays out block elements on every token delta, causing visible text jumps and jerky scrolling. This adds an opt-in `ALTIMATE_SMOOTH_STREAMING` feature flag with four optimizations: - Use `<code filetype="markdown">` during streaming (no layout-shifting blocks), swap to `<markdown>` after message completion for rich rendering - Pre-merge consecutive `message.part.delta` events in the SDK 16ms batch window (N tokens → 1 store update per part per frame) - Replace `produce()` with direct store path updates on the delta hot path - Reduce `toBottom()` scroll delay from 50ms to 0ms - Memoize `trim()` in `TextPart` (unconditional, no flag needed) Enable with: `ALTIMATE_SMOOTH_STREAMING=true` * docs: add `ALTIMATE_SMOOTH_STREAMING` to experimental flags table * fix: address code review — delta merge ordering and `streaming` regression - Clear `deltaMap` on non-delta events to preserve causal ordering (prevents folding deltas across intervening `message.part.updated` events) - Clone event objects instead of mutating in-place during delta merge - Restore dynamic `streaming` prop in fallback `<markdown>`/`<code>` blocks (`!props.message.time.completed`) to avoid regression for non-opt-in users * feat: add `ALTIMATE_CALM_MODE` — unified Claude Code-like streaming experience Combines three streaming optimizations under one flag: - **Smooth streaming** (`ALTIMATE_SMOOTH_STREAMING`): renders with `<code>` during streaming to avoid markdown layout jumps, swaps to `<markdown>` after message completion - **Line buffering** (`ALTIMATE_LINE_STREAMING`): buffers deltas and flushes only on `\n` (complete lines). Remaining text flushes on message completion. No partial lines ever appear. - **Width cap** (`ALTIMATE_CONTENT_MAX_WIDTH`): caps text at 100 columns for readability. Automatically disabled on small screens where the cap would exceed available width. `ALTIMATE_CALM_MODE=true` enables all three with sensible defaults. Individual flags still work independently for fine-grained control. Includes 38 unit tests covering delta merging, line buffering, flag composition, and width capping edge cases (small screens, empty buffers, consecutive newlines, cross-message isolation). * fix: clean up line buffers on message removal, add calm mode docs - Flush/delete line buffer entries in `message.removed` handler to prevent memory leaks when messages are aborted or removed without `time.completed` - Add clarifying comment explaining line streaming / smooth streaming interaction (line streaming branch handles its own direct store updates) - Add "Calm Mode Quick Start" section to CLI docs with usage examples - Update `ALTIMATE_LINE_STREAMING` doc to mention abort cleanup * fix: discard line buffer on `message.part.updated` and `message.part.removed` When streaming ends, `message.part.updated` writes the full final text via `reconcile()`. Without clearing the line buffer first, `flushAllBuffersForMessage` on `message.updated` would append the remaining buffered text on top of the already-complete content — duplicating the trailing partial line. Fix: discard all line buffer entries for a part when `message.part.updated` fires (the server's content is authoritative). Also clear on `message.part.removed` to prevent orphaned buffer entries.
1 parent 77aad83 commit 69bd0d6

6 files changed

Lines changed: 748 additions & 74 deletions

File tree

‎docs/docs/usage/cli.md‎

Lines changed: 79 additions & 54 deletions
Original file line numberDiff line numberDiff line change
@@ -19,82 +19,107 @@ altimate --agent analyst
1919
2020
## Subcommands
2121

22-
| Command | Description |
23-
|---------|------------|
24-
| `run` | Run a prompt non-interactively |
25-
| `serve` | Start the HTTP API server |
26-
| `web` | Start the web UI |
27-
| `agent` | Agent management |
28-
| `auth` | Authentication |
29-
| `mcp` | Model Context Protocol tools |
30-
| `acp` | Agent Communication Protocol |
31-
| `models` | List available models |
32-
| `stats` | Usage statistics |
33-
| `export` | Export session data |
34-
| `import` | Import session data |
35-
| `session` | Session management |
36-
| `trace` | List and view session traces |
37-
| `github` | GitHub integration |
38-
| `pr` | Pull request tools |
39-
| `upgrade` | Upgrade to latest version |
40-
| `uninstall` | Uninstall altimate |
22+
| Command | Description |
23+
| ----------- | ------------------------------ |
24+
| `run` | Run a prompt non-interactively |
25+
| `serve` | Start the HTTP API server |
26+
| `web` | Start the web UI |
27+
| `agent` | Agent management |
28+
| `auth` | Authentication |
29+
| `mcp` | Model Context Protocol tools |
30+
| `acp` | Agent Communication Protocol |
31+
| `models` | List available models |
32+
| `stats` | Usage statistics |
33+
| `export` | Export session data |
34+
| `import` | Import session data |
35+
| `session` | Session management |
36+
| `trace` | List and view session traces |
37+
| `github` | GitHub integration |
38+
| `pr` | Pull request tools |
39+
| `upgrade` | Upgrade to latest version |
40+
| `uninstall` | Uninstall altimate |
4141

4242
## Global Flags
4343

44-
| Flag | Description |
45-
|------|------------|
46-
| `--model <provider/model>` | Override the default model |
47-
| `--agent <name>` | Start with a specific agent |
48-
| `--print-logs` | Print logs to stderr |
49-
| `--log-level <level>` | Set log level: `DEBUG`, `INFO`, `WARN`, `ERROR` |
50-
| `--help`, `-h` | Show help |
51-
| `--version`, `-v` | Show version |
44+
| Flag | Description |
45+
| -------------------------- | ----------------------------------------------- |
46+
| `--model <provider/model>` | Override the default model |
47+
| `--agent <name>` | Start with a specific agent |
48+
| `--print-logs` | Print logs to stderr |
49+
| `--log-level <level>` | Set log level: `DEBUG`, `INFO`, `WARN`, `ERROR` |
50+
| `--help`, `-h` | Show help |
51+
| `--version`, `-v` | Show version |
5252

5353
## Environment Variables
5454

5555
Configuration can be controlled via environment variables:
5656

5757
### Core Configuration
5858

59-
| Variable | Description |
60-
|----------|------------|
61-
| `ALTIMATE_CLI_CONFIG` | Path to custom config file |
62-
| `ALTIMATE_CLI_CONFIG_DIR` | Custom config directory |
59+
| Variable | Description |
60+
| ----------------------------- | ---------------------------- |
61+
| `ALTIMATE_CLI_CONFIG` | Path to custom config file |
62+
| `ALTIMATE_CLI_CONFIG_DIR` | Custom config directory |
6363
| `ALTIMATE_CLI_CONFIG_CONTENT` | Inline config as JSON string |
64-
| `ALTIMATE_CLI_GIT_BASH_PATH` | Path to Git Bash (Windows) |
64+
| `ALTIMATE_CLI_GIT_BASH_PATH` | Path to Git Bash (Windows) |
6565

6666
### Feature Toggles
6767

68-
| Variable | Description |
69-
|----------|------------|
70-
| `ALTIMATE_CLI_DISABLE_AUTOUPDATE` | Disable automatic updates |
71-
| `ALTIMATE_CLI_DISABLE_LSP_DOWNLOAD` | Don't auto-download LSP servers |
72-
| `ALTIMATE_CLI_DISABLE_AUTOCOMPACT` | Disable automatic context compaction |
73-
| `ALTIMATE_CLI_DISABLE_DEFAULT_PLUGINS` | Skip loading default plugins |
74-
| `ALTIMATE_CLI_DISABLE_EXTERNAL_SKILLS` | Disable external skill discovery |
75-
| `ALTIMATE_CLI_DISABLE_PROJECT_CONFIG` | Ignore project-level config files |
76-
| `ALTIMATE_CLI_DISABLE_TERMINAL_TITLE` | Don't set terminal title |
77-
| `ALTIMATE_CLI_DISABLE_PRUNE` | Disable database pruning |
78-
| `ALTIMATE_CLI_DISABLE_MODELS_FETCH` | Don't fetch models from models.dev |
68+
| Variable | Description |
69+
| -------------------------------------- | ------------------------------------ |
70+
| `ALTIMATE_CLI_DISABLE_AUTOUPDATE` | Disable automatic updates |
71+
| `ALTIMATE_CLI_DISABLE_LSP_DOWNLOAD` | Don't auto-download LSP servers |
72+
| `ALTIMATE_CLI_DISABLE_AUTOCOMPACT` | Disable automatic context compaction |
73+
| `ALTIMATE_CLI_DISABLE_DEFAULT_PLUGINS` | Skip loading default plugins |
74+
| `ALTIMATE_CLI_DISABLE_EXTERNAL_SKILLS` | Disable external skill discovery |
75+
| `ALTIMATE_CLI_DISABLE_PROJECT_CONFIG` | Ignore project-level config files |
76+
| `ALTIMATE_CLI_DISABLE_TERMINAL_TITLE` | Don't set terminal title |
77+
| `ALTIMATE_CLI_DISABLE_PRUNE` | Disable database pruning |
78+
| `ALTIMATE_CLI_DISABLE_MODELS_FETCH` | Don't fetch models from models.dev |
7979

8080
### Server & Security
8181

82-
| Variable | Description |
83-
|----------|------------|
82+
| Variable | Description |
83+
| ------------------------------ | ------------------------------- |
8484
| `ALTIMATE_CLI_SERVER_USERNAME` | Server HTTP basic auth username |
8585
| `ALTIMATE_CLI_SERVER_PASSWORD` | Server HTTP basic auth password |
86-
| `ALTIMATE_CLI_PERMISSION` | Permission config as JSON |
86+
| `ALTIMATE_CLI_PERMISSION` | Permission config as JSON |
8787

8888
### Experimental
8989

90-
| Variable | Description |
91-
|----------|------------|
92-
| `ALTIMATE_CLI_EXPERIMENTAL` | Enable all experimental features |
93-
| `ALTIMATE_CLI_EXPERIMENTAL_FILEWATCHER` | Enable file watcher |
94-
| `ALTIMATE_CLI_EXPERIMENTAL_BASH_DEFAULT_TIMEOUT_MS` | Custom bash timeout (ms) |
95-
| `ALTIMATE_CLI_EXPERIMENTAL_OUTPUT_TOKEN_MAX` | Max output tokens |
96-
| `ALTIMATE_CLI_EXPERIMENTAL_PLAN_MODE` | Enable plan mode |
97-
| `ALTIMATE_CLI_ENABLE_EXA` | Enable Exa web search |
90+
| Variable | Description |
91+
| --------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
92+
| `ALTIMATE_CLI_EXPERIMENTAL` | Enable all experimental features |
93+
| `ALTIMATE_CLI_EXPERIMENTAL_FILEWATCHER` | Enable file watcher |
94+
| `ALTIMATE_CLI_EXPERIMENTAL_BASH_DEFAULT_TIMEOUT_MS` | Custom bash timeout (ms) |
95+
| `ALTIMATE_CLI_EXPERIMENTAL_OUTPUT_TOKEN_MAX` | Max output tokens |
96+
| `ALTIMATE_CLI_EXPERIMENTAL_PLAN_MODE` | Enable plan mode |
97+
| `ALTIMATE_CLI_ENABLE_EXA` | Enable Exa web search |
98+
| `ALTIMATE_CALM_MODE` | Enables all streaming optimizations: smooth rendering, line-at-a-time buffering, and 100-column width cap. Recommended for a Claude Code-like experience. Equivalent to setting `ALTIMATE_SMOOTH_STREAMING=true ALTIMATE_LINE_STREAMING=true ALTIMATE_CONTENT_MAX_WIDTH=100`. |
99+
| `ALTIMATE_SMOOTH_STREAMING` | Uses lightweight `<code>` rendering during LLM streaming, then swaps to rich markdown after completion. Reduces text jumps and scroll jitter. Included in `ALTIMATE_CALM_MODE`. |
100+
| `ALTIMATE_LINE_STREAMING` | Buffers LLM output and reveals one complete line at a time (on `\n`). Gives a calm, steady flow. Remaining text flushes on message completion or abort. Included in `ALTIMATE_CALM_MODE`. |
101+
| `ALTIMATE_CONTENT_MAX_WIDTH` | Cap text content width in columns (e.g. `100`). Improves readability on wide screens. Automatically disabled on small terminals. Set to `100` by `ALTIMATE_CALM_MODE`. |
102+
103+
#### Calm Mode Quick Start
104+
105+
For a Claude Code-like streaming experience, add to your shell profile:
106+
107+
```bash
108+
export ALTIMATE_CALM_MODE=true
109+
```
110+
111+
Or use individual flags for fine-grained control:
112+
113+
```bash
114+
# Smooth rendering only (no line buffering)
115+
export ALTIMATE_SMOOTH_STREAMING=true
116+
117+
# Line buffering only (no rendering changes)
118+
export ALTIMATE_LINE_STREAMING=true
119+
120+
# Custom width cap (e.g., 80 columns)
121+
export ALTIMATE_CONTENT_MAX_WIDTH=80
122+
```
98123

99124
## Non-interactive Usage
100125

‎packages/opencode/src/cli/cmd/tui/context/sdk.tsx‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,9 @@ import { createOpencodeClient, type Event } from "@opencode-ai/sdk/v2"
22
import { createSimpleContext } from "./helper"
33
import { createGlobalEmitter } from "@solid-primitives/event-bus"
44
import { batch, onCleanup, onMount } from "solid-js"
5+
// altimate_change start - smooth streaming
6+
import { Flag } from "@/flag/flag"
7+
// altimate_change end
58

69
export type EventSource = {
710
on: (handler: (event: Event) => void) => () => void
@@ -48,6 +51,44 @@ export const { use: useSDK, provider: SDKProvider } = createSimpleContext({
4851
queue = []
4952
timer = undefined
5053
last = Date.now()
54+
55+
// altimate_change start - smooth streaming: pre-merge delta events
56+
// When enabled, merge consecutive delta events for the same part+field
57+
// to reduce store updates from N-per-part to 1-per-part per flush cycle.
58+
if (Flag.ALTIMATE_SMOOTH_STREAMING) {
59+
const merged: Event[] = []
60+
const deltaMap = new Map<string, number>()
61+
for (const event of events) {
62+
if (event.type === "message.part.delta") {
63+
const props = event.properties as { messageID: string; partID: string; field: string; delta: string }
64+
const key = `${props.messageID}:${props.partID}:${props.field}`
65+
const existing = deltaMap.get(key)
66+
if (existing !== undefined) {
67+
const prev = merged[existing] as typeof event
68+
merged[existing] = {
69+
...prev,
70+
properties: {
71+
...prev.properties,
72+
delta: (prev.properties as typeof props).delta + props.delta,
73+
},
74+
} as Event
75+
continue
76+
}
77+
deltaMap.set(key, merged.length)
78+
} else {
79+
deltaMap.clear()
80+
}
81+
merged.push(event)
82+
}
83+
batch(() => {
84+
for (const event of merged) {
85+
emitter.emit(event.type, event)
86+
}
87+
})
88+
return
89+
}
90+
// altimate_change end
91+
5192
// Batch all event emissions so all store updates result in a single render
5293
batch(() => {
5394
for (const event of events) {

‎packages/opencode/src/cli/cmd/tui/context/sync.tsx‎

Lines changed: 107 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -116,6 +116,43 @@ export const { use: useSync, provider: SyncProvider } = createSimpleContext({
116116
setStore("workspaceList", reconcile(result.data))
117117
}
118118

119+
// altimate_change start - line streaming: buffer deltas, flush only on \n or message completion
120+
const lineBuffer = new Map<string, string>()
121+
122+
function flushLineBuffer(messageID: string, partID: string, field: string, forceAll: boolean) {
123+
const key = `${messageID}:${partID}:${field}`
124+
const buffer = lineBuffer.get(key)
125+
if (!buffer) return
126+
let textToFlush: string
127+
if (forceAll) {
128+
textToFlush = buffer
129+
lineBuffer.delete(key)
130+
} else {
131+
const lastNewline = buffer.lastIndexOf("\n")
132+
if (lastNewline === -1) return
133+
textToFlush = buffer.slice(0, lastNewline + 1)
134+
const remainder = buffer.slice(lastNewline + 1)
135+
if (remainder) lineBuffer.set(key, remainder)
136+
else lineBuffer.delete(key)
137+
}
138+
if (!textToFlush) return
139+
const parts = store.part[messageID]
140+
if (!parts) return
141+
const result = Binary.search(parts, partID, (p) => p.id)
142+
if (!result.found) return
143+
const existing = parts[result.index][field as keyof (typeof parts)[number]] as string | undefined
144+
setStore("part", messageID, result.index, field as any, ((existing ?? "") + textToFlush) as any)
145+
}
146+
147+
function flushAllBuffersForMessage(messageID: string) {
148+
for (const [key] of lineBuffer) {
149+
if (!key.startsWith(messageID + ":")) continue
150+
const [, partID, field] = key.split(":")
151+
flushLineBuffer(messageID, partID, field, true)
152+
}
153+
}
154+
// altimate_change end
155+
119156
sdk.event.listen((e) => {
120157
const event = e.details
121158
switch (event.type) {
@@ -254,6 +291,15 @@ export const { use: useSync, provider: SyncProvider } = createSimpleContext({
254291
}
255292

256293
case "message.updated": {
294+
// altimate_change start - line streaming: flush remaining buffer when message completes
295+
if (
296+
Flag.ALTIMATE_LINE_STREAMING &&
297+
"completed" in event.properties.info.time &&
298+
event.properties.info.time.completed
299+
) {
300+
flushAllBuffersForMessage(event.properties.info.id)
301+
}
302+
// altimate_change end
257303
const messages = store.message[event.properties.info.sessionID]
258304
if (!messages) {
259305
setStore("message", event.properties.info.sessionID, [event.properties.info])
@@ -293,6 +339,11 @@ export const { use: useSync, provider: SyncProvider } = createSimpleContext({
293339
break
294340
}
295341
case "message.removed": {
342+
// altimate_change start - line streaming: clean up buffers for removed/aborted messages
343+
if (Flag.ALTIMATE_LINE_STREAMING) {
344+
flushAllBuffersForMessage(event.properties.messageID)
345+
}
346+
// altimate_change end
296347
const messages = store.message[event.properties.sessionID]
297348
const result = Binary.search(messages, event.properties.messageID, (m) => m.id)
298349
if (result.found) {
@@ -307,6 +358,17 @@ export const { use: useSync, provider: SyncProvider } = createSimpleContext({
307358
break
308359
}
309360
case "message.part.updated": {
361+
// altimate_change start - line streaming: discard buffered text when part is
362+
// authoritatively set by the server (via reconcile). Without this, the buffer
363+
// would append stale text on top of the server's complete content, duplicating
364+
// the trailing partial line.
365+
if (Flag.ALTIMATE_LINE_STREAMING) {
366+
const { messageID, id: partID } = event.properties.part
367+
for (const key of lineBuffer.keys()) {
368+
if (key.startsWith(`${messageID}:${partID}:`)) lineBuffer.delete(key)
369+
}
370+
}
371+
// altimate_change end
310372
const parts = store.part[event.properties.part.messageID]
311373
if (!parts) {
312374
setStore("part", event.properties.part.messageID, [event.properties.part])
@@ -332,20 +394,55 @@ export const { use: useSync, provider: SyncProvider } = createSimpleContext({
332394
if (!parts) break
333395
const result = Binary.search(parts, event.properties.partID, (p) => p.id)
334396
if (!result.found) break
335-
setStore(
336-
"part",
337-
event.properties.messageID,
338-
produce((draft) => {
339-
const part = draft[result.index]
340-
const field = event.properties.field as keyof typeof part
341-
const existing = part[field] as string | undefined
342-
;(part[field] as string) = (existing ?? "") + event.properties.delta
343-
}),
344-
)
397+
// altimate_change start - line streaming: buffer deltas, flush only on \n
398+
// Note: when line streaming is enabled (including via calm mode), this branch
399+
// handles all delta processing and breaks — the smooth streaming branch below
400+
// is not reached. This is intentional: flushLineBuffer already does direct
401+
// store path updates, so the produce() bypass is not needed.
402+
if (Flag.ALTIMATE_LINE_STREAMING) {
403+
const { messageID, partID, field, delta } = event.properties
404+
const key = `${messageID}:${partID}:${field}`
405+
lineBuffer.set(key, (lineBuffer.get(key) ?? "") + delta)
406+
flushLineBuffer(messageID, partID, field, false)
407+
break
408+
}
409+
// altimate_change end
410+
// altimate_change start - smooth streaming: direct path update avoids produce() proxy overhead
411+
if (Flag.ALTIMATE_SMOOTH_STREAMING) {
412+
const field = event.properties.field as keyof (typeof parts)[number]
413+
const existing = parts[result.index][field] as string | undefined
414+
setStore(
415+
"part",
416+
event.properties.messageID,
417+
result.index,
418+
field as any,
419+
((existing ?? "") + event.properties.delta) as any,
420+
)
421+
} else {
422+
setStore(
423+
"part",
424+
event.properties.messageID,
425+
produce((draft) => {
426+
const part = draft[result.index]
427+
const field = event.properties.field as keyof typeof part
428+
const existing = part[field] as string | undefined
429+
;(part[field] as string) = (existing ?? "") + event.properties.delta
430+
}),
431+
)
432+
}
433+
// altimate_change end
345434
break
346435
}
347436

348437
case "message.part.removed": {
438+
// altimate_change start - line streaming: discard buffers for removed parts
439+
if (Flag.ALTIMATE_LINE_STREAMING) {
440+
const { messageID, partID } = event.properties
441+
for (const key of lineBuffer.keys()) {
442+
if (key.startsWith(`${messageID}:${partID}:`)) lineBuffer.delete(key)
443+
}
444+
}
445+
// altimate_change end
349446
const parts = store.part[event.properties.messageID]
350447
const result = Binary.search(parts, event.properties.partID, (p) => p.id)
351448
if (result.found)

0 commit comments

Comments
 (0)