Skip to content

Commit 78d62bc

Browse files
committed
Real inference for agents via the pi agent harness
bun run cli infer [-m model] [--tools] [--json] "prompt" — ask a real model a question (or let it use coding tools in a scratch dir) through an actual agent harness rather than a bespoke loop. pi (@earendil-works/pi-coding-agent) runs headless (--mode json -p, clean typed JSONL events, --no-tools by default) with a throwaway config dir and no session, so nothing touches anyone's pi state. The model is reached through the machine's OpenCode Zen subscription: the gateway is OpenAI-compatible, so a generated pi provider extension points at it with the key from opencode's auth.json via env — no new credentials. runAgent() in e2e/src/clients/agent.ts is the programmatic primitive (answerText, thinkingText, toolCalls, usage, full event stream) for future eval harnesses. The OpenCode client (src/clients/opencode.ts) keeps its role as the MCP-native real-client actor — pi is the inference workhorse, OpenCode is the client-behavior probe. Supersedes the inference half of PR #972's approach (the OpenCode binary as a one-shot inference vehicle): same concept, but through a harness whose headless mode is built for it — no auth.json copying, no open(1) shim, no PWD-leak workarounds, ~2s per call. Verified live: plain answers on deepseek-v4-flash + glm-5.1, a tool-using run (write + read in the scratch dir), the missing-binary and missing-credential error paths, repo lint/typecheck/format.
1 parent 2d86581 commit 78d62bc

3 files changed

Lines changed: 333 additions & 0 deletions

File tree

RUNNING.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,7 @@ bun run cli identity selfhost # fresh identity (headers / cookies / creds)
7676
bun run cli api selfhost tools.list
7777
bun run cli mcp selfhost call execute '{"code":"return 1+1;"}'
7878
bun run cli ledger cloud workos # what hit the emulator
79+
bun run cli infer "..." # real model inference (see below)
7980
bun run cli down selfhost # tear down (also removes tailscale serves)
8081
```
8182

@@ -94,6 +95,23 @@ sides (its `baseUrl` and the app's `WORKOS_API_URL` — the browser-facing
9495
authorize URL derives from the latter), and Vite must allow the public
9596
hostname (`__VITE_ADDITIONAL_SERVER_ALLOWED_HOSTS`).
9697

98+
## Real inference for development
99+
100+
`bun run cli infer [-m model] [--tools] [--json] "prompt"` runs a real model
101+
through the **pi agent harness** (`@earendil-works/pi-coding-agent`,
102+
headless `--mode json`), reached via the machine's OpenCode Zen subscription
103+
(key auto-discovered from `~/.local/share/opencode/auth.json`; run
104+
`opencode auth login` once if missing). Every run is hermetic: throwaway pi
105+
config dir, no session persisted, and `--tools` confines the model's
106+
read/bash/edit/write to a fresh scratch dir. Default model is cheap+fast
107+
(deepseek-v4-flash); `-m glm-5.1` etc. selects others on the subscription.
108+
`--json` prints the distilled result (answer, thinking, tool calls, usage,
109+
full event stream) for programmatic use; the same primitive is importable as
110+
`runAgent()` from `e2e/src/clients/agent.ts` for eval-style harnesses.
111+
OpenCode (`e2e/src/clients/opencode.ts`) remains the MCP-native real-client
112+
actor for scenarios that test OAuth/tool-discovery behavior; pi is the
113+
inference workhorse.
114+
97115
## Environment gotchas (learned the hard way)
98116

99117
- The shell is fish, and the working directory resets between Bash calls.

e2e/scripts/cli.ts

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -496,6 +496,38 @@ const logs = (targetName: string) => {
496496
console.log(readFileSync(state.logFile, "utf8"));
497497
};
498498

499+
// Real inference through the pi agent harness (Zen subscription). Plain
500+
// question→answer by default; --tools gives the model pi's coding tools in a
501+
// scratch dir; --json dumps the full distilled result for grading.
502+
const infer = async (args: ReadonlyArray<string>, flags: ReadonlySet<string>) => {
503+
const { runAgent, DEFAULT_MODEL, hasInferenceCredential } = await import("../src/clients/agent");
504+
if (!hasInferenceCredential()) {
505+
throw new Error(
506+
"infer: no inference credential — run `opencode auth login` once on this machine",
507+
);
508+
}
509+
let model = DEFAULT_MODEL;
510+
const prompt: string[] = [];
511+
for (let index = 0; index < args.length; index++) {
512+
if (args[index] === "-m") model = args[++index] ?? model;
513+
else prompt.push(args[index]!);
514+
}
515+
if (prompt.length === 0) throw new Error('usage: infer [-m model] [--tools] [--json] "prompt"');
516+
const result = await runAgent({
517+
prompt: prompt.join(" "),
518+
model,
519+
tools: flags.has("--tools"),
520+
});
521+
if (flags.has("--json")) {
522+
console.log(JSON.stringify(result, null, 2));
523+
} else {
524+
if (result.exitCode !== 0 && !result.answerText) {
525+
throw new Error(`infer: pi exited ${result.exitCode}\n${result.stderr.slice(0, 800)}`);
526+
}
527+
console.log(result.answerText);
528+
}
529+
};
530+
499531
// --- main ------------------------------------------------------------------
500532

501533
const HELP = `e2e dev CLI — the scenario primitives, interactive (see e2e/AGENTS.md)
@@ -508,6 +540,9 @@ const HELP = `e2e dev CLI — the scenario primitives, interactive (see e2e/AGEN
508540
api <target> <group.endpoint> [json] typed API call as a fresh identity
509541
mcp <target> tools | call <tool> [json] MCP session call
510542
ledger <target> [workos|autumn] the emulator's request ledger (cloud)
543+
infer [-m model] [--tools] [--json] "prompt" real inference via the pi
544+
agent harness (Zen subscription; --tools gives
545+
the model coding tools in a scratch dir)
511546
logs <target> dump the instance's dev-server log
512547
down <target> tear down (kills servers, removes tailscale serves)
513548
@@ -534,6 +569,8 @@ const main = async () => {
534569
return mcpCall(args[0] ?? "", args[1], args.slice(2));
535570
case "ledger":
536571
return ledger(args[0] ?? "", args[1]);
572+
case "infer":
573+
return infer(args, flags);
537574
case "logs":
538575
return logs(args[0] ?? "");
539576
case "down":

e2e/src/clients/agent.ts

Lines changed: 278 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,278 @@
1+
// Dev-time inference through a REAL agent harness: drive pi
2+
// (@earendil-works/pi-coding-agent) headless, one shot, hermetically. The
3+
// model is reached through the machine's OpenCode Zen subscription (the
4+
// gateway is OpenAI-compatible; pi gets it via a generated provider
5+
// extension), so an agent working in this repo can ask a real model a
6+
// question — or let it use tools in a scratch dir — without any new
7+
// credentials and without touching anyone's pi/OpenCode state.
8+
//
9+
// Consumers:
10+
// - `bun run cli infer "..."` — the interactive command (scripts/cli.ts).
11+
// - future eval tiers — fan out runAgent() trials and grade the results.
12+
//
13+
// This is deliberately NOT the OpenCode binary: pi's headless mode is built
14+
// for this (clean JSONL events, --no-tools, hermetic config dir via env),
15+
// while OpenCode stays what it already is in this suite — the MCP-native
16+
// real-client actor (src/clients/opencode.ts).
17+
import { spawn } from "node:child_process";
18+
import { mkdirSync, mkdtempSync, readFileSync, writeFileSync } from "node:fs";
19+
import { homedir, tmpdir } from "node:os";
20+
import { join } from "node:path";
21+
22+
// ---------------------------------------------------------------------------
23+
// The subscription credential (OpenCode Zen — an OpenAI-compatible gateway)
24+
// ---------------------------------------------------------------------------
25+
26+
const ZEN_BASE_URL = "https://opencode.ai/zen/v1";
27+
28+
const zenAuthFile = (): string => join(homedir(), ".local", "share", "opencode", "auth.json");
29+
30+
export const zenApiKey = (): string | undefined => {
31+
// oxlint-disable-next-line executor/no-try-catch-or-throw, executor/no-json-parse -- boundary: optional host credential file
32+
try {
33+
const auth = JSON.parse(readFileSync(zenAuthFile(), "utf8")) as {
34+
opencode?: { key?: string };
35+
};
36+
return auth.opencode?.key;
37+
} catch {
38+
return undefined;
39+
}
40+
};
41+
42+
export const hasInferenceCredential = (): boolean => zenApiKey() !== undefined;
43+
44+
/** Models on the Zen subscription (see the gateway's own docs for quotas).
45+
* Any other id is passed through untouched — the registry just supplies
46+
* display metadata for the known ones. */
47+
export const ZEN_MODELS = [
48+
"deepseek-v4-flash", // cheap + fast: the default
49+
"glm-5.1",
50+
"kimi-k2.5",
51+
"minimax-m2.5",
52+
] as const;
53+
54+
export const DEFAULT_MODEL: string = ZEN_MODELS[0];
55+
56+
// ---------------------------------------------------------------------------
57+
// Result shape — distilled from pi's JSONL event stream
58+
// ---------------------------------------------------------------------------
59+
60+
export interface AgentToolCall {
61+
readonly name: string;
62+
readonly args: unknown;
63+
readonly result?: unknown;
64+
readonly isError?: boolean;
65+
}
66+
67+
export interface AgentRunResult {
68+
/** All assistant text parts of the final answer, joined. */
69+
readonly answerText: string;
70+
/** The model's reasoning text, when the model emits it. */
71+
readonly thinkingText: string;
72+
readonly toolCalls: readonly AgentToolCall[];
73+
/** Every JSONL event pi emitted, parsed, in order — for grading/artifacts. */
74+
readonly events: readonly Record<string, unknown>[];
75+
readonly usage?: { readonly input: number; readonly output: number };
76+
readonly exitCode: number | null;
77+
readonly durationMs: number;
78+
/** Raw stderr — populated on failures (provider errors land here). */
79+
readonly stderr: string;
80+
}
81+
82+
export interface AgentRunOptions {
83+
readonly prompt: string;
84+
/** Zen model id (see ZEN_MODELS). Default: cheap + fast. */
85+
readonly model?: string;
86+
/** Give the agent pi's coding tools (read/bash/edit/write) in `cwd`.
87+
* Default false — plain question → answer, no side effects possible. */
88+
readonly tools?: boolean;
89+
/** Working directory for tool use. Default: a fresh empty temp dir, so a
90+
* tool-using model explores nothing it wasn't given. */
91+
readonly cwd?: string;
92+
/** Replace pi's coding system prompt. */
93+
readonly systemPrompt?: string;
94+
readonly timeoutMs?: number;
95+
}
96+
97+
const DEFAULT_TIMEOUT_MS = 120_000;
98+
99+
// ---------------------------------------------------------------------------
100+
// Run
101+
// ---------------------------------------------------------------------------
102+
103+
// The provider extension pi loads at startup. The key travels via env
104+
// (referenced as $OPENCODE_ZEN_API_KEY) — never written to disk.
105+
const providerExtension = (modelIds: readonly string[]): string => `export default function (pi) {
106+
pi.registerProvider("zen", {
107+
name: "OpenCode Zen",
108+
baseUrl: ${JSON.stringify(ZEN_BASE_URL)},
109+
apiKey: "$OPENCODE_ZEN_API_KEY",
110+
api: "openai-completions",
111+
models: ${JSON.stringify(modelIds)}.map((id) => ({
112+
id,
113+
name: id,
114+
reasoning: false,
115+
input: ["text"],
116+
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
117+
contextWindow: 128000,
118+
maxTokens: 8192,
119+
})),
120+
});
121+
}
122+
`;
123+
124+
export const runAgent = async (options: AgentRunOptions): Promise<AgentRunResult> => {
125+
const key = zenApiKey();
126+
if (!key) {
127+
throw new Error(
128+
`inference: no Zen credential at ${zenAuthFile()} — run \`opencode auth login\` once on this machine`,
129+
);
130+
}
131+
const model = options.model ?? DEFAULT_MODEL;
132+
133+
// Hermetic everything: pi's config dir, the session-less run, and (unless
134+
// the caller passes cwd) an empty project dir, so a tool-using model acts
135+
// only on what it was given instead of wandering a repo.
136+
const scratch = mkdtempSync(join(tmpdir(), "executor-agent-"));
137+
const configDir = join(scratch, "pi");
138+
mkdirSync(configDir, { recursive: true });
139+
const extensionPath = join(scratch, "zen-provider.mjs");
140+
// Unknown model ids still work — register them alongside the known set.
141+
const modelIds = (ZEN_MODELS as readonly string[]).includes(model)
142+
? ZEN_MODELS
143+
: [...ZEN_MODELS, model];
144+
writeFileSync(extensionPath, providerExtension(modelIds));
145+
const cwd = options.cwd ?? join(scratch, "project");
146+
mkdirSync(cwd, { recursive: true });
147+
148+
const args = [
149+
"--extension",
150+
extensionPath,
151+
"--provider",
152+
"zen",
153+
"--model",
154+
model,
155+
"--no-session",
156+
"--mode",
157+
"json",
158+
...(options.tools ? [] : ["--no-tools"]),
159+
...(options.systemPrompt ? ["--system-prompt", options.systemPrompt] : []),
160+
"-p",
161+
options.prompt,
162+
];
163+
164+
const startedAt = Date.now();
165+
const child = spawn("pi", args, {
166+
cwd,
167+
env: {
168+
...process.env,
169+
PWD: cwd,
170+
PI_CODING_AGENT_DIR: configDir,
171+
PI_OFFLINE: "1",
172+
PI_SKIP_VERSION_CHECK: "1",
173+
OPENCODE_ZEN_API_KEY: key,
174+
},
175+
stdio: ["ignore", "pipe", "pipe"],
176+
});
177+
178+
let stdout = "";
179+
let stderr = "";
180+
child.stdout.on("data", (chunk: Buffer) => (stdout += chunk.toString("utf8")));
181+
child.stderr.on("data", (chunk: Buffer) => (stderr += chunk.toString("utf8")));
182+
183+
let spawnError: NodeJS.ErrnoException | undefined;
184+
const exitCode = await new Promise<number | null>((resolve) => {
185+
const killer = setTimeout(() => child.kill("SIGKILL"), options.timeoutMs ?? DEFAULT_TIMEOUT_MS);
186+
child.once("error", (error: NodeJS.ErrnoException) => {
187+
clearTimeout(killer);
188+
spawnError = error;
189+
resolve(null);
190+
});
191+
child.once("exit", (code) => {
192+
clearTimeout(killer);
193+
resolve(code);
194+
});
195+
});
196+
if (spawnError) {
197+
throw spawnError.code === "ENOENT"
198+
? new Error(
199+
"inference: the `pi` binary is not installed — `npm install -g @earendil-works/pi-coding-agent`",
200+
)
201+
: spawnError;
202+
}
203+
204+
return { ...distill(stdout), exitCode, durationMs: Date.now() - startedAt, stderr };
205+
};
206+
207+
// ---------------------------------------------------------------------------
208+
// Event distillation
209+
// ---------------------------------------------------------------------------
210+
211+
interface MessagePart {
212+
readonly type?: string;
213+
readonly text?: string;
214+
readonly thinking?: string;
215+
}
216+
217+
interface AgentEndEvent {
218+
readonly type?: string;
219+
readonly messages?: ReadonlyArray<{
220+
readonly role?: string;
221+
readonly content?: readonly MessagePart[];
222+
readonly usage?: { readonly input?: number; readonly output?: number };
223+
}>;
224+
readonly toolCallId?: string;
225+
readonly toolName?: string;
226+
readonly args?: unknown;
227+
readonly result?: unknown;
228+
readonly isError?: boolean;
229+
}
230+
231+
const distill = (
232+
jsonl: string,
233+
): Pick<AgentRunResult, "answerText" | "thinkingText" | "toolCalls" | "events" | "usage"> => {
234+
const events: Record<string, unknown>[] = [];
235+
for (const line of jsonl.split("\n")) {
236+
if (!line.trim()) continue;
237+
// oxlint-disable-next-line executor/no-try-catch-or-throw, executor/no-json-parse -- boundary: tolerant parse of pi's JSONL stream
238+
try {
239+
events.push(JSON.parse(line) as Record<string, unknown>);
240+
} catch {
241+
// Non-JSON line (warning/banner) — keep going.
242+
}
243+
}
244+
245+
const toolCalls = new Map<string, AgentToolCall>();
246+
let answerText = "";
247+
let thinkingText = "";
248+
let usage: AgentRunResult["usage"];
249+
250+
for (const event of events as readonly AgentEndEvent[]) {
251+
if (event.type === "tool_execution_start" && event.toolCallId) {
252+
toolCalls.set(event.toolCallId, { name: event.toolName ?? "?", args: event.args });
253+
}
254+
if (event.type === "tool_execution_end" && event.toolCallId) {
255+
const started = toolCalls.get(event.toolCallId);
256+
toolCalls.set(event.toolCallId, {
257+
name: event.toolName ?? started?.name ?? "?",
258+
args: started?.args,
259+
result: event.result,
260+
isError: event.isError,
261+
});
262+
}
263+
if (event.type === "agent_end") {
264+
for (const message of event.messages ?? []) {
265+
if (message.role !== "assistant") continue;
266+
for (const part of message.content ?? []) {
267+
if (part.type === "text" && part.text) answerText += part.text;
268+
if (part.type === "thinking" && part.thinking) thinkingText += part.thinking;
269+
}
270+
if (message.usage) {
271+
usage = { input: message.usage.input ?? 0, output: message.usage.output ?? 0 };
272+
}
273+
}
274+
}
275+
}
276+
277+
return { answerText, thinkingText, toolCalls: [...toolCalls.values()], events, usage };
278+
};

0 commit comments

Comments
 (0)