Skip to content

Commit d58a256

Browse files
authored
fix(agent): make spoken narration strictly opt-in
Spoken narration defaulted on for all cloud runs via resolveSpokenNarration falling back to isCloudRun(meta). That loaded the `speak` tool and its prompt instructions — and prompted the agent to call it — on headless runs (Slack threads, Signals scouts) where nothing can play audio, costing tokens with no listener and regardless of the feature flag, user setting, or ElevenLabs key. Make narration strictly opt-in: resolveSpokenNarration now returns true only when a caller explicitly sets spokenNarration. The desktop is the only client that can play audio and knows the flag/setting/key, so it computes that boolean and passes it; everything else stays silent and never loads the tool. Also require an ElevenLabs key alongside the setting on the desktop start/reconnect paths, and refresh the now-stale comments. Generated-By: PostHog Code Task-Id: 7a65b8a4-b69d-472f-9753-67d59715e332
1 parent ed6cdfb commit d58a256

5 files changed

Lines changed: 44 additions & 40 deletions

File tree

packages/agent/src/adapters/local-tools/tools/speak.ts

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -48,10 +48,11 @@ export const SPEAK_TOOL_DESCRIPTION =
4848
* speakers, so it just acknowledges. The desktop renderer observes the
4949
* surfaced `tool_call` (carrying `text`/`needsUser` in its rawInput) and routes
5050
* it to the speech queue, exactly like completion/permission notifications are
51-
* pure side effects off the event stream. Gated on `spokenNarration`: local
52-
* sessions pass the user's setting at session start, while cloud sessions
53-
* resolve it to true at the adapter (the sandbox can't know which clients are
54-
* listening, so cloud emits always and consumers gate playback).
51+
* pure side effects off the event stream. Gated on `spokenNarration`, which is
52+
* strictly opt-in (see `resolveSpokenNarration`): the desktop passes it true
53+
* only when the feature flag, the user's setting, and an ElevenLabs key are all
54+
* in place. Headless cloud runs (Slack threads, Signals scouts) never enable
55+
* it, so the tool and its instructions never load and never cost tokens there.
5556
*/
5657
export const speakTool = defineLocalTool({
5758
name: SPEAK_TOOL_NAME,

packages/agent/src/adapters/session-meta.test.ts

Lines changed: 15 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -6,25 +6,18 @@ describe("resolveSpokenNarration", () => {
66
vi.unstubAllEnvs();
77
});
88

9+
// Narration is strictly opt-in: only an explicit `spokenNarration: true`
10+
// enables it. Cloud/sandbox runs no longer default on, so headless runs
11+
// (Slack threads, Signals scouts) never load the tool or its instructions.
912
it.each([
1013
{
11-
name: "explicit true on local",
12-
meta: { environment: "local" as const, spokenNarration: true },
14+
name: "explicit true",
15+
meta: { spokenNarration: true },
1316
expected: true,
1417
},
1518
{
16-
name: "explicit false on cloud",
17-
meta: { environment: "cloud" as const, spokenNarration: false },
18-
expected: false,
19-
},
20-
{
21-
name: "cloud default",
22-
meta: { environment: "cloud" as const },
23-
expected: true,
24-
},
25-
{
26-
name: "local default",
27-
meta: { environment: "local" as const },
19+
name: "explicit false",
20+
meta: { spokenNarration: false },
2821
expected: false,
2922
},
3023
{ name: "no meta", meta: undefined, expected: false },
@@ -35,20 +28,20 @@ describe("resolveSpokenNarration", () => {
3528
});
3629

3730
it.each([
38-
{ name: "no meta", meta: undefined, expected: true },
39-
{ name: "empty meta", meta: {}, expected: true },
31+
{ name: "no meta", meta: undefined, expected: false },
32+
{ name: "empty meta", meta: {}, expected: false },
4033
{
41-
name: "explicit false",
42-
meta: { spokenNarration: false },
43-
expected: false,
34+
name: "explicit true",
35+
meta: { spokenNarration: true },
36+
expected: true,
4437
},
4538
{
46-
name: "explicit local environment",
47-
meta: { environment: "local" as const },
39+
name: "explicit false",
40+
meta: { spokenNarration: false },
4841
expected: false,
4942
},
5043
])(
51-
"resolves $name to $expected in a sandbox without an environment tag",
44+
"resolves $name to $expected in a sandbox (no default-on)",
5245
({ meta, expected }) => {
5346
vi.stubEnv("IS_SANDBOX", "1");
5447
expect(resolveSpokenNarration(meta)).toBe(expected);

packages/agent/src/adapters/session-meta.ts

Lines changed: 9 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,3 @@
1-
import { isCloudRun } from "../utils/common";
2-
31
/** Minimal shape needed to resolve the effective task id from session meta. */
42
interface TaskIdSource {
53
taskId?: string;
@@ -19,18 +17,21 @@ export function resolveTaskId(
1917

2018
/** Minimal shape needed to resolve spoken narration from session meta. */
2119
interface SpokenNarrationSource {
22-
environment?: "local" | "cloud";
2320
spokenNarration?: boolean;
2421
}
2522

2623
/**
27-
* An explicit setting wins; otherwise cloud runs (including sandbox runs
28-
* detected via `IS_SANDBOX`) default to on because the sandbox can't know
29-
* which clients are listening, so consumers gate playback. Local runs stay
30-
* silent. Shared by the Claude and Codex adapters.
24+
* Spoken narration is strictly opt-in: it is on only when a caller explicitly
25+
* sets `spokenNarration` true at session start. The desktop is the only client
26+
* that can play audio and the only place that knows the feature flag, the
27+
* user's setting, and whether an ElevenLabs key is configured — so it computes
28+
* that boolean and passes it. Everything else (headless cloud runs like Slack
29+
* threads and Signals scouts, sandboxes, local runs without the setting) stays
30+
* silent, so the `speak` tool and its instructions never load and never cost
31+
* tokens where nothing is listening. Shared by the Claude and Codex adapters.
3132
*/
3233
export function resolveSpokenNarration(
3334
meta: SpokenNarrationSource | undefined,
3435
): boolean {
35-
return meta?.spokenNarration ?? isCloudRun(meta);
36+
return meta?.spokenNarration === true;
3637
}

packages/core/src/sessions/sessionService.ts

Lines changed: 12 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -336,6 +336,7 @@ export interface SessionServiceDeps {
336336
rtkEnabledLocal?: boolean;
337337
rtkEnabledCloud?: boolean;
338338
spokenNotifications?: boolean;
339+
elevenLabsKeyConfigured?: boolean;
339340
};
340341
usageLimit: { show: (...args: any[]) => any };
341342
readonly addDirectoryDialog: { open: boolean };
@@ -1087,14 +1088,19 @@ export class SessionService {
10871088
this.d.log.warn("Failed to verify workspace", { taskId, err });
10881089
});
10891090

1090-
const { customInstructions, rtkEnabledLocal, spokenNotifications } =
1091-
this.d.settings;
1091+
const {
1092+
customInstructions,
1093+
rtkEnabledLocal,
1094+
spokenNotifications,
1095+
elevenLabsKeyConfigured,
1096+
} = this.d.settings;
10921097
const result = await this.d.trpc.agent.reconnect.mutate({
10931098
taskId,
10941099
taskRunId,
10951100
repoPath,
10961101
rtkEnabled: rtkEnabledLocal,
1097-
spokenNarration: spokenNotifications === true,
1102+
spokenNarration:
1103+
spokenNotifications === true && elevenLabsKeyConfigured === true,
10981104
apiHost: auth.apiHost,
10991105
projectId: auth.projectId,
11001106
logUrl,
@@ -1418,6 +1424,7 @@ export class SessionService {
14181424
customInstructions: startCustomInstructions,
14191425
rtkEnabledLocal,
14201426
spokenNotifications,
1427+
elevenLabsKeyConfigured,
14211428
} = this.d.settings;
14221429
const preferredModel = model ?? this.d.DEFAULT_GATEWAY_MODEL;
14231430
const result = await this.d.trpc.agent.start.mutate({
@@ -1430,7 +1437,8 @@ export class SessionService {
14301437
adapter,
14311438
customInstructions: startCustomInstructions || undefined,
14321439
rtkEnabled: rtkEnabledLocal,
1433-
spokenNarration: spokenNotifications === true,
1440+
spokenNarration:
1441+
spokenNotifications === true && elevenLabsKeyConfigured === true,
14341442
effort: effortLevelSchema.safeParse(reasoningLevel).success
14351443
? (reasoningLevel as EffortLevel)
14361444
: undefined,

packages/workspace-server/src/services/agent/schemas.ts

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -93,8 +93,9 @@ export const startSessionInput = z.object({
9393
rtkEnabled: z.boolean().optional(),
9494
/**
9595
* The user's spoken-narration setting at session start. Gates the agent's
96-
* speak tool and its prompt instructions; when absent the adapter defaults
97-
* by environment (cloud on, local off).
96+
* speak tool and its prompt instructions. Strictly opt-in: only the desktop
97+
* sets it true (feature flag + setting + ElevenLabs key); when absent the
98+
* adapter leaves narration off, so headless runs never load the tool.
9899
*/
99100
spokenNarration: z.boolean().optional(),
100101
});

0 commit comments

Comments
 (0)