Skip to content

Commit 39920b0

Browse files
committed
docs: add telemetry instrumentation guide
1 parent 52a24ce commit 39920b0

3 files changed

Lines changed: 166 additions & 0 deletions

File tree

.github/harness/prompts/review.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,3 +25,5 @@ comment on the PR saying it looks good to merge (or that all issues have already
2525
- **Excessive mocking** — Avoid excessive mocking; it couples tests to implementation details, provides weaker
2626
guarantees, and often points to mismanaged dependencies. Prefer real dependencies (e.g. temp directories over fs
2727
mocks) and only mock at true I/O boundaries (e.g., network calls, AWS SDK clients, HTTP requests).
28+
- **Missing telemetry** — New features should include telemetry instrumentation. See `src/cli/telemetry/README.md` for
29+
guidance on what and how to instrument.

AGENTS.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -143,6 +143,10 @@ See `docs/TESTING.md` for details.
143143
- Always look for existing types before creating a new type inline.
144144
- Re-usable constants must be defined in a constants file in the closest sensible subdirectory.
145145

146+
## Telemetry
147+
148+
New features must include telemetry instrumentation. See `src/cli/telemetry/README.md` for how to add metrics.
149+
146150
## Multi-Partition Support (GovCloud, China)
147151

148152
The CLI supports multiple AWS partitions (commercial, GovCloud, China) through a central utility at

src/cli/telemetry/README.md

Lines changed: 160 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,160 @@
1+
# Adding New Telemetry Metrics
2+
3+
## Overview
4+
5+
Every CLI command emits a `command_run` metric with a command key, exit reason, and command-specific attributes. This
6+
guide shows how to add telemetry to a new command.
7+
8+
## Step 1: Register the command in `schemas/command-run.ts`
9+
10+
Add an entry to `COMMAND_SCHEMAS`. If the command has attributes, define a schema; otherwise use `NoAttrs`.
11+
12+
```ts
13+
// No attributes (e.g. remove commands):
14+
'remove.widget': NoAttrs,
15+
16+
// With attributes:
17+
const AddWidgetAttrs = safeSchema({
18+
widget_type: WidgetType, // must be z.enum(), z.boolean(), or z.number()
19+
count: Count,
20+
});
21+
22+
// Then register:
23+
'add.widget': AddWidgetAttrs,
24+
```
25+
26+
**Rules:**
27+
28+
- Only `z.enum()`, `z.boolean()`, `z.number()`, and `z.literal()` are allowed as field types (`safeSchema` enforces this
29+
at compile time).
30+
- No `z.string()` — prevents PII leakage.
31+
32+
## Step 2: Add any new enums to `schemas/common-shapes.ts`
33+
34+
If your attributes need a new enum value:
35+
36+
```ts
37+
export const WidgetType = z.enum(['basic', 'advanced']);
38+
```
39+
40+
Export it so command handlers can import and use `standardize()` to normalize CLI input:
41+
42+
```ts
43+
import { WidgetType, standardize } from '../telemetry/schemas/common-shapes.js';
44+
45+
const type = standardize(WidgetType, userInput); // lowercases + validates
46+
```
47+
48+
## Step 3: Instrument the command handler
49+
50+
Pick the right helper from `cli-command-run.ts`:
51+
52+
| Helper | Use case |
53+
| --------------------------------------------- | ------------------------------------------------- |
54+
| `runCliCommand(command, json, fn)` | CLI handlers that print output and `process.exit` |
55+
| `withCommandRunTelemetry(command, attrs, fn)` | CLI/TUI handlers that return a `Result` |
56+
| `withAddTelemetry(command, attrs, fn)` | TUI hooks wrapping `.add()` calls |
57+
58+
### CLI path — `runCliCommand`
59+
60+
Wrap the command body. Throw on failure; return attrs on success.
61+
62+
```ts
63+
import { runCliCommand } from '../telemetry/cli-command-run.js';
64+
65+
await runCliCommand('remove.widget', !!options.json, async () => {
66+
const result = await widgetPrimitive.remove(options.name);
67+
if (!result.success) throw new Error(result.error);
68+
console.log(JSON.stringify(result));
69+
return {}; // attrs (NoAttrs → empty object)
70+
});
71+
```
72+
73+
### CLI path with attributes — `runCliCommand`
74+
75+
```ts
76+
import { runCliCommand } from '../telemetry/cli-command-run.js';
77+
import { WidgetType, standardize } from '../telemetry/schemas/common-shapes.js';
78+
79+
await runCliCommand('add.widget', !!options.json, async () => {
80+
const result = await widgetPrimitive.add(options);
81+
if (!result.success) throw new Error(result.error);
82+
console.log(JSON.stringify(result));
83+
return {
84+
widget_type: standardize(WidgetType, options.type),
85+
count: options.items.length,
86+
};
87+
});
88+
```
89+
90+
### TUI path — `withCommandRunTelemetry`
91+
92+
Returns the `Result` to the caller instead of exiting.
93+
94+
```ts
95+
import { withCommandRunTelemetry } from '../telemetry/cli-command-run.js';
96+
97+
const result = await withCommandRunTelemetry('remove.widget', {}, () => widgetPrimitive.remove(name));
98+
```
99+
100+
### TUI path with attributes — `withAddTelemetry`
101+
102+
```ts
103+
import { withAddTelemetry } from '../telemetry/cli-command-run.js';
104+
import { WidgetType, standardize } from '../telemetry/schemas/common-shapes.js';
105+
106+
const result = await withAddTelemetry(
107+
'add.widget',
108+
{ widget_type: standardize(WidgetType, config.type), count: config.items.length },
109+
() => widgetPrimitive.add(config)
110+
);
111+
```
112+
113+
## Concrete Example: `add.credential`
114+
115+
1. **Schema** (`schemas/command-run.ts`):
116+
117+
```ts
118+
const AddCredentialAttrs = safeSchema({ credential_type: CredentialType });
119+
120+
export const COMMAND_SCHEMAS = {
121+
'add.credential': AddCredentialAttrs,
122+
// ...
123+
};
124+
```
125+
126+
2. **Enum** (`schemas/common-shapes.ts`):
127+
128+
```ts
129+
export const CredentialType = z.enum(['api-key', 'oauth']);
130+
```
131+
132+
3. **CLI handler** (`primitives/CredentialPrimitive.tsx`):
133+
134+
```ts
135+
await runCliCommand('add.credential', !!cliOptions.json, async () => {
136+
const result = await this.add(addOptions);
137+
if (!result.success) throw new Error(result.error);
138+
console.log(JSON.stringify(result));
139+
return {
140+
credential_type: standardize(CredentialType, cliOptions.type ?? 'api-key'),
141+
};
142+
});
143+
```
144+
145+
4. **TUI hook** (`tui/hooks/useCreateCredential.ts`):
146+
```ts
147+
const result = await withAddTelemetry(
148+
'add.credential',
149+
{ credential_type: standardize(CredentialType, config.type) },
150+
() => credentialPrimitive.add(config)
151+
);
152+
```
153+
154+
## Key Points
155+
156+
- Telemetry never crashes the CLI — `standardize()` falls back gracefully, and `resilientParse` defaults invalid fields
157+
to `'unknown'`.
158+
- The callback in `runCliCommand` must **throw** on failure (not return an error). The helper catches, records failure
159+
telemetry, prints the error, and exits.
160+
- `withCommandRunTelemetry` and `withAddTelemetry` return the `Result` to the caller for further handling.

0 commit comments

Comments
 (0)