Skip to content

Commit a4c1316

Browse files
abrichrclaude
andauthored
docs: telemetry guide (disable with DO_NOT_TRACK=1) (#232)
* fix: TelemetryCallback __bases__ crash + 12 TRL integration tests The dynamic __bases__ assignment to inject TrainerCallback as a base class fails in Python: "deallocator differs from object". Fixed by creating a proper subclass at definition time instead. 12 new tests: - Mock rollout_func: correct keys, count, reward variance - Config separation: TrainingConfig has no TRL fields, wrapper accepts trl_config - Wrapper construction: all callback combinations, trl_config passthrough - TelemetryCallback: importable, fires events Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: telemetry guide — how to disable globally with one line DO_NOT_TRACK=1 in .env disables all telemetry. Already supported, just needed documentation. Covers what we collect, what we don't, privacy scrubbing, CI behavior, and source code links. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 82179b2 commit a4c1316

1 file changed

Lines changed: 82 additions & 0 deletions

File tree

docs/TELEMETRY.md

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
# Telemetry
2+
3+
OpenAdapt collects anonymous usage telemetry to understand how the tools are used and prioritize development. No screenshots, text content, API keys, or personally identifiable information is ever collected.
4+
5+
## Disabling telemetry
6+
7+
Add one line to your `.env` file:
8+
9+
```env
10+
DO_NOT_TRACK=1
11+
```
12+
13+
That's it. All telemetry is disabled globally — no events are sent, no background threads are started, no network connections are made.
14+
15+
This works across all OpenAdapt packages (openadapt-evals, openadapt-ml, openadapt-capture) because they all check this variable before sending any data.
16+
17+
### Alternative
18+
19+
```env
20+
OPENADAPT_TELEMETRY_ENABLED=false
21+
```
22+
23+
Both `DO_NOT_TRACK=1` and `OPENADAPT_TELEMETRY_ENABLED=false` have the same effect. `DO_NOT_TRACK` follows the [Console Do Not Track](https://consoledonottrack.com/) standard used by Homebrew, Gatsby, Next.js, and other OSS projects.
24+
25+
### Environment variable (no .env file)
26+
27+
```bash
28+
export DO_NOT_TRACK=1
29+
```
30+
31+
Or per-command:
32+
33+
```bash
34+
DO_NOT_TRACK=1 python scripts/train_trl_grpo.py --task-dir tasks/ ...
35+
```
36+
37+
## What we collect
38+
39+
When telemetry is enabled, we collect:
40+
41+
- **Event counts**: how many training runs, agent episodes, demos recorded
42+
- **Timing**: how long operations take (not wall-clock timestamps)
43+
- **Package metadata**: package version, Python version, OS (no hostname or IP)
44+
- **Training metrics**: reward mean, loss value, gradient norm (no model weights or training data)
45+
46+
We do **not** collect:
47+
48+
- Screenshots or images
49+
- Text content, prompts, or model outputs
50+
- API keys, passwords, or credentials
51+
- File paths with usernames
52+
- IP addresses or hostnames
53+
- Any data that could identify a person or organization
54+
55+
## How it works
56+
57+
Telemetry events are sent to [PostHog](https://posthog.com/) via a background thread. Events are batched and sent asynchronously — telemetry never blocks your training or evaluation. If the network is unavailable, events are silently dropped.
58+
59+
The telemetry client generates a random anonymous ID stored locally at `~/.openadapt/telemetry_distinct_id`. This ID is not linked to any personal information.
60+
61+
## Automatic scrubbing
62+
63+
Even if telemetry is enabled, a privacy scrubber automatically removes:
64+
65+
- Email addresses and phone numbers (pattern-matched)
66+
- API keys and tokens (key name detection)
67+
- File paths containing usernames (sanitized to `~`)
68+
- Any field named `password`, `token`, `secret`, `api_key`, etc.
69+
70+
## CI environments
71+
72+
Telemetry is automatically disabled in CI environments unless explicitly enabled:
73+
74+
```bash
75+
OPENADAPT_TELEMETRY_IN_CI=true # opt-in for CI
76+
```
77+
78+
## Source code
79+
80+
- Privacy scrubber: [openadapt-telemetry/privacy.py](https://github.com/OpenAdaptAI/openadapt-telemetry)
81+
- PostHog client: [openadapt-telemetry/posthog.py](https://github.com/OpenAdaptAI/openadapt-telemetry)
82+
- Event definitions: [openadapt-evals/telemetry.py](https://github.com/OpenAdaptAI/openadapt-evals/blob/main/openadapt_evals/telemetry.py)

0 commit comments

Comments
 (0)