Skip to content

Commit 776c739

Browse files
Dumbrisclaude
andauthored
feat(telemetry): add non-reversible machine_id to heartbeat for install dedup (#796)
The telemetry anonymous_id is a UUID persisted in the config file, so ephemeral environments (layered Docker builds, throwaway HOMEs, CI) mint a fresh id per run — one CI block produced 1,231 ids from 3 IPv6 /64s. The dashboard currently dedups via a lossy normalized_ip|os|arch|edition heuristic; its dedup.ts header explicitly asks for "a stable hashed machine id in the payload". This adds machine_id (schema v6): HMAC-SHA256 of the OS machine id scoped by an mcpproxy-specific app key (denisbrodbeck/machineid ProtectedID pattern) — non-reversible, uncorrelatable with other apps' telemetry, and never the raw id. It resolves once per process (cached, stable across builds) and gracefully falls back to empty (omitted) when the OS machine id is unreadable, so the heartbeat is never blocked. It rides the existing opt-out gate. schema_version bumped 5->6: the ingest worker stores payload_json wholesale and only validates fields for schema_version >= 3/>=4 (it already receives v5 while validating only v4), so a higher version with an additive field cannot break ingestion. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
1 parent 204b4bc commit 776c739

10 files changed

Lines changed: 338 additions & 19 deletions

File tree

docs/features/telemetry.md

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,11 +4,12 @@ MCPProxy collects anonymous usage statistics to help improve the product. This p
44

55
## What is collected
66

7-
MCPProxy sends a **daily heartbeat** containing only aggregate, non-identifying information. The current schema is **version 5** (`schema_version: 5` in the JSON payload); the schema is forward-compatible so older consumers simply ignore fields they don't recognize.
7+
MCPProxy sends a **daily heartbeat** containing only aggregate, non-identifying information. The current schema is **version 6** (`schema_version: 6` in the JSON payload); the schema is forward-compatible so older consumers simply ignore fields they don't recognize.
88

99
| Field | Example | Purpose |
1010
|-------|---------|---------|
1111
| `anonymous_id` | `550e8400-...` | Random UUID for deduplication (not linked to you) |
12+
| `machine_id` | `9f86d081...` (64-hex) | Stable, **non-reversible** salted hash of the OS machine id — dedups ephemeral installs whose `anonymous_id` churns every run (schema v6). Empty/omitted when unreadable. Never the raw machine id |
1213
| `version` | `0.21.3` | Track version adoption |
1314
| `edition` | `personal` | Understand edition usage |
1415
| `os` | `darwin` | Platform distribution |
@@ -43,13 +44,27 @@ The following is **never** collected:
4344
- User identity, email, or account information
4445
- Tool call content, arguments, or responses
4546
- Any user-generated content
47+
- The **raw** OS machine id or any reversible hardware identifier (only the salted, non-reversible `machine_id` hash is sent — see below)
4648

4749
## Anonymous ID
4850

4951
The anonymous ID is a random UUID (v4) generated on first run. It has **no correlation** to your hardware, user account, or identity. It exists solely to deduplicate heartbeats (so we don't count the same install twice in a day).
5052

5153
You can delete it by removing the `telemetry.anonymous_id` field from your config — a new random ID will be generated on next startup.
5254

55+
## Machine ID (schema v6)
56+
57+
The `anonymous_id` above is a UUID persisted in the config file. In **ephemeral environments** — throwaway `HOME`s, layered Docker builds, CI runners — the config (and therefore the UUID) is regenerated on every run, so a single machine can masquerade as hundreds of distinct installs. That inflates our install counts and defeats deduplication.
58+
59+
`machine_id` fixes this without collecting anything identifying:
60+
61+
- It is a **salted, non-reversible hash**`HMAC-SHA256` keyed by the OS machine id, scoped by an mcpproxy-specific application key. The **raw machine id is never transmitted**; only the hash leaves your machine.
62+
- The application-specific key means the value **cannot be correlated** with any other application's telemetry that hashes the same OS machine id.
63+
- It is **stable per physical machine**, so ephemeral installs collapse to one identity for counting.
64+
- If the OS machine id **cannot be read** (a container without `/etc/machine-id`, a permission error, or an exotic platform), the field is simply **omitted** — the heartbeat is never blocked, and the backend treats an absent value as "unknown".
65+
66+
`machine_id` respects the **same opt-out** as every other field: when telemetry is disabled (see below), the entire heartbeat — including `machine_id` — is never sent.
67+
5368
## One-time opt-out signal
5469

5570
When telemetry transitions from **enabled to disabled** (via the CLI, the config

go.mod

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ require (
99
github.com/blevesearch/bleve/v2 v2.6.0
1010
github.com/charmbracelet/bubbletea v1.3.10
1111
github.com/charmbracelet/lipgloss v1.1.0
12+
github.com/denisbrodbeck/machineid v1.0.1
1213
github.com/dop251/goja v0.0.0-20260305124333-6a7976c22267
1314
github.com/evanw/esbuild v0.28.1
1415
github.com/gen2brain/beeep v0.11.2
@@ -39,6 +40,7 @@ require (
3940
golang.org/x/sync v0.21.0
4041
golang.org/x/sys v0.46.0
4142
golang.org/x/term v0.44.0
43+
golang.org/x/text v0.37.0
4244
gopkg.in/natefinch/lumberjack.v2 v2.2.1
4345
gopkg.in/yaml.v3 v3.0.1
4446
pgregory.net/rapid v1.3.0
@@ -142,7 +144,6 @@ require (
142144
go.yaml.in/yaml/v2 v2.4.2 // indirect
143145
go.yaml.in/yaml/v3 v3.0.4 // indirect
144146
golang.org/x/net v0.55.0 // indirect
145-
golang.org/x/text v0.37.0 // indirect
146147
golang.org/x/tools v0.45.0 // indirect
147148
google.golang.org/genproto/googleapis/api v0.0.0-20260526163538-3dc84a4a5aaa // indirect
148149
google.golang.org/genproto/googleapis/rpc v0.0.0-20260526163538-3dc84a4a5aaa // indirect

go.sum

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -85,6 +85,8 @@ github.com/danieljoos/wincred v1.2.3/go.mod h1:6qqX0WNrS4RzPZ1tnroDzq9kY3fu1KwE7
8585
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
8686
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
8787
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
88+
github.com/denisbrodbeck/machineid v1.0.1 h1:geKr9qtkB876mXguW2X6TU4ZynleN6ezuMSRhl4D7AQ=
89+
github.com/denisbrodbeck/machineid v1.0.1/go.mod h1:dJUwb7PTidGDeYyUBmXZ2GphQBbjJCrnectwCyxcUSI=
8890
github.com/dlclark/regexp2 v1.11.4 h1:rPYF9/LECdNymJufQKmri9gV604RvvABwgOA8un7yAo=
8991
github.com/dlclark/regexp2 v1.11.4/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=
9092
github.com/dop251/goja v0.0.0-20260305124333-6a7976c22267 h1:Kfmq11A6DLHD8XoOeljWjzWg/rrujeaLHWSb8u7+2qQ=

internal/telemetry/anonymity_test.go

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,8 @@ import (
44
"errors"
55
"strings"
66
"testing"
7+
8+
"github.com/denisbrodbeck/machineid"
79
)
810

911
// TestScanForPII_BlockedPrefix_UsersPath asserts that a payload containing a
@@ -152,6 +154,41 @@ func TestPopulateBlockedValuesFrom(t *testing.T) {
152154
}
153155
}
154156

157+
// TestScanForPII_RawMachineIDBlocked asserts that when the raw OS machine id is
158+
// added to BlockedValues (defense-in-depth, as the telemetry service does for
159+
// hostname/tokens), a payload accidentally carrying the raw id is rejected. The
160+
// hashed machine_id we DO emit is unrelated to the raw value and passes. Skips
161+
// when the OS machine id is unreadable on this host.
162+
func TestScanForPII_RawMachineIDBlocked(t *testing.T) {
163+
raw, err := machineid.ID()
164+
if err != nil || len(raw) < 3 {
165+
t.Skipf("OS machine id unavailable on this host (%v); skipping", err)
166+
}
167+
168+
prev := BlockedValues
169+
BlockedValues = []string{raw}
170+
defer func() { BlockedValues = prev }()
171+
172+
// A payload leaking the raw id must be caught.
173+
leak := []byte(`{"anonymous_id":"abc","machine_id":"` + raw + `"}`)
174+
if scanErr := ScanForPII(leak); scanErr == nil {
175+
t.Fatalf("expected violation when raw machine id present, got nil")
176+
} else if !errors.Is(scanErr, ErrAnonymityViolation) {
177+
t.Fatalf("expected ErrAnonymityViolation, got %v", scanErr)
178+
}
179+
180+
// The hashed value we actually emit must NOT contain the raw id and must
181+
// pass the scan.
182+
hashed := protectedMachineID()
183+
if hashed != "" && strings.Contains(hashed, raw) {
184+
t.Fatalf("hashed machine_id leaked the raw id")
185+
}
186+
clean := []byte(`{"anonymous_id":"abc","machine_id":"` + hashed + `"}`)
187+
if scanErr := ScanForPII(clean); scanErr != nil {
188+
t.Errorf("hashed machine_id payload should pass scan, got %v", scanErr)
189+
}
190+
}
191+
155192
// TestScanForPII_AllBlockedPrefixes covers each of the default blocked
156193
// prefixes to guard against a regression that silently drops one.
157194
func TestScanForPII_AllBlockedPrefixes(t *testing.T) {

internal/telemetry/machine_id.go

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
package telemetry
2+
3+
import (
4+
"sync"
5+
6+
"github.com/denisbrodbeck/machineid"
7+
)
8+
9+
// machineIDAppKey scopes the hashed machine id to mcpproxy. machineid.ProtectedID
10+
// feeds it as the HMAC-SHA256 message keyed by the OS machine id, so the emitted
11+
// value is a non-reversible per-app hash: it cannot be reversed to the raw id and
12+
// cannot be correlated with any other application that hashes the same OS machine
13+
// id with a different key.
14+
const machineIDAppKey = "mcpproxy-telemetry"
15+
16+
// machineIDProvider is the seam used to obtain the app-scoped, non-reversible
17+
// machine-id hash. Production points it at protectedMachineID; tests override it
18+
// to inject deterministic values or simulate an unreadable machine id. It mirrors
19+
// the function-pointer injection style used elsewhere in this package
20+
// (see populateBlockedValuesFrom, DetectEnvKind).
21+
var machineIDProvider = protectedMachineID
22+
23+
// protectedMachineID returns HMAC-SHA256(osMachineID, machineIDAppKey) as a
24+
// lowercase hex string (64 chars). On any failure to read the OS machine id
25+
// (permission errors, missing /etc/machine-id in a minimal container, an exotic
26+
// platform) it returns "" — the caller then omits the field, and the backend
27+
// treats empty as "unknown". The raw OS machine id is NEVER returned or
28+
// transmitted, only the salted hash.
29+
func protectedMachineID() string {
30+
id, err := machineid.ProtectedID(machineIDAppKey)
31+
if err != nil {
32+
return ""
33+
}
34+
return id
35+
}
36+
37+
// machineIDOnce guards the cached machine-id hash so repeated heartbeats reuse
38+
// one value without re-probing the OS each time.
39+
var (
40+
machineIDOnce sync.Once
41+
machineIDCached string
42+
)
43+
44+
// resolveMachineID computes the app-scoped machine-id hash once per process and
45+
// caches it. Subsequent heartbeats reuse the cached value, guaranteeing the
46+
// field is stable across payload builds without re-probing the OS. Empty string
47+
// when the OS machine id is unavailable.
48+
func resolveMachineID() string {
49+
machineIDOnce.Do(func() {
50+
machineIDCached = machineIDProvider()
51+
})
52+
return machineIDCached
53+
}
54+
55+
// resetMachineIDForTest clears the cached machine-id hash so tests can re-run
56+
// resolveMachineID with a freshly injected machineIDProvider. MUST NOT be called
57+
// from production code — guard calls behind _test.go files only.
58+
func resetMachineIDForTest() {
59+
machineIDOnce = sync.Once{}
60+
machineIDCached = ""
61+
}
Lines changed: 179 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,179 @@
1+
package telemetry
2+
3+
import (
4+
"encoding/json"
5+
"strings"
6+
"testing"
7+
"time"
8+
9+
"github.com/denisbrodbeck/machineid"
10+
"go.uber.org/zap"
11+
12+
"github.com/smart-mcp-proxy/mcpproxy-go/internal/config"
13+
)
14+
15+
// isLowerHex reports whether s is a non-empty string of lowercase hex digits.
16+
func isLowerHex(s string) bool {
17+
if s == "" {
18+
return false
19+
}
20+
for _, r := range s {
21+
switch {
22+
case r >= '0' && r <= '9':
23+
case r >= 'a' && r <= 'f':
24+
default:
25+
return false
26+
}
27+
}
28+
return true
29+
}
30+
31+
// newMachineIDTestService builds a minimal telemetry Service suitable for
32+
// exercising BuildPayload in machine-id tests.
33+
func newMachineIDTestService(t *testing.T) *Service {
34+
t.Helper()
35+
cfg := &config.Config{
36+
Telemetry: &config.TelemetryConfig{
37+
AnonymousID: "fixed-id",
38+
AnonymousIDCreatedAt: time.Now().UTC().Format(time.RFC3339),
39+
},
40+
}
41+
return New(cfg, "", "v1.2.3", "personal", zap.NewNop())
42+
}
43+
44+
// TestMachineID_PresentAndStable asserts the machine_id field is populated and
45+
// identical across two payload builds on the same machine.
46+
func TestMachineID_PresentAndStable(t *testing.T) {
47+
const fixed = "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"
48+
prev := machineIDProvider
49+
machineIDProvider = func() string { return fixed }
50+
resetMachineIDForTest()
51+
defer func() {
52+
machineIDProvider = prev
53+
resetMachineIDForTest()
54+
}()
55+
56+
svc := newMachineIDTestService(t)
57+
58+
p1 := svc.BuildPayload()
59+
p2 := svc.BuildPayload()
60+
61+
if p1.MachineID == "" {
62+
t.Fatalf("machine_id is empty, want %q", fixed)
63+
}
64+
if p1.MachineID != fixed {
65+
t.Errorf("machine_id = %q, want %q", p1.MachineID, fixed)
66+
}
67+
if p1.MachineID != p2.MachineID {
68+
t.Errorf("machine_id not stable across builds: %q != %q", p1.MachineID, p2.MachineID)
69+
}
70+
}
71+
72+
// TestMachineID_OmittedWhenUnavailable asserts that when the OS machine id
73+
// cannot be read, the field is empty and omitted from the serialized JSON
74+
// (never failing the heartbeat).
75+
func TestMachineID_OmittedWhenUnavailable(t *testing.T) {
76+
prev := machineIDProvider
77+
machineIDProvider = func() string { return "" }
78+
resetMachineIDForTest()
79+
defer func() {
80+
machineIDProvider = prev
81+
resetMachineIDForTest()
82+
}()
83+
84+
svc := newMachineIDTestService(t)
85+
payload := svc.BuildPayload()
86+
87+
if payload.MachineID != "" {
88+
t.Errorf("machine_id = %q, want empty when unavailable", payload.MachineID)
89+
}
90+
91+
data, err := json.Marshal(payload)
92+
if err != nil {
93+
t.Fatalf("marshal failed: %v", err)
94+
}
95+
if strings.Contains(string(data), "machine_id") {
96+
t.Errorf("empty machine_id should be omitted from JSON, got: %s", data)
97+
}
98+
}
99+
100+
// TestResolveMachineID_CachedOncePerProcess asserts resolveMachineID probes the
101+
// provider at most once and caches the result.
102+
func TestResolveMachineID_CachedOncePerProcess(t *testing.T) {
103+
calls := 0
104+
prev := machineIDProvider
105+
machineIDProvider = func() string {
106+
calls++
107+
return "cached-value"
108+
}
109+
resetMachineIDForTest()
110+
defer func() {
111+
machineIDProvider = prev
112+
resetMachineIDForTest()
113+
}()
114+
115+
_ = resolveMachineID()
116+
_ = resolveMachineID()
117+
_ = resolveMachineID()
118+
119+
if calls != 1 {
120+
t.Errorf("machineIDProvider called %d times, want 1 (result must be cached)", calls)
121+
}
122+
}
123+
124+
// TestProtectedMachineID_NotRawAndHex verifies the production hash is a
125+
// lowercase-hex string, is not the raw OS machine id, and is scoped to the app
126+
// key (differs from the raw id and from a bare hash of another app key). Skips
127+
// when the OS machine id is unreadable (e.g. CI containers without
128+
// /etc/machine-id) — the graceful-fallback path is covered separately.
129+
func TestProtectedMachineID_NotRawAndHex(t *testing.T) {
130+
raw, err := machineid.ID()
131+
if err != nil || raw == "" {
132+
t.Skipf("OS machine id unavailable on this host (%v); skipping raw-comparison", err)
133+
}
134+
135+
got := protectedMachineID()
136+
if got == "" {
137+
t.Fatal("protectedMachineID returned empty despite a readable OS machine id")
138+
}
139+
if !isLowerHex(got) {
140+
t.Errorf("protectedMachineID = %q, want lowercase hex", got)
141+
}
142+
// HMAC-SHA256 hex is 64 chars.
143+
if len(got) != 64 {
144+
t.Errorf("protectedMachineID length = %d, want 64 (SHA-256 hex)", len(got))
145+
}
146+
if strings.Contains(got, raw) || got == raw {
147+
t.Errorf("protectedMachineID leaked the raw machine id")
148+
}
149+
}
150+
151+
// TestProtectedMachineID_RawNeverInPayload builds a real payload (production
152+
// provider) and asserts the raw OS machine id never appears in the serialized
153+
// JSON, while the hashed machine_id does. Skips when the raw id is unreadable.
154+
func TestProtectedMachineID_RawNeverInPayload(t *testing.T) {
155+
raw, err := machineid.ID()
156+
if err != nil || raw == "" {
157+
t.Skipf("OS machine id unavailable on this host (%v); skipping", err)
158+
}
159+
160+
// Ensure the production provider is in effect and the cache is fresh.
161+
resetMachineIDForTest()
162+
defer resetMachineIDForTest()
163+
164+
svc := newMachineIDTestService(t)
165+
payload := svc.BuildPayload()
166+
167+
data, err := json.Marshal(payload)
168+
if err != nil {
169+
t.Fatalf("marshal failed: %v", err)
170+
}
171+
js := string(data)
172+
173+
if strings.Contains(js, raw) {
174+
t.Errorf("PRIVACY VIOLATION: payload contains the raw machine id")
175+
}
176+
if payload.MachineID != "" && !strings.Contains(js, payload.MachineID) {
177+
t.Errorf("hashed machine_id missing from serialized payload")
178+
}
179+
}

internal/telemetry/payload_privacy_test.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -144,7 +144,7 @@ func TestPayloadHasNoForbiddenSubstrings(t *testing.T) {
144144
// Sanity check: the payload should still contain the legitimate fields,
145145
// otherwise we've over-redacted.
146146
for _, required := range []string{
147-
`"schema_version":5`,
147+
`"schema_version":6`,
148148
`"surface_requests"`,
149149
`"builtin_tool_calls"`,
150150
`"upstream_tool_call_count_bucket"`,

internal/telemetry/payload_v2_test.go

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -88,8 +88,8 @@ func TestHeartbeatPayloadV2Marshal(t *testing.T) {
8888

8989
payload := svc.BuildPayload()
9090

91-
if payload.SchemaVersion != 5 {
92-
t.Errorf("schema_version = %d, want 5", payload.SchemaVersion)
91+
if payload.SchemaVersion != 6 {
92+
t.Errorf("schema_version = %d, want 6", payload.SchemaVersion)
9393
}
9494
if payload.AnonymousID != "fixed-id" {
9595
t.Errorf("anonymous_id = %q", payload.AnonymousID)
@@ -135,7 +135,7 @@ func TestHeartbeatPayloadV2Marshal(t *testing.T) {
135135
}
136136
js := string(data)
137137
for _, key := range []string{
138-
`"schema_version":5`,
138+
`"schema_version":6`,
139139
`"surface_requests"`,
140140
`"builtin_tool_calls"`,
141141
`"upstream_tool_call_count_bucket":"11-100"`,

0 commit comments

Comments
 (0)