Skip to content

feat(telemetry): add non-reversible machine_id to heartbeat for install dedup#796

Merged
Dumbris merged 1 commit into
mainfrom
feat/telemetry-machine-id
Jul 2, 2026
Merged

feat(telemetry): add non-reversible machine_id to heartbeat for install dedup#796
Dumbris merged 1 commit into
mainfrom
feat/telemetry-machine-id

Conversation

@Dumbris

@Dumbris Dumbris commented Jul 2, 2026

Copy link
Copy Markdown
Member

Motivation

Telemetry anonymous_id is a UUIDv4 persisted in the config file. In ephemeral environments — layered Docker builds, throwaway HOMEs, CI runners — the config (and therefore the UUID) is regenerated on every run, so a single physical machine masquerades as many distinct installs. One CI block alone produced 1,231 anonymous_ids from 3 IPv6 /64s, inflating install counts.

The dashboard (mcpproxy-dash/src/lib/shared/dedup.ts) currently dedups via a lossy normalized_ip|os|arch|edition heuristic, and its header comment explicitly asks for the fix this PR delivers:

"The only way to fix both would be a stable hashed machine id in the payload (see telemetry client TODO)."

What this adds

A new heartbeat field machine_id (schema v6): a stable, non-reversible hash of the OS machine id.

  • Hash schemeHMAC-SHA256(osMachineID, appKey) via denisbrodbeck/machineid's ProtectedID pattern, hex-encoded (64 chars). The mcpproxy-specific appKey scopes the value so it cannot be correlated with any other application that hashes the same OS machine id. The raw machine id is never returned or transmitted — only the salted hash.
  • Stability — resolved once per process and cached (resolveMachineID), so the value is identical across every heartbeat build on a machine.
  • Graceful fallback — if the OS machine id is unreadable (container without /etc/machine-id, permission error, exotic platform), protectedMachineID returns "", the field is omitted (omitempty), and the heartbeat is never blocked. The backend treats empty as "unknown".
  • Opt-out — no new gate needed: machine_id is built inside buildHeartbeat, which is only reached through the existing opt-out-gated Start/sendHeartbeat path (MCPPROXY_TELEMETRY=false, DO_NOT_TRACK, config disable, and the mid-flight optedOut latch all cover it).

Design decisions

Hash scheme. ProtectedID keys the HMAC with the machine id and uses the app key as the message, yielding a per-app, non-reversible digest — exactly the "ProtectedID" pattern requested. Chosen over a bare SHA256(machineID) because the app-key scoping prevents cross-application correlation.

Dependency — added github.com/denisbrodbeck/machineid v1.0.1 (yes). Nothing in-tree or in existing deps reads the OS machine id (checked repo + go.mod). Rolling it by hand is non-trivial cross-platform (macOS needs ioreg, Windows the registry MachineGuid, Linux /etc/machine-id + dbus fallback). The library is tiny, stdlib-only (no transitive deps), and is the standard answer. Per CLAUDE.md ("avoid new deps without clear need"), the need is clear and the footprint minimal. Now a direct dependency in go.mod.

Schema version — bumped 5 → 6 (safe, verified against the worker). I read the ingest worker (mcpproxy-telemetry, read-only) to confirm this cannot break ingestion:

  • src/index.ts / src/validation.ts only validate fields conditionally on schema_version >= 3 and >= 4; there is no upper-bound check and no unknown-field rejection.
  • The daemon already sends schema_version: 5 while the worker validates only up to v4 — empirical proof it tolerates higher-than-known versions.
  • schema.sql stores the whole payload in payload_json wholesale.

So a higher version carrying one additive field is ignored by older consumers and preserved in payload_json. Bumping (vs. keeping v5) matches this codebase's convention — every prior field set bumped the version — and keeps the version→fields contract honest for the dashboard.

Privacy

  • Only the salted, non-reversible hash is sent; the raw machine id never leaves the process.
  • App-key scoping blocks cross-application correlation.
  • docs/features/telemetry.md updated: new field row, a dedicated "Machine ID (schema v6)" section, and a "what is NOT collected" bullet (raw machine id / reversible hardware ids).
  • Tests assert the raw id never appears in the serialized payload and that the field is omitted when unavailable; anonymity_test.go extended so the anonymity scanner catches a raw-id leak while the hashed value passes.

Tests (TDD, failing-first)

internal/telemetry/machine_id_test.go + extended anonymity_test.go:

  • field present & stable across two payload builds;
  • production hash is 64-char lowercase hex and ≠ raw machine id;
  • empty/omitted when the underlying id is unavailable (injected via the machineIDProvider seam, matching the package's function-pointer test style);
  • resolver caches (provider probed once);
  • raw id never in the serialized payload; anonymity scanner blocks a raw-id leak.

Results:

  • go build ./... — OK
  • go test -race ./internal/telemetry/...ok (raw-comparison tests ran, not skipped)
  • golangci-lint run --config .github/.golangci.yml ./...0 issues
  • gofmt/goimports — clean

Follow-ups (not in this PR)

  1. Worker migration (mcpproxy-telemetry): add a machine_id TEXT column + extraction in validateV*Payload/index.ts and an index, so dedup can query it directly rather than digging into payload_json. Not required for ingestion today (stored wholesale).
  2. Dashboard (mcpproxy-dash): update identityExpr in dedup.ts to prefer machine_id when present, falling back to the current normalized_ip|os|arch|edition heuristic for pre-v6 rows. Resolves the TODO that motivated this PR.

🤖 Generated with Claude Code

…ll dedup

The telemetry anonymous_id is a UUID persisted in the config file, so
ephemeral environments (layered Docker builds, throwaway HOMEs, CI) mint a
fresh id per run — one CI block produced 1,231 ids from 3 IPv6 /64s. The
dashboard currently dedups via a lossy normalized_ip|os|arch|edition
heuristic; its dedup.ts header explicitly asks for "a stable hashed machine
id in the payload".

This adds machine_id (schema v6): HMAC-SHA256 of the OS machine id scoped by
an mcpproxy-specific app key (denisbrodbeck/machineid ProtectedID pattern) —
non-reversible, uncorrelatable with other apps' telemetry, and never the raw
id. It resolves once per process (cached, stable across builds) and gracefully
falls back to empty (omitted) when the OS machine id is unreadable, so the
heartbeat is never blocked. It rides the existing opt-out gate.

schema_version bumped 5->6: the ingest worker stores payload_json wholesale
and only validates fields for schema_version >= 3/>=4 (it already receives v5
while validating only v4), so a higher version with an additive field cannot
break ingestion.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying mcpproxy-docs with  Cloudflare Pages  Cloudflare Pages

Latest commit: a2d1f5c
Status: ✅  Deploy successful!
Preview URL: https://c9ac6d8c.mcpproxy-docs.pages.dev
Branch Preview URL: https://feat-telemetry-machine-id.mcpproxy-docs.pages.dev

View logs

@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 85.71429% with 2 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
internal/telemetry/machine_id.go 84.61% 1 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown

📦 Build Artifacts

Workflow Run: View Run
Branch: feat/telemetry-machine-id

Available Artifacts

  • archive-darwin-amd64 (28 MB)
  • archive-darwin-arm64 (25 MB)
  • archive-linux-amd64 (16 MB)
  • archive-linux-arm64 (14 MB)
  • archive-windows-amd64 (28 MB)
  • archive-windows-arm64 (25 MB)
  • frontend-dist-pr (0 MB)
  • installer-dmg-darwin-amd64 (21 MB)
  • installer-dmg-darwin-arm64 (19 MB)

How to Download

Option 1: GitHub Web UI (easiest)

  1. Go to the workflow run page linked above
  2. Scroll to the bottom "Artifacts" section
  3. Click on the artifact you want to download

Option 2: GitHub CLI

gh run download 28582395495 --repo smart-mcp-proxy/mcpproxy-go

Note: Artifacts expire in 14 days.

@Dumbris Dumbris merged commit 776c739 into main Jul 2, 2026
36 checks passed
@Dumbris Dumbris deleted the feat/telemetry-machine-id branch July 2, 2026 10:48
Dumbris added a commit that referenced this pull request Jul 2, 2026
…ne, connect-trust/upgrade-nudge progress (#803)

check-github now passes with 0 errors: scanner-simplification epic
complete (#786/#792/#793/#794 incl. deep-scan trust fixes + docs
sweep); connect-trust US1 preview (#802) + backup visibility (#799)
done; upgrade-nudge status/log slice (#798) split out as done with
the banner+config remainder tracked separately; telemetry machine_id
client (#796) and hygiene check-github (#800) done. Remaining
warnings are the known windows-tray no-PR-evidence items.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants