feat(042): telemetry tier 2 — privacy-respecting usage signals#373
Merged
Conversation
… telemetry tier 2 Related #42 Initial spec artifacts for the Tier 2 telemetry expansion: - spec.md: 10 user stories (P1-P3), 44 functional requirements, privacy constraints, success criteria, assumptions, out-of-scope - plan.md: technical context, constitution gate, project structure, complexity tracking - research.md: 16 technical decisions resolved (counter primitives, surface classification, REST template extraction, env var precedence, ID rotation, schema versioning, etc.) - data-model.md: HeartbeatPayloadV2, CounterRegistry, ErrorCategory enum, FeatureFlagSnapshot, extended TelemetryConfig - contracts/heartbeat-v2.schema.json: full JSON Schema for the new payload - quickstart.md: build/run/verify instructions - tasks.md: 91 dependency-ordered sub-tasks across 13 phases Spec deliberately follows the autonomous-mode constraint from CLAUDE.md: no [NEEDS CLARIFICATION] markers; all ambiguous decisions captured in the Assumptions section. TDD is mandatory: every implementation task is preceded by a failing test.
…tegories, env overrides Related #42 Phase 2 of spec 042. Adds the in-memory CounterRegistry that aggregates Tier 2 telemetry counters, the fixed ErrorCategory enum, env-var-based opt-out, and the extended HeartbeatPayload schema. Hooks ID rotation, upgrade funnel, and counter reset into the existing telemetry Service so successful sends advance state and failures preserve it. ## Changes - internal/config/config.go: extend TelemetryConfig with AnonymousIDCreatedAt, LastReportedVersion, LastStartupOutcome, NoticeShown - internal/telemetry/error_categories.go: 11-value ErrorCategory enum + helper - internal/telemetry/registry.go: CounterRegistry with atomic surface counters, RWMutex-protected built-in/REST/error/doctor maps, Snapshot, Reset, surface parser, OAuth provider sort/dedupe helper - internal/telemetry/feature_flags.go: BuildFeatureFlagSnapshot derives the config-flag matrix and OAuth provider type list (no client IDs/URLs ever) - internal/telemetry/env_overrides.go: DO_NOT_TRACK, CI, MCPPROXY_TELEMETRY precedence chain - internal/telemetry/telemetry.go: HeartbeatPayload extended with all Tier 2 fields; Service captures envDisabledReason at construction; Start respects it; sendHeartbeat resets counters and advances upgrade funnel only on 2xx; Snapshot triggers annual ID rotation; legacy installs get created_at initialized non-destructively ## Testing All telemetry tests pass under -race. Personal and server editions both build clean.
Related #42 Wires Tier 2 counters into: - HTTP middleware (surface classifier, REST endpoint histogram with Chi templates) - MCP server (built-in tool histogram, MCP surface, upstream bucketed counter) - CLI client (X-MCPProxy-Client: cli/<version> via transport wrapper) - serve startup (first-run notice, startup outcome categorization) - telemetry CLI (new show-payload subcommand) All counters are nil-safe so the registry can be attached after route setup. Upstream tool names are NEVER recorded — only a bucketed total. Built-in tools come from a fixed allow-list. REST templates come from chi.RouteContext to prevent path-parameter leakage. All tests pass under -race. Both editions build clean.
…ntend, swift Related #42 Final integration pass: error category counters at OAuth refresh, tool quarantine, and upstream tool call error sites; doctor result aggregation; X-MCPProxy-Client header on web UI fetch and macOS tray URLRequest; canonical privacy substring regression test.
Related #42 Removes EnableTray from FeatureFlagSnapshot because the underlying config field is documented as deprecated/unused. The snapshot should only report flags that have runtime effect. ## Changes - internal/telemetry/feature_flags.go: drop EnableTray field and assignment - internal/telemetry/feature_flags_test.go: drop EnableTray assertions - internal/telemetry/payload_v2_test.go: drop EnableTray - internal/telemetry/payload_privacy_test.go: drop EnableTray
Related #42 Previously `mcpproxy telemetry status` only checked MCPPROXY_TELEMETRY=false for the env override flag. Now it uses the same precedence chain as the service itself (DO_NOT_TRACK > CI > MCPPROXY_TELEMETRY) and reports the specific override reason in both the table and JSON outputs.
Related #42 Run of make swagger. Picks up the new TelemetryConfig fields from spec 042: anonymous_id_created_at, last_reported_version, last_startup_outcome, notice_shown. No runtime change.
Deploying mcpproxy-docs with
|
| Latest commit: |
29f68e0
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://fffee297.mcpproxy-docs.pages.dev |
| Branch Preview URL: | https://042-telemetry-tier2.mcpproxy-docs.pages.dev |
TestHeartbeatSend and TestMultipleHeartbeats instantiate the telemetry service, which captures the env-var disable reason at New() time. GitHub Actions sets CI=true, which caused Start() to early-return before sending any heartbeats — failing both tests on CI but passing locally. Use t.Setenv to clear CI, DO_NOT_TRACK, and MCPPROXY_TELEMETRY for the duration of these tests so telemetry actually runs. Other tests in the package don't call Start() so they're unaffected. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
📦 Build ArtifactsWorkflow Run: View Run Available Artifacts
How to DownloadOption 1: GitHub Web UI (easiest)
Option 2: GitHub CLI gh run download 24274206218 --repo smart-mcp-proxy/mcpproxy-go
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Additive expansion of the v1 anonymous telemetry heartbeat with 12 new privacy-respecting signals that answer the product questions we cannot answer today: "which client surface do users prefer?", "which built-in MCP tools matter?", "are installs hitting startup failures?", "where do errors cluster?". Implements spec
042-telemetry-tier2.schema_version: 2,surface_requests,builtin_tool_calls,upstream_tool_call_count_bucket,rest_endpoint_calls,feature_flags,last_startup_outcome,previous_version/current_version,error_category_counts,doctor_checks,anonymous_id_created_at.mcpproxy telemetry show-payload(no network call, audit tool).DO_NOT_TRACK(consoledonottrack.com convention),CI(auto-disable in CI).mcpproxy serveruns.X-MCPProxy-Clientheader set by the CLI HTTP client (cli/<version>), the web UI fetch (webui/web), and the macOS tray URLRequest (tray/<bundle version>).Privacy contract
The spec's hard privacy constraints are enforced by
internal/telemetry/payload_privacy_test.go::TestPayloadHasNoForbiddenSubstrings, which builds a fully populated payload with deliberately distinctive canary inputs (upstream server namedMY-CANARY-SERVER, path-like URL/Users/alice/private-token-store, attempt to leak an upstream tool name) and asserts the rendered JSON contains none of: server names, upstream tool names, file paths, hostnames, IPs, bearer tokens, client secrets, or free-text error messages. Also asserts payload size is under 8 KB.The canary test passes. Privacy contract is verifiable.
Test plan
go test -race ./internal/telemetry/...→ pass (14 test files)go test -race ./internal/httpapi/...→ pass (new surface + REST histogram middleware tests)go test -race ./internal/cliclient/...→ pass (new header-injection test)go test -race -run "TestHandleUpstream|TestHandleCallTool|TestHandleRetrieveTools|TestHandleQuarantine" ./internal/server/→ passgo test -race ./cmd/mcpproxy/...→ pass (new startup-outcome mapping test)./scripts/run-linter.sh→ 0 issuesgo build ./cmd/mcpproxy(personal edition) → passgo build -tags server ./cmd/mcpproxy(server edition) → pass./mcpproxy telemetry show-payloademits schema_version=2 and all new fields with no network callDO_NOT_TRACK=1 ./mcpproxy telemetry statusreportsOverride: DO_NOT_TRACK./scripts/test-api-e2e.sh→ 61 pass, 10 fail (all 10 failures are pre-existing on main, verified by running the same script on main — same test names, same failures, unrelated to telemetry)Documents
specs/042-telemetry-tier2/spec.md— 10 user stories, 44 functional requirements, privacy constraints, 15-item Assumptions section, out-of-scope. Zero[NEEDS CLARIFICATION]markers per autonomous-mode override.specs/042-telemetry-tier2/plan.md— technical context, constitution gate evaluation (one justifiedsync.RWMutexexception documented in Complexity Tracking), project structure.specs/042-telemetry-tier2/research.md— 16 technical decisions.specs/042-telemetry-tier2/data-model.md— all entities and invariants.specs/042-telemetry-tier2/contracts/heartbeat-v2.schema.json— JSON Schema for the new payload.specs/042-telemetry-tier2/tasks.md— 91 sub-tasks across 13 phases.autonomous_summary.md— end-to-end execution record.Follow-ups (out of scope)
telemetry.mcpproxy.appneeds a separate PR to accept the newschema_version: 2fields.https://mcpproxy.app/telemetryprivacy policy page referenced in the first-run notice.docs/features/telemetry.mdfull Tier 2 field inventory.Related #42