docs: ADR-0027 feature-usage bitmask in the User-Agent#6500
docs: ADR-0027 feature-usage bitmask in the User-Agent#6500eavanvalkenburg wants to merge 6 commits into
Conversation
Add an ADR, design spec, and per-language bit registry for a lightweight feature-usage signal: a 64-bit mask, emitted as a `(feat=vN.<hex>)` User-Agent comment, stamped per request on first-party (Azure/Foundry) clients only. - docs/decisions/0027-feature-usage-bitmask-user-agent.md — ADR (options-first, with Limitations, Open Questions, and v1->v2 migration) - docs/specs/002-feature-usage-telemetry.md — design spec + implementation plan - docs/specs/feature-usage-bit-registry.md — per-language bit tables + governance Granularity is per package with core broken out per feature (each orchestration pattern and built-in context/history provider). Registries are per language (decoder selects by the language already in the UA). OpenTelemetry emission is deferred (privacy). Docs only; no code changes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The registry JSON was consolidated into feature-usage-bit-registry.md; point the ADR's two remaining links at the markdown instead of the deleted file (fixes markdown-link-check 404s). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Adds a docs-only proposal describing how Agent Framework SDKs can emit a privacy-scoped, per-process “feature usage” signal via a 64-bit bitmask encoded as a (feat=vN.<hex>) User-Agent comment, including versioning/governance guidance and an implementation plan for Python and .NET.
Changes:
- Adds ADR-0027 documenting the decision (first-party-only UA emission, per-language registries, 64-bit mask, hex encoding, opt-out reuse).
- Adds SPEC-002 detailing the planned mechanism/API surface and per-request stamping approach (httpx hooks + azure-core policy, plus .NET parity notes).
- Adds a per-language bit registry table and governance rules for allocating/maintaining bits.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| docs/decisions/0027-feature-usage-bitmask-user-agent.md | ADR documenting the decision rationale, options, limitations, and open questions for UA-based feature-usage telemetry. |
| docs/specs/002-feature-usage-telemetry.md | Companion design/implementation plan describing accumulator/marking API and per-request first-party-only emission. |
| docs/specs/feature-usage-bit-registry.md | Human-readable per-language bit tables plus encoding/decoding rules, opt-out behavior, and governance. |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Automated Code Review
Reviewers: 5 | Confidence: 68%
✓ Correctness
Docs-only PR adding ADR-027, SPEC-002, and a feature-usage bit registry. The design is internally consistent and the hex/bit examples are mathematically correct. The main issue is two broken cross-reference links in the ADR that point to a non-existent JSON file (
../feature-usage-bit-registry.json) when the actual registry file isdocs/specs/feature-usage-bit-registry.md.
✓ Security Reliability
Docs-only ADR and spec for feature-usage bitmask telemetry. The design is well-reasoned about privacy (first-party-only emission, opt-out, coarse granularity, privacy review as blocking precondition). However, the ADR contains two broken links referencing a non-existent
docs/feature-usage-bit-registry.jsonfile — the actual registry is the markdown file atdocs/specs/feature-usage-bit-registry.md. The registry doc itself confirms 'No machine-readable registry file ships today,' contradicting the ADR's JSON link.
✓ Test Coverage
This is a docs-only PR (ADR, design spec, and bit registry) with no code changes. Since no runtime behavior is introduced or modified, there are no test coverage concerns. The spec itself appropriately describes the tests that should accompany the future implementation (step 5 of the implementation plan mentions 'tests for the UA opt-out, first-party scoping, and the live (non-frozen) UA'), which is the correct approach for a design document.
✓ Failure Modes
This is a well-structured docs-only PR (ADR + spec + registry) with no code changes. The only concrete issue found is broken cross-reference links in the ADR: lines 23 and 277 reference a non-existent
../feature-usage-bit-registry.jsonfile, while the actual registry lives at../specs/feature-usage-bit-registry.md. The ADR itself correctly links to the registry on line 278 using the right path, making the inconsistency clear. No runtime failure modes apply since this is documentation only.
✗ Design Approach
The overall direction is reasonable, but the .NET mapping section currently specifies accumulator math that would encode the raw bit index instead of setting that bit, so the emitted mask would decode to the wrong features. I also found two internal contract inconsistencies in the docs: one sentence implies cross-language bit meanings are shared even though the registry explicitly makes them language-specific, and the ADR points readers to a JSON registry file even though this proposal says no machine-readable registry ships in v1.
Flagged Issues
- docs/specs/002-feature-usage-telemetry.md:273-277 defines
FeatureBitas the source-of-truth bit list, but then usesInterlocked.Or(ref _mask, (long)bit), which ORs the ordinal itself into the mask. For example, marking bit 22 would emit0x16instead of setting bit 22, so decoders would silently attribute the request to the wrong features.
Suggestions
- docs/specs/002-feature-usage-telemetry.md:284-285 should not say the decoded numbers mean the same thing in both SDKs; docs/specs/feature-usage-bit-registry.md:31-35 explicitly makes the bit tables language-specific.
- docs/decisions/0027-feature-usage-bitmask-user-agent.md:21-23 should point to the Markdown registry, not
docs/feature-usage-bit-registry.json, because docs/specs/feature-usage-bit-registry.md:208-210 says no machine-readable registry file ships in v1.
Automated review by eavanvalkenburg's agents
|
Flagged issue docs/specs/002-feature-usage-telemetry.md:273-277 defines Source: automated DevFlow PR review |
… decode - ADR option J: the parity test compares the enum against the per-language table in the registry doc, not a (now-removed) JSON file. - Spec .NET mapping: the wire format is shared, but the mask is decoded per-language (select the table via the UA product token) — fixes the "decoded numbers mean the same thing in both SDKs" wording that conflicted with the per-language, non-synchronized bit indexes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…E_MASK_DISABLED)
Re-introduce a dedicated opt-out that disables only the feature mask while keeping
the base agent-framework-<lang>/{version} User-Agent, alongside the existing
AGENT_FRAMEWORK_USER_AGENT_DISABLED (whole UA). Updates the spec accumulator gate,
API surface, opt-out table and examples; the registry opt-out section; and the
ADR (decision outcome, consequences, open questions -> decided).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add a Prior art section to ADR-0027 surveying how comparable SDKs encode identity/usage in the User-Agent or sidecar headers, with citations: - AWS botocore `m/` feature-code list — the direct analog (per-request, usage-based feature flags in the UA); contrasts short-code set vs our hex bitmask. - OpenAI/Anthropic Stainless `X-Stainless-*` headers (static identity). - Azure azure-core UserAgentPolicy + AZURE_TELEMETRY_DISABLED. - Google x-goog-api-client; LangSmith version token + tracing opt-in. Also add an Open Question on honoring the cross-tool DO_NOT_TRACK convention. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
eavanvalkenburg
temporarily deployed
to
integration
GitHub Actions
Inactive
eavanvalkenburg
temporarily deployed
to
integration
GitHub Actions
Inactive
botocore's m/ feature list scopes features to a per-request contextvars set that resets between calls — clean per-call attribution, but it assumes every feature lives inside a service request. That holds for an SDK natively bound to its own services; it does not for us, where many features (agent/workflow/provider construction, session setup) are not bound to any request. - ADR: add Accumulation scope options — P (process-global monotonic, chosen) vs Q (botocore per-request set, rejected) with the request-binding rationale; reference P in the decision; reframe the "no per-call attribution" limitation as a deliberate scope choice. - ADR Prior art: bitmask gives bounded token size for free (vs botocore's 1024-byte cap + truncation); mechanism is private, wire format is the contract; fix a duplicated phrase. - Spec: note the mask is process-global, monotonic, never reset (intentional, lock/Interlocked.Or-safe), the token is safe-by-construction (no sanitization), and the helpers are private API. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
eavanvalkenburg
temporarily deployed
to
integration
GitHub Actions
Inactive
eavanvalkenburg
temporarily deployed
to
integration
GitHub Actions
Inactive
Motivation and Context
We can see which Agent Framework packages are installed and that some framework call happened (via the existing
agent-framework-python/{version}User-Agent), but we have no usage-based signal about which features are actually exercised at runtime, nor which are used together (e.g. workflows + MCP + Foundry). This proposes a lightweight, privacy-respecting way to collect that signal for the traffic we can actually read, without standing up new event pipelines.Docs-only (ADR + design spec + registry); no code changes.
Description
Adds:
docs/decisions/0027-feature-usage-bitmask-user-agent.md— ADR (options-first, then decision), including Prior art, Limitations, Open Questions (privacy review is a blocking precondition), and a v1→v2 migration note.docs/specs/002-feature-usage-telemetry.md— design spec + implementation plan.docs/specs/feature-usage-bit-registry.md— per-language bit tables + governance.Key design decisions:
(feat=vN.<hex>)User-Agent comment, stamped per request on first-party (Azure/Foundry) clients only — never sent to third-party providers (no fingerprint leak into logs we can't read). OpenTelemetry emission is deferred, primarily on privacy grounds, left open behind the version prefix.FeatureBitenum is the source of truth (no codegen).AGENT_FRAMEWORK_FEATURE_MASK_DISABLED(drops only the mask, keeps the base SDK identity/version UA) and the existingAGENT_FRAMEWORK_USER_AGENT_DISABLED(drops the whole contribution).Prior art is surveyed in the ADR with citations — most relevant is AWS botocore's
m/User-Agent feature list, the closest production analog (per-request, usage-based feature codes in the UA). Key contrasts captured: our fixed-width hex bitmask gives bounded token size for free (botocore needs a 1024-byte cap + truncation), and botocore's per-requestcontextvarsscope works only because every feature there lives inside a service request — which is not true for us. Also surveyed: OpenAI/Anthropic StainlessX-Stainless-*headers, Azureazure-coreUserAgentPolicy, Googlex-goog-api-client, LangSmith, and the cross-toolDO_NOT_TRACKconvention.Open items for deciders are called out in the ADR (privacy review; if/when to add OTel; whether to also honor
DO_NOT_TRACK; and .NET maintainers confirming the .NET bit table).Contribution Checklist