Skip to content

feat(contrib/mark3labs/mcp-go): skip telemetry injection for UI-only tools#4942

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 1 commit into
mainfrom
jb/mcp-skip-ui-only-tools
Jul 3, 2026
Merged

feat(contrib/mark3labs/mcp-go): skip telemetry injection for UI-only tools#4942
gh-worker-dd-mergequeue-cf854d[bot] merged 1 commit into
mainfrom
jb/mcp-skip-ui-only-tools

Conversation

@jboolean

@jboolean jboolean commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

This PR stack ports functionality added in the clone of this contrib in dd-source back to dd-trace-go.

Summary

The intent-capture ListTools hook unconditionally adds telemetry as a required field to every tool's input schema. That breaks tools annotated as UI-only (_meta.ui.visibility excluding "model"): they are invoked by the app UI, not by the model, so the UI caller has no intent to supply. OpenAI's MCP client strictly validates tool arguments against the advertised schema and rejects every UI-driven call as a result.

This change adds an isModelCallable helper that inspects *mcp.Meta.AdditionalFields["ui"]["visibility"] (handling both []string and []any shapes from JSON round-trips) and short-circuits the injection when the model is not in the visibility list. Tools with no _meta.ui.visibility continue to default to model-callable per the MCP Apps spec.

Ported from internal dd-source PR #420911.

Test plan

  • TestIntentCaptureSkipsUIOnlyTools — three tools (model-only, UI-only [app], dual [app, model]); asserts telemetry is injected for model and dual but not for UI-only.
  • Existing intent-capture tests still pass.

Stacked on #4941.

MCP standard reference

The _meta.ui.visibility field is part of the official MCP Apps extension. Values: ["model", "app"] (default), ["model"], or ["app"] — they control which actor may invoke the tool. UI-only tools (["app"]) are invoked by the host shell, never by the model. This PR's skip logic respects that standard.


Parallel port in the modelcontextprotocol/go-sdk contrib: #4949

@datadog-prod-us1-3

datadog-prod-us1-3 Bot commented Jun 24, 2026

Copy link
Copy Markdown

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: 21.74%
Overall Coverage: 62.78% (+11.47%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 2ff4a7b | Docs | Datadog PR Page | Give us feedback!

@pr-commenter

pr-commenter Bot commented Jun 24, 2026

Copy link
Copy Markdown

Benchmarks

Benchmark execution time: 2026-07-01 16:38:29

Comparing candidate commit 2ff4a7b in PR branch jb/mcp-skip-ui-only-tools with baseline commit 2de53a9 in branch jb/mcp-intent-from-context.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 321 metrics, 2 unstable metrics, 1 flaky benchmarks without significant changes.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

Known flaky benchmarks

These benchmarks are marked as flaky and will not trigger a failure. Modify FLAKY_BENCHMARKS_REGEX to control which benchmarks are marked as flaky.

Known flaky benchmarks without significant changes:

  • scenario:BenchmarkOTLPTraceWriterFlush

Comment thread contrib/mark3labs/mcp-go/intent_capture.go Outdated
@jboolean jboolean force-pushed the jb/mcp-intent-from-context branch from 0f308e2 to 301b597 Compare June 24, 2026 14:43
@jboolean jboolean force-pushed the jb/mcp-skip-ui-only-tools branch from d0e5464 to 0d1c005 Compare June 24, 2026 14:43
@jboolean jboolean requested a review from kitfre June 24, 2026 14:46
@jboolean jboolean force-pushed the jb/mcp-skip-ui-only-tools branch from 0d1c005 to a8261f9 Compare June 24, 2026 15:08
@jboolean jboolean force-pushed the jb/mcp-intent-from-context branch from 301b597 to ce524bf Compare June 24, 2026 15:09
@jboolean jboolean marked this pull request as ready for review June 24, 2026 15:17
@jboolean jboolean requested review from a team as code owners June 24, 2026 15:17
@jboolean jboolean requested review from rarguelloF and removed request for a team June 24, 2026 15:17

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a8261f9663

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread contrib/mark3labs/mcp-go/tool_meta.go Outdated
if meta == nil {
return true
}
uiMap, ok := meta.AdditionalFields["ui"].(map[string]any)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Handle typed UI metadata before defaulting callable

When a server constructs the same valid _meta.ui.visibility wire format with a typed Go value, such as a struct or map[string][]string, this assertion falls through and treats the tool as model-callable. mcp.Meta.AdditionalFields accepts arbitrary values and will marshal those typed values correctly, so app-only tools declared that way still get required telemetry injected in tools/list, leaving the UI calls rejected for the scenario this change is meant to fix.

Useful? React with 👍 / 👎.

@jboolean jboolean force-pushed the jb/mcp-intent-from-context branch from ce524bf to ec14544 Compare June 24, 2026 15:28
@jboolean jboolean force-pushed the jb/mcp-skip-ui-only-tools branch 2 times, most recently from a850ceb to cb69328 Compare June 24, 2026 16:48
@jboolean jboolean force-pushed the jb/mcp-intent-from-context branch from ec14544 to 0c444d5 Compare June 24, 2026 16:48
@codecov

codecov Bot commented Jun 24, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (jb/mcp-intent-from-context@0c444d5). Learn more about missing BASE report.

Additional details and impacted files
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d Bot merged commit 2ec11b5 into main Jul 3, 2026
916 of 956 checks passed
@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d Bot deleted the jb/mcp-skip-ui-only-tools branch July 3, 2026 19:31
gh-worker-dd-mergequeue-cf854d Bot added a commit that referenced this pull request Jul 3, 2026
… time (#4943)

This PR stack ports functionality added in the clone of this contrib in dd-source back to dd-trace-go.

## Summary

Pass `llmobs.WithSessionID` at `StartToolSpan` time so LLMObs groups tool spans under their MCP session natively. The session is read from `server.ClientSessionFromContext(ctx)` if present.

The existing `mcp_session_id` tag is unchanged — they are complementary: the tag exposes the session ID to span search/filtering, while `WithSessionID` populates the LLMObs session field used for trace grouping.

## Provenance

No single dd-source PR — the change landed as part of a series of LLMObs-session changes. The current dd-source middleware lives at [`mcp-go/tracing.go#L36-L44`](https://github.com/DataDog/dd-source/blob/d10d624c374b69aa02168983be725532a38562b1/domains/mcp_services/libs/go/mcp/pre-release-external/dd-trace-go/contrib/mark3labs/mcp-go/tracing.go#L36-L44):

```go
var toolHandlerMiddleware = func(next server.ToolHandlerFunc) server.ToolHandlerFunc {
    return func(ctx context.Context, request mcp.CallToolRequest) (*mcp.CallToolResult, error) {
        // Note: WithIntegration is not yet in released llmobs package and cannot be used here
        var startOpts []llmobs.StartSpanOption
        if session := server.ClientSessionFromContext(ctx); session != nil {
            startOpts = append(startOpts, llmobs.WithSessionID(session.SessionID()))
        }
        toolSpan, ctx := llmobs.StartToolSpan(ctx, request.Params.Name, startOpts...)
```

This PR ports that exact pattern, additionally keeping the existing `llmobs.WithIntegration(...)` call since the released llmobs package now has it.

## Test plan

- [x] Extended `TestIntegrationToolCallSuccess` to assert `toolSpan.SessionID` matches the MCP client session.
- [x] Existing tracing tests still pass.

Stacked on #4942.

Co-authored-by: jboolean <julian.boilen@datadoghq.com>
gh-worker-dd-mergequeue-cf854d Bot added a commit that referenced this pull request Jul 6, 2026
…-only tools (#4949)

This PR stack ports functionality added in the clone of this contrib in dd-source back to dd-trace-go (Rapid clone at `domains/api_platform/libs/go/mcp/tracing`).

## Summary

Skips telemetry/intent injection for tools whose `_meta.ui.visibility` excludes `"model"`. UI-only tools cannot be invoked by the model, so a required `telemetry` parameter would be useless noise.

`_meta.ui.visibility` is part of the MCP Apps extension (see https://github.com/modelcontextprotocol/modelcontextprotocol/blob/main/SEP-1865.md). Absent visibility list ⇒ model-callable (spec default).

## What this enables in dd-source

Same behavior already shipping in the Rapid clone — UI-only tools registered through dd-source's MCP server are silently skipped today; this PR brings that to the public contrib. Source PR: DataDog/dd-source#420911

## Test plan

- [x] `TestIntentCaptureSkipsUIOnlyTools` — three tools (no meta, ui-only, dual-visibility): only the UI-only one is skipped.
- [x] All existing intent-capture tests still pass.

Stacked on #4948.


---

Parallel port in the mark3labs/mcp-go contrib: #4942


Co-authored-by: jboolean <julian.boilen@datadoghq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

apm:ecosystem contrib/* related feature requests or bugs mergequeue-status: done

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants