Skip to content

Add FileInput/FileOutput well-known types for file transfer between MCP tools#42

Open
birdayz wants to merge 2 commits into
mainfrom
jb/file-input-output
Open

Add FileInput/FileOutput well-known types for file transfer between MCP tools#42
birdayz wants to merge 2 commits into
mainfrom
jb/file-input-output

Conversation

@birdayz
Copy link
Copy Markdown
Contributor

@birdayz birdayz commented May 12, 2026

What

Add mcp.v1.FileInput and mcp.v1.FileOutput as well-known protobuf types with oneof source/destination for deployment-aware file transfer in MCP tools.

Why

MCP tools that move files need different transports depending on deployment topology. Inline bytes work for small files in hosted mode. Filesystem paths avoid context-window bloat in sandbox deployments. And S3 presigned URLs solve the MCP-to-MCP file transfer problem in production without a shared filesystem -- MCP A uploads to S3, emits a presigned GET URL, MCP B downloads directly, and the LLM just passes a URL string instead of carrying megabytes through its context window.

Without a shared abstraction every file-moving MCP reinvents this, and MCP-to-MCP file passing doesn't work at all in pure MCP (no sandbox) because the protocol has no mechanism for direct server-to-server transfer.

Implementation details

Proto shape

message S3Reference {
  string presigned_url = 1;
}

message FileInput {
  oneof source {
    bytes content = 1;     // inline (hosted, small files)
    string path = 2;       // filesystem (sandbox)
    S3Reference s3 = 3;    // presigned URL (cloud-native)
  }
  string filename = 4;
  string mime_type = 5;
}

message FileOutput {
  oneof destination {
    bytes content = 1;
    string path = 2;
    S3Reference s3 = 3;
  }
  string filename = 4;
  string mime_type = 5;
  int64 size_bytes = 6;
}

The oneof makes the transport explicit at the proto level. Common metadata (filename, mime_type, size_bytes) sits outside the oneof -- always present regardless of transport.

Generator (pkg/gen/schema.go)

Recognizes mcp.v1.FileInput, mcp.v1.FileOutput, and mcp.v1.S3Reference by FQN in messageFieldSchema() -- same mechanism as google.protobuf.Timestamp. Emits JSON Schemas with an x-mcp-file-mode extension marker that the runtime uses to locate file-typed properties.

Runtime (pkg/runtime/file_mode.go)

Four modes via WithFileMode(mode) registration option:

  • FileModeInline -- exposes only the content oneof variant (hosted)
  • FileModePath -- exposes only path (sandbox)
  • FileModeS3 -- exposes only s3 (cloud-native)
  • FileModeAll -- all variants visible, agent picks (hybrid)

Schema rewriting strips non-matching oneof variants at registration time. The x-mcp-file-mode marker is removed from the final schema.

Framework contract

MCP handlers always produce/consume raw bytes. The framework layer (in the consumer repo) intercepts and rewrites to the configured transport:

Handler returns FileOutput{content: <bytes>}
         |
    Framework intercepts
         |
   ┌─────┼─────┐
   v     v     v
 inline path   S3
        write  upload, mint
        to     presigned
        disk   GET URL

Handlers never touch S3 or the filesystem directly.

S3 presigned URL flow

S3Reference carries a presigned URL. The framework (not the handler) mints it:

  • Output: framework uploads bytes to a scratch bucket, returns a presigned GET URL
  • Input: framework downloads from the presigned URL, passes bytes to the handler

Presigned URLs are time-limited and scoped to a single object -- no IAM cross-wiring between MCPs. This makes MCP-to-MCP file transfer work without the LLM carrying bytes through its context window.

S3 wiring is not yet implemented in any consumer -- this PR defines the proto shape and schema machinery so the framework can wire it when ready.

References

First consumer: SharePoint managed MCP in cloudv2 (https://github.com/redpanda-data/cloudv2/pull/26291).

birdayz and others added 2 commits May 12, 2026 19:59
… rewriting

MCP tools that move files need different schemas depending on whether
the deployment shares a filesystem between aigw and the agent (sandbox)
or not (hosted). Rather than each MCP reinventing this, define it once.

The generator recognizes mcp.v1.FileInput and mcp.v1.FileOutput by FQN
(same mechanism as google.protobuf.Timestamp) and emits schemas with an
x-mcp-file-mode marker. At registration time, WithFileMode(mode) rewrites
the schema:

  - FileModeInline: strips file_path, requires content (hosted)
  - FileModePath:   strips content, requires file_path (sandbox)
  - FileModeAll:    keeps both, agent picks (hybrid)

When no mode is set, the marker stays and all fields are exposed.

The google/* testdata changes are just buf pulling newer BSR versions
during regeneration -- no functional change there.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…iant

The flat field layout forced runtime schema rewriting to strip individual
fields. The oneof pattern is cleaner -- each transport (inline bytes,
filesystem path, S3 presigned URL) is a distinct variant, and the runtime
exposes only the one matching the deployment mode.

The S3Reference type carries a presigned URL. The framework (not the MCP
handler) is responsible for uploading/downloading from S3 and minting
the URLs. This means MCP handlers always produce raw bytes and never
touch object storage directly -- the framework intercepts and rewrites
to the configured transport.

Four modes:
  - FileModeInline: only content variant (hosted, small files)
  - FileModePath:   only path variant (sandbox, shared filesystem)
  - FileModeS3:     only s3 variant (cloud-native, presigned URLs)
  - FileModeAll:    all variants exposed, agent picks

S3 wiring is not yet implemented in any consumer -- this commit defines
the proto shape and schema machinery so the framework layer in cloudv2
can wire it when ready.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Comment on lines +45 to +46
// Path on the shared filesystem (sandbox mode).
string path = 2;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need to validate the path so that nobody references wrong location like local fs?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants