Skip to content

fix: MCP tools report only an exit code and discard the command output #1611

@ryzizub

Description

@ryzizub

Description

The test, packages_get, and packages_check_licenses MCP tools run the CLI in-process and surface only a synthesized line such as "test" failed with exit code 69. in the CallToolResult. The actual command output — failing test names, assertion messages, compile errors, coverage details — is written by the in-process mason Logger to the MCP server's stdout/stderr, which under the stdio transport is shared with the JSON-RPC message stream. Non-MCP lines on stdout violate the stdio transport contract (the server MUST NOT write anything to stdout that is not a valid MCP message) and are dropped by the client, so the agent only ever sees the opaque exit code and cannot diagnose the failure.

Exit code 69 (ExitCode.unavailable) is especially unhelpful: the test command collapses every failure mode — compile error, failed assertion, unmet --min-coverage, thrown exception — into that single code (lib/src/commands/test/test.dart, lib/src/cli/test_cli_runner.dart).

Steps To Reproduce

  1. Run the very_good_cli MCP test tool on a package whose tests fail (or fail to compile).
  2. Observe the tool result: "test" failed with exit code 69. with no failing test names, error messages, or compile output.

Expected Behavior

The tool result should include the captured command output (the real failure detail) in the CallToolResult content, with isError: true, so the agent can act on it. Per the MCP spec, tool-execution failures are reported inside the result with isError: true and the human-readable detail in content — not as a bare status line.

Additional Context

  • Lives in the same handler (_runToolCommand in lib/src/mcp/mcp_server.dart) as the directory bug (vgv-ai-flutter-plugin feat #70 doesn't work #1599 / fix(mcp): apply directory argument as the working directory #1600).
  • Related: fix: very_good_cli MCP 'test' tool hangs vgv-ai-flutter-plugin#94 — agents get stuck because the MCP test result is opaque.
  • Possible approaches:
    • Capture the in-process Logger output via an IOOverrides stdout/stderr redirect and include a bounded tail in content. Note: the Logger must be constructed inside the zone, since mason captures IOOverrides.current at construction time.
    • Or run the CLI as a subprocess with piped stdout/stderr (cleaner separation from the JSON-RPC stream, at the cost of in-process speed/testability).
    • Output should be size-bounded and scrubbed of secrets (e.g. --dart-define values, tokens in stack traces).
    • Optionally stream progress via the MCP logging capability (notifications/message) rather than the process stderr.

Metadata

Metadata

Assignees

Labels

bugSomething isn't working as expected

Type

No type
No fields configured for issues without a type.

Projects

Status
In Progress

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions