Skip to content

[UPDATE PRIMITIVE] Report all validation errors at once instead of one-at-a-time#227

Merged
data-douser merged 5 commits intomainfrom
copilot/report-all-validation-errors
Apr 7, 2026
Merged

[UPDATE PRIMITIVE] Report all validation errors at once instead of one-at-a-time#227
data-douser merged 5 commits intomainfrom
copilot/report-all-validation-errors

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 7, 2026

📝 Update Information

Primitive Details

  • Type: Tool (all tools — applies server-wide)
  • Name: Tool input validation (McpServer.validateToolInput)
  • Update Category: Bug Fix

⚠️ CRITICAL: PR SCOPE VALIDATION

ALLOWED FILES:

  • Server implementation files (server/src/**/*.ts)
  • Modified supporting library files (server/src/lib/*.ts)
  • Updated or new test files (server/test/**/*.ts)

🚫 FORBIDDEN FILES: None included.


🛑 MANDATORY PR VALIDATION CHECKLIST

  • ONLY server implementation files are included
  • NO temporary or output files are included
  • NO unrelated configuration files are included
  • ALL existing tests continue to pass
  • NEW functionality is properly tested

  • Impact Scope: Extensive — affects validation for every registered tool

Update Metadata

  • Breaking Changes: No
  • API Compatibility: Enhanced — error messages are now more informative
  • Performance Impact: Neutral

🎯 Changes Description

Current Behavior

The MCP SDK's getParseErrorMessage() extracts only the first Zod issue when validation fails. Missing three required fields forces three round-trips:

1. {} → "must have required property 'owner'"
2. {owner} → "must have required property 'repo'"
3. {owner, repo} → "must have required property 'sourceLocation'"

Updated Behavior

All violations are reported in one response:

{} → "must have required properties: 'owner', 'repo', 'sourceLocation'"

Mixed error types are semicolon-separated:

{count: "abc"} → "must have required property 'owner'; count: Expected number, received string"

Motivation

The SDK's getParseErrorMessage() checks error.message first (a raw JSON blob from ZodError) then falls back to error.issues[0] — either way only one issue is surfaced. Zod already collects all errors; the bottleneck is formatting.

🔄 Before vs. After Comparison

Functionality Changes

// BEFORE (SDK default): only first issue surfaced
getParseErrorMessage(error) {
  if ('message' in error) return error.message;        // raw JSON blob
  if ('issues' in error)  return error.issues[0].message; // single issue
}

// AFTER: all issues aggregated into human-readable message
formatAllValidationErrors(error: ZodError): string {
  // Groups missing-required into: "must have required properties: 'a', 'b'"
  // Appends type/custom errors: "field: Expected number, received string"
  // Joins with "; "
}

API Changes

No schema changes. The McpError thrown on validation failure now contains an aggregated message instead of a single-issue message. The error code (InvalidParams / -32602) is unchanged.

Output Format Changes

// BEFORE (3 separate tool calls needed):
{"error": {"code": -32602, "message": "...must have required property 'owner'"}}

// AFTER (single tool call):
{"error": {"code": -32602, "message": "...must have required properties: 'owner', 'repo', 'sourceLocation'"}}

🧪 Testing & Validation

Test Coverage Updates

  • Existing Tests: All 1300 tests pass across 59 files
  • New Test Cases: 19 tests for the new module
  • Regression Tests: E2E tests via InMemoryTransport verify full MCP protocol flow
  • Edge Case Tests: Empty issues, nested paths, mixed error types, single vs plural, unrecognized schemas

Validation Scenarios

  1. Backward Compatibility: Valid tool calls produce identical results; only error formatting changes
  2. New Functionality: Multiple missing fields → single aggregated message
  3. Error Handling: Type errors, custom refinements, nested object paths all formatted correctly
  4. Performance: No measurable impact — same Zod safeParseAsync call, different message formatting

Test Results

  • Unit Tests: All pass (1300/1300)
  • Integration Tests: 3 new E2E tests via InMemoryTransport
  • Manual Testing: Validated with realistic tool schemas
  • Performance Testing: No regressions detected

📋 Implementation Details

Files Modified

  • Core Implementation: server/src/lib/tool-validation.ts (new, 133 lines)
  • Server Entry: server/src/codeql-development-mcp-server.ts (+5 lines)
  • Tests: server/test/src/lib/tool-validation.test.ts (new, 19 tests)

Code Changes Summary

  • Error Handling: Aggregates all Zod issues into one clear message
  • Input Validation: Handles raw Zod shapes, ZodObject, and ZodEffects schemas

Dependencies

  • No New Dependencies: Uses existing zod and @modelcontextprotocol/sdk exports

🔍 Quality Improvements

Bug Fixes (if applicable)

  • Issue: Validation errors reported one-at-a-time, forcing iterative trial-and-error
  • Root Cause: SDK's getParseErrorMessage() returns error.issues[0] despite Zod collecting all issues
  • Solution: Override validateToolInput on the server instance with aggregated error formatting
  • Prevention: The patch is applied at server startup, covering all current and future tools

Code Quality Enhancements

  • Readability: Clear separation of error formatting, schema resolution, and patching
  • Testability: formatAllValidationErrors exported for direct unit testing
  • Reusability: resolveZodSchema handles all schema types the SDK supports

🔗 References

Related Issues/PRs

🚀 Compatibility & Migration

Backward Compatibility

  • Fully Compatible: No breaking changes — only error message formatting improves

API Evolution

  • Better Error Messages: All violations in one response instead of one-at-a-time
  • Maintained Contracts: Same error code, same McpError type, same protocol behavior

👥 Review Guidelines

For Reviewers

  • ⚠️ SCOPE COMPLIANCE: 3 source files only
  • ⚠️ NO UNRELATED FILES: Clean diff
  • ⚠️ BACKWARD COMPATIBILITY: Only error message text changes; valid calls unaffected

Testing Instructions

npm run build-and-test    # Full suite
cd server && npm test     # Server unit tests (1300 tests)
cd server && npx vitest run test/src/lib/tool-validation.test.ts  # Targeted

📊 Impact Assessment

Server Impact

  • Startup Time: Negligible — one function assignment
  • Runtime Stability: No impact — same validation logic, better formatting
  • Resource Usage: No change
  • Concurrent Usage: Safe — stateless formatting function

AI Assistant Impact

  • Enhanced Accuracy: LLMs can fix all missing params in one retry instead of N retries
  • Improved Reliability: Fewer wasted tool invocations on validation discovery

Copilot AI and others added 2 commits April 7, 2026 01:21
- Create server/src/lib/tool-validation.ts with formatAllValidationErrors()
  and patchValidateToolInput() that overrides the SDK's one-at-a-time error
  reporting
- Integrate patchValidateToolInput into server startup
- Add comprehensive unit tests in server/test/src/lib/tool-validation.test.ts

Agent-Logs-Url: https://github.com/advanced-security/codeql-development-mcp-server/sessions/306c50e7-31e0-4a39-bba5-9cacb4dd1674

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
Copilot AI changed the title [WIP] Add feature to report all validation errors at once [UPDATE PRIMITIVE] Report all validation errors at once instead of one-at-a-time Apr 7, 2026
Copilot AI requested a review from data-douser April 7, 2026 01:29
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 7, 2026

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Snapshot Warnings

⚠️: No snapshots were found for the head SHA 0e71d73.
Ensure that dependencies are being submitted on PR branches and consider enabling retry-on-snapshot-warnings. See the documentation for more information and troubleshooting advice.

Scanned Files

None

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR enhances MCP tool input validation so that Zod validation failures report all issues in a single, human-readable error message (instead of only the first issue), applied server-wide by patching the server instance’s validateToolInput.

Changes:

  • Adds server/src/lib/tool-validation.ts to aggregate Zod issues into one message and patch McpServer.validateToolInput.
  • Applies the patch during server startup in server/src/codeql-development-mcp-server.ts.
  • Adds unit + E2E (InMemoryTransport) tests for aggregated validation messaging and updates the built bundle.
Show a summary per file
File Description
server/src/lib/tool-validation.ts New validation formatter + server instance patch to aggregate Zod issues.
server/src/codeql-development-mcp-server.ts Calls patchValidateToolInput(server) during startup to enable the behavior server-wide.
server/test/src/lib/tool-validation.test.ts New unit + E2E tests covering missing required fields, mixed errors, nested paths, and protocol behavior.
server/dist/codeql-development-mcp-server.js Rebuilt bundle reflecting the new validation behavior.

Copilot's findings

  • Files reviewed: 3/5 changed files
  • Comments generated: 3

@data-douser data-douser marked this pull request as ready for review April 7, 2026 02:40
@data-douser data-douser requested review from a team and enyil as code owners April 7, 2026 02:40
Copilot AI review requested due to automatic review settings April 7, 2026 02:40
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves server-wide MCP tool input validation by overriding the MCP SDK’s default validateToolInput behavior to report all Zod validation issues in a single error message, reducing iterative retry loops for missing/invalid parameters.

Changes:

  • Added aggregated Zod issue formatting (formatAllValidationErrors) and an instance patch (patchValidateToolInput) to override McpServer.validateToolInput.
  • Applied the patch during server startup so it affects all registered tools consistently.
  • Added unit and E2E (InMemoryTransport) tests to verify aggregated error reporting through both direct calls and the MCP protocol.
Show a summary per file
File Description
server/src/lib/tool-validation.ts New module implementing aggregated validation error formatting and the validateToolInput instance patch.
server/src/codeql-development-mcp-server.ts Applies the validation patch at server startup to enable server-wide behavior change.
server/test/src/lib/tool-validation.test.ts Adds unit + E2E tests validating aggregated errors and patched behavior.
server/dist/codeql-development-mcp-server.js Rebuilt bundle including the new validation behavior.

Copilot's findings

  • Files reviewed: 3/5 changed files
  • Comments generated: 2

@data-douser data-douser merged commit 1ffd305 into main Apr 7, 2026
22 checks passed
@data-douser data-douser deleted the copilot/report-all-validation-errors branch April 7, 2026 02:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Report all validation errors at once instead of one-at-a-time

3 participants