Skip to content

start_debugging needs additional parameters for real-world .NET test framework execution #81

Description

@mikejohnstonPremierinc

Feature Request:

Problem

When bug #80 is fixed and createTestDebugConfig becomes reachable for coreclr, the auto-generated config will be:

args: ['test', '--filter', `FullyQualifiedName~${testName}`, '--no-build']

This is insufficient for real-world .NET test automation frameworks. Enterprise test projects require additional dotnet test parameters that have no corresponding start_debugging input:

Required Parameter Purpose Example
--settings <path> Specifies runsettings file (environment URL, credentials, timeouts) --settings RunSettings/project.qa.runsettings
Project path (.csproj) Specifies which test project to run (solutions contain multiple projects) dotnet test MyApp.UITest/MyApp.UITest.csproj
Additional --filter criteria Browser selection, fixture type, device exclusion --filter "Name=MyTest&TestCategory=Remote.chrome&FullyQualifiedName!~iPhone"
--environment / env vars Runtime environment variables needed by the test framework BROWSER=chrome, credential injection

Real-World Example

Here's what a real dotnet test command looks like for an enterprise NUnit/Selenium test framework:

dotnet test MyApp.UITest/MyApp.UITest.csproj \
  --filter "Name=LoginTest&TestCategory=Local.chrome&FullyQualifiedName!~iPhone" \
  --settings MyApp.UITest/RunSettings/myapp.qa.runsettings

Breaking this down:

  • MyApp.UITest.csproj - The solution has 4+ projects; only one is the test project
  • Name=LoginTest - The specific test method (this is what testName maps to today)
  • TestCategory=Local.chrome - Run with Local Chrome fixture only (not Remote, not Edge, not Firefox)
  • FullyQualifiedName!~iPhone - Exclude iPhone emulation device tests (Chrome has two variants: standard + iPhone emulation)
  • --settings ...qa.runsettings - Points to the QA environment config containing application URL, credentials, timeouts, and other environment-specific settings

Without these parameters, the test either:

  1. Fails immediately (no runsettings = no application URL to navigate to)
  2. Runs ALL browser/fixture combinations simultaneously (chrome + edge + remote + local = resource explosion)
  3. Runs against the wrong project or fails to find the test entirely

Proposed Solution

Add these parameters to start_debugging:

// Option A: Named parameters (better UX for AI agents)
start_debugging({
  fileFullPath: "path/to/TestFile.cs",
  testName: "LoginTest",
  workingDirectory: "path/to/workspace",
  projectPath: "MyApp.UITest/MyApp.UITest.csproj",        // NEW
  testSettings: "MyApp.UITest/RunSettings/myapp.qa.runsettings",  // NEW
  testFilterExpression: "Name=LoginTest&TestCategory=Local.chrome&FullyQualifiedName!~iPhone",  // NEW (overrides testName-based filter)
  environmentVariables: { "BROWSER": "chrome" }            // NEW
})

// Option B: Generic additionalArgs (simpler implementation, covers all cases)
start_debugging({
  fileFullPath: "path/to/TestFile.cs",
  testName: "LoginTest",
  workingDirectory: "path/to/workspace",
  additionalTestArgs: [                                    // NEW
    "--settings", "MyApp.UITest/RunSettings/myapp.qa.runsettings",
    "--", "NUnit.NumberOfTestWorkers=1"
  ],
  testProjectPath: "MyApp.UITest/MyApp.UITest.csproj"     // NEW
})

Option A is more explicit and easier for AI agents to reason about. Option B is more flexible and handles edge cases we haven't thought of.

Either way, the generated args array in createTestDebugConfig would become:

case 'coreclr':
    const args = ['test'];
    if (projectPath) args.push(projectPath);
    args.push('--filter', testFilterExpression || `FullyQualifiedName~${testName}`);
    if (testSettings) args.push('--settings', testSettings);
    args.push('--no-build');
    if (additionalTestArgs) args.push(...additionalTestArgs);
    return {
        type: 'coreclr', request: 'launch',
        name: `DebugMCP .NET Test: ${testName}`,
        program: 'dotnet',
        args: args,
        console: 'integratedTerminal',
        cwd: cwd,
        stopAtEntry: false,
        env: environmentVariables || {}
    };

Why This Matters for AI-Assisted Debugging

The value proposition of DebugMCP is autonomous AI debugging - the agent decides what to debug and does it without human intervention. For .NET test projects, an AI agent already knows (from project context):

  • Which .csproj contains the test
  • Which runsettings file to use (from prior test runs or project conventions)
  • Which browser/fixture to target (from the test fixture attributes)
  • What filter expression to construct

The agent has all this information. It just needs a way to PASS it to start_debugging. Without these parameters, the only option is the configurationName workaround (manually creating launch.json configs), which:

  • Requires per-project setup that doesn't scale
  • Has hardcoded paths that break across environments
  • Cannot dynamically target different tests without modifying files each time
  • Defeats the "zero configuration" promise of DebugMCP

Environment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions