feat(launcher): local launcher for HTTP/SSE upstreams (spec 046)

[electrolobzik]

Summary

Implements spec 046 — let mcpproxy spawn the upstream's local process before connecting via HTTP / SSE / streamable-HTTP, instead of only stdio. Decouples launcher (how the process starts) from transport (how the protocol bytes flow) so {command, url, protocol: "http"|"sse"|"streamable-http"} is a first-class config combo.

See specs/046-local-launcher-for-http-sse/plan.md (already on this branch) for the design rationale, codebase reconnaissance, gotchas, and definition of done. specs/046-local-launcher-for-http-sse/execution_log.md documents the deviations from the plan and the bugs found during verification.

What's new for users

{
  "name": "local-http-mcp",
  "protocol": "http",
  "url": "http://127.0.0.1:9999/mcp",
  "command": "node",
  "args": ["./server.js", "--port", "9999"],
  "working_dir": "/path/to/repo",
  "launcher_wait_timeout": "15s",
  "enabled": true
}

mcpproxy now: spawns node, polls 127.0.0.1:9999 via TCP-dial until it accepts, then runs the HTTP transport handshake. Disconnect / restart / disable / shutdown reap the child with SIGTERM → grace → SIGKILL. Child stdout+stderr land in the per-server log, so mcpproxy upstream logs <name> shows them.

docs/configuration.md has the full behaviour matrix including the back-compat note: under protocol: "auto", command still wins over url and forces stdio — you must set protocol explicitly to opt into the launcher.

What's new internally

New package internal/upstream/launcher/ — owns the child's lifecycle:

• Spec / Handle / Spawn(ctx, spec, log) (Handle, error). Stop is idempotent and blocks until the child is reaped, so callers can rely on "Stop returned → port is free."
• WaitForURL(ctx, url, timeout) — TCP-dial polling, not http.Get (SSE GETs stream forever — gotcha Enhance configuration management and signal handling in mcpproxy #2 in the plan). Infers default ports for http/https/ws/wss.
• Unix: child placed in its own pgroup via Setpgid: true, so signals reach grandchildren spawned by sh -c … and docker run …. Windows: best-effort stubs matching the existing process_windows.go TODO.

Wiring:

• internal/upstream/core/connection.go — pre-transport launcher dispatch for http/sse/streamable-http when Command != "". Stdio is excluded because mcp-go's Stdio transport spawns through its own CommandFunc; running the launcher there would double-spawn.
• internal/upstream/core/connection_launcher.go — connectWithLauncher, stopLauncher, watchLauncher, buildLauncherCmd. Reuses the existing setupDockerIsolation, wrapWithUserShell, injectEnvVarsIntoDockerArgs, insertCidfileIntoShellDockerCommand helpers so Docker isolation lives in one place (gotcha Remove unused isDatabaseOpen method from Manager struct to streamline code and improve maintainability. #4).
• internal/upstream/core/connection_lifecycle.go — stopLauncher is called after the MCP-client close in Disconnect, so the child sees the network drop before SIGTERM.
• internal/config/{config,merge}.go — ServerConfig.LauncherWaitTimeout Duration (default 30s), copied by CopyServerConfig.

Deviation from plan: Phase 0 originally contemplated rerouting stdio through launcher.Spawn as well. That would require patching mcp-go (the Stdio transport spawns via its own CommandFunc and there's no way to feed in an externally-spawned child). To honor the spirit — "Docker-isolation logic must live in one place" — both paths share the command-prep helpers instead. Stdio behaviour is unchanged. See execution_log.md for the full reasoning.

Test plan

• go vet ./internal/upstream/launcher/... ./internal/upstream/core/... ./internal/config/... — clean.
• go test ./internal/upstream/launcher/... — 15/15 pass. Covers immediately-bound / late-bound / never-bound / ctx-canceled / bad-URL WaitForURL, graceful SIGTERM exit, SIGKILL fallback after StopGrace, natural-exit code capture, idempotent Stop under concurrent callers, LogSink capture, and an integration test that combines Spawn + WaitForURL with a python subprocess.
• go test ./internal/upstream/... ./internal/config/... — all green. TestClient_Connect_SSE_NotSupported surfaced a deadlock in the connect-failure cleanup that was fixed before commit.
• E2E lifecycle test: scripts/test-api-e2e.sh now builds a small in-tree HTTP MCP fixture (test/launcher-server) and runs six sub-assertions via the REST API — tools/list reaches the launched server, pgrep finds the child, restart yields a new PID, disable reaps within 8s, enable respawns with a fresh PID, per-server log captures the launcher banner. Verified end-to-end on a macOS developer host (all six pass with real PIDs).
• Manual smoke test with a real Python HTTP MCP server (out-of-tree helper) — Successfully connected and initialized reaches mcpproxy upstream logs, repeated tools/list polls succeed, restart/disable/enable cycles reap and respawn cleanly on a macOS developer host.
• go build -tags server ./cmd/mcpproxy — sandbox-blocked locally (CDN policy); CI will confirm.
• go test -race ./internal/upstream/launcher/... — sandbox can't install gcc; CI will confirm.

Out of scope / future work

• {port} templating in args/url for ephemeral ports.
• Per-launcher custom health probes (currently always TCP-dial).
• Exponential backoff for repeated launcher crashes (currently relies on the transport-level reconnect path).
• Replace the integration test's python3 -c … with a Go test-binary helper in internal/upstream/launcher/testdata/.
• Windows: real Job-Objects support for grandchild cleanup (currently best-effort like the existing stdio path).
• Improve connectHTTP's auth-fallback ladder so a non-401 transport failure doesn't surface as "OAuth authentication required" (separate concern; predates this PR).

---

🤖 Generated with Claude Code

[Dumbris]

Thanks Roman, this is a really nice piece of work. 🙏

The launcher/transport decoupling is the right factoring, and {command, url, protocol: http|sse|streamable-http} becoming a first-class config combo removes a whole category of "I forgot to start the server" friction. The TCP-dial readiness gate + SIGTERM → grace → SIGKILL reap, per-server launcher_wait_timeout, and routing child stdout/stderr into the per-server log so mcpproxy upstream logs <name> Just Works are all the right calls. Spec/plan/execution-log discipline is excellent too.

Threat-model-wise this is a UX expansion, not a security expansion: the launcher reuses the same ServerConfig.Command surface that stdio already trusts, Docker isolation gates apply the same way, and mcpproxy itself only TCP-dials the configured URL.

CI is red on TestSaveServerSyncFieldCoverage (the reflection guard tripped because LauncherWaitTimeout wasn't whitelisted) and on the OpenAPI artifact-drift check (oas/swagger.yaml + oas/docs.go not regenerated). Since the org-owned fork blocks maintainer pushes, I'm landing this as-is and shipping the test/OAS fix in an immediate follow-up PR so main goes green again within minutes. Authorship of every commit stays yours.

Merging. Thank you again — this is a really useful feature.

[Dumbris]

Quick heads-up @electrolobzik — while pushing the test fix in #454, the CI on Linux runners caught a second issue I'd missed in the original review: TestSpawn_LogSinkCaptured flakes because reap() calls cmd.Wait() before pumpWG.Wait(), and per exec.Cmd docs Wait closes the parent-side pipes — so the pump scanners can race against that close and lose buffered child output. Locally everything passes because timing favors the pumps; the slower CI runners hit it.

Flipped the order in #454 (pumps drain first, then Wait). 20x with -race green locally. Just FYI so you have the fix in your mental model — no action needed.

Thanks again for the feature.