Skip to content

Commit 07eaa94

Browse files
altimate-harness-bot[bot]saravmajesticsaravmajesticclaude
authored
feat: IDE-aware datamate transport, enabled-state persistence, and /mcps command (#893)
* feat: datamate stdio local transport + extension single-gateway mode - Add stdio transport support for datamate MCP (vs HTTP-only before) - Single-gateway mode: when .vscode/mcp.json has "datamate" key, always use it as server name — prevents duplicate tool sets from extension - syncDatamateUrlFromVscodeMcp: use updatedAt field as change signal for the "datamate" entry (works for both stdio and HTTP), URL comparison for all other remote entries - Strip ALTIMATE_EXTENSION_RPC from persisted mcp-discover configs to avoid stale socket paths across VS Code sessions - persistMcpEnabled: write enabled/disabled flag to disk on MCP connect/disconnect so it survives session restarts - Add /altimate/mcp/reload-datamate endpoint to re-sync and reconnect without full server restart - MCP.ToolsChanged subscription in prompt loop for traceability - Merge main: preserve trace consumer in serve.ts, restore exports on isAnthropicLikeModel and insertReminders * feat: [AI-6948] register mcps slash command in server command list (#885) Co-authored-by: saravmajestic <saravmajestic@altimate.ai> * fix: address review comments + stale config cache in reload-datamate + improve logging * refactor: replace hardcoded IDE paths with glob scan for mcp.json Instead of checking .vscode/mcp.json, .cursor/mcp.json, and .github/copilot/mcp.json separately, scan all mcp.json files under the project root and use the first one that contains a datamate entry. This is IDE-agnostic and will work with any editor that writes an mcp.json. - Remove IDE_MCP_SOURCES constant - Add findAllMcpJsonFiles() using Glob.scan(**/mcp.json) - Add extractServersMap() helper to try both "servers" and "mcpServers" keys - Update readDatamateTransportFromIde() and syncDatamateUrlFromVscodeMcp() to use the new scan approach * fix(mcps): bypass LLM for /mcps list, enable, disable commands Add direct handler in SessionPrompt.command for the mcps slash command. Instead of routing through the AI template, the handler calls MCP.status(), MCP.connect(), and MCP.disconnect() directly and returns a structured MessageV2.WithParts response, matching the TUI HTTP API behavior. - /mcps (no args): queries MCP.status() and renders a markdown table - /mcps enable <name>: calls MCP.connect(name) and reports result - /mcps disable <name>: calls MCP.disconnect(name) and confirms * fix(mcps): move /mcps bypass handler before Command.get() check The /mcps direct handler was added after the Command.get() call and "Command not found" error, so it never fired — /mcps was not in the registered commands list and would throw before reaching the bypass. Move the handler to run before Command.get() so it short-circuits cleanly for the built-in /mcps, /mcps enable, and /mcps disable commands without requiring a registered command entry. * fix(mcp-discover): strip enabled:false when explicitly adding server to config * fix(mcp-discover): set enabled:true when user explicitly adds server to config When the user asks to add an MCP server via /discover-and-add-mcps, the config entry should have enabled:true so /mcps shows it as connected. Previously the strip of the auto-discovery enabled:false left no enabled field, which caused ambiguity in the UI even though mcp/index.ts treats missing as enabled=true at runtime. * fix(mcp-discover): add updatedAt timestamp and connect MCP immediately after config write - Add updatedAt: new Date().toISOString() when writing MCP server to config so clients can detect config changes without restarting - Call MCP.connect(name) immediately after addMcpToConfig so the server becomes active in the current session — /mcps now shows connected without restart * chore: move isAnthropicLikeModel NOTE docblock back above function * chore: strip altimate_change markers from altimate/ subtree, inline DATAMATE_KEY * fix: address review comments — glob ignore, type guard, persist from disk, dedup respond() * fix: add model param to respond() helper — was undefined in closure causing no-reply on /mcps * fix: [AI-6948] discover MCPs from project root instead of git worktree `/discover-and-add-mcps` found no servers (and auto-discovery added none) for non-git projects: `Instance.worktree` resolves to "/" when there is no `.git`, so the scan looked under the filesystem root and missed the workspace's `.vscode/mcp.json` (datamate). The `add` action also resolved its project-scoped config path against "/". - `mcp-discover.ts`: use `Instance.directory` (project root) instead of `Instance.worktree` for the discovery scan, persisted-name read, and the project-scoped config write target - `config.ts`: same `Instance.worktree` -> `Instance.directory` for the background auto-discovery caller - `discover.ts`: replace the hardcoded IDE paths with a recursive `**/mcp.json` glob scan rooted at the project directory (IDE-agnostic, trying both `servers` and `mcpServers` keys), keeping the non-`mcp.json` sources (`.mcp.json`, `.gemini/settings.json`, `~/.claude.json`) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: [AI-6948] address PR review — preserve MCP config fields on disk writes Resolves the review findings on the `/mcps` enable/disable persistence and datamate transport sync paths. - `mcp/config.ts`: `readMcpEntryFromDisk` now uses `getNodeValue` instead of a manual children walk. `jsonc-parser` only populates `Node.value` for primitives, so the old walk silently dropped `command` arrays and `environment`/`headers` objects — corrupting local stdio (and remote-with- headers) entries on the next write via `persistMcpEnabled`. - `altimate/datamate-transport.ts`: same `getNodeValue` fix for the two sibling walkers; preserve non-transport fields (`enabled`, `timeout`, `oauth`, …) when rebuilding the synced datamate entry; widen the `**/mcp.json` ignore list (`target`, `.next`, `out`, `vendor`, `coverage`, `.venv`, `.turbo`). - `mcp/index.ts`: serialize `persistMcpEnabled` (promise chain) so concurrent enable/disable can't interleave read-modify-write and clobber each other. - `server/server.ts`: `reload-datamate` returns HTTP 500 (not 200) on error so the extension can distinguish failure. - `session/prompt.ts`: `/mcps enable|disable` validates the server name against config and reports clearly when it's unknown (was silent). - `test/mcp/config.test.ts`: add `readMcpEntryFromDisk` round-trip tests, including the persist-enabled flow that previously corrupted local entries. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: [AI-6948] preserve IDE source precedence in mcp.json glob scan The `**/mcp.json` glob scan sorted purely alphabetically, which let `.cursor/mcp.json` override `.vscode/mcp.json` for same-named servers — a regression vs the previous fixed-source order (.vscode > .cursor > .github/copilot). Caught by the `discover-adversarial` precedence test. Order globbed files by IDE precedence first, then alphabetically, restoring first-source-wins semantics while staying IDE-agnostic for any other mcp.json. Also align the ignore list with `datamate-transport.ts`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: [AI-6948] balance altimate_change markers (CI Marker Guard + integrity tests) Two pre-existing marker imbalances on the branch failed the "Require-markers regression backstop" job and the `altimate_change marker integrity` tests (origin/main is balanced; these were introduced by earlier merge commits on this branch): - `cli/cmd/serve.ts` (6 start / 4 end): close the trace-import region after its import, and close the branding region after the rebranded log line (the sync-datamate region stays nested inside it). - `session/prompt.ts` (43 start / 44 end): remove a duplicate `altimate_change end` left by a merge — the MCP-ToolsChanged region is already closed before `while (true)`, and the SessionStatus region right after has its own pair. Comment-only changes; no behavior impact. Verified with `bun run script/upstream/analyze.ts --require-markers --strict` (38/38) and the marker-integrity suites (118/0). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: saravmajestic <saravmajestic@altimate.ai> Co-authored-by: altimate-harness-bot[bot] <274995457+altimate-harness-bot[bot]@users.noreply.github.com> Co-authored-by: Saravanan <sarav.majestic@gmail.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent ae69109 commit 07eaa94

13 files changed

Lines changed: 906 additions & 34 deletions

File tree

Lines changed: 326 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,326 @@
1+
import { readFile } from "fs/promises"
2+
import path from "path"
3+
import { parseTree, findNodeAtLocation, getNodeValue } from "jsonc-parser"
4+
import { resolveConfigPath, addMcpToConfig, readMcpEntryFromDisk } from "../mcp/config"
5+
import { Filesystem } from "../util/filesystem"
6+
import { Glob } from "../util/glob"
7+
import { Log } from "../util/log"
8+
import type { Config } from "../config/config"
9+
10+
const log = Log.create({ service: "datamate-transport" })
11+
12+
export const DATAMATE_KEY = "datamate"
13+
14+
/**
15+
* Top-level keys that MCP config files use to map server name → entry.
16+
* VS Code 1.99+ uses "servers"; older VS Code and Cursor use "mcpServers".
17+
* We try both so the scan works regardless of which IDE wrote the file.
18+
*/
19+
const MCP_SERVERS_KEYS = ["servers", "mcpServers"] as const
20+
21+
22+
export type DatamateTransport =
23+
| { type: "remote"; url: string }
24+
| { type: "local"; command: string[] }
25+
26+
/**
27+
* Parse a single mcp.json file and return the servers map, trying each of the
28+
* known top-level key names in order.
29+
*/
30+
function extractServersMap(
31+
parsed: Record<string, unknown>,
32+
): Record<string, Record<string, unknown>> {
33+
for (const key of MCP_SERVERS_KEYS) {
34+
const candidate = parsed[key]
35+
if (candidate && typeof candidate === "object" && !Array.isArray(candidate)) {
36+
return candidate as Record<string, Record<string, unknown>>
37+
}
38+
}
39+
return {}
40+
}
41+
42+
/**
43+
* Find all mcp.json files under projectRootDir (excluding noise directories)
44+
* and return the paths sorted for deterministic ordering.
45+
*/
46+
async function findAllMcpJsonFiles(projectRootDir: string): Promise<string[]> {
47+
try {
48+
const paths = await Glob.scan("**/mcp.json", {
49+
cwd: projectRootDir,
50+
absolute: true,
51+
dot: true,
52+
// Exclude build/dependency/output trees. command + args from a discovered
53+
// mcp.json are passed to StdioClientTransport, so keep the scan to source the
54+
// user actually authors and out of vendored/generated directories.
55+
ignore: [
56+
"**/node_modules/**",
57+
"**/.git/**",
58+
"**/dist/**",
59+
"**/build/**",
60+
"**/.pnpm/**",
61+
"**/target/**",
62+
"**/.next/**",
63+
"**/out/**",
64+
"**/vendor/**",
65+
"**/coverage/**",
66+
"**/.venv/**",
67+
"**/.turbo/**",
68+
],
69+
})
70+
return paths.sort()
71+
} catch {
72+
log.warn("findAllMcpJsonFiles: glob scan failed", { cwd: projectRootDir })
73+
return []
74+
}
75+
}
76+
77+
/**
78+
* Scan all mcp.json files under projectRootDir and return the transport type
79+
* for the first "datamate" server entry found.
80+
*
81+
* Returns null if no mcp.json contains a "datamate" entry — the caller should
82+
* fall back to the cloud config.
83+
*
84+
* Reuses the exact command from the IDE config so altimate-code spawns the
85+
* same process the extension already started, rather than a second one.
86+
*/
87+
export async function readDatamateTransportFromIde(
88+
projectRootDir: string,
89+
): Promise<DatamateTransport | null> {
90+
const mcpJsonPaths = await findAllMcpJsonFiles(projectRootDir)
91+
92+
for (const mcpJsonPath of mcpJsonPaths) {
93+
const relPath = path.relative(projectRootDir, mcpJsonPath)
94+
try {
95+
const text = await readFile(mcpJsonPath, "utf-8")
96+
const parsed = JSON.parse(text) as Record<string, unknown>
97+
const serversMap = extractServersMap(parsed)
98+
const entry = serversMap[DATAMATE_KEY]
99+
if (!entry) continue
100+
101+
log.info("readDatamateTransportFromIde: found entry", {
102+
source: relPath,
103+
type: entry["type"] ?? "(no type)",
104+
})
105+
106+
if (typeof entry["url"] === "string") {
107+
return { type: "remote", url: entry["url"] }
108+
}
109+
110+
// stdio entry — reuse the exact command + args the extension registered
111+
const cmd = typeof entry["command"] === "string" ? entry["command"] : undefined
112+
const args = Array.isArray(entry["args"]) ? (entry["args"] as string[]) : []
113+
if (cmd) {
114+
return { type: "local", command: [cmd, ...args] }
115+
}
116+
117+
// Entry exists but has no usable command — treat as local marker
118+
return { type: "local", command: [DATAMATE_KEY, "start-stdio"] }
119+
} catch {
120+
log.warn("readDatamateTransportFromIde: failed to parse", { source: relPath })
121+
}
122+
}
123+
124+
log.info("readDatamateTransportFromIde: no IDE entry found, falling back to cloud config")
125+
return null
126+
}
127+
128+
/**
129+
* Sync the "datamate" entry (and other remote MCP entries) from the first
130+
* mcp.json that contains a "datamate" key to altimate-code.json.
131+
*
132+
* Uses `updatedAt` as the change signal for the datamate entry (covers both
133+
* stdio and HTTP transport), and URL comparison for all other remote entries.
134+
*
135+
* Fire-and-forget friendly: errors are logged but never thrown.
136+
* Returns the list of MCP server names whose config was updated on disk.
137+
*/
138+
export async function syncDatamateUrlFromVscodeMcp(cwd: string): Promise<string[]> {
139+
const updated: string[] = []
140+
try {
141+
log.info("syncDatamateUrlFromVscodeMcp: start", { cwd })
142+
143+
// Find the first mcp.json that contains a "datamate" entry.
144+
const mcpJsonPaths = await findAllMcpJsonFiles(cwd)
145+
let mcpJsonPath: string | undefined
146+
let serversMap: Record<string, Record<string, unknown>> = {}
147+
148+
for (const candidate of mcpJsonPaths) {
149+
try {
150+
const text = await readFile(candidate, "utf-8")
151+
const parsed = JSON.parse(text) as Record<string, unknown>
152+
const map = extractServersMap(parsed)
153+
if (map[DATAMATE_KEY]) {
154+
mcpJsonPath = candidate
155+
serversMap = map
156+
break
157+
}
158+
} catch {
159+
// Unparseable — skip
160+
}
161+
}
162+
163+
if (!mcpJsonPath) {
164+
log.info("syncDatamateUrlFromVscodeMcp: no mcp.json with datamate entry found, skipping sync")
165+
return updated
166+
}
167+
168+
log.info("syncDatamateUrlFromVscodeMcp: using config", {
169+
source: path.relative(cwd, mcpJsonPath),
170+
})
171+
172+
// ── "datamate" entry: sync by updatedAt (works for stdio + HTTP) ────────
173+
const datamateVscode = serversMap[DATAMATE_KEY]
174+
const vscodeUpdatedAt =
175+
datamateVscode && typeof datamateVscode["updatedAt"] === "string"
176+
? (datamateVscode["updatedAt"] as string)
177+
: undefined
178+
179+
if (datamateVscode && vscodeUpdatedAt) {
180+
const configPath = await resolveConfigPath(cwd)
181+
if (await Filesystem.exists(configPath)) {
182+
const configText = await Filesystem.readText(configPath)
183+
const existingTree = parseTree(configText)
184+
const existingNode = existingTree
185+
? findNodeAtLocation(existingTree, ["mcp", DATAMATE_KEY])
186+
: undefined
187+
188+
if (existingNode) {
189+
// getNodeValue reconstructs the full entry (a manual children walk reading
190+
// `prop.children[1].value` drops array/object fields — jsonc-parser only
191+
// populates `Node.value` for primitives).
192+
const existingEntry =
193+
existingNode.type === "object"
194+
? (getNodeValue(existingNode) as Record<string, unknown>)
195+
: {}
196+
const existingUpdatedAt =
197+
typeof existingEntry["updatedAt"] === "string" ? existingEntry["updatedAt"] : undefined
198+
199+
if (vscodeUpdatedAt === existingUpdatedAt) {
200+
log.info("syncDatamateUrlFromVscodeMcp: datamate entry already up to date", {
201+
updatedAt: vscodeUpdatedAt,
202+
})
203+
} else {
204+
// Preserve fields the IDE doesn't manage (enabled, timeout, oauth, …) by
205+
// carrying forward everything except the transport-identity fields, which
206+
// we re-derive below. IDE config uses "stdio"/"http"/"streamable-http"/"sse";
207+
// altimate-code.json uses "local"/"remote".
208+
const TRANSPORT_FIELDS = new Set([
209+
"type",
210+
"command",
211+
"args",
212+
"environment",
213+
"url",
214+
"updatedAt",
215+
])
216+
const preserved: Record<string, unknown> = {}
217+
for (const [k, v] of Object.entries(existingEntry)) {
218+
if (!TRANSPORT_FIELDS.has(k)) preserved[k] = v
219+
}
220+
221+
let newEntry: Record<string, unknown>
222+
if ("command" in datamateVscode) {
223+
const env = datamateVscode["env"] as Record<string, string> | undefined
224+
const { ALTIMATE_EXTENSION_RPC: _rpc, ...restEnv } = env ?? {}
225+
const cmd =
226+
typeof datamateVscode["command"] === "string"
227+
? (datamateVscode["command"] as string)
228+
: DATAMATE_KEY
229+
newEntry = {
230+
...preserved,
231+
type: "local",
232+
command: [cmd, ...((datamateVscode["args"] as string[]) ?? [])],
233+
...(Object.keys(restEnv).length > 0 ? { environment: restEnv } : {}),
234+
updatedAt: vscodeUpdatedAt,
235+
}
236+
} else {
237+
// http / streamable-http / sse → remote
238+
newEntry = {
239+
...preserved,
240+
type: "remote",
241+
url: datamateVscode["url"] as string,
242+
updatedAt: vscodeUpdatedAt,
243+
}
244+
}
245+
246+
await addMcpToConfig(
247+
DATAMATE_KEY,
248+
newEntry as Parameters<typeof addMcpToConfig>[1],
249+
configPath,
250+
)
251+
log.info("syncDatamateUrlFromVscodeMcp: datamate entry synced", {
252+
type: datamateVscode["type"],
253+
updatedAt: vscodeUpdatedAt,
254+
})
255+
updated.push(DATAMATE_KEY)
256+
}
257+
}
258+
}
259+
}
260+
261+
// ── All other remote MCP entries: existing URL-comparison logic ──────────
262+
const httpEntries: Array<{ key: string; url: string }> = []
263+
for (const [key, entry] of Object.entries(serversMap)) {
264+
if (key === DATAMATE_KEY) continue
265+
if (typeof entry["url"] === "string") {
266+
httpEntries.push({ key, url: entry["url"] })
267+
}
268+
}
269+
270+
if (httpEntries.length > 0) {
271+
const configPath = await resolveConfigPath(cwd)
272+
if (await Filesystem.exists(configPath)) {
273+
const configText = await Filesystem.readText(configPath)
274+
const tree = parseTree(configText)
275+
const mcpNode = tree ? findNodeAtLocation(tree, ["mcp"]) : undefined
276+
277+
if (tree && mcpNode && mcpNode.type === "object" && mcpNode.children) {
278+
const remoteMcpEntries: Array<{ name: string; url: string }> = []
279+
for (const child of mcpNode.children) {
280+
if (child.type !== "property" || !child.children) continue
281+
const nameNode = child.children[0]
282+
const valueNode = child.children[1]
283+
if (!nameNode || !valueNode || valueNode.type !== "object" || !valueNode.children) continue
284+
const typeNode = findNodeAtLocation(valueNode, ["type"])
285+
const urlNode = findNodeAtLocation(valueNode, ["url"])
286+
if (typeNode?.value === "remote" && typeof urlNode?.value === "string") {
287+
remoteMcpEntries.push({ name: nameNode.value as string, url: urlNode.value })
288+
}
289+
}
290+
291+
for (const remote of remoteMcpEntries) {
292+
const match = httpEntries.find((e) => e.key === remote.name)
293+
if (match && match.url !== remote.url) {
294+
const entryNode = findNodeAtLocation(tree, ["mcp", remote.name])
295+
if (!entryNode || entryNode.type !== "object") continue
296+
// getNodeValue preserves headers/oauth/timeout; a children walk reading
297+
// `prop.children[1].value` would strip them (object/array nodes).
298+
const entry = getNodeValue(entryNode) as Record<string, unknown>
299+
entry["url"] = match.url
300+
entry["updatedAt"] = new Date().toISOString()
301+
await addMcpToConfig(
302+
remote.name,
303+
entry as Parameters<typeof addMcpToConfig>[1],
304+
configPath,
305+
)
306+
log.info("syncDatamateUrlFromVscodeMcp: remote entry updated", {
307+
name: remote.name,
308+
oldUrl: remote.url,
309+
newUrl: match.url,
310+
})
311+
updated.push(remote.name)
312+
}
313+
}
314+
}
315+
}
316+
}
317+
318+
if (updated.length === 0) log.info("syncDatamateUrlFromVscodeMcp: no changes detected")
319+
} catch (err) {
320+
log.warn("syncDatamateUrlFromVscodeMcp: failed (non-fatal)", {
321+
error: err instanceof Error ? err.message : String(err),
322+
})
323+
}
324+
return updated
325+
}
326+

0 commit comments

Comments
 (0)