fix: dedup dependency auto-apply to prevent duplicate tool mints (Gap #10)

Targeted assistant pushes minted duplicate dashboard tools when bootstrap
pull stored an existing dashboard tool under a name-slugged state key
(e.g. `end-call-67aea057`) instead of the user's original local key. The
exact-key lookup in `ensureToolExists` / `ensureStructuredOutputExists`
missed and POSTed a fresh duplicate. Each subsequent targeted push
repeated the cycle, accumulating dashboard orphans.

This adds a name-based dedup check between the exact-key short-circuit
and the create path, in two layers:

  1. State-side: scan state for any key whose `extractBaseSlug` matches
     the local payload's slugified name (handles bootstrap-renamed keys).
  2. Dashboard-side: lazy-fetch the live `/tool` (and `/structured-output`)
     list once per push and check for a remote resource with the same
     canonical name.

When >1 distinct UUID matches the same name (real on-dashboard duplicates
from prior bug runs), pick the lex-smallest UUID for stable adoption,
warn naming the loser UUIDs, and point at `npm run cleanup`. Never mint
another duplicate.

Adoption flow:
  - Re-key state to the adopted UUID under the local resourceId.
  - Drop other state keys pointing at the same UUID and mark them
    touched, so a subsequent full push doesn't orphan-delete the
    adopted dashboard resource (Stack J / mergeScoped flushes the
    deletion).
  - Route through `applyTool`/`applyStructuredOutput` so the local
    payload PATCHes the dashboard with the standard drift-check flow,
    instead of recording a fake `lastPushedHash` that would silently
    drop a locally-edited dependency.

Tests: 12 unit tests for `findExistingResourceByName` covering state-only,
dashboard-only, both-agree, ambiguous (state-vs-state, state-vs-dashboard),
no-name, exact-key-excluded, no-match. All 114 suites pass.

Refs: improvements.md §10

Author: dhruva-vapi

src/dep-dedup.ts

@@ -0,0 +1,137 @@
+// Gap #10 — dependency deduplication helpers.
+//
+// On a targeted assistant push, `ensureToolExists` / `ensureStructuredOutputExists`
+// previously skipped auto-create only when (a) the dep id was UUID-shaped,
+// (b) `state.tools[depId]` was an exact key match, or (c) we'd already
+// auto-applied this id in the current run. Bootstrap pull (pull.ts:269-273)
+// stores resources under `<slug>-<uuid8>` keys (e.g. `end-call-67aea057`),
+// so a local file referencing `b2b-invoice-end-call` would miss the exact-key
+// check and POST a duplicate dashboard tool. Repeated targeted pushes
+// accumulated orphans on the dashboard.
+//
+// This module's helpers detect those collisions BEFORE create:
+//   - `findExistingResourceByName` — match local payload's canonical name
+//     against existing state entries (renamed/state-only keys) and the live
+//     dashboard list. Returns the UUID to adopt, plus an `ambiguous` flag
+//     when multiple distinct UUIDs share the same name (real on-dashboard
+//     duplicates from prior bug runs — the caller should warn and surface
+//     the loser UUIDs so a follow-up `npm run cleanup` can prune them).
+//
+// NOTE on duplication: `slugify` and `extractBaseSlug` here mirror the
+// definitions in `src/pull.ts` (lines ~253-278). pull.ts imports config.ts,
+// which calls `parseEnvironment()` at module load and `process.exit(1)`s
+// without a CLI env arg — making it impossible to import in a unit test.
+// This module imports ONLY from `./types.ts` so it stays testable in
+// isolation. Five lines duplicated is the right tradeoff; do not "DRY"
+// these back into pull.ts.
+
+import type { ResourceState } from "./types.ts";
+
+export interface RemoteResource {
+  id: string;
+  name?: string;
+  function?: { name?: string };
+}
+
+export interface DedupMatch {
+  uuid: string;
+  source: "state" | "dashboard" | "both";
+  ambiguous: boolean;
+  // Other distinct UUIDs we saw under the same canonical name. Empty when
+  // `ambiguous` is false. Caller should surface these in a warning so the
+  // user can run `npm run cleanup` to prune the duplicates.
+  duplicateUuids: string[];
+}
+
+export function slugify(name: string): string {
+  return name
+    .toLowerCase()
+    .replace(/[^a-z0-9]+/g, "-")
+    .replace(/^-+|-+$/g, "")
+    .replace(/-+/g, "-");
+}
+
+export function extractBaseSlug(resourceId: string): string {
+  const match = resourceId.match(/^(.*)-([a-f0-9]{8})$/i);
+  return match?.[1] ?? resourceId;
+}
+
+// Pulls the canonical name from a tool / SO / assistant payload.
+// For tools: `function.name` is the canonical name (per pull.ts:261-267).
+// For SOs / assistants / etc.: top-level `name`. Top-level wins when both
+// are present.
+export function extractResourceName(
+  payload: Record<string, unknown>,
+): string | undefined {
+  if (typeof payload.name === "string" && payload.name) return payload.name;
+  const fn = payload.function as Record<string, unknown> | undefined;
+  if (fn && typeof fn.name === "string" && fn.name) return fn.name;
+  return undefined;
+}
+
+// Find an existing resource (in state or dashboard) whose canonical name
+// matches the local payload's canonical name. Used by ensureToolExists /
+// ensureStructuredOutputExists to avoid creating a duplicate when bootstrap
+// pull has already stored the same dashboard resource under a different
+// state key.
+//
+// `localResourceId` is the key the engine WANTS to use in state. If a state
+// entry already exists under that exact key, the caller short-circuits BEFORE
+// calling this — so we exclude it here as a safety belt.
+//
+// Tiebreaker for >1 distinct UUID matching the same slugified name (i.e.
+// real on-dashboard duplicates from prior bug runs): pick lexically smallest
+// UUID for stable, deterministic adoption. Set `ambiguous=true` and surface
+// the loser UUIDs in `duplicateUuids` so the caller can warn.
+export function findExistingResourceByName(args: {
+  localResourceId: string;
+  localPayload: Record<string, unknown>;
+  stateSection: Record<string, ResourceState>;
+  remoteList?: RemoteResource[];
+}): DedupMatch | undefined {
+  const localName = extractResourceName(args.localPayload);
+  if (!localName) return undefined;
+  const localSlug = slugify(localName);
+
+  // uuid -> set of source labels (state:<key>, dashboard:<id>)
+  const matches = new Map<string, Set<string>>();
+
+  for (const [stateKey, entry] of Object.entries(args.stateSection)) {
+    if (stateKey === args.localResourceId) continue;
+    if (extractBaseSlug(stateKey) === localSlug) {
+      const set = matches.get(entry.uuid) ?? new Set<string>();
+      set.add(`state:${stateKey}`);
+      matches.set(entry.uuid, set);
+    }
+  }
+
+  for (const remote of args.remoteList ?? []) {
+    const remoteName =
+      (typeof remote.name === "string" && remote.name) || remote.function?.name;
+    if (!remoteName) continue;
+    if (slugify(remoteName) === localSlug) {
+      const set = matches.get(remote.id) ?? new Set<string>();
+      set.add(`dashboard:${remote.id}`);
+      matches.set(remote.id, set);
+    }
+  }
+
+  if (matches.size === 0) return undefined;
+
+  const sorted = [...matches.keys()].sort();
+  const winner = sorted[0]!;
+  const winnerSources = matches.get(winner)!;
+  const hasState = [...winnerSources].some((s) => s.startsWith("state:"));
+  const hasDashboard = [...winnerSources].some((s) =>
+    s.startsWith("dashboard:"),
+  );
+  const source: DedupMatch["source"] =
+    hasState && hasDashboard ? "both" : hasState ? "state" : "dashboard";
+
+  return {
+    uuid: winner,
+    source,
+    ambiguous: matches.size > 1,
+    duplicateUuids: sorted.slice(1),
+  };
+}

src/push.ts

@@ -16,6 +16,10 @@ import { summarizeFindings, validateResources } from "./validate.ts";
 import { checkDriftForUpdate } from "./drift.ts";
 import { writeSnapshot } from "./snapshot.ts";
 import { mergeScoped } from "./state-merge.ts";
+import {
+  findExistingResourceByName,
+  type RemoteResource,
+} from "./dep-dedup.ts";
 
 // Map a resource label to its state-file key. Used for snapshotting (Stack H)
 // — snapshot directories are keyed by the same names the state file uses.
@@ -827,6 +831,66 @@ interface DependencyContext {
   autoApplied: Set<string>;
   autoAppliedTools: ResourceFile<Record<string, unknown>>[];
   autoAppliedStructuredOutputs: ResourceFile<Record<string, unknown>>[];
+  // Stack — Gap #10 dedup. Lazy-fetched at most once per push when an
+  // auto-apply path needs to verify the tool/SO doesn't already exist on
+  // the dashboard under a different state key. `undefined` = not yet
+  // fetched; an empty array means "we tried, dashboard returned 0".
+  existingRemoteTools?: RemoteResource[];
+  existingRemoteStructuredOutputs?: RemoteResource[];
+  // Touched sets — adoption needs to mark touched so scoped state-merge
+  // (#15 / Stack J) persists the adopted UUID under the local resourceId.
+  touched: TouchedSets;
+}
+
+// Gap #10 — lazy-fetch the live dashboard tool list once per push. Used by
+// `ensureToolExists` to detect bootstrap-renamed entries (e.g. local key
+// `b2b-invoice-end-call` vs state key `end-call-67aea057`) and live
+// dashboard duplicates from prior bug runs before minting another POST.
+async function getExistingRemoteTools(
+  ctx: DependencyContext,
+): Promise<RemoteResource[]> {
+  if (ctx.existingRemoteTools !== undefined) return ctx.existingRemoteTools;
+  if (DRY_RUN) {
+    // Honor dry-run: no API calls. Empty list = "no dashboard match
+    // possible" → falls through to existing create-skip log.
+    ctx.existingRemoteTools = [];
+    return ctx.existingRemoteTools;
+  }
+  try {
+    const remote = await fetchAllResources("tools");
+    ctx.existingRemoteTools = remote as unknown as RemoteResource[];
+  } catch (err) {
+    console.warn(
+      `   ⚠️  Could not fetch dashboard tools for dedup check: ${
+        err instanceof Error ? err.message : String(err)
+      }. Falling back to state-only dedup.`,
+    );
+    ctx.existingRemoteTools = [];
+  }
+  return ctx.existingRemoteTools;
+}
+
+async function getExistingRemoteStructuredOutputs(
+  ctx: DependencyContext,
+): Promise<RemoteResource[]> {
+  if (ctx.existingRemoteStructuredOutputs !== undefined)
+    return ctx.existingRemoteStructuredOutputs;
+  if (DRY_RUN) {
+    ctx.existingRemoteStructuredOutputs = [];
+    return ctx.existingRemoteStructuredOutputs;
+  }
+  try {
+    const remote = await fetchAllResources("structuredOutputs");
+    ctx.existingRemoteStructuredOutputs = remote as unknown as RemoteResource[];
+  } catch (err) {
+    console.warn(
+      `   ⚠️  Could not fetch dashboard structured outputs for dedup check: ${
+        err instanceof Error ? err.message : String(err)
+      }. Falling back to state-only dedup.`,
+    );
+    ctx.existingRemoteStructuredOutputs = [];
+  }
+  return ctx.existingRemoteStructuredOutputs;
 }
 
 async function ensureToolExists(
@@ -843,6 +907,73 @@ async function ensureToolExists(
   const tool = ctx.allTools.find((t) => t.resourceId === toolId);
   if (!tool) return;
 
+  // Gap #10 dedup — before creating, check if a state entry under a
+  // different key (bootstrap-generated like `end-call-<uuid8>`) or a
+  // live dashboard tool already represents this same logical tool.
+  // Adopt instead of mint a duplicate.
+  const remoteList = await getExistingRemoteTools(ctx);
+  const match = findExistingResourceByName({
+    localResourceId: toolId,
+    localPayload: tool.data as Record<string, unknown>,
+    stateSection: ctx.state.tools,
+    remoteList,
+  });
+  if (match) {
+    if (match.ambiguous) {
+      const payload = tool.data as Record<string, unknown>;
+      const fn = payload.function as Record<string, unknown> | undefined;
+      const displayName =
+        (typeof fn?.name === "string" && fn.name) ||
+        (typeof payload.name === "string" && payload.name) ||
+        toolId;
+      console.warn(
+        `  ⚠️  Multiple dashboard tools share the name "${displayName}" — adopting ${match.uuid} (lex-smallest). Other UUIDs: ${match.duplicateUuids.join(", ")}. Run \`npm run cleanup -- ${VAPI_ENV}\` to prune duplicates.`,
+      );
+    }
+    console.log(
+      `  🔁 Reusing existing tool: ${toolId} → ${match.uuid} (matched via ${match.source})`,
+    );
+
+    // Re-key state to point at the adopted UUID under the local resourceId.
+    // No hash yet — applyTool below will PATCH with the local payload and
+    // record the post-PATCH hash, exercising the standard drift-check flow.
+    upsertState(ctx.state.tools, tool.resourceId, { uuid: match.uuid });
+
+    // Orphan-deletion guard — drop other state keys pointing at the SAME uuid
+    // so a subsequent full push doesn't see them as "tracked but no local file"
+    // and DELETE the dashboard resource we just adopted. Mark them touched
+    // so mergeScoped (Stack J) flushes the deletion. Do NOT touch entries
+    // pointing at match.duplicateUuids — those are SEPARATE dashboard duplicates
+    // and `npm run cleanup` is the right tool for those.
+    for (const [staleKey, entry] of Object.entries(ctx.state.tools)) {
+      if (staleKey !== tool.resourceId && entry.uuid === match.uuid) {
+        delete ctx.state.tools[staleKey];
+        ctx.touched.tools.add(staleKey);
+      }
+    }
+
+    // Now PATCH the dashboard with the local payload. applyTool's
+    // upsertResourceWithStateRecovery branch picks PATCH because state.tools
+    // now has existingUuid set. Drift check fires (no-baseline → log + proceed
+    // when lastPulledHash is undefined; full check when it isn't).
+    try {
+      const uuid = await applyTool(tool, ctx.state);
+      ctx.autoApplied.add(`tools:${toolId}`);
+      if (!uuid) return;
+      upsertState(ctx.state.tools, tool.resourceId, {
+        uuid,
+        lastPushedHash: hashPayload(tool.data),
+      });
+      ctx.applied.tools++;
+      ctx.autoAppliedTools.push(tool);
+      ctx.touched.tools.add(tool.resourceId);
+    } catch (error) {
+      console.error(formatApiError(toolId, error));
+      throw error;
+    }
+    return;
+  }
+
   console.log(`  📦 Auto-applying dependency → tool: ${toolId}`);
   try {
     const uuid = await applyTool(tool, ctx.state);
@@ -854,6 +985,7 @@ async function ensureToolExists(
     });
     ctx.applied.tools++;
     ctx.autoAppliedTools.push(tool);
+    ctx.touched.tools.add(tool.resourceId);
   } catch (error) {
     console.error(formatApiError(toolId, error));
     throw error;
@@ -876,6 +1008,72 @@ async function ensureStructuredOutputExists(
   );
   if (!output) return;
 
+  // Gap #10 dedup — same as ensureToolExists but against the SO state
+  // section + dashboard SO list.
+  const remoteList = await getExistingRemoteStructuredOutputs(ctx);
+  const match = findExistingResourceByName({
+    localResourceId: outputId,
+    localPayload: output.data as Record<string, unknown>,
+    stateSection: ctx.state.structuredOutputs,
+    remoteList,
+  });
+  if (match) {
+    if (match.ambiguous) {
+      const payload = output.data as Record<string, unknown>;
+      const displayName =
+        (typeof payload.name === "string" && payload.name) || outputId;
+      console.warn(
+        `  ⚠️  Multiple dashboard structured outputs share the name "${displayName}" — adopting ${match.uuid} (lex-smallest). Other UUIDs: ${match.duplicateUuids.join(", ")}. Run \`npm run cleanup -- ${VAPI_ENV}\` to prune duplicates.`,
+      );
+    }
+    console.log(
+      `  🔁 Reusing existing structured output: ${outputId} → ${match.uuid} (matched via ${match.source})`,
+    );
+
+    // Re-key state to point at the adopted UUID under the local resourceId.
+    // No hash yet — applyStructuredOutput below will PATCH with the local
+    // payload and record the post-PATCH hash, exercising the standard
+    // drift-check flow.
+    upsertState(ctx.state.structuredOutputs, output.resourceId, {
+      uuid: match.uuid,
+    });
+
+    // Orphan-deletion guard — drop other state keys pointing at the SAME uuid
+    // so a subsequent full push doesn't see them as "tracked but no local file"
+    // and DELETE the dashboard resource we just adopted. Mark them touched
+    // so mergeScoped (Stack J) flushes the deletion. Do NOT touch entries
+    // pointing at match.duplicateUuids — those are SEPARATE dashboard duplicates
+    // and `npm run cleanup` is the right tool for those.
+    for (const [staleKey, entry] of Object.entries(ctx.state.structuredOutputs)) {
+      if (staleKey !== output.resourceId && entry.uuid === match.uuid) {
+        delete ctx.state.structuredOutputs[staleKey];
+        ctx.touched.structuredOutputs.add(staleKey);
+      }
+    }
+
+    // Now PATCH the dashboard with the local payload. applyStructuredOutput's
+    // upsertResourceWithStateRecovery branch picks PATCH because
+    // state.structuredOutputs now has existingUuid set. Drift check fires
+    // (no-baseline → log + proceed when lastPulledHash is undefined; full
+    // check when it isn't).
+    try {
+      const uuid = await applyStructuredOutput(output, ctx.state);
+      ctx.autoApplied.add(`structuredOutputs:${outputId}`);
+      if (!uuid) return;
+      upsertState(ctx.state.structuredOutputs, output.resourceId, {
+        uuid,
+        lastPushedHash: hashPayload(output.data),
+      });
+      ctx.applied.structuredOutputs++;
+      ctx.autoAppliedStructuredOutputs.push(output);
+      ctx.touched.structuredOutputs.add(output.resourceId);
+    } catch (error) {
+      console.error(formatApiError(outputId, error));
+      throw error;
+    }
+    return;
+  }
+
   console.log(`  📦 Auto-applying dependency → structured output: ${outputId}`);
   try {
     const uuid = await applyStructuredOutput(output, ctx.state);
@@ -887,6 +1085,7 @@ async function ensureStructuredOutputExists(
     });
     ctx.applied.structuredOutputs++;
     ctx.autoAppliedStructuredOutputs.push(output);
+    ctx.touched.structuredOutputs.add(output.resourceId);
   } catch (error) {
     console.error(formatApiError(outputId, error));
     throw error;
@@ -1178,6 +1377,7 @@ async function main(): Promise<void> {
     autoApplied,
     autoAppliedTools,
     autoAppliedStructuredOutputs,
+    touched,
   };
 
   // Determine which types to check for orphaned deletions