Skip to content

fix: Migrate MCPB build to esbuild with ESM compatibility fixes #360

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wonderwhy-er wants to merge 1 commit into from
Closed

fix: Migrate MCPB build to esbuild with ESM compatibility fixes #360

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
163 changes: 111 additions & 52 deletions scripts/build-mcpb.cjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,12 @@
* Build script for creating Desktop Commander MCPB bundle
*
* This script:
* 1. Builds the TypeScript project
* 2. Creates a bundle directory structure
* 3. Generates a proper MCPB manifest.json
* 4. Copies the built server and dependencies
* 5. Uses mcpb CLI to create the final .mcpb bundle
* 1. Uses esbuild to create a single-file ESM bundle
* 2. Post-processes the bundle to fix esbuild ESM compatibility issues
* 3. Creates a bundle directory structure
* 4. Generates a proper MCPB manifest.json
* 5. Copies native dependencies that can't be bundled
* 6. Uses mcpb CLI to create the final .mcpb bundle
*/

const fs = require('fs');
Expand All @@ -31,73 +32,136 @@ try {
process.exit(1);
}

// Step 1: Build the TypeScript project
console.log('📦 Building TypeScript project...');
// Step 1: Clean and create bundle directory
console.log('🧹 Cleaning bundle directory...');
if (fs.existsSync(BUNDLE_DIR)) {
fs.rmSync(BUNDLE_DIR, { recursive: true });
}
fs.mkdirSync(BUNDLE_DIR, { recursive: true });

// Step 2: Bundle with esbuild into a single ESM file
// Native modules (sharp, ripgrep) and CJS-only packages (pdf2md, pdf-lib) are kept external
console.log('📦 Bundling with esbuild...');
try {
execSync('npm run build', { cwd: PROJECT_ROOT, stdio: 'inherit' });
console.log('✅ TypeScript build completed');
const distDir = path.join(BUNDLE_DIR, 'dist');
fs.mkdirSync(distDir, { recursive: true });

const externals = ['sharp', '@vscode/ripgrep', 'vscode', '@opendocsg/pdf2md', 'pdf-lib'];
const externalFlags = externals.map(e => `--external:${e}`).join(' ');
execSync(`npx esbuild src/index.ts --bundle --platform=node --target=node18 --format=esm --outfile=mcpb-bundle/dist/index.js ${externalFlags}`, { cwd: PROJECT_ROOT, stdio: 'inherit' });
console.log('✅ esbuild bundling completed');

// Step 2b: Post-process the bundle to fix esbuild ESM compatibility issues
console.log('🔧 Post-processing bundle for ESM compatibility...');
const bundlePath = path.join(BUNDLE_DIR, 'dist', 'index.js');
let bundleContent = fs.readFileSync(bundlePath, 'utf8');

// Fix 1: Inject ESM compatibility shims
// esbuild's ESM format wraps CJS require() calls in a shim that throws at runtime.
// We inject createRequire so Node.js built-ins (fs, path, child_process, etc.) work.
// Also inject __dirname/__filename which CJS modules expect but ESM doesn't provide.
const shebanLine = '#!/usr/bin/env node';
const esmShims = [
shebanLine,
'import { createRequire as __bundleCreateRequire } from "module";',
'import { fileURLToPath as __bundleFileURLToPath } from "url";',
'import { dirname as __bundleDirname } from "path";',
'var require = __bundleCreateRequire(import.meta.url);',
'var __filename = __bundleFileURLToPath(import.meta.url);',
'var __dirname = __bundleDirname(__filename);',
].join('\n');
if (bundleContent.startsWith(shebanLine)) {
bundleContent = bundleContent.replace(shebanLine, esmShims);
} else {
bundleContent = esmShims.replace(shebanLine + '\n', '') + '\n' + bundleContent;
}
console.log(' ✅ Injected ESM compatibility shims (createRequire, __dirname, __filename)');

// Fix 2: Propagate async to __esm callbacks that contain await
// esbuild bug: when module A has top-level await and module B imports from A,
// esbuild correctly makes A's __esm callback async but fails to make B's async too.
// This causes "SyntaxError: Unexpected reserved word" for await in non-async functions.
// Fix: scan all __esm blocks and add async to any that contain await in their body.
let fixCount = 0;
const esmBlockRegex = /var (\w+) = __esm\(\{\n(\s+)(async )?"([^"]+)"\(\)/g;
const fixes = [];
let match;
while ((match = esmBlockRegex.exec(bundleContent)) !== null) {
if (match[3]) continue; // Already async
const modName = match[4];
const blockStart = match.index;
const nextInit = bundleContent.indexOf('\nvar init_', blockStart + match[0].length);
const blockEnd = nextInit !== -1 ? nextInit : blockStart + 10000;
const blockBody = bundleContent.slice(blockStart, blockEnd);
if (blockBody.includes('await ')) {
fixes.push({
position: match.index + match[0].indexOf(`"${modName}"()`),
oldText: `"${modName}"()`,
newText: `async "${modName}"()`
});
}
}
Comment on lines +89 to +103
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot Feb 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Magic fallback blockStart + 10000 can silently miss await in the last/large __esm block.

If the final __esm block in the bundle has no subsequent \nvar init_ marker and its body exceeds 10,000 characters, the blockBody slice will be truncated and any await past that point won't be detected — leaving a non-async callback that will throw at runtime.

Consider using bundleContent.length as the fallback instead:

Proposed fix 
-        const blockEnd = nextInit !== -1 ? nextInit : blockStart + 10000;
+        const blockEnd = nextInit !== -1 ? nextInit : bundleContent.length;
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@scripts/build-mcpb.cjs` around lines 89 - 103, The code uses a magic fallback
blockEnd = blockStart + 10000 when no nextInit is found, which can truncate the
final __esm block and miss await checks; update the logic in the loop that uses
esmBlockRegex and bundleContent so that when nextInit === -1 you use
bundleContent.length (e.g., blockEnd = bundleContent.length) instead of
blockStart + 10000, ensuring blockBody contains the full remaining content
before checking for 'await' and pushing fixes into the fixes array.

for (let i = fixes.length - 1; i >= 0; i--) {
const fix = fixes[i];
bundleContent = bundleContent.slice(0, fix.position) + fix.newText + bundleContent.slice(fix.position + fix.oldText.length);
fixCount++;
}
if (fixCount > 0) {
console.log(` ✅ Fixed ${fixCount} non-async __esm callbacks containing await`);
}

// Fix 3: Remove the isMainModule auto-start block from device.ts
// device.ts has a module-level guard: if (import.meta.url === process.argv[1]) { device.start() }
// In the single-file bundle, import.meta.url always matches process.argv[1], so this
// incorrectly starts the remote device on every run, outputting debug messages to stdout
// which breaks Claude Desktop's JSON-RPC transport.
const isMainModulePattern = /var isMainModule = process\.argv\[1\][\s\S]*?if \(isMainModule\) \{[\s\S]*?\n\}/;
if (bundleContent.match(isMainModulePattern)) {
bundleContent = bundleContent.replace(isMainModulePattern, '// [MCPB] Removed isMainModule auto-start block (not applicable in bundle context)');
console.log(' ✅ Removed isMainModule auto-start block from device.ts');
}

fs.writeFileSync(bundlePath, bundleContent);
console.log('✅ Post-processing completed');
} catch (error) {
console.error('❌ TypeScript build failed:', error.message);
console.error('❌ esbuild bundling/post-processing failed:', error.message);
process.exit(1);
}

// Step 2: Clean and create bundle directory
if (fs.existsSync(BUNDLE_DIR)) {
fs.rmSync(BUNDLE_DIR, { recursive: true });
}fs.mkdirSync(BUNDLE_DIR, { recursive: true });

// Step 3: Read package.json for version and metadata
const packageJson = JSON.parse(fs.readFileSync(path.join(PROJECT_ROOT, 'package.json'), 'utf8'));

// Step 4: Load and process manifest template
console.log('📝 Processing manifest template...');

const manifestTemplatePath = path.join(PROJECT_ROOT, 'manifest.template.json');
console.log(`📄 Using manifest: manifest.template.json`);

let manifestTemplate;
try {
manifestTemplate = fs.readFileSync(manifestTemplatePath, 'utf8');
} catch (error) {
console.error('❌ Failed to read manifest template:', manifestTemplatePath);
process.exit(1);
}

// Replace template variables
const manifestContent = manifestTemplate.replace('{{VERSION}}', packageJson.version);

// Parse and validate the resulting manifest
let manifest;
try {
manifest = JSON.parse(manifestContent);
} catch (error) {
console.error('❌ Invalid JSON in manifest template:', error.message);
process.exit(1);
}

// Write manifest
fs.writeFileSync(MANIFEST_PATH, JSON.stringify(manifest, null, 2));
console.log('✅ Created manifest.json');
// Step 5: Copy necessary files
const filesToCopy = [
'dist',
'package.json',
'README.md',
'LICENSE',
'PRIVACY.md',
'icon.png'
];

// Step 5: Copy necessary files (dist is generated by esbuild, not copied)
const filesToCopy = ['package.json', 'README.md', 'LICENSE', 'PRIVACY.md', 'icon.png'];
filesToCopy.forEach(file => {
const srcPath = path.join(PROJECT_ROOT, file);
const destPath = path.join(BUNDLE_DIR, file);

if (fs.existsSync(srcPath)) {
if (fs.statSync(srcPath).isDirectory()) {
// Copy directory recursively
fs.cpSync(srcPath, destPath, { recursive: true });
} else {
// Copy file
fs.copyFileSync(srcPath, destPath);
}
console.log(`✅ Copied ${file}`);
Expand All @@ -106,22 +170,26 @@ filesToCopy.forEach(file => {
}
});

// Step 6: Create package.json in bundle with production dependencies from main package.json
// This ensures MCPB bundle always has the same dependencies as the npm package
// Step 6: Create bundle package.json with only native/CJS-only dependencies
// Everything else is bundled by esbuild into the single dist/index.js file
const bundlePackageJson = {
name: manifest.name,
version: manifest.version,
description: manifest.description,
type: "module", // Required for ESM - without this, Node.js defaults to CommonJS and shows warnings
type: "module",
main: "dist/index.js",
author: manifest.author,
license: manifest.license,
repository: manifest.repository,
dependencies: packageJson.dependencies // Use dependencies directly from package.json
dependencies: {
"sharp": packageJson.dependencies.sharp,
"@vscode/ripgrep": packageJson.dependencies["@vscode/ripgrep"],
"@opendocsg/pdf2md": packageJson.dependencies["@opendocsg/pdf2md"],
"pdf-lib": packageJson.dependencies["pdf-lib"]
}
};

fs.writeFileSync(
path.join(BUNDLE_DIR, 'package.json'),
path.join(BUNDLE_DIR, 'package.json'),
JSON.stringify(bundlePackageJson, null, 2)
);

Expand All @@ -142,28 +210,19 @@ try {
const ripgrepBinDest = path.join(BUNDLE_DIR, 'node_modules/@vscode/ripgrep/bin');
const ripgrepWrapperSrc = path.join(PROJECT_ROOT, 'scripts/ripgrep-wrapper.js');
const ripgrepIndexDest = path.join(BUNDLE_DIR, 'node_modules/@vscode/ripgrep/lib/index.js');

// Ensure bin directory exists
if (!fs.existsSync(ripgrepBinDest)) {
fs.mkdirSync(ripgrepBinDest, { recursive: true });
}

// Copy all platform-specific ripgrep binaries
const binaries = fs.readdirSync(ripgrepBinSrc).filter(f => f.startsWith('rg-'));
binaries.forEach(binary => {
const src = path.join(ripgrepBinSrc, binary);
const dest = path.join(ripgrepBinDest, binary);
fs.copyFileSync(src, dest);
// Set executable permissions (for development/testing)
// Note: Zip archives don't preserve permissions reliably, so ripgrep-wrapper.js
// also sets permissions at runtime to ensure compatibility after extraction
if (!binary.endsWith('.exe')) {
fs.chmodSync(dest, 0o755);
}
});
console.log(`✅ Copied ${binaries.length} ripgrep binaries`);

// Replace index.js with our wrapper
fs.copyFileSync(ripgrepWrapperSrc, ripgrepIndexDest);
console.log('✅ Installed ripgrep runtime wrapper');
} catch (error) {
Expand All @@ -180,10 +239,10 @@ try {
console.error('❌ Manifest validation failed:', error.message);
process.exit(1);
}

// Step 8: Pack the bundle
console.log('📦 Creating .mcpb bundle...');
const outputFile = path.join(PROJECT_ROOT, `${manifest.name}-${manifest.version}.mcpb`);

try {
execSync(`npx @anthropic-ai/mcpb pack "${BUNDLE_DIR}" "${outputFile}"`, { stdio: 'inherit' });
console.log('✅ MCPB bundle created successfully!');
Expand All @@ -205,4 +264,4 @@ console.log('');
console.log('To submit to Anthropic directory:');
console.log('- Ensure privacy policy is accessible at the GitHub URL');
console.log('- Complete destructive operation annotations (✅ Done)');
console.log('- Submit via Anthropic desktop extensions interest form');
console.log('- Submit via Anthropic desktop extensions interest form');
5 changes: 3 additions & 2 deletions src/utils/trackTools.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,10 @@ import * as fs from 'fs';
import * as path from 'path';
import { TOOL_CALL_FILE, TOOL_CALL_FILE_MAX_SIZE } from '../config.js';

// Ensure the directory for the log file exists
// Ensure the directory for the log file exists (use sync to avoid top-level await
// which breaks esbuild ESM bundling due to async propagation issues in __esm wrappers)
const logDir = path.dirname(TOOL_CALL_FILE);
await fs.promises.mkdir(logDir, { recursive: true });
fs.mkdirSync(logDir, { recursive: true });

/**
* Track tool calls and save them to a log file
Expand Down