Retire MCP packages and keep Skills support (#2694)

* refactor(workflow): retire mcp packages

* test(playground): fix local execution adapter mock types

* docs(site): note final mcp-supported version

Author: quanru

CONTRIBUTING.md

@@ -87,7 +87,7 @@ git config user.email "SOME_EMAIL@example.com"
   Playwright/Puppeteer integration and main web e2e coverage. The package name is `@midscene/web`.
 - `packages/shared`: shared utilities used across the monorepo. The package name is `@midscene/shared`.
 - `packages/{android,ios,computer,harmony}`: platform runtimes. Matching
-  `*-mcp` and `*-playground` packages live alongside them. The package names are `@midscene/android`, `@midscene/ios`, `@midscene/computer`, `@midscene/harmony`.
+  `*-playground` packages live alongside them. The package names are `@midscene/android`, `@midscene/ios`, `@midscene/computer`, `@midscene/harmony`.
 - `packages/visualizer` and `apps/report`: report rendering and viewer UI.
 - `apps/site`: documentation site. The Nx project name is `doc`, not `site`.
 - `apps/chrome-extension`, `apps/playground`, `apps/report`,
@@ -288,14 +288,13 @@ Every commit **must** include a scope. The scope must be one of the following:
 *   `llm`
 *   `playwright`
 *   `puppeteer`
-*   `mcp`
 *   `bridge`
 *   *(All top-level directories in the apps and packages directories)*
 *   *(Consider adding other relevant top-level packages or areas here if needed)*
 
 **Examples:**
 
-*   `feat(mcp): add screenshot tool with element selection`
+*   `feat(bridge): add screenshot tool with element selection`
 *   `fix(android): correct adb connection issue on windows`
 *   `refactor(llm): simplify prompt generation logic`
 *   `chore(workflow): update commitlint configuration`

README.md

@@ -40,7 +40,6 @@ Use [Midscene Skills](https://github.com/web-infra-dev/midscene-skills) to contr
 * [iOS Automation - Auto-like the first @midscene_ai tweet](https://midscenejs.com/showcases#ios)
 * [Android Automation - DCar: Xiaomi SU7 specs](https://midscenejs.com/showcases#android)
 * [Android Automation - Booking a hotel for Christmas](https://midscenejs.com/showcases#android)
-* [MCP Integration - Midscene MCP UI prepatch release](https://midscenejs.com/showcases#mcp)
 * [robotic arm + vision + voice for in-vehicle testing](https://midscenejs.com/showcases#community-showcases)
 
 ## 💡 Why Midscene
@@ -50,13 +49,13 @@ Most UI automation — including AI tools that read the DOM or the accessibility
 - **Less maintenance** — no selectors to chase when the UI changes.
 - **Reach every element and surface** — if a human can see it, Midscene can target it, even with no semantic annotations, on `<canvas>`, native apps, and cross-origin iframes.
 - **Assert what users actually see** — verify colors, highlights, layout, and rendered state, not just whether a DOM node exists.
-- **Two ways to test** — add Midscene to your [Playwright](https://midscenejs.com/integrate-with-playwright) / Vitest suite, or let an AI agent test autonomously via [Skills](https://midscenejs.com/skills) and [MCP](https://midscenejs.com/mcp).
+- **Two ways to test** — add Midscene to your [Playwright](https://midscenejs.com/integrate-with-playwright) / Vitest suite, or let an AI agent test autonomously via [Skills](https://midscenejs.com/skills).
 
 Midscene is built for UI testing first, but the same vision-driven engine handles any UI automation task.
 
 ## 💡 What you can automate
 
-Midscene works anywhere you can take a screenshot — web browsers, Android, iOS, HarmonyOS, desktop apps, and [any custom interface](https://midscenejs.com/integrate-with-any-interface) — all through one API. Write automation with the JavaScript SDK or in YAML, hand it to AI agents via [Skills](https://midscenejs.com/skills) and [MCP](https://midscenejs.com/mcp), and look up every method (`aiAct`, `aiQuery`, `aiAssert`, and more) in the [API reference](https://midscenejs.com/api).
+Midscene works anywhere you can take a screenshot — web browsers, Android, iOS, HarmonyOS, desktop apps, and [any custom interface](https://midscenejs.com/integrate-with-any-interface) — all through one API. Write automation with the JavaScript SDK or in YAML, hand it to AI agents via [Skills](https://midscenejs.com/skills), and look up every method (`aiAct`, `aiQuery`, `aiAssert`, and more) in the [API reference](https://midscenejs.com/api).
 
 ## 🚀 Get started
 

README.zh.md

@@ -40,7 +40,6 @@
 * [iOS 自动化 - 自动点赞 @midscene_ai 的第一条推文](https://midscenejs.com/zh/showcases#ios)
 * [Android 自动化 - 懂车帝：查看小米 SU7 参数](https://midscenejs.com/zh/showcases#android)
 * [Android 自动化 - 预订圣诞节酒店](https://midscenejs.com/zh/showcases#android)
-* [MCP 集成 - Midscene MCP UI prepatch 版本发布](https://midscenejs.com/zh/showcases#mcp)
 * [车机测试中的机械臂 + 视觉 + 语音方案](https://midscenejs.com/zh/showcases#community-showcases)
 
 ## 💡 为什么选择 Midscene
@@ -52,15 +51,15 @@ Midscene 仅凭截图工作。你只需用自然语言描述每一步：
 - **更低的维护成本**：UI 变化时，无需再追着改选择器。
 - **触达每个元素与界面**：只要人眼能看到，Midscene 就能定位。即使元素没有语义化标注，或位于 `<canvas>`、原生应用、跨域 iframe 上，也可以定位。
 - **校验用户真正看到的效果**：验证颜色、高亮、布局与渲染状态，而不只是判断 DOM 节点是否存在。
-- **两种测试方式**：接入你的 [Playwright](https://midscenejs.com/zh/integrate-with-playwright) / Vitest 测试，或让 AI Agent 通过 [Skills](https://midscenejs.com/zh/skills) 与 [MCP](https://midscenejs.com/zh/mcp) 自主测试。
+- **两种测试方式**：接入你的 [Playwright](https://midscenejs.com/zh/integrate-with-playwright) / Vitest 测试，或让 AI Agent 通过 [Skills](https://midscenejs.com/zh/skills) 自主测试。
 
 Midscene 首先为 UI 测试而生，但同一套视觉驱动引擎也能胜任任意 UI 自动化任务。
 
 ## 💡 能自动化什么
 
 只要能截图，Midscene 就能工作。Web 浏览器、Android、iOS、HarmonyOS、桌面应用，以及[任意自定义界面](https://midscenejs.com/zh/integrate-with-any-interface)，全部通过同一套 API。
 
-你可以用 JavaScript SDK 或 YAML 编写自动化，也可以通过 [Skills](https://midscenejs.com/zh/skills) 与 [MCP](https://midscenejs.com/zh/mcp) 交给 AI Agent。所有方法都可以在 [API 参考](https://midscenejs.com/zh/api) 中查阅，包括 `aiAct`、`aiQuery` 和 `aiAssert`。
+你可以用 JavaScript SDK 或 YAML 编写自动化，也可以通过 [Skills](https://midscenejs.com/zh/skills) 交给 AI Agent。所有方法都可以在 [API 参考](https://midscenejs.com/zh/api) 中查阅，包括 `aiAct`、`aiQuery` 和 `aiAssert`。
 
 ## 🚀 开始使用
 

apps/chrome-extension/src/scripts/worker.ts

@@ -27,7 +27,7 @@ self.addEventListener('unhandledrejection', (event) => {
   console.error('[ServiceWorker] Unhandled promise rejection:', event.reason);
 });
 
-// Background Bridge for MCP connection
+// Background Bridge for external automation connections
 const BRIDGE_PERMISSION_KEY = 'midscene_bridge_permission';
 const BRIDGE_STOPPED_KEY = 'midscene_bridge_stopped';
 let backgroundBridge: BridgeConnector | null = null;

apps/report/rsbuild.config.ts

@@ -7,6 +7,13 @@ import { pluginNodePolyfill } from '@rsbuild/plugin-node-polyfill';
 import { pluginReact } from '@rsbuild/plugin-react';
 import { pluginSvgr } from '@rsbuild/plugin-svgr';
 import { pluginWorkspaceDev } from 'rsbuild-plugin-workspace-dev';
+import {
+  buildReportTemplateInjection,
+  isReportTemplateInjectableFile,
+  reportTemplateMagicString,
+  reportTemplateReplacedMark,
+  reportTemplateReplacementRegExp,
+} from '../../scripts/report-template-utils.mjs';
 import {
   commonIgnoreWarnings,
   createTypeCheckPlugin,
@@ -35,20 +42,14 @@ const copyReportTemplate = () => ({
     onAfterBuild: (arg0: ({ compiler }: { compiler: any }) => void) => void;
   }) {
     api.onAfterBuild(({ compiler }) => {
-      const magicString = 'REPLACE_ME_WITH_REPORT_HTML';
-      const replacedMark = '/*REPORT_HTML_REPLACED*/';
-      const regExpForReplace = /\/\*REPORT_HTML_REPLACED\*\/.*/;
-
       // read the template file
       const srcPath = path.join(__dirname, 'dist', 'index.html');
-      const tplFileContent = fs
-        .readFileSync(srcPath, 'utf-8')
-        .replaceAll(magicString, '');
+      const { sanitizedTplFileContent, finalContent } =
+        buildReportTemplateInjection(fs.readFileSync(srcPath, 'utf-8'));
       assert(
-        !tplFileContent.includes(magicString),
+        !sanitizedTplFileContent.includes(reportTemplateMagicString),
         'magic string should not be in the template file',
       );
-      const finalContent = `${replacedMark}${JSON.stringify(tplFileContent)}`;
 
       // find the core package
       const corePkgDir = path.join(__dirname, '..', '..', 'packages', 'core');
@@ -65,35 +66,33 @@ const copyReportTemplate = () => ({
       const jsFiles = fs.readdirSync(corePkgDistDir, { recursive: true });
       let replacedCount = 0;
       for (const file of jsFiles) {
-        if (
-          typeof file === 'string' &&
-          (file.endsWith('.js') || file.endsWith('.mjs'))
-        ) {
+        if (isReportTemplateInjectableFile(file)) {
           const filePath = path.join(corePkgDistDir, file.toString());
           const fileContent = fs.readFileSync(filePath, 'utf-8');
-          if (fileContent.includes(replacedMark)) {
+          if (fileContent.includes(reportTemplateReplacedMark)) {
             assert(
-              regExpForReplace.test(fileContent),
+              reportTemplateReplacementRegExp.test(fileContent),
               'a replaced mark is found but cannot match',
             );
 
             const replacedContent = fileContent.replace(
-              regExpForReplace,
+              reportTemplateReplacementRegExp,
               () => finalContent,
             );
             fs.writeFileSync(filePath, replacedContent);
             replacedCount++;
             console.log(`Template updated in file ${filePath}`);
-          } else if (fileContent.includes(magicString)) {
+          } else if (fileContent.includes(reportTemplateMagicString)) {
             const magicStringCount = (
-              fileContent.match(new RegExp(magicString, 'g')) || []
+              fileContent.match(new RegExp(reportTemplateMagicString, 'g')) ||
+              []
             ).length;
             assert(
               magicStringCount === 1,
               'magic string shows more than once in the file, cannot process',
             );
             const replacedContent = fileContent.replace(
-              `'${magicString}'`,
+              `'${reportTemplateMagicString}'`,
               () => finalContent, // there are some $- code in the tpl, so we have to use a function as the second argument
             );
             fs.writeFileSync(filePath, replacedContent);

apps/report/scripts/inject-report-template.mjs

@@ -3,30 +3,32 @@ import assert from 'node:assert';
 import fs from 'node:fs';
 import path from 'node:path';
 import { fileURLToPath } from 'node:url';
+import {
+  buildReportTemplateInjection,
+  isReportTemplateInjectableFile,
+  reportTemplateMagicString,
+  reportTemplateReplacedMark,
+  reportTemplateReplacementRegExp,
+} from '../../../scripts/report-template-utils.mjs';
 
 const __dirname = path.dirname(fileURLToPath(import.meta.url));
 const reportRoot = path.resolve(__dirname, '..');
 const repoRoot = path.resolve(reportRoot, '..', '..');
 
-const magicString = 'REPLACE_ME_WITH_REPORT_HTML';
-const replacedMark = '/*REPORT_HTML_REPLACED*/';
-const regExpForReplace = /\/\*REPORT_HTML_REPLACED\*\/.*/;
-
 const srcPath = path.join(reportRoot, 'dist', 'index.html');
 if (!fs.existsSync(srcPath)) {
   throw new Error(
     `Report template not found at ${srcPath}. Run "nx build @midscene/report" first.`,
   );
 }
 
-const tplFileContent = fs
-  .readFileSync(srcPath, 'utf-8')
-  .replaceAll(magicString, '');
+const { sanitizedTplFileContent, finalContent } = buildReportTemplateInjection(
+  fs.readFileSync(srcPath, 'utf-8'),
+);
 assert(
-  !tplFileContent.includes(magicString),
+  !sanitizedTplFileContent.includes(reportTemplateMagicString),
   'magic string should not be in the template file',
 );
-const finalContent = `${replacedMark}${JSON.stringify(tplFileContent)}`;
 
 const corePkgDir = path.join(repoRoot, 'packages', 'core');
 const corePkgJson = JSON.parse(
@@ -41,34 +43,31 @@ const corePkgDistDir = path.join(corePkgDir, 'dist');
 const jsFiles = fs.readdirSync(corePkgDistDir, { recursive: true });
 let replacedCount = 0;
 for (const file of jsFiles) {
-  if (
-    typeof file === 'string' &&
-    (file.endsWith('.js') || file.endsWith('.mjs'))
-  ) {
+  if (isReportTemplateInjectableFile(file)) {
     const filePath = path.join(corePkgDistDir, file);
     const fileContent = fs.readFileSync(filePath, 'utf-8');
-    if (fileContent.includes(replacedMark)) {
+    if (fileContent.includes(reportTemplateReplacedMark)) {
       assert(
-        regExpForReplace.test(fileContent),
+        reportTemplateReplacementRegExp.test(fileContent),
         'a replaced mark is found but cannot match',
       );
       const replacedContent = fileContent.replace(
-        regExpForReplace,
+        reportTemplateReplacementRegExp,
         () => finalContent,
       );
       fs.writeFileSync(filePath, replacedContent);
       replacedCount++;
       console.log(`Template updated in file ${filePath}`);
-    } else if (fileContent.includes(magicString)) {
+    } else if (fileContent.includes(reportTemplateMagicString)) {
       const magicStringCount = (
-        fileContent.match(new RegExp(magicString, 'g')) || []
+        fileContent.match(new RegExp(reportTemplateMagicString, 'g')) || []
       ).length;
       assert(
         magicStringCount === 1,
         'magic string shows more than once in the file, cannot process',
       );
       const replacedContent = fileContent.replace(
-        `'${magicString}'`,
+        `'${reportTemplateMagicString}'`,
         () => finalContent,
       );
       fs.writeFileSync(filePath, replacedContent);

apps/report/test-data/mcp-release-workflow.json

@@ -0,0 +1,30 @@
+import { describe, expect, it } from 'vitest';
+import {
+  buildReportTemplateInjection,
+  reportTemplateMagicString,
+  reportTemplateReplacedMark,
+  reportTemplateReplacementRegExp,
+} from '../../../scripts/report-template-utils.mjs';
+
+describe('report template utils', () => {
+  it('should remove placeholders and sanitize nested injected report templates', () => {
+    const html = [
+      '<html>',
+      `${reportTemplateMagicString}<body>latest report</body>`,
+      `<script>window.__REPORT__=${reportTemplateReplacedMark}"<html>old report</html>"</script>`,
+      '</html>',
+    ].join('');
+
+    const { sanitizedTplFileContent, finalContent } =
+      buildReportTemplateInjection(html);
+
+    expect(sanitizedTplFileContent).not.toContain(reportTemplateMagicString);
+    expect(sanitizedTplFileContent).toContain(
+      `${reportTemplateReplacedMark}""`,
+    );
+    expect(finalContent).toMatch(reportTemplateReplacementRegExp);
+    expect(
+      JSON.parse(finalContent.slice(reportTemplateReplacedMark.length)),
+    ).toBe(sanitizedTplFileContent);
+  });
+});

apps/report/tests/report-template-utils.test.ts

@@ -39,7 +39,7 @@ All agents share these base options:
   - For mobile devices, setting `screenshotShrinkFactor` to 2 can reduce token consumption while maintaining clarity, but it is not recommended to set it higher than 3, as this may cause the image to be too blurry and affect the AI model's understanding.
   - For web pages, if the content is complex or contains a lot of details, it is not recommended to set `screenshotShrinkFactor` to avoid overly blurry screenshots. Additionally, if you want higher clarity for web page screenshots, you can configure Puppeteer or Playwright's `deviceScaleFactor` to 2, which will allow Puppeteer or Playwright to render the page as if it were a high-definition screen.
 
-The MCP tools and device CLIs expose these same Agent behavior options per call. In CLIs, convert the camelCase API option to a bare kebab-case flag, such as `waitAfterAction` -> `--wait-after-action`. In MCP calls, keep the camelCase option under the platform namespace, such as `android.waitAfterAction` or `web.waitAfterAction`. See [Configure Agent behavior per call](./mcp#configure-agent-behavior-per-call) for examples.
+The device CLIs expose these same Agent behavior options per call. Convert the camelCase API option to a bare kebab-case flag, such as `waitAfterAction` -> `--wait-after-action`. See [Skills](./skills) for the platform CLI entry points.
 
 ### Custom model configuration
 

apps/site/docs/en/api.mdx

@@ -14,18 +14,26 @@ Midscene now includes an AndroidWorld benchmark report. With v1.9.5, Midscene ac
 
 ### Model and Planning Updates
 
+- `aiAct` now supports image prompts.
 - `MIDSCENE_MODEL_REASONING_ENABLED` now supports `default`, so Midscene can follow each model family's default thinking behavior.
+- Gemini thinking content and GPT-5 reasoning configuration are now handled more completely.
 - `aiAct` falls back to model planning when a cached plan becomes invalid, and clears the corresponding cache entry.
+- AI request errors now include retry-attempt details, parse failures preserve raw model responses, and parsed locate results are validated before use.
+- Model dumps now expose more model response metadata, including raw choice messages and response model names in usage data.
 - The `deepLocate` search area is now displayed in reports.
 
 ### Chrome Extension
 
 - Chrome extension Bridge mode now supports file upload.
+- Bridge-mode file uploads now honor file chooser accept filters and WSL file paths.
 
 ### YAML, CLI, and MCP
 
+- `1.9.8` is the final Midscene version that includes MCP support. Later versions retire MCP server packages in favor of Skills and the platform CLIs.
+- Platform CLIs now accept agent behavior init args.
 - YAML scripts now support the HarmonyOS target in the CLI, so HarmonyOS automation can be driven through the same script runner workflow as Web, Android, iOS, and Computer. See: [Automate with Scripts in YAML](./automate-with-scripts-in-yaml), [HarmonyOS API](./harmony-api-reference)
 - YAML Web config supports custom HTTP headers for browser automation.
+- YAML Web config supports `downloadPath` for browser downloads.
 - YAML runs now surface real execution errors instead of ending as a silent "not executed" case.
 - YAML batch runs can retry failed cases.
 - Successful YAML runs now print the report path.
@@ -36,13 +44,26 @@ Midscene now includes an AndroidWorld benchmark report. With v1.9.5, Midscene ac
 
 ### Reports
 
+- `recordToReport` now supports custom screenshots.
 - Report exports keep image paths aligned with exported screenshots.
 - Reports include a JSON tree view for inspecting structured task and model data.
+- Report screenshots, labels, playground server origin handling, and context spacing were refined.
+
+### Studio and Recorder
+
+- Studio recorder descriptions and preview input coalescing were stabilized.
+- Studio now handles invalid model environment configuration more safely.
+- Recorder workflows can generate Markdown replay output.
 
 ### Android Automation
 
 - Android action controls and planning guidance were improved for native mobile automation flows.
 
+### Computer Automation
+
+- Computer desktop automation now ships Intel packaging.
+- Libnut scrolling now emits one full wheel delta per scroll tick.
+
 ### Bug Fixes
 
 - Fixed `longPress` duration in Web integration being capped at 600ms.