fix: Generate desktop runtime core lock (#125)

* fix: generate desktop runtime core lock

* fix: harden desktop runtime core lock handling

Reuse the launcher env constant and only suppress missing top_level.txt metadata so unexpected lock-generation failures stay visible.

* fix: honor desktop runtime core lock overrides

Preserve explicit launcher lock path overrides for tests and advanced setups, and let the Python lock generator own output directory creation.

* docs: fix dependency boundary design wording

Clarify the Cua failure-mode sentence in the desktop plugin dependency boundary design by replacing the comma splice with clearer punctuation.

* fix: harden runtime core lock generation

Replace brittle runtime core lock source-inspection tests with behavior checks, add generator context to failure messages, and tolerate undecodable top-level metadata. Remove the tool-specific execution note from the implementation plan doc.

* fix: verify runtime core lock generator output

Report signal-terminated generator failures clearly and reject invalid generated lock files so broken build artifacts fail fast with actionable diagnostics.

Author: zouyonghe

docs/plans/2026-04-28-desktop-plugin-dependency-boundary-design.md

@@ -0,0 +1,77 @@
+# Desktop Plugin Dependency Boundary Design
+
+## Context
+
+Packaged desktop builds run AstrBot inside a bundled CPython runtime. Runtime plugin
+dependencies are installed into `ASTRBOT_ROOT/data/site-packages` and are then
+prepended to `sys.path` by AstrBot's pip installer.
+
+This allows plugin-only dependencies to work, but it also allows plugin installs to
+shadow packages already used by the bundled backend runtime. Packages such as
+`openai`, `pydantic`, `fastapi`, `numpy`, and PyObjC modules are unsafe to hot-reload
+inside a running backend process. Installing Cua exposed this failure mode: pip
+installed a second dependency set into plugin `site-packages`; the installer tried to
+prefer those modules in-process, and the running OpenAI client later saw incompatible
+class identities.
+
+## Goals
+
+- Keep packaged desktop plugin installs from replacing bundled core runtime modules.
+- Preserve normal plugin dependency installation when dependencies are not core
+  runtime packages.
+- Make the desktop bundle self-describing enough that runtime code can distinguish
+  bundled core dependencies from plugin dependencies.
+- Favor explicit incompatibility or restart-required behavior over in-process module
+  replacement.
+
+## Non-Goals
+
+- Full per-plugin virtual environment isolation.
+- Rewriting AstrBot's plugin manager.
+- Guaranteeing that every PyPI package can coexist in the backend process.
+
+## Design
+
+The desktop backend build writes a `runtime-core-lock.json` file into
+`resources/backend/app`. The lock captures distributions installed in the bundled
+runtime, their versions, and top-level import modules. The launcher exposes this file
+to the backend through `ASTRBOT_DESKTOP_CORE_LOCK_PATH`.
+
+When running in packaged desktop mode, AstrBot's pip installer reads the lock and uses
+it for two protections:
+
+1. Add constraints for locked distributions so plugin pip installs cannot upgrade or
+   downgrade bundled runtime packages.
+2. Avoid preferring locked top-level modules from `data/site-packages` after install.
+
+If pip cannot satisfy a plugin because it truly requires a different core dependency
+version, installation fails with the existing dependency conflict path. This is
+intentional: a clear install-time incompatibility is safer than corrupting the live
+backend process.
+
+## Data Flow
+
+1. `scripts/backend/build-backend.mjs` installs backend requirements into the copied
+   runtime.
+2. The build script runs a small Python helper against that runtime to enumerate
+   installed distributions.
+3. The helper writes `app/runtime-core-lock.json`.
+4. `launch_backend.py` sets `ASTRBOT_DESKTOP_CORE_LOCK_PATH` before running
+   `main.py`.
+5. The packaged backend pip installer reads the lock while installing plugin
+   requirements and while attempting dependency preference.
+
+## Error Handling
+
+- If the lock cannot be generated during resource preparation, the backend build
+  fails. A packaged desktop runtime without this boundary is not considered safe.
+- If the lock cannot be read at runtime, AstrBot falls back to existing behavior and
+  logs a warning. This keeps old custom launch setups from breaking.
+- If a plugin conflicts with locked core dependencies, the install fails with a
+  dependency conflict message.
+
+## Testing
+
+- Script tests cover lock generation and launcher environment wiring.
+- AstrBot pip installer tests cover adding lock constraints and skipping locked
+  modules during post-install preference.

docs/plans/2026-04-28-desktop-plugin-dependency-boundary.md

@@ -0,0 +1,132 @@
+# Desktop Plugin Dependency Boundary Implementation Plan
+
+**Goal:** Prevent packaged desktop plugin installs from replacing bundled backend runtime dependencies.
+
+**Architecture:** Generate a runtime core dependency lock during desktop backend packaging, expose it through an environment variable at launch, and teach AstrBot's packaged pip installer to constrain and skip locked core modules. This keeps plugin-only dependencies installable while preventing unsafe in-process replacement of backend dependencies.
+
+**Tech Stack:** Node.js resource preparation scripts, Python launcher/runtime code, AstrBot pip installer tests with pytest, Node `node:test`.
+
+---
+
+### Task 1: Generate the Desktop Runtime Core Lock
+
+**Files:**
+- Modify: `scripts/backend/build-backend.mjs`
+- Create: `scripts/backend/templates/generate_runtime_core_lock.py`
+- Test: `scripts/backend/runtime-core-lock.test.mjs`
+
+**Step 1: Write the failing test**
+
+Add a Node test that creates a fake runtime Python command, runs the lock generator entry point, and expects `runtime-core-lock.json` to be written under the backend app directory.
+
+**Step 2: Run test to verify it fails**
+
+Run: `node --test scripts/backend/runtime-core-lock.test.mjs`
+Expected: FAIL because the helper/export does not exist yet.
+
+**Step 3: Write minimal implementation**
+
+Add a Python helper that uses `importlib.metadata.distributions()` to collect:
+- distribution name
+- distribution version
+- `top_level.txt` modules when present
+
+Call it from `build-backend.mjs` after runtime dependency installation and before the manifest is written.
+
+**Step 4: Run test to verify it passes**
+
+Run: `node --test scripts/backend/runtime-core-lock.test.mjs`
+Expected: PASS.
+
+### Task 2: Expose the Lock to the Packaged Backend
+
+**Files:**
+- Modify: `scripts/backend/templates/launch_backend.py`
+- Test: `scripts/prepare-resources/backend-runtime.test.mjs` or a focused launcher template test
+
+**Step 1: Write the failing test**
+
+Add a test that verifies the launcher template contains `ASTRBOT_DESKTOP_CORE_LOCK_PATH` and points it at `APP_DIR / "runtime-core-lock.json"` only when the file exists.
+
+**Step 2: Run test to verify it fails**
+
+Run: `pnpm run test:prepare-resources`
+Expected: FAIL because the launcher does not set the env var yet.
+
+**Step 3: Write minimal implementation**
+
+Set `os.environ["ASTRBOT_DESKTOP_CORE_LOCK_PATH"]` in `launch_backend.py` before `runpy.run_path()` when the lock file exists.
+
+**Step 4: Run test to verify it passes**
+
+Run: `pnpm run test:prepare-resources`
+Expected: PASS.
+
+### Task 3: Apply Lock Constraints in the Pip Installer
+
+**Files:**
+- Modify: `vendor/AstrBot/astrbot/core/utils/core_constraints.py`
+- Modify: `vendor/AstrBot/astrbot/core/utils/pip_installer.py`
+- Test: `vendor/AstrBot/tests/test_pip_installer.py`
+
+**Step 1: Write the failing test**
+
+Add a test that sets `ASTRBOT_DESKTOP_CORE_LOCK_PATH` to a JSON lock containing `openai==2.32.0`, then verifies `PipInstaller.install(package_name="Cua")` adds `-c <constraints file>` and the constraints include `openai==2.32.0`.
+
+**Step 2: Run test to verify it fails**
+
+Run: `cd vendor/AstrBot && uv run pytest tests/test_pip_installer.py::<test_name> -q`
+Expected: FAIL because the lock is ignored.
+
+**Step 3: Write minimal implementation**
+
+Teach `CoreConstraintsProvider` to merge existing core metadata constraints with locked desktop runtime constraints in packaged mode.
+
+**Step 4: Run test to verify it passes**
+
+Run: `cd vendor/AstrBot && uv run pytest tests/test_pip_installer.py::<test_name> -q`
+Expected: PASS.
+
+### Task 4: Skip Locked Modules During Post-Install Preference
+
+**Files:**
+- Modify: `vendor/AstrBot/astrbot/core/utils/pip_installer.py`
+- Test: `vendor/AstrBot/tests/test_pip_installer.py`
+
+**Step 1: Write the failing test**
+
+Add a test where candidate modules include `openai` and `cua_agent`, the lock marks `openai` as a core module, and `_ensure_plugin_dependencies_preferred` calls `_ensure_preferred_modules` only with `cua_agent`.
+
+**Step 2: Run test to verify it fails**
+
+Run: `cd vendor/AstrBot && uv run pytest tests/test_pip_installer.py::<test_name> -q`
+Expected: FAIL because locked modules are still preferred.
+
+**Step 3: Write minimal implementation**
+
+Filter candidate modules using the lock's top-level module set before attempting `_prefer_module_from_site_packages`.
+
+**Step 4: Run test to verify it passes**
+
+Run: `cd vendor/AstrBot && uv run pytest tests/test_pip_installer.py::<test_name> -q`
+Expected: PASS.
+
+### Task 5: Full Verification and PR
+
+**Files:**
+- All touched files
+
+**Step 1: Run focused tests**
+
+Run:
+- `pnpm run test:prepare-resources`
+- `cd vendor/AstrBot && uv run pytest tests/test_pip_installer.py -q`
+
+**Step 2: Run repository tests if feasible**
+
+Run:
+- `make test`
+
+**Step 3: Commit and open PR**
+
+Commit only intentional files, push `codex/desktop-plugin-dependency-lock`, and create a draft PR against `AstrBotDevs/AstrBot-desktop:main`.

scripts/backend/build-backend.mjs

@@ -19,6 +19,7 @@ import {
   pruneLinuxTkinterRuntime,
 } from './runtime-linux-compat-utils.mjs';
 import { isWindowsArm64BundledRuntime } from './runtime-arch-utils.mjs';
+import { generateRuntimeCoreLock } from './runtime-core-lock.mjs';
 
 const __dirname = path.dirname(fileURLToPath(import.meta.url));
 const projectRoot = path.resolve(__dirname, '..', '..');
@@ -29,6 +30,7 @@ const outputDir = path.join(projectRoot, 'resources', 'backend');
 const appDir = path.join(outputDir, 'app');
 const runtimeDir = path.join(outputDir, 'python');
 const manifestPath = path.join(outputDir, 'runtime-manifest.json');
+const runtimeCoreLockPath = path.join(appDir, 'runtime-core-lock.json');
 const launcherPath = path.join(outputDir, 'launch_backend.py');
 const launcherTemplatePath = path.join(__dirname, 'templates', 'launch_backend.py');
 const importScannerScriptPath = path.join(__dirname, 'tools', 'scan_imports.py');
@@ -613,6 +615,10 @@ const main = () => {
   copyAppSources(resolvedSourceDir);
   const runtimePython = prepareRuntimeExecutable(runtimeSourceReal);
   installRuntimeDependencies(runtimePython);
+  generateRuntimeCoreLock({
+    runtimePython,
+    outputPath: runtimeCoreLockPath,
+  });
   pruneLinuxTkinterRuntime(runtimeDir);
   patchLinuxRuntimeRpaths(runtimeDir);
   writeLauncherScript();

scripts/backend/runtime-core-lock.mjs

@@ -0,0 +1,68 @@
+import fs from 'node:fs';
+import path from 'node:path';
+import { spawnSync } from 'node:child_process';
+import { fileURLToPath } from 'node:url';
+
+const __dirname = path.dirname(fileURLToPath(import.meta.url));
+const generatorScriptPath = path.join(__dirname, 'tools', 'generate_runtime_core_lock.py');
+const formatGeneratorContext = (pythonPath) =>
+  `(python: ${pythonPath ?? 'undefined'}, script: ${generatorScriptPath})`;
+
+export const generateRuntimeCoreLock = ({ runtimePython, outputPath }) => {
+  if (!runtimePython?.absolute) {
+    throw new Error(
+      `Missing runtime Python executable for runtime core lock generation ${formatGeneratorContext(runtimePython?.absolute)}.`,
+    );
+  }
+  if (!outputPath) {
+    throw new Error(
+      `Missing output path for runtime core lock generation ${formatGeneratorContext(runtimePython.absolute)}.`,
+    );
+  }
+
+  const result = spawnSync(
+    runtimePython.absolute,
+    [generatorScriptPath, '--output', outputPath],
+    {
+      encoding: 'utf8',
+      stdio: ['ignore', 'pipe', 'pipe'],
+      windowsHide: true,
+    },
+  );
+
+  if (result.error) {
+    throw new Error(
+      `Failed to generate runtime core lock ${formatGeneratorContext(runtimePython.absolute)}: ${result.error.message}`,
+    );
+  }
+  if (result.status !== 0) {
+    let detail;
+    if (result.signal) {
+      detail = `terminated by signal ${result.signal}`;
+    } else {
+      detail = result.stderr?.trim() || result.stdout?.trim() || `exit code ${result.status}`;
+    }
+    throw new Error(
+      `Runtime core lock generation failed ${formatGeneratorContext(runtimePython.absolute)}: ${detail}`,
+    );
+  }
+
+  {
+    const context = formatGeneratorContext(runtimePython.absolute);
+    const baseMessage = `Runtime core lock generator did not create valid ${outputPath} ${context}.`;
+
+    if (!fs.existsSync(outputPath)) {
+      throw new Error(baseMessage);
+    }
+
+    try {
+      const contents = fs.readFileSync(outputPath, 'utf8');
+      if (!contents.trim()) {
+        throw new Error('empty runtime core lock output');
+      }
+      JSON.parse(contents);
+    } catch {
+      throw new Error(baseMessage);
+    }
+  }
+};