Merge branch 'beads/bd-6rczoll3-ts-packages-build-step'

Author: cscheid

CLAUDE.md

@@ -460,8 +460,13 @@ cargo xtask verify           # Full verification (Rust + hub-client builds + tes
 This runs:
 1. `cargo build --workspace` - Build all Rust crates
 2. `cargo nextest run --workspace` - Run all Rust tests
-3. `cd hub-client && npm run build:all` - Build hub-client (includes WASM)
-4. `cd hub-client && npm run test:ci` - Run hub-client tests
+3. `npm run build --if-present -w ts-packages/...` - Build the ts-packages
+   workspaces, then smoke-check the quarto-hub-mcp server (`node
+   dist/index.js --help` must survive ESM module resolution). These `dist/`
+   outputs are consumed at runtime by Node consumers (the MCP server);
+   hub-client bundles ts-packages from source, so nothing else builds them.
+4. `cd hub-client && npm run build:all` - Build hub-client (includes WASM)
+5. `cd hub-client && npm run test:ci` - Run hub-client tests
 
 **Skip options** (for faster iteration):
 ```bash

claude-notes/plans/2026-06-12-ts-packages-build-step.md

@@ -0,0 +1,135 @@
+# ts-packages build step for xtask build-all / verify (bd-6rczoll3)
+
+## Overview
+
+`cargo xtask build-all` documents itself as the fresh-clone source of truth,
+but no step in it (nor in `cargo xtask verify`) ever builds `ts-packages/*`.
+This goes unnoticed because hub-client consumes ts-packages **from source**:
+every ts-package's exports map points `types`/`source` at `./src/index.ts`,
+hub-client's `tsc -b` has no project references into ts-packages, and Vite
+bundles from source. The one runtime consumer of ts-package `dist/` output is
+the **quarto-hub-mcp server**, which Node executes directly and which resolves
+workspace imports through the `"import": "./dist/index.js"` export condition.
+A missing/stale dependency dist (observed: `@quarto/quarto-sync-client` had no
+`dist/` at all) kills the MCP at startup with `ERR_MODULE_NOT_FOUND`, which
+the MCP harness surfaces as a `-32000` error.
+
+Observed tree state at diagnosis time (2026-06-12): 7 of 9 ts-packages had no
+`dist/` (`annotated-qmd`, `pandoc-types`, `preview-renderer`,
+`preview-runtime`, `quarto-sync-client`, `sync-test-harness`,
+`wasm-js-bridge`); the two that did had been built by hand.
+
+## Design
+
+1. **New shared helper** `crates/xtask/src/ts_packages.rs`:
+   - `package_dirs(project_root) -> Vec<PathBuf>`: sorted list of
+     `ts-packages/*` subdirectories containing a `package.json`; empty when
+     `ts-packages/` is absent (old branches). Unit-tested with tempdir
+     fixtures.
+   - The build invocation is one root-level npm run:
+     `npm run build --if-present -w ts-packages/<a> -w ts-packages/<b> ...`
+     `--if-present` covers packages without a `build` script
+     (`sync-test-harness`, `wasm-js-bridge`). Build **order doesn't matter**:
+     since types resolve via `src/`, each package's `tsc` compiles without its
+     dependencies' `dist/` present.
+2. **build-all**: new step between `npm install` and the hub-client build,
+   with a `--skip-ts-packages-build` flag.
+3. **verify**: new step 6 (before the hub-client build; `TOTAL_STEPS` 12→13,
+   later steps renumbered), same `--skip-ts-packages-build` flag. The step
+   builds ts-packages, then **smoke-checks the MCP server**:
+   `node ts-packages/quarto-hub-mcp/dist/index.js --help` must exit 0. The
+   `--help` path runs after the whole ESM graph links, prints usage, and
+   exits 0 — no hang, no network, no auth env needed. A broken module graph
+   exits 1 with `ERR_MODULE_NOT_FOUND` before any code runs.
+4. **Cross-platform fix**: quarto-hub-mcp's build script is
+   `tsc && chmod +x dist/index.js`, which fails on Windows (no `chmod`).
+   Replace with a portable node one-liner
+   (`node -e "fs.chmodSync(...)"`-style, no-op semantics on Windows) so the
+   new verify/build-all step doesn't break Windows.
+5. **Docs**: update step lists in `build_all.rs` / `verify.rs` module docs,
+   the `Verify` / `BuildAll` doc comments in `main.rs`, and the verify
+   description in `CLAUDE.md`.
+
+Out of scope (deliberately): TypeScript project references between
+ts-packages and hub-client; building hub-client/trace-viewer/q2-preview-spa
+from the root `npm run build --workspaces` script (they keep their dedicated
+steps, which include WASM and embedding concerns the root script doesn't
+know about).
+
+## Work Items
+
+### Phase 1: Red (failing checks first)
+
+- [x] Reproduce the failure mechanically: `node
+      ts-packages/quarto-hub-mcp/dist/index.js --help` exits 1 with
+      `ERR_MODULE_NOT_FOUND` for `@quarto/quarto-sync-client/dist/index.js`
+      (recorded below)
+- [x] Write unit tests for `ts_packages::workspace_paths` (tempdir fixtures:
+      sorted result, skips dirs without package.json, missing ts-packages/
+      → empty, ignores plain files)
+
+### Phase 2: Implementation
+
+- [x] Add `crates/xtask/src/ts_packages.rs` helper + tests
+- [x] Make quarto-hub-mcp's `build` script cross-platform (drop bare `chmod`;
+      now `node -e "import('node:fs').then(fs => fs.chmodSync(...))"`)
+- [x] build-all: add ts-packages step + `--skip-ts-packages-build` flag
+- [x] verify: add step 6 (build + MCP smoke check), renumber steps 6–12 →
+      7–13, add `--skip-ts-packages-build` flag
+- [x] Update doc comments in `main.rs`, module docs, `CLAUDE.md`
+
+### Phase 3: Verification
+
+- [x] `cargo nextest run -p xtask` — 4 new unit tests pass
+- [x] End-to-end: verify with all other steps skipped → ts-packages step
+      builds 7 packages (2 have no build script, covered by `--if-present`)
+      and the MCP smoke check passes; same for the build-all step
+- [x] End-to-end: `node ts-packages/quarto-hub-mcp/dist/index.js --help`
+      exits 0 and prints usage (the original failing command from the bug
+      report)
+- [x] `cargo build --workspace` (clean) + `cargo nextest run --workspace`
+      (9967 passed, 196 skipped)
+- [x] `cargo xtask verify --skip-hub-build` (also exercises the new step
+      for real) — all steps passed, exit 0
+- [ ] Close bd-6rczoll3
+
+## Evidence
+
+### Red state (before fix, 2026-06-12)
+
+```
+$ node ts-packages/quarto-hub-mcp/dist/index.js --help
+Error [ERR_MODULE_NOT_FOUND]: Cannot find module
+'/Users/cscheid/repos/github/quarto-dev/q2/node_modules/@quarto/quarto-sync-client/dist/index.js'
+imported from .../ts-packages/quarto-hub-mcp/dist/connection-manager.js
+exit code: 1
+```
+
+(Tree state: 7 of 9 ts-packages had no `dist/`. The `--help` flag still
+fails because ESM module resolution happens before any code runs — which is
+exactly why it works as a smoke check.)
+
+### Green state (after fix)
+
+```
+$ cargo xtask verify --skip-rust-build --skip-rust-tests --skip-hub-build \
+    --skip-hub-tests --skip-trace-viewer-build --skip-trace-viewer-tests \
+    --skip-treesitter-tests --skip-shared-package-tests --skip-q2-preview-spa-build
+━━━ Step 6/13: Building ts-packages workspaces ━━━
+> @quarto/annotated-qmd@0.1.1 build ... (7 packages built via
+  npm run build --if-present -w ts-packages/<pkg> ...)
+  ↳ Smoke-checking quarto-hub-mcp module graph (node dist/index.js --help)...
+Usage: quarto-hub-mcp --server <url> [--read-only] [--redirect-port <N>]
+  ✓ quarto-hub-mcp module graph loads
+✓ ts-packages build complete
+...
+✓ All verification steps passed!
+
+$ node ts-packages/quarto-hub-mcp/dist/index.js --help
+Usage: quarto-hub-mcp --server <url> [--read-only] [--redirect-port <N>]
+MCP --help exit code: 0
+```
+
+Output inspected: usage text printed, exit 0, and
+`ts-packages/quarto-sync-client/dist/index.js` (the originally missing
+module) now exists.

crates/xtask/src/build_all.rs

@@ -5,10 +5,12 @@
 //! to produce a working build:
 //!
 //! 1. `npm install` at the repo root (npm workspaces)
-//! 2. Build hub-client (includes WASM via `npm run build:all`)
-//! 3. Build trace-viewer (if present; Phase 4.3+)
-//! 4. Build q2-preview-spa (if present; q2-preview Phase A.4 / bd-501n)
-//! 5. Build the Rust workspace (`cargo build --workspace`)
+//! 2. Build the ts-packages workspaces (`dist/` for Node consumers like
+//!    the quarto-hub-mcp server; bd-6rczoll3 — see `ts_packages.rs`)
+//! 3. Build hub-client (includes WASM via `npm run build:all`)
+//! 4. Build trace-viewer (if present; Phase 4.3+)
+//! 5. Build q2-preview-spa (if present; q2-preview Phase A.4 / bd-501n)
+//! 6. Build the Rust workspace (`cargo build --workspace`)
 //!
 //! Both SPAs (trace-viewer + q2-preview-spa) must build *before* the
 //! Rust workspace because `quarto-trace-server` and `quarto-preview`
@@ -23,6 +25,8 @@ pub struct BuildAllConfig {
     /// Skip `npm install`. Useful when running in a loop where dependencies
     /// haven't changed.
     pub skip_npm_install: bool,
+    /// Skip the ts-packages build step. No-op when `ts-packages/` is absent.
+    pub skip_ts_packages_build: bool,
     /// Skip the hub-client build step.
     pub skip_hub_build: bool,
     /// Skip the trace-viewer build step. No-op until Phase 4.3 lands.
@@ -40,6 +44,7 @@ impl Default for BuildAllConfig {
     fn default() -> Self {
         Self {
             skip_npm_install: false,
+            skip_ts_packages_build: false,
             skip_hub_build: false,
             skip_trace_viewer_build: false,
             skip_q2_preview_spa_build: false,
@@ -52,9 +57,14 @@ impl Default for BuildAllConfig {
 /// Run the build-all command.
 pub fn run(config: &BuildAllConfig) -> Result<()> {
     let project_root = find_project_root()?;
+    let ts_workspaces = crate::ts_packages::workspace_paths(&project_root);
 
     let steps: Vec<(&str, bool)> = vec![
         ("npm install (root workspaces)", !config.skip_npm_install),
+        (
+            "ts-packages build",
+            !config.skip_ts_packages_build && !ts_workspaces.is_empty(),
+        ),
         ("hub-client build (WASM + TS)", !config.skip_hub_build),
         (
             "trace-viewer build",
@@ -85,6 +95,29 @@ pub fn run(config: &BuildAllConfig) -> Result<()> {
         println!("✓ npm install complete");
     }
 
+    // Step: ts-packages build (bd-6rczoll3)
+    //
+    // Emits the `dist/` output that Node consumers (the quarto-hub-mcp
+    // server) resolve at runtime; nothing else in this sequence builds it
+    // because hub-client bundles ts-packages from source.
+    if !config.skip_ts_packages_build && !ts_workspaces.is_empty() {
+        step_idx += 1;
+        banner(step_idx, total, "Building ts-packages workspaces");
+        let mut npm_args: Vec<&str> = vec!["run", "build", "--if-present"];
+        for workspace in &ts_workspaces {
+            npm_args.push("-w");
+            npm_args.push(workspace);
+        }
+        run_command(
+            "npm",
+            &npm_args,
+            &project_root,
+            None,
+            "ts-packages build failed",
+        )?;
+        println!("✓ ts-packages build complete");
+    }
+
     // Step: hub-client build (WASM + TS)
     if !config.skip_hub_build {
         step_idx += 1;

crates/xtask/src/main.rs

@@ -27,6 +27,7 @@ mod stage_doc_examples;
 mod switch_task;
 mod test;
 mod treesitter_crlf;
+mod ts_packages;
 mod util;
 mod verify;
 
@@ -138,8 +139,9 @@ enum Command {
     /// 3. Build all Rust crates (cargo build --workspace, with -D warnings)
     /// 4. Test tree-sitter grammars (tree-sitter test)
     /// 5. Run all Rust tests (cargo nextest run --workspace, with -D warnings)
-    /// 6. Build hub-client including WASM (npm run build:all)
-    /// 7. Run hub-client tests (npm run test:ci)
+    /// 6. Build ts-packages workspaces + quarto-hub-mcp smoke check
+    /// 7. Build hub-client including WASM (npm run build:all)
+    /// 8. Run hub-client tests (npm run test:ci)
     ///
     /// Use this before pushing to ensure nothing will fail in CI.
     Verify {
@@ -151,6 +153,10 @@ enum Command {
         #[arg(long)]
         skip_rust_tests: bool,
 
+        /// Skip the ts-packages build + quarto-hub-mcp smoke check.
+        #[arg(long)]
+        skip_ts_packages_build: bool,
+
         /// Skip hub-client build.
         #[arg(long)]
         skip_hub_build: bool,
@@ -221,15 +227,20 @@ enum Command {
     /// of truth for what a fresh checkout (or CI) needs to produce a working
     /// build:
     /// 1. npm install at the repo root (npm workspaces)
-    /// 2. hub-client build (includes WASM)
-    /// 3. trace-viewer build (if present; Phase 4.3+)
-    /// 4. q2-preview-spa build (if present; q2-preview Phase A.4)
-    /// 5. cargo build --workspace
+    /// 2. ts-packages build (dist/ for Node consumers like quarto-hub-mcp)
+    /// 3. hub-client build (includes WASM)
+    /// 4. trace-viewer build (if present; Phase 4.3+)
+    /// 5. q2-preview-spa build (if present; q2-preview Phase A.4)
+    /// 6. cargo build --workspace
     BuildAll {
         /// Skip `npm install`.
         #[arg(long)]
         skip_npm_install: bool,
 
+        /// Skip the ts-packages build.
+        #[arg(long)]
+        skip_ts_packages_build: bool,
+
         /// Skip the hub-client build.
         #[arg(long)]
         skip_hub_build: bool,
@@ -288,6 +299,7 @@ fn main() -> Result<()> {
         Command::Verify {
             skip_rust_build,
             skip_rust_tests,
+            skip_ts_packages_build,
             skip_hub_build,
             skip_hub_tests,
             skip_trace_viewer_build,
@@ -302,6 +314,7 @@ fn main() -> Result<()> {
             let config = verify::VerifyConfig {
                 skip_rust_build,
                 skip_rust_tests,
+                skip_ts_packages_build,
                 skip_hub_build,
                 skip_hub_tests,
                 skip_trace_viewer_build,
@@ -320,6 +333,7 @@ fn main() -> Result<()> {
         Command::BuildQ2PreviewSpa {} => build_q2_preview_spa::run(),
         Command::BuildAll {
             skip_npm_install,
+            skip_ts_packages_build,
             skip_hub_build,
             skip_trace_viewer_build,
             skip_q2_preview_spa_build,
@@ -328,6 +342,7 @@ fn main() -> Result<()> {
         } => {
             let config = build_all::BuildAllConfig {
                 skip_npm_install,
+                skip_ts_packages_build,
                 skip_hub_build,
                 skip_trace_viewer_build,
                 skip_q2_preview_spa_build,