Skip to content

feat(pm): resolve workspace:/catalog: protocols in packed manifests#3102

Merged
elrrrrrrr merged 7 commits into
nextfrom
fix/pm-pack-workspace-catalog-3094
Jun 3, 2026
Merged

feat(pm): resolve workspace:/catalog: protocols in packed manifests#3102
elrrrrrrr merged 7 commits into
nextfrom
fix/pm-pack-workspace-catalog-3094

Conversation

@elrrrrrrr

Copy link
Copy Markdown
Contributor

Summary

Closes #3094.

ut pm-pack / utoo publish emitted tarballs whose package.json kept the literal workspace:* and catalog: specifiers. Any downstream npm install of such a tarball fails immediately with EUNSUPPORTEDPROTOCOL, which forced monorepos migrating to utoo (e.g. eggjs/egg#5959) to keep pnpm around just for the pack step.

This rewrites both protocols at pack/publish time the way pnpm/bun do:

spec result (member version 1.2.3)
workspace:* / workspace: 1.2.3
workspace:~ ~1.2.3
workspace:^ ^1.2.3
workspace:^1.2.0 ^1.2.0 (prefix stripped)
workspace:./path 1.2.3
catalog: / catalog:<name> version pinned in the catalog

Design

  • New resolve_workspace_spec in crates/ruborist/src/spec/mod.rs, symmetric to the existing resolve_catalog_spec.
  • New crates/pm/src/service/publish_manifest.rs orchestrates the rewrite:
    • No-op (returns None) when the manifest has neither protocol, so standalone packages are packed byte-for-byte.
    • Resolves against the discovered workspace rootpm-pack runs inside a member dir, so workspace: versions come from sibling members and catalog: from the root .utoo.toml (migrated from pnpm-workspace.yaml), not the cwd.
    • Unresolvable specifiers bail with an actionable message rather than shipping a broken tarball.
  • pm_pack::pack substitutes the rewritten manifest into the tarball entry (append_override); the on-disk package.json is never mutated (pnpm parity).
  • publish now builds the payload from pack_result.manifest, so registry metadata and tarball contents can't drift.

Test plan

  • Unit: resolve_workspace_spec (all forms), rewrite_dep_specs across all four dependency kinds, unresolvable workspace/catalog errors, protocol detection. — pm publish_manifest (5) + ruborist spec pass.
  • Existing pm_pack (11) and full utoo-pm (248) suites green; cargo fmt --check + cargo clippy --all-targets -D warnings clean.
  • Manual E2E: a 2-member workspace with workspace:^/workspace:* + catalog: → tarball package.json rewritten to concrete ranges, unknown fields (repository/keywords) preserved, source manifest unchanged.

Follow-ups (out of scope)

🤖 Generated with Claude Code

…3094)

`ut pm-pack`/`utoo publish` emitted tarballs whose package.json kept the
literal `workspace:*` and `catalog:` specifiers, so any downstream
`npm install` of the tarball failed with EUNSUPPORTEDPROTOCOL. This forced
monorepos migrating to utoo (e.g. eggjs/egg #5959) to keep pnpm around just
for the pack step.

Rewrite both protocols at pack/publish time the way pnpm/bun do:
- `workspace:*`/`^`/`~`/`<range>` -> the linked workspace member's version
- `catalog:`/`catalog:<name>` -> the version pinned in the workspace-root
  `.utoo.toml` (migrated from pnpm-workspace.yaml)

Resolution happens against the discovered workspace root (pm-pack runs inside
a member dir, not the root). The on-disk package.json is left untouched; only
the tarball entry and the publish payload carry the rewritten manifest, and
the payload now reuses the packed manifest so registry metadata and tarball
contents can't drift. Unresolvable specifiers fail loudly rather than ship a
broken tarball. Packages without these protocols are packed verbatim.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new publish_manifest service to normalize package manifests by rewriting workspace: and catalog: dependency protocols to concrete semver ranges during packing and publishing. Feedback focuses on improving error handling by propagating I/O and configuration loading errors instead of silently ignoring them, and refactoring the protocol detection to utilize the existing Protocol abstraction rather than relying on raw string prefix checks.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread crates/pm/src/service/pm_pack.rs Outdated
Comment on lines +148 to +153
async fn load_catalogs(root: &Path) -> Catalogs {
Config::load_from_path(&root.join(".utoo.toml"))
.await
.map(|c| c.catalogs())
.unwrap_or_default()
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Silently ignoring config loading errors with unwrap_or_default() can hide syntax or permission errors in .utoo.toml. We should check if the file exists using try_exists and propagate any other I/O or parsing errors.

async fn load_catalogs(root: &Path) -> Result<Catalogs> {
    let path = root.join(".utoo.toml");
    if path.try_exists()? {
        let config = Config::load_from_path(&path)
            .await
            .context("failed to load .utoo.toml")?;
        Ok(config.catalogs())
    } else {
        Ok(Catalogs::new())
    }
}
References
  1. Propagate I/O errors (e.g., from try_exists) using ? rather than silently ignoring them with unwrap_or(false), to ensure permission or other critical errors fail early and clearly.

Comment thread crates/pm/src/service/publish_manifest.rs
Comment on lines +22 to +23
use utoo_ruborist::manifest::PackageJson;
use utoo_ruborist::spec::{Catalogs, resolve_catalog_spec, resolve_workspace_spec};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Import Protocol from utoo_ruborist::spec to use the established codebase abstraction instead of raw string prefix checks.

Suggested change
use utoo_ruborist::manifest::PackageJson;
use utoo_ruborist::spec::{Catalogs, resolve_catalog_spec, resolve_workspace_spec};
use utoo_ruborist::manifest::PackageJson;
use utoo_ruborist::spec::{Catalogs, Protocol, resolve_catalog_spec, resolve_workspace_spec};

Comment on lines +41 to +45
let needs_workspace = has_spec_with_prefix(pkg, WORKSPACE_PREFIX);
let needs_catalog = has_spec_with_prefix(pkg, CATALOG_PREFIX);
if !needs_workspace && !needs_catalog {
return Ok(None);
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Use the established Protocol abstraction to check for workspace and catalog dependencies instead of raw string prefix checks.

Suggested change
let needs_workspace = has_spec_with_prefix(pkg, WORKSPACE_PREFIX);
let needs_catalog = has_spec_with_prefix(pkg, CATALOG_PREFIX);
if !needs_workspace && !needs_catalog {
return Ok(None);
}
let needs_workspace = has_protocol(pkg, Protocol::Workspace);
let needs_catalog = has_protocol(pkg, Protocol::Catalog);
if !needs_workspace && !needs_catalog {
return Ok(None);
}
References
  1. Prefer using established codebase abstractions (such as Protocol parsing) over raw string prefix checks (like starts_with) for classifying URLs to maintain consistency across the codebase.

Comment thread crates/pm/src/service/publish_manifest.rs
Comment on lines +156 to +167
fn has_spec_with_prefix(pkg: &PackageJson, prefix: &str) -> bool {
[
&pkg.dependencies,
&pkg.dev_dependencies,
&pkg.peer_dependencies,
&pkg.optional_dependencies,
]
.into_iter()
.flatten()
.flat_map(|m| m.values())
.any(|spec| spec.starts_with(prefix))
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Refactor has_spec_with_prefix to has_protocol using the established Protocol abstraction to avoid raw string prefix checks.

fn has_protocol(pkg: &PackageJson, target_proto: Protocol) -> bool {
    [
        &pkg.dependencies,
        &pkg.dev_dependencies,
        &pkg.peer_dependencies,
        &pkg.optional_dependencies,
    ]
    .into_iter()
    .flatten()
    .flat_map(|m| m.values())
    .any(|spec| {
        Protocol::strip_prefix(spec)
            .map(|(proto, _)| proto == target_proto)
            .unwrap_or(false)
    })
}
References
  1. Prefer using established codebase abstractions (such as Protocol parsing) over raw string prefix checks (like starts_with) for classifying URLs to maintain consistency across the codebase.

elrrrrrrr and others added 6 commits June 3, 2026 22:56
`pack()` computed the workspace:/catalog: rewrite (and serialized the
override bytes) before running the `prepack` lifecycle script. npm/pnpm pack
the post-`prepack` manifest, so a prepack script that mutates package.json
(version bump, stripped fields) was silently dropped from the rewritten
tarball entry and the reused publish payload.

Run prepack first, then re-read the manifest fresh (get_or_load is cached
from before the script) with a write-through cache refresh, and normalize
that. Verified: a prepack version bump now lands in both the tarball
package.json and the tarball filename while workspace:/catalog: stay rewritten.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…nfig errors

Gemini review follow-ups on the workspace:/catalog: rewrite:
- Detect and dispatch dependency protocols via `Protocol::strip_prefix`
  instead of raw `starts_with("workspace:")`/`"catalog:"` checks, matching
  the codebase's established protocol abstraction.
- `load_catalogs` now returns `Result`: a malformed/unreadable `.utoo.toml`
  fails loudly instead of silently yielding an empty catalog (which read as
  a confusing "no matching catalog entry"). Missing file still maps to empty.
- `append_override` propagates non-NotFound stat errors on package.json
  rather than swallowing them; only a racy NotFound falls back to defaults.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The post-prepack manifest re-read fed a `set_package_json` write-through to
the in-memory cache, but nothing on the pack/publish path reads that cache
afterwards (publish uses pack_result.manifest; collect_pack_files uses the
local pkg). The write-through was a pointless shared-state mutation. Drop it
and read the fresh manifest into a plain typed binding.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… loop

The match-on-protocol logic was nested four levels deep (for/for/match/arm).
Collapse the two dependency-map loops into one via flat_map and extract the
per-spec rewrite into `rewrite_spec`, so the match sits at the top level of
its own function.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…mports

`std::fs::metadata`, `std::io::ErrorKind::NotFound`, `tar::Header` and
`tar::EntryType` were used fully-qualified inline, against the project's
top-use convention. Import them (fs, ErrorKind, Header, EntryType) and
reference the short names.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
New fixture e2e/pm/pack-protocols: a workspace member depends on a sibling
via workspace:^ / workspace:~ / workspace:* and on a catalog: entry. Case 10c
runs `utoo pm-pack` and asserts the packed package.json carries concrete
versions (^2.4.1 / ~2.4.1 / 2.4.1 / ^4.17.21), no raw workspace:/catalog:
specifiers survive, and the on-disk source manifest is left untouched.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@elrrrrrrr elrrrrrrr marked this pull request as ready for review June 3, 2026 15:47
@elrrrrrrr elrrrrrrr requested review from fireairforce and killagu June 3, 2026 15:48

@killagu killagu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@elrrrrrrr elrrrrrrr merged commit 1f48ea9 into next Jun 3, 2026
32 checks passed
@elrrrrrrr elrrrrrrr deleted the fix/pm-pack-workspace-catalog-3094 branch June 3, 2026 15:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ut pm-pack: resolve workspace: and catalog: protocols in emitted manifests

2 participants