Skip to content

Commit 58cd8f5

Browse files
feat: .npm-extension transformManifest for imperative manifest repairs (#9586)
Implements the accepted RFC [npm/rfcs#903](npm/rfcs#903): a root-owned `.npm-extension.mjs` / `.npm-extension.cjs` file exporting `transformManifest(pkg, context)` that imperatively repairs third-party dependency manifests **before** Arborist finalizes the ideal tree. It is the imperative counterpart to [`packageExtensions` (#9496)](#9496) and operates in the same pre-resolution phase, running **before** `packageExtensions`. ```js // .npm-extension.mjs export function transformManifest(pkg, context) { if (pkg.name === "foo" && pkg.version.startsWith("1.")) { pkg.dependencies = { ...pkg.dependencies, bar: "^2.0.0" }; context.log("added bar to foo@1"); } return pkg; } ``` ## Why `packageExtensions` is declarative JSON: it cannot carry comments or issue links, repeats itself across many packages, is add-only, and lives in `package.json` (so public packages cannot publish while it is present). `.npm-extension` covers the gap for advanced projects that need conditional repairs, deletion, range rewrites, repeated rules expressed as code, stale-repair guards, and a policy location outside the published manifest. ## What it does - **Discovery** — one root `.npm-extension.mjs` or `.npm-extension.cjs` (both present is an error). The `extension-file` config overrides discovery with a project-local path that must resolve inside the project root and use `.mjs`/`.cjs`. - **`transformManifest(pkg, context)`** — receives a deeply isolated copy of the normalized manifest; `context` exposes `log`, `root`, and `extensionPoint`. Must return a manifest synchronously; `null`/primitive/array/promise returns and throws fail the install with a `.npm-extension`-named error. - **Allowlist** — only `dependencies`, `optionalDependencies`, `peerDependencies`, and `peerDependenciesMeta` may change (add, replace, or delete). Any other changed field (`scripts`, `bin`, …) is rejected. pacote's cached manifest is never mutated. - **Caching** — runs at most once per resolved package identity (integrity, else resolved source + name@version); the entry file is hash-keyed so a changed file is reloaded rather than served stale from the module cache. - **Lockfile** — the root entry records `npmExtensionHash` (a format-tagged digest of the file bytes); affected entries record minimal `npmExtensionApplied` provenance. Extension state reuses the existing `lockfileVersion: 4` threshold. - **Re-resolution** — changing or removing the file re-resolves the affected packages on the next `npm install`, reverting transforms that no longer apply. - **`npm ci`** — never imports or executes the file; it validates the recorded hash and reifies the locked graph (which already carries the extension-influenced edges). - **Configs** — `ignore-extension` disables import/execution; `ignore-scripts` implies it; `extension-file` is honored only from project config or the command line, never from user/global/builtin sources. - **Workspaces** — a `.npm-extension` file in a non-root workspace is ignored with a warning; only the workspace root's file is honored. - **Visibility** — `npm explain` annotates extension-changed edges and `npm ls` (human + `--json`) surfaces the provenance. - **Publish** — companion change in `npm-packlist` force-excludes root `.npm-extension.{mjs,cjs}` from package tarballs. ## Companion change Requires [npm/npm-packlist#294](npm/npm-packlist#294) to exclude root `.npm-extension.{mjs,cjs}` from tarballs. `pacote`/CLI will pick this up via a version bump once that publishes. ## Notes / out of scope for this PR One item is deferred for a genuine structural reason the RFC itself flags: - **Local `file:`/`link:`/directory sources.** `transformManifest` applies to fetched manifests (registry, git, remote tarball, `file:` tarballs) and is re-derived on the installed tree across all install strategies including `install-strategy=linked`. It is **not yet** applied to local sources that create `Link` nodes directly and bypass the fetch phase — the RFC flags this as net-new wiring ("npm must add an analogous pre-edge-read transform path for the `Link` target"). Follow-up. ## References Implements npm/rfcs#903 Builds on #9496 Companion: npm/npm-packlist#294
1 parent ce11dcf commit 58cd8f5

34 files changed

Lines changed: 1598 additions & 21 deletions

docs/lib/build.js

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -107,6 +107,7 @@ const generateNav = async (contentPath, navPath) => {
107107
'/configuring-npm/npmrc',
108108
'/configuring-npm/package-json',
109109
'/configuring-npm/package-lock-json',
110+
'/configuring-npm/npm-extension',
110111
]
111112

112113
// Hardcoded order for using-npm section (only urls - title/description come from frontmatter)
Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
---
2+
title: .npm-extension
3+
section: 5
4+
description: Imperative, root-owned manifest repairs
5+
---
6+
7+
### Description
8+
9+
A root-owned `.npm-extension.mjs` or `.npm-extension.cjs` file lets a project imperatively repair the manifests of third-party dependencies before npm resolves the dependency tree. It exports a `transformManifest(pkg, context)` function that receives a candidate dependency manifest and returns the effective manifest npm should use.
10+
11+
`.npm-extension` is the imperative counterpart to the declarative [`packageExtensions`](/configuring-npm/package-json#packageextensions) field, and runs in the same pre-resolution phase, **before** `packageExtensions`. Prefer `packageExtensions` for simple, data-only repairs; reach for `.npm-extension` when you need comments and links explaining a repair, conditional logic, repeated repairs expressed as code, deletion or range rewrites, stale-repair guards, or a policy location outside `package.json`.
12+
13+
### Example
14+
15+
```js
16+
// .npm-extension.mjs
17+
export function transformManifest (pkg, context) {
18+
if (pkg.name === 'foo' && pkg.version.startsWith('1.')) {
19+
pkg.dependencies = { ...pkg.dependencies, bar: '^2.0.0' }
20+
context.log(`added bar to ${pkg.name}@${pkg.version}`)
21+
}
22+
return pkg
23+
}
24+
```
25+
26+
The `.cjs` form uses CommonJS exports instead:
27+
28+
```js
29+
// .npm-extension.cjs
30+
module.exports = {
31+
transformManifest (pkg, context) {
32+
return pkg
33+
},
34+
}
35+
```
36+
37+
### The `transformManifest` function
38+
39+
`transformManifest(pkg, context)` receives a deeply isolated copy of a candidate dependency manifest. It may mutate and return that copy, or return a new manifest object. It **must** return a manifest object synchronously; returning `null`, `undefined`, a primitive, an array, or a promise fails the install.
40+
41+
The `context` argument is intentionally small:
42+
43+
* `context.log(message)` writes an npm debug log message.
44+
* `context.root` is the absolute path to the project root.
45+
* `context.extensionPoint` is the string `"transformManifest"`.
46+
47+
npm provides no registry, fetch, lockfile, or extraction helpers. Keep the extension file self-contained or limited to Node builtins; npm does not guarantee that project dependencies are available when the file is loaded.
48+
49+
### Supported mutations
50+
51+
Only the four resolution-affecting fields may change:
52+
53+
* `dependencies`
54+
* `optionalDependencies`
55+
* `peerDependencies`
56+
* `peerDependenciesMeta`
57+
58+
Within those fields you may add, replace, or delete entries. Changing any other field (such as `scripts`, `bin`, `engines`, `os`, `cpu`, `exports`, or `main`) is rejected, and the install fails with an error naming `.npm-extension` and the package being processed. The package tarball and the installed `node_modules/<pkg>/package.json` are never rewritten.
59+
60+
### Discovery and `extension-file`
61+
62+
npm looks for a single `.npm-extension.mjs` or `.npm-extension.cjs` at the project root (the workspace root in a workspace project). Having both files present is an error. A `.npm-extension` file in a dependency or in a non-root workspace is ignored; a non-root workspace file produces a warning.
63+
64+
The [`extension-file`](/using-npm/config#extension-file) config selects a different project-local file. It must resolve inside the project root and use a `.mjs` or `.cjs` extension, and it is honored only from project config or the command line — never from user, global, or builtin config.
65+
66+
### Interaction with `packageExtensions` and `overrides`
67+
68+
When both are present, `transformManifest` runs first and `packageExtensions` is applied to its output. Avoid targeting the same package with both unless you intend to rely on that ordering. `overrides` still controls the final resolution target of any edge, including edges created by `transformManifest`.
69+
70+
### Lockfile and `npm ci`
71+
72+
A lockfile influenced by `.npm-extension` records an `npmExtensionHash` (a digest of the selected file's bytes and module format) on its root entry, and minimal `npmExtensionApplied` provenance on each affected package entry. Extension state requires `lockfileVersion: 4`.
73+
74+
Changing the file's contents makes `npm install` re-resolve the affected packages. `npm ci` does **not** import or execute `.npm-extension`; it verifies the recorded hash against the file and reifies the locked graph, failing if the file and lockfile disagree (or if one has extension state and the other does not).
75+
76+
The hash proves only that the install uses the same extension file bytes that generated the lockfile. It does not make arbitrary JavaScript deterministic: extension output that depends on environment variables, the network, the clock, or files imported by the extension can still produce non-reproducible installs. Treat `.npm-extension` as trusted, deterministic project code, and only enable it in repositories you trust.
77+
78+
### Disabling
79+
80+
Set [`ignore-extension`](/using-npm/config#ignore-extension) to skip importing and executing `.npm-extension`. [`ignore-scripts`](/using-npm/config#ignore-scripts) implies `ignore-extension`, since both disable root-owned install-time code. `npm ci` still verifies the file hash even when execution is disabled.
81+
82+
### Publishing
83+
84+
`.npm-extension.mjs` and `.npm-extension.cjs` are project configuration, not package contents. npm excludes the root file from the package tarball produced by `npm pack` and `npm publish`, even when the package's `files` list would include it, so a public package can keep `.npm-extension` in its repository for local use without publishing it.
85+
86+
### See also
87+
88+
* [package.json `packageExtensions`](/configuring-npm/package-json#packageextensions)
89+
* [package-lock.json](/configuring-npm/package-lock-json)
90+
* [config](/using-npm/config)

docs/lib/content/nav.yml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -232,6 +232,9 @@
232232
- title: package-lock.json
233233
url: /configuring-npm/package-lock-json
234234
description: A manifestation of the manifest
235+
- title: .npm-extension
236+
url: /configuring-npm/npm-extension
237+
description: Imperative, root-owned manifest repairs
235238
- title: Using npm
236239
shortName: Using
237240
url: /using-npm

lib/commands/ci.js

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ const fs = require('node:fs/promises')
66
const path = require('node:path')
77
const { log, time } = require('proc-log')
88
const validateLockfile = require('../utils/validate-lockfile.js')
9-
const { validatePackageExtensions } = require('../utils/validate-lockfile.js')
9+
const { validatePackageExtensions, validateNpmExtension } = require('../utils/validate-lockfile.js')
1010
const ArboristWorkspaceCmd = require('../arborist-cmd.js')
1111
const getWorkspaces = require('../utils/get-workspaces.js')
1212

@@ -66,6 +66,9 @@ class CI extends ArboristWorkspaceCmd {
6666
save: false, // npm ci should never modify the lockfile or package.json
6767
workspaces: this.workspaceNames,
6868
allowScripts: allowScriptsPolicy,
69+
// npm ci reifies the locked graph, which already carries extension-influenced edges, so it must never import or execute .npm-extension.
70+
// The extension file hash is still validated below, independent of execution.
71+
ignoreExtension: true,
6972
}
7073

7174
// generate an inventory from the virtual tree in the lockfile
@@ -92,6 +95,16 @@ class CI extends ArboristWorkspaceCmd {
9295
const errors = validateLockfile(virtualInventory, arb.idealTree.inventory)
9396
// Verifies that the root packageExtensions state matches the lockfile and is still consistent with the locked tree.
9497
errors.push(...validatePackageExtensions(virtualArb.virtualTree, arb.idealTree))
98+
// Verifies that the root .npm-extension file matches the lockfile hash.
99+
// The hash comes from discovering the file (no import or execution), so this holds even under ignore-extension/ignore-scripts.
100+
const { NpmExtension } = require('@npmcli/arborist')
101+
let fileHash = null
102+
try {
103+
fileHash = new NpmExtension({ root: where, extensionFile: opts.extensionFile }).hash
104+
} catch (err) {
105+
errors.push(`Invalid: ${err.message}`)
106+
}
107+
errors.push(...validateNpmExtension(virtualArb.virtualTree, fileHash))
95108
if (errors.length) {
96109
throw this.usageError(
97110
'`npm ci` can only install packages when your package.json and package-lock.json are in sync. ' +

lib/commands/ls.js

Lines changed: 13 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -278,8 +278,8 @@ const augmentItemWithIncludeMetadata = (node, item) => {
278278
return item
279279
}
280280

281-
// Render a node's packageExtensions provenance as a short "field.name" list, empty when none.
282-
const formatPackageExtensions = (applied) => {
281+
// Render a manifest-extension provenance object as a short "field.name" list, empty when none.
282+
const formatExtensionApplied = (applied) => {
283283
if (!applied) {
284284
return ''
285285
}
@@ -354,8 +354,13 @@ const getHumanOutputItem = (node, { args, chalk, global, long }) => {
354354
: ''
355355
) +
356356
(
357-
formatPackageExtensions(node.packageExtensionsApplied)
358-
? ' ' + chalk.dim(`packageExtensions: ${formatPackageExtensions(node.packageExtensionsApplied)}`)
357+
formatExtensionApplied(node.packageExtensionsApplied)
358+
? ' ' + chalk.dim(`packageExtensions: ${formatExtensionApplied(node.packageExtensionsApplied)}`)
359+
: ''
360+
) +
361+
(
362+
formatExtensionApplied(node.npmExtensionApplied)
363+
? ' ' + chalk.dim(`.npm-extension: ${formatExtensionApplied(node.npmExtensionApplied)}`)
359364
: ''
360365
) +
361366
(isGitNode(node) ? ` (${node.resolved})` : '') +
@@ -386,6 +391,10 @@ const getJsonOutputItem = (node, { global, long }) => {
386391
item.packageExtensionsApplied = node.packageExtensionsApplied
387392
}
388393

394+
if (node.npmExtensionApplied) {
395+
item.npmExtensionApplied = node.npmExtensionApplied
396+
}
397+
389398
item[_name] = node.name
390399

391400
// special formatting for top-level package name

lib/npm.js

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -118,6 +118,19 @@ class Npm {
118118
return { exec: false }
119119
}
120120

121+
// extension-file selects root-owned install-time code, so it is only honored from project config or the command line.
122+
// This is checked after #display.load() so the error is surfaced to the user instead of being swallowed during early config loading.
123+
const extensionFile = this.config.get('extension-file')
124+
if (extensionFile != null) {
125+
const where = this.config.find('extension-file')
126+
if (!['cli', 'project', 'default'].includes(where)) {
127+
throw Object.assign(
128+
new Error(`\`extension-file\` may only be set in project config or on the command line, not from ${where} config`),
129+
{ code: 'ENPMEXTENSIONCONFIG' }
130+
)
131+
}
132+
}
133+
121134
// mkdir this separately since the logs dir can be set to a different location.
122135
// if this fails, then we don't have a cache dir, but we don't want to fail immediately since the command might not need a cache dir (like `npm --version`)
123136
await time.start('npm:load:mkdirpcache', () =>

lib/utils/explain-dep.js

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -76,7 +76,7 @@ const explainDependents = ({ dependents }, depth, chalk, seen) => {
7676
}
7777

7878
const explainEdge = (
79-
{ name, type, bundled, from, spec, rawSpec, overridden, packageExtensions },
79+
{ name, type, bundled, from, spec, rawSpec, overridden, packageExtensions, npmExtension },
8080
depth, chalk, seen = new Set()
8181
) => {
8282
let dep = type === 'workspace'
@@ -93,9 +93,14 @@ const explainEdge = (
9393
? chalk.dim(` (added by packageExtensions["${packageExtensions.selector}"].${packageExtensions.field}.${name})`)
9494
: ''
9595

96+
// note an edge created or changed by a root .npm-extension repair
97+
const npmExtMsg = npmExtension
98+
? chalk.dim(` (changed by .npm-extension ${npmExtension.extensionPoint} ${npmExtension.field}.${name})`)
99+
: ''
100+
96101
return (type === 'prod' ? '' : `${colorType(type, chalk)} `) +
97102
(bundled ? `${colorType('bundled', chalk)} ` : '') +
98-
`${dep}${fromMsg}${extMsg}`
103+
`${dep}${fromMsg}${extMsg}${npmExtMsg}`
99104
}
100105

101106
const explainFrom = (from, depth, chalk, seen) => {

lib/utils/validate-lockfile.js

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -98,5 +98,33 @@ function validatePackageExtensions (virtualTree, idealTree) {
9898
return errors
9999
}
100100

101+
// validates that the .npm-extension state recorded in the lockfile still matches the selected extension file.
102+
// Validation is hash-based: arbitrary code has no selector to re-check, so a matching hash is trusted and a mismatch fails.
103+
// fileHash is computed from the on-disk file (discovery only, no execution), so this holds even under ignore-extension/ignore-scripts.
104+
// The lockfile carries extension state if it records a root hash or any per-package npmExtensionApplied provenance.
105+
// Returns an array of human-readable error strings, empty when valid.
106+
function validateNpmExtension (virtualTree, fileHash) {
107+
const lockHash = virtualTree?.meta?.npmExtensionHash || null
108+
const hasProvenance = !!virtualTree &&
109+
[...virtualTree.inventory.values()].some(node => node.npmExtensionApplied)
110+
fileHash = fileHash || null
111+
112+
if (fileHash) {
113+
if (!lockHash) {
114+
return ['Missing: .npm-extension state from lock file']
115+
}
116+
if (lockHash !== fileHash) {
117+
return ['Invalid: .npm-extension file does not match the lock file']
118+
}
119+
return []
120+
}
121+
// no extension file present
122+
if (lockHash || hasProvenance) {
123+
return ['Invalid: lock file records .npm-extension state but no .npm-extension file is present']
124+
}
125+
return []
126+
}
127+
101128
module.exports = validateLockfile
102129
module.exports.validatePackageExtensions = validatePackageExtensions
130+
module.exports.validateNpmExtension = validateNpmExtension

tap-snapshots/test/lib/commands/config.js.test.cjs

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,7 @@ exports[`test/lib/commands/config.js TAP config list --json > output matches sna
5858
"expect-result-count": null,
5959
"expect-results": null,
6060
"expires": null,
61+
"extension-file": null,
6162
"fetch-retries": 2,
6263
"fetch-retry-factor": 10,
6364
"fetch-retry-maxtimeout": 60000,
@@ -76,6 +77,7 @@ exports[`test/lib/commands/config.js TAP config list --json > output matches sna
7677
"heading": "npm",
7778
"https-proxy": null,
7879
"if-present": false,
80+
"ignore-extension": false,
7981
"ignore-scripts": false,
8082
"include": [],
8183
"include-staged": false,
@@ -254,6 +256,7 @@ engine-strict = false
254256
expect-result-count = null
255257
expect-results = null
256258
expires = null
259+
extension-file = null
257260
fetch-retries = 2
258261
fetch-retry-factor = 10
259262
fetch-retry-maxtimeout = 60000
@@ -273,6 +276,7 @@ heading = "npm"
273276
https-proxy = null
274277
if-present = false
275278
ignore-existing = false
279+
ignore-extension = false
276280
ignore-patch-failures = false
277281
ignore-scripts = false
278282
include = []

tap-snapshots/test/lib/commands/install.js.test.cjs

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -135,8 +135,8 @@ verbose stack Error: The developer of this package has specified the following t
135135
verbose stack Invalid devEngines.runtime
136136
verbose stack Invalid name "nondescript" does not match "node" for "runtime"
137137
verbose stack at Install.checkDevEngines ({CWD}/lib/base-cmd.js:249:27)
138-
verbose stack at MockNpm.execCommandClass ({CWD}/lib/npm.js:281:7)
139-
verbose stack at MockNpm.exec ({CWD}/lib/npm.js:181:9)
138+
verbose stack at MockNpm.execCommandClass ({CWD}/lib/npm.js:294:7)
139+
verbose stack at MockNpm.exec ({CWD}/lib/npm.js:194:9)
140140
error code EBADDEVENGINES
141141
error EBADDEVENGINES The developer of this package has specified the following through devEngines
142142
error EBADDEVENGINES Invalid devEngines.runtime
@@ -200,8 +200,8 @@ verbose stack Error: The developer of this package has specified the following t
200200
verbose stack Invalid devEngines.runtime
201201
verbose stack Invalid name "nondescript" does not match "node" for "runtime"
202202
verbose stack at Install.checkDevEngines ({CWD}/lib/base-cmd.js:249:27)
203-
verbose stack at MockNpm.execCommandClass ({CWD}/lib/npm.js:281:7)
204-
verbose stack at MockNpm.exec ({CWD}/lib/npm.js:181:9)
203+
verbose stack at MockNpm.execCommandClass ({CWD}/lib/npm.js:294:7)
204+
verbose stack at MockNpm.exec ({CWD}/lib/npm.js:194:9)
205205
error code EBADDEVENGINES
206206
error EBADDEVENGINES The developer of this package has specified the following through devEngines
207207
error EBADDEVENGINES Invalid devEngines.runtime
@@ -226,8 +226,8 @@ verbose stack Error: The developer of this package has specified the following t
226226
verbose stack Invalid devEngines.runtime
227227
verbose stack Invalid name "nondescript" does not match "node" for "runtime"
228228
verbose stack at Install.checkDevEngines ({CWD}/lib/base-cmd.js:249:27)
229-
verbose stack at MockNpm.execCommandClass ({CWD}/lib/npm.js:281:7)
230-
verbose stack at MockNpm.exec ({CWD}/lib/npm.js:181:9)
229+
verbose stack at MockNpm.execCommandClass ({CWD}/lib/npm.js:294:7)
230+
verbose stack at MockNpm.exec ({CWD}/lib/npm.js:194:9)
231231
error code EBADDEVENGINES
232232
error EBADDEVENGINES The developer of this package has specified the following through devEngines
233233
error EBADDEVENGINES Invalid devEngines.runtime

0 commit comments

Comments
 (0)