Skip to content

Commit 562d4c7

Browse files
X-GuardianSimon Heather
andauthored
fix(provider-generator): copy only publishable files into the jsii compile bundle (#292)
### Description The `@cdktn/provider-generator:build` step (its `postbuild` runs the edge-provider-schema CLI, which generates bindings for all languages) intermittently fails in CI with: ``` [Error: ENOENT: no such file or directory, stat '/__w/cdk-terrain/cdk-terrain/packages/cdktn/.pack-staging/node_modules/semver/ranges/outside.js'] ``` Example failed job: https://github.com/open-constructs/cdk-terrain/actions/runs/28436357153/job/84263776058?pr=285 The cause is in `generateJsiiLanguage` ([`constructs-maker.ts`](packages/@cdktn/provider-generator/src/get/constructs-maker.ts)). To build the jsii compile bundle, it copies each compile dependency into a staging `node_modules`. One of those deps is the `cdktn` package directory itself, and the copy was a blanket `fs.copy(dir, targetdir, { dereference: true })` of the **whole directory**. `packages/cdktn/.pack-staging` is a transient tree that the `pnpm package` step (`packages/cdktn/scripts/pack.mjs`) creates and rewrites — it `rmSync`s it and runs `npm install` into it. Because `.pack-staging` is a subdirectory of `packages/cdktn`, the blanket copy recursed into it. When the two steps overlapped in CI, `fs.copy` listed a file and then `stat`ed it after `pack.mjs` had already removed it mid-install — hence the `ENOENT`. Beyond the race, copying `.pack-staging` (and `dist`) into the bundle is pure waste: they are build outputs, never jsii compile inputs. The fix copies only each dependency's **publishable** files — the set `npm pack` would ship — instead of the whole directory: - **Add `resolvePublishableFiles(dir)`** ([`constructs-maker.ts`](packages/@cdktn/provider-generator/src/get/constructs-maker.ts)): runs `npm pack --dry-run --json` for the dep and returns its publishable file list (dropping the `..`-escape paths npm emits when following pnpm's `node_modules` symlinks). This is the same technique already used by `packages/cdktn/scripts/pack.mjs`. - **Copy that file list into staging** instead of the whole directory. Build outputs — `.pack-staging`, `dist`, `node_modules`, `*.tsbuildinfo` — fall away because they are not in the package's published surface, so the copy no longer walks `.pack-staging` and the race is gone. **Verification:** rebuilt `@cdktn/provider-generator` end-to-end (`tsc` + the `postbuild` edge-CLI that was crashing) with `.pack-staging` present locally — it passes, and all four language targets (Python, Java, C#, Go) plus the TypeScript providers are generated, confirming nothing needed was dropped from the copy. ### Checklist - [x] I have updated the PR title to match [CDKTN's style guide](https://github.com/open-constructs/cdk-terrain/blob/main/CONTRIBUTING.md#pull-requests-1) - [x] I have run the linter on my code locally - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the [documentation](https://github.com/open-constructs/cdk-terrain-docs/tree/main/content) if applicable - [x] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works if applicable - [x] New and existing unit tests pass locally with my changes Co-authored-by: Simon Heather <simon.heather@yulife.com>
1 parent d5af07d commit 562d4c7

1 file changed

Lines changed: 25 additions & 2 deletions

File tree

packages/@cdktn/provider-generator/src/get/constructs-maker.ts

Lines changed: 25 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,23 @@ import { readSchema } from "@cdktn/provider-schema";
2525
const pacmakModule = require.resolve("jsii-pacmak/bin/jsii-pacmak");
2626
const jsiiModule = require.resolve("jsii/bin/jsii");
2727

28+
/**
29+
* Returns the files `npm pack` would publish for the package at `dir`, as paths relative to `dir`. This is the set
30+
* jsii needs to compile against a dependency — its sources and declared metadata — and excludes build outputs
31+
* (`dist`, `.pack-staging`, `node_modules`, tsbuildinfo, ...) that a blanket directory copy would otherwise pull in.
32+
*
33+
* @param dir - Absolute path to the package directory to inspect (must contain a package.json).
34+
* @returns Publishable file paths, relative to `dir`, with `..`-escape paths (from following pnpm symlinks) dropped.
35+
*/
36+
async function resolvePublishableFiles(dir: string): Promise<string[]> {
37+
const stdout = await exec("npm", ["pack", "--dry-run", "--json"], {
38+
cwd: dir,
39+
});
40+
const packInfo = JSON.parse(stdout) as Array<{ files: { path: string }[] }>;
41+
// Drop `..`-escape paths npm emits when following pnpm's node_modules symlinks.
42+
return packInfo[0].files.map((f) => f.path).filter((p) => !p.includes(".."));
43+
}
44+
2845
export interface GenerateJSIIOptions {
2946
entrypoint: string;
3047
deps: string[];
@@ -115,8 +132,14 @@ export async function generateJsiiLanguage(
115132
path.join(staging, "node_modules"),
116133
moduleName,
117134
);
118-
await fs.mkdirp(path.dirname(targetdir));
119-
await fs.copy(dir, targetdir, { dereference: true });
135+
await fs.mkdirp(targetdir);
136+
// Copy only the dep's publishable files (what `npm pack` would ship) into the jsii compile bundle.
137+
for (const relPath of await resolvePublishableFiles(dir)) {
138+
const src = path.join(dir, relPath);
139+
const dst = path.join(targetdir, relPath);
140+
await fs.mkdirp(path.dirname(dst));
141+
await fs.copy(src, dst, { dereference: true });
142+
}
120143

121144
// add to "deps" and "peer deps"
122145
if (!moduleName.startsWith("@types/")) {

0 commit comments

Comments
 (0)