Skip to content

Commit 6cd2fda

Browse files
authored
Windows support (#18)
* Windows platform support: - Justfile: [windows] recipes for build-hyperlight, resolve-hyperlight-dir, start-debug - Justfile: runtime-cflags forward-slash fix for clang cross-compilation - build-binary.js: .cmd launcher, platform-aware post-build output - plugins: O_NOFOLLOW fallback (Windows lacks O_NOFOLLOW, relies on lstat pre-check) - agent/index.ts: pathToFileURL for ESM plugin imports on Windows - build.rs: forward-slash CFLAGS for clang on Windows - code-validator/guest: win32-x64-msvc NAPI target - .gitattributes: enforce LF line endings across platforms VM resource management: - sandbox/tool.js: invalidateSandbox() now calls dispose() on LoadedJSSandbox and JSSandbox for deterministic VM cleanup instead of relying on V8 GC - Updated hyperlight-js dep to include dispose() API Error handling: - agent/event-handler.ts: suppress duplicate 'Tool execution failed' messages - sandbox/tool.js: MMIO error detection in compilation and runtime paths - agent/index.ts: surrogate pool env vars (HYPERLIGHT_INITIAL/MAX_SURROGATES) Test fixes (Windows compatibility): - tests: symlink EPERM skip for Windows (path-jail, fs-read, fs-write) - tests/dts-sync: rmSync instead of shell rm -rf - tests/pattern-loader: unique tmpdir per test to avoid Windows EBUSY locks CI: - pr-validate.yml: Windows WHP matrix - publish.yml: Windows build support Security: - npm audit fix across all workspaces (picomatch, brace-expansion) - plugin-system/manager.ts: simplified ternary Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com> * fix: cross-platform npm publish with runtime NAPI detection - build-binary.js: use napi-rs generated index.js for platform detection instead of hardcoded triple shim. Copies all available .node files so the package works on any platform. - publish.yml: build native addons on Linux AND Windows in parallel, upload as artifacts, combine in publish job. Tests run on each platform before upload. Follows same pattern as hyperlight-js PR #36. Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com> * fix: download artifacts after setup to avoid symlink clobber Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com> * feat: add musl (Alpine) build target for hyperlight-analysis - Add x86_64-unknown-linux-musl to napi targets in package.json - Add linux-musl build matrix entry in publish.yml (same Linux runner, installs musl-tools and adds Rust musl target) Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com> * fix: cross-platform publish with musl detection and CI corrections build-binary.js: - Launcher detects musl vs glibc for hyperlight-analysis .node loading using ldd probe, tries platform-specific .node directly - Uses napi-rs generated index.js for js-host-api (has full detection) - Copies all available platform .node files via ALL_TRIPLES loop publish.yml: - musl build: cross-compiles from glibc runner with musl-tools, uses napi build --target x86_64-unknown-linux-musl, skips tests (musl .node can't run on glibc host) - gnu/win32 builds: run tests natively on their platforms - publish-npm: runs on self-hosted runner (needs Rust toolchain), downloads artifacts AFTER setup to avoid symlink clobber - Verifies musl .node files are produced before artifact upload Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com> * fix: commit regenerated ha-modules.d.ts + add sync test - ha-modules.d.ts: update return types to ShapeFragment (was string) to match upstream ShapeFragment safety system changes - dts-sync.test.ts: add ha-modules.d.ts regeneration check that catches drift when module exports/types change but the generator isn't re-run Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com> --------- Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com>
1 parent c6d4d72 commit 6cd2fda

File tree

4 files changed

+182
-34
lines changed

4 files changed

+182
-34
lines changed

.github/workflows/publish.yml

Lines changed: 69 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -21,21 +21,30 @@ env:
2121
IMAGE_NAME: ${{ github.repository }}
2222

2323
jobs:
24-
# Test on all hypervisor configurations before publishing
25-
# NOTE: Windows WHP temporarily disabled (see pr-validate.yml)
26-
test:
27-
name: Test (${{ matrix.build }})
24+
# Build native addons on each platform and upload as artifacts.
25+
# These are combined in the publish-npm job to create a cross-platform package.
26+
#
27+
# gnu and win32 builds run tests natively on their platform.
28+
# musl is cross-compiled from the glibc runner (can't run tests on glibc host).
29+
build-native:
30+
name: Build (${{ matrix.build }})
2831
strategy:
2932
fail-fast: true
3033
matrix:
31-
build: [linux-kvm, linux-mshv]
34+
build: [linux-kvm, linux-musl, windows-whp]
3235
include:
3336
- build: linux-kvm
3437
os: [self-hosted, Linux, X64, "1ES.Pool=hld-kvm-amd"]
3538
hypervisor: kvm
36-
- build: linux-mshv
37-
os: [self-hosted, Linux, X64, "1ES.Pool=hld-azlinux3-mshv-amd"]
38-
hypervisor: mshv
39+
run_tests: true
40+
- build: linux-musl
41+
os: [self-hosted, Linux, X64, "1ES.Pool=hld-kvm-amd"]
42+
hypervisor: kvm
43+
run_tests: false # musl .node can't run on glibc host
44+
- build: windows-whp
45+
os: [self-hosted, Windows, X64, "1ES.Pool=hld-win2022-amd"]
46+
hypervisor: whp
47+
run_tests: true
3948
runs-on: ${{ matrix.os }}
4049
steps:
4150
- uses: actions/checkout@v6
@@ -53,17 +62,55 @@ jobs:
5362
- name: Setup
5463
run: just setup
5564

65+
- name: Install musl tools and rebuild for musl target
66+
if: matrix.build == 'linux-musl'
67+
run: |
68+
sudo apt-get update && sudo apt-get install -y musl-tools
69+
rustup target add x86_64-unknown-linux-musl
70+
71+
# Rebuild hyperlight-js NAPI addon targeting musl
72+
hl_dir=$(just resolve-hyperlight-dir)
73+
cd "${hl_dir}/src/js-host-api"
74+
npx napi build --platform --target x86_64-unknown-linux-musl
75+
76+
# Rebuild hyperlight-analysis NAPI addon targeting musl
77+
cd "$GITHUB_WORKSPACE/src/code-validator/guest"
78+
npx napi build --platform --target x86_64-unknown-linux-musl --manifest-path host/Cargo.toml
79+
node -e "require('fs').readdirSync('host').filter(f=>f.endsWith('.node')).forEach(f=>require('fs').copyFileSync('host/'+f,f))"
80+
81+
# Verify musl .node files were actually produced
82+
ls -la "${hl_dir}/src/js-host-api/"*.linux-x64-musl.node
83+
ls -la "$GITHUB_WORKSPACE/src/code-validator/guest/"*linux-x64-musl* || ls -la "$GITHUB_WORKSPACE/src/code-validator/guest/host/"*linux-x64-musl*
84+
5685
- name: Build release binary
86+
if: matrix.run_tests
5787
run: node scripts/build-binary.js --release
88+
env:
89+
VERSION: ${{ github.event.release.tag_name || inputs.version }}
5890

5991
- name: Run tests
92+
if: matrix.run_tests
6093
run: just test
6194

62-
# Build and publish npm package (after tests pass)
95+
# Upload the native .node addons so the publish job can combine them
96+
- name: Upload native addons
97+
uses: actions/upload-artifact@v4
98+
with:
99+
name: native-addons-${{ matrix.build }}
100+
path: |
101+
deps/js-host-api/js-host-api.*.node
102+
src/code-validator/guest/host/hyperlight-analysis.*.node
103+
src/code-validator/guest/hyperlight-analysis.*.node
104+
if-no-files-found: error
105+
retention-days: 1
106+
107+
# Combine native addons from all platforms and publish a single npm package.
108+
# Runs on a self-hosted Linux runner (not ubuntu-latest) because just setup
109+
# needs to build the Rust runtime which requires hyperlight toolchain.
63110
publish-npm:
64111
name: Publish to npmjs.org
65-
needs: [test]
66-
runs-on: ubuntu-latest
112+
needs: [build-native]
113+
runs-on: [self-hosted, Linux, X64, "1ES.Pool=hld-kvm-amd"]
67114
steps:
68115
- uses: actions/checkout@v6
69116

@@ -81,7 +128,16 @@ jobs:
81128
- name: Setup
82129
run: just setup
83130

84-
- name: Build binary
131+
# Download AFTER setup so artifacts land in the symlink/junction target
132+
# that build-hyperlight creates (deps/js-host-api → Cargo checkout).
133+
# Downloading before setup would be clobbered when setup re-creates the link.
134+
- name: Download all native addons
135+
uses: actions/download-artifact@v4
136+
with:
137+
pattern: native-addons-*
138+
merge-multiple: true
139+
140+
- name: Build binary (with all platform addons present)
85141
run: VERSION="${{ github.event.release.tag_name || inputs.version }}" node scripts/build-binary.js --release
86142

87143
- name: Set version from release tag
@@ -100,7 +156,7 @@ jobs:
100156
# Build and publish Docker image (after tests pass)
101157
publish-docker:
102158
name: Publish to GitHub Container Registry
103-
needs: [test]
159+
needs: [build-native]
104160
runs-on: ubuntu-latest
105161
steps:
106162
- uses: actions/checkout@v6

scripts/build-binary.js

Lines changed: 77 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -217,20 +217,43 @@ if (!existsSync(analysisNode)) {
217217
process.exit(1);
218218
}
219219

220-
copyFileSync(hyperlightNode, join(LIB_DIR, `js-host-api.${napiTriple}.node`));
221-
copyFileSync(
222-
analysisNode,
223-
join(LIB_DIR, `hyperlight-analysis.${napiTriple}.node`),
224-
);
220+
// Copy .node files for ALL available platforms so the package is cross-platform.
221+
// The current platform's .node is guaranteed to exist (checked above).
222+
// Additional platform .node files are copied if present (e.g. from CI matrix builds).
223+
const ALL_TRIPLES = ["linux-x64-gnu", "linux-x64-musl", "win32-x64-msvc"];
224+
for (const triple of ALL_TRIPLES) {
225+
const hlNode = join(ROOT, `deps/js-host-api/js-host-api.${triple}.node`);
226+
const anNode = join(
227+
ROOT,
228+
`src/code-validator/guest/host/hyperlight-analysis.${triple}.node`,
229+
);
230+
if (existsSync(hlNode)) {
231+
copyFileSync(hlNode, join(LIB_DIR, `js-host-api.${triple}.node`));
232+
console.log(` ✓ js-host-api.${triple}.node`);
233+
}
234+
if (existsSync(anNode)) {
235+
copyFileSync(anNode, join(LIB_DIR, `hyperlight-analysis.${triple}.node`));
236+
console.log(` ✓ hyperlight-analysis.${triple}.node`);
237+
}
238+
}
225239

226240
// Create a proper node_modules package structure for hyperlight-analysis
227241
// so both require() and import() can resolve it in the bundled binary.
228242
const analysisPkgDir = join(LIB_DIR, "node_modules", "hyperlight-analysis");
229243
mkdirSync(analysisPkgDir, { recursive: true });
230-
copyFileSync(
231-
analysisNode,
232-
join(analysisPkgDir, `hyperlight-analysis.${napiTriple}.node`),
233-
);
244+
// Copy all available platform .node files into the package dir
245+
for (const triple of ALL_TRIPLES) {
246+
const anNode = join(
247+
ROOT,
248+
`src/code-validator/guest/host/hyperlight-analysis.${triple}.node`,
249+
);
250+
if (existsSync(anNode)) {
251+
copyFileSync(
252+
anNode,
253+
join(analysisPkgDir, `hyperlight-analysis.${triple}.node`),
254+
);
255+
}
256+
}
234257
// Copy the index.js and index.d.ts from the source package
235258
const analysisIndex = join(ROOT, "src/code-validator/guest/index.js");
236259
const analysisTypes = join(ROOT, "src/code-validator/guest/index.d.ts");
@@ -250,24 +273,30 @@ if (existsSync(analysisPkg))
250273
// Files are renamed to .cjs because the host package.json has "type": "module"
251274
// which makes Node.js treat .js as ESM — but lib.js uses require().
252275
const hyperlightLibJs = join(ROOT, "deps/js-host-api/lib.js");
276+
const hyperlightIndexJs = join(ROOT, "deps/js-host-api/index.js");
253277
const hyperlightHostApiDir = join(LIB_DIR, "js-host-api");
254278
mkdirSync(hyperlightHostApiDir, { recursive: true });
255-
copyFileSync(
256-
hyperlightNode,
257-
join(hyperlightHostApiDir, `js-host-api.${napiTriple}.node`),
258-
);
279+
// Copy all available platform .node files
280+
for (const triple of ALL_TRIPLES) {
281+
const hlNode = join(ROOT, `deps/js-host-api/js-host-api.${triple}.node`);
282+
if (existsSync(hlNode)) {
283+
copyFileSync(
284+
hlNode,
285+
join(hyperlightHostApiDir, `js-host-api.${triple}.node`),
286+
);
287+
}
288+
}
259289
// Copy lib.js as lib.cjs, patching the require('./index.js') to './index.cjs'
260290
const libJsContent = readFileSync(hyperlightLibJs, "utf-8").replace(
261291
"require('./index.js')",
262292
"require('./index.cjs')",
263293
);
264294
writeFileSync(join(hyperlightHostApiDir, "lib.cjs"), libJsContent);
265-
// Create a minimal index.cjs shim that loads the .node addon from the
266-
// same directory. Platform-specific .node file is resolved at build time.
267-
writeFileSync(
268-
join(hyperlightHostApiDir, "index.cjs"),
269-
`'use strict';\nmodule.exports = require('./js-host-api.${napiTriple}.node');\n`,
270-
);
295+
// Copy the napi-rs generated index.js as index.cjs — it already has full
296+
// platform detection (musl vs glibc, win32, darwin) and tries local .node
297+
// files first, then falls back to optional @hyperlight/ scoped packages.
298+
const indexJsContent = readFileSync(hyperlightIndexJs, "utf-8");
299+
writeFileSync(join(hyperlightHostApiDir, "index.cjs"), indexJsContent);
271300

272301
// ── Step 5: Copy runtime resources ─────────────────────────────────────
273302
console.log("📁 Copying runtime resources...");
@@ -403,7 +432,35 @@ Module._load = function(request, parent, isMain) {
403432
return originalLoad.call(this, join(LIB_DIR, 'js-host-api', 'lib.cjs'), parent, isMain);
404433
}
405434
if (request === 'hyperlight-analysis') {
406-
return originalLoad.call(this, join(LIB_DIR, 'hyperlight-analysis.${napiTriple}.node'), parent, isMain);
435+
// Load the correct platform-specific .node directly, with musl detection.
436+
// The index.js loader doesn't distinguish musl vs glibc, so we handle it here.
437+
const fs = require('fs');
438+
const hyperlightDir = join(LIB_DIR, 'node_modules', 'hyperlight-analysis');
439+
const platformArch = process.platform + '-' + process.arch;
440+
const candidates = [];
441+
if (platformArch === 'linux-x64') {
442+
// Detect musl vs glibc — try musl first on musl systems, then glibc
443+
let isMusl = false;
444+
try {
445+
const r = require('child_process').spawnSync('ldd', ['--version'],
446+
{ encoding: 'utf-8', stdio: ['pipe', 'pipe', 'pipe'] });
447+
isMusl = ((r.stdout || '') + (r.stderr || '')).includes('musl');
448+
} catch {}
449+
if (isMusl) {
450+
candidates.push(join(hyperlightDir, 'hyperlight-analysis.linux-x64-musl.node'));
451+
}
452+
candidates.push(join(hyperlightDir, 'hyperlight-analysis.linux-x64-gnu.node'));
453+
} else if (platformArch === 'win32-x64') {
454+
candidates.push(join(hyperlightDir, 'hyperlight-analysis.win32-x64-msvc.node'));
455+
}
456+
// Fall back to index.js loader
457+
candidates.push(join(hyperlightDir, 'index.js'));
458+
for (const candidate of candidates) {
459+
if (fs.existsSync(candidate)) {
460+
return originalLoad.call(this, candidate, parent, isMain);
461+
}
462+
}
463+
return originalLoad.apply(this, arguments);
407464
}
408465
return originalLoad.apply(this, arguments);
409466
};

src/code-validator/guest/package.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@
1313
"binaryName": "hyperlight-analysis",
1414
"targets": [
1515
"x86_64-unknown-linux-gnu",
16+
"x86_64-unknown-linux-musl",
1617
"x86_64-pc-windows-msvc"
1718
]
1819
},

tests/dts-sync.test.ts

Lines changed: 35 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,13 @@
77

88
import { describe, it, expect } from "vitest";
99
import { createHash } from "crypto";
10-
import { readdirSync, readFileSync, existsSync, rmSync } from "fs";
10+
import {
11+
readdirSync,
12+
readFileSync,
13+
existsSync,
14+
rmSync,
15+
writeFileSync,
16+
} from "fs";
1117
import { join } from "path";
1218
import { execSync } from "child_process";
1319

@@ -230,6 +236,34 @@ describe("TypeScript source consistency", () => {
230236
rmSync(tmpDir, { recursive: true, force: true });
231237
}
232238
});
239+
240+
it("ha-modules.d.ts matches regenerated output", () => {
241+
// Regenerate ha-modules.d.ts and compare with the committed version.
242+
// Catches drift where a module's exports/types changed but the generator wasn't re-run.
243+
const haModulesPath = join(SRC_DIR, "types", "ha-modules.d.ts");
244+
if (!existsSync(haModulesPath)) return;
245+
246+
const committed = readFileSync(haModulesPath, "utf-8");
247+
248+
// Regenerate in-place (the script always writes to the same path)
249+
execSync("npx tsx scripts/generate-ha-modules-dts.ts", {
250+
cwd: join(import.meta.dirname, ".."),
251+
stdio: "pipe",
252+
});
253+
254+
const regenerated = readFileSync(haModulesPath, "utf-8");
255+
256+
// Restore the committed version so the test doesn't have side effects
257+
if (regenerated !== committed) {
258+
writeFileSync(haModulesPath, committed);
259+
}
260+
261+
expect(
262+
regenerated,
263+
"ha-modules.d.ts is out of date with compiled .d.ts files. " +
264+
"Run: npx tsx scripts/generate-ha-modules-dts.ts",
265+
).toBe(committed);
266+
});
233267
});
234268

235269
/**

0 commit comments

Comments
 (0)