feat: nix support for the nix folks#3924
feat: nix support for the nix folks#3924rekram1-node merged 86 commits intoanomalyco:devfrom Alb-O:nix-support
Conversation
|
I'm in no position to properly review this, but I'd have two suggestions:
|
|
thanks for ur feedback @lrittel I think some people in the discord are also taking looks at it. |
|
Currently working on the github workflow to automatically update hashes of build deps in flake.nix. Any preferences around this, i.e. triggers or commits from bot? |
|
prolly the same way we do it for automatic version updates of package.json files? woudl that work |
|
Oh boy it was fun trying to get the node modules perfectly reproducible with Bun’s symlink tree, think I found a way though, it builds in github CI and on my local machine now. Next up is support for darwin builds and configuring the workflow automation |
|
amazing thank you @Alb-O |
|
Thank you for your work on this! It looks amazing, and I'm really looking forward to it. Resources like this are incredibly helpful as I work on improving the nixpkgs release - something I've been struggling with lately. |
|
Thank you for the work on this, can't wait to have a proper non-hacked/patched solution for nix! |
|
@rekram1-node Ready for reviewing/testing now (especially Mac+Nix users would be good)
For specific architectures:
|
On > nix run github:Alb-O/opencode/nix-support
error: hash mismatch in fixed-output derivation '/nix/store/09wqyqc7dvm6s8va3hbwc176py3wrnbi-opencode-node_modules-1.0.39.drv':
specified: sha256-AwTmHj8CvTBllQUnP4atM58b0bVMjvXi9yRZt1YTVbE=
got: sha256-Vu1drIvA5tImD4ajk152AE4HQ6yGE/8SaEgsKwgC4zM=
error: Cannot build '/nix/store/pl7kjk0sb72vq2qhzvwyvxp2fm5w2px6-opencode-1.0.39.drv'.
Reason: 1 dependency failed.
Output paths:
/nix/store/r8sqgkwycljr82kfnpjmqgww9yvpamzl-opencode-1.0.39It seems something's not right. Running it with yields a different hash.
|
Thank you for testing, these tiny, tiny, differences between the GitHub runner environment and actual user hardware that results in a different hash is driving me a bit mental, but I'll look into it nonetheless. I had it working well on linux x86_64 (exact same hashes between runner and my machine), so maybe an OS/darwin issue? Would be good to know if linux builds are working consistently Edit: I just realized you meant there was a hash difference between the two different commands which really should give the exact same result. Hm this is a good clue |
|
Ok, may have found the culprit, hopefully this extra optional package pinning solves this. Please test again, same build/run commands as before |
The situation has reversed: > nix run github:Alb-O/opencode/nix-support#packages.x86_64-darwin.default
error: hash mismatch in fixed-output derivation '/nix/store/n8vbgmr3lsn5qixq9gk9y73i673rq3rc-opencode-node_modules-1.0.39.drv':
specified: sha256-MTZm3ogq4XgsglNzaUCFD3RoHFsjGkaTnyoU4fEn+rI=
got: sha256-q1uEMWquAaV1Xd0bzKZKm+S4/uW3xpwadBQU3RPl53U=
error: Cannot build '/nix/store/0yxv492hr4zf4dyjps9m5g5m1wlvngp6-opencode-1.0.39.drv'.
Reason: 1 dependency failed.
Output paths:
/nix/store/231qf56dni35fx7176vd8x5pilc9fpbi-opencode-1.0.39The |
|
aarch64-darwin is failing on the models.dev api.json hash as well: Given that models.dev is external to this repo, I don't think you can have a hard match to api.json that will be sustainable. I think you just have to accept the non-declarativeness of fetching it and taking whatever you're given, maybe with a null fallback, since if you're trying to use opencode in an airgapped environment you probably either know what you're doing or shouldn't be doing it anyway. |
|
I encountered this exact issue when I first packaged OpenCode for nixpkgs. The problem stems from fetching from a link which doesn't pin a specific version - whenever I solved this by packaging |
|
I don't understand the opencode architecture choice, but it seems like (I just asked an agent; I didn't verify:) opencode will crash on first run if api.json isn't accessible (e.g. offline). As a result, it's included in the build even though it's not really needed (or even desirable, frankly). I think the right answer is to fix the application to check for connectivity and cleanly error on first-run instead of forcing a download on build, but that's well beyond the scope of making a nix flake. For the moment, since it doesn't matter, you just need to pick a deterministic way of giving opencode something on first-run, whether you refer to nixpkgs models-dev or just pass a null. |
|
@ndrwstn the macro is needed so there is a copy of available models when you run opencode |
|
Ready to be tested again, models.dev API JSON is now sourced from nixpkgs, and I'm doing extra hash validation on other dependency packages Same commands as before:
For specific architectures:
|
Alb-O
changed the title
nix: improve flake tooling and workflow robustness - Add fetch-depth: 0 to workflow checkout to enable git rebase - Run canonicalizer directly from source instead of copying - Support dynamic branch names in update workflow using GITHUB_REF_NAME The workflow previously failed on rebase because shallow clones lack history. Running the canonicalizer in-place keeps the build tree cleaner and avoids drift. Dynamic branch support allows the workflow to run on any branch instead of hardcoding 'dev'. nix: add commented-out workspace symlink rewriting option Current .bun/node_modules rewriting provides sufficient stability. Added optional extension to rewrite all workspace-level symlinks through the canonical tree if needed in the future.
|
Hash inconsistency issues seem to be finally resolved, pinpointing the issue to symlinked binaries in |
rekram1-node
changed the title
|
@Alb-O is the goat |
|
/oc run |
|
Build completed and changes committed. The SDK build script ran successfully and updated the generated client utilities. |
Point opencode input to the official sst/opencode dev branch (PR #3924 merged) and refresh flake.lock so pinned revisions and narHashes are updated across inputs (nixpkgs, nixpkgs-darwin, nixpkgs-unstable, nixvim, NUR, opencode). Refs anomalyco/opencode#3924
|
Triggeres the same bug on macOS that I have seen before when doing my own packaging. I opened #4575 for discussion. |
|
Something is not quite right... this is what happens on an M3 MBP, how do you generate the hash for the fixed output derivation? Only for one architecture? |
|
I think you just got in between the job that generates the hash for nix not having run yet, at least for me if I wait a little it usually works, the release is out before the new hash is updated. (this is my understanding of the process by looking at the commits and personal experience) |
|
I don't know anything about nix but I know why it keeps behaving weird to some degree Ill try to fix it today, think it just involves moving stuff around |
|
I guess for now I will stick to my own nix build of opencode until this one is more stable. Note: it did build after quite a while even with the error. My approach is to use the pre-built binaries from the release page as non flake inputs, which avoids the entire build and also avoids any fixed output derivation hashes. |
|
Yes, you can simply fetch/patch release tarballs. But this flake is targeted mainly for contributor usage where building and running the dev work tree which doesn't have a release yet is a must. It also helps out the nixpkgs maintainers who prefer packages to be built from source (and cached) |
yes, this flake should definitely become the source of the upstream nix package. |
10 participants
Adds Nix flake (+ supporting CI tools) to build OpenCode from source.
To enable this, a few tweaks were done to the core TS codebase that shouldn't impact non-nix builds/runtimes:
packages/opencode/src/cli/cmd/tui/thread.tsopencode-worker.js) next to the executable. The new logic checks for this bundled worker and prefers it over the development-timeworker.tsURL.packages/opencode/src/provider/models-macro.tsMODELS_DEV_API_JSONenv var pointing to a pre-fetched copy of the models API JSON. During Nix builds, process the pre-fetched json and inject its path via this env var. Allows builds to complete without network calls.packages/sdk/js/src/gen/client/utils.gen.tsconfig.baseUrlto handles cases wherebaseUrlmight be a URL object or other non-string type during Nix builds, where bundling/compilation may preserve different object types.packages/opencode/src/tool/bash.tsGitHub workflow CI:
update-nix-hashes.ymltriggers whenever updates to package/lock files are pushed/merged.hashes.jsonrelates to:
#3540
#3739
#462
#1903