Skip to content

Commit ff047c0

Browse files
committed
Codex Infinity v1.3.41 - merge upstream + maintain custom features
2 parents cc81016 + 60b45d9 commit ff047c0

102 files changed

Lines changed: 5755 additions & 963 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
---
2+
name: update-v8-version
3+
description: Update Codex's pinned `v8` / `rusty_v8` versions, validate the release-candidate path, and investigate failed V8 canary or artifact builds. Use when asked to bump V8, update `rusty_v8` artifacts, prepare or validate a V8 release candidate, check `v8-canary`, or diagnose why a V8 version update no longer builds.
4+
---
5+
6+
# Update V8 Version
7+
8+
## Core Workflow
9+
10+
1. Read `third_party/v8/README.md` and follow its version-bump sequence. Treat
11+
that document as the release-process source of truth.
12+
2. Inspect and update the concrete repo surfaces that carry the pin:
13+
- `codex-rs/Cargo.toml`
14+
- `codex-rs/Cargo.lock`
15+
- `MODULE.bazel`
16+
- `third_party/v8/BUILD.bazel`
17+
- `third_party/v8/README.md`
18+
- the matching `third_party/v8/rusty_v8_<version>.sha256` manifest when the
19+
remaining prebuilt inputs change
20+
3. Keep the existing checksum helpers in the loop:
21+
22+
```bash
23+
python3 .github/scripts/rusty_v8_bazel.py update-module-bazel
24+
python3 .github/scripts/rusty_v8_bazel.py check-module-bazel
25+
python3 -m unittest discover -s .github/scripts -p test_rusty_v8_bazel.py
26+
```
27+
28+
4. Validate the release-candidate path before broadening the work:
29+
- Prefer checking the `v8-canary` CI result for the candidate branch or PR
30+
when one exists, using GitHub check tooling or `gh` as appropriate.
31+
- If CI is unavailable or the user asked for a local-only check, run the
32+
closest local validation that is practical for the changed surface and say
33+
explicitly that it is a local substitute, not the full hosted canary.
34+
5. If the canary path passes, stop there. Summarize the result and encourage the
35+
user to commit the candidate changes or proceed with the release flow they
36+
requested. Do not publish tags, releases, or pushes unless the user asked.
37+
38+
## Failure Path
39+
40+
Enter this path only when the canary or local build path fails.
41+
42+
1. Capture the failing target, workflow job, and first actionable error.
43+
2. Compare the currently pinned version with the target version at the relevant
44+
upstream tag or SHA. Inspect both:
45+
- `denoland/rusty_v8`
46+
- upstream V8 source at the target Bazel-pinned version
47+
3. Track build-relevant deltas rather than broad source churn:
48+
- generated binding layout changes
49+
- archive or asset naming changes
50+
- GN/Bazel target changes
51+
- custom libc++ / libc++abi / llvm-libc inputs
52+
- sandbox or pointer-compression feature relationships
53+
- patch hunks in `patches/` that no longer apply or no longer match upstream
54+
4. Trace each failing delta back into Codex's build graph:
55+
- `MODULE.bazel`
56+
- `third_party/v8/BUILD.bazel`
57+
- `.github/scripts/rusty_v8_bazel.py`
58+
- `.github/workflows/v8-canary.yml`
59+
- `.github/workflows/rusty-v8-release.yml`
60+
5. Update only the pieces required to restore the target version's build and
61+
artifact contract. Keep patch explanations and doc changes close to the
62+
affected files.
63+
6. Re-run the focused validation. If it becomes green, return to the normal
64+
workflow and stop with a concise summary plus the remaining release step.
65+
66+
## Reporting
67+
68+
- Say whether validation came from hosted `v8-canary` or from a local
69+
substitute.
70+
- Distinguish "version bump complete" from "release published".
71+
- When blocked, report the upstream delta that matters, the Codex file it hits,
72+
and the next concrete fix to try.
Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
interface:
2+
display_name: "Update V8 Version"
3+
short_description: "Guide V8 bumps and release validation"
4+
default_prompt: "Use $update-v8-version to update Codex to a new v8 release and validate the release-candidate path."
Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
name: setup-msvc-env
2+
description: Expose an MSVC developer environment for the requested Windows target.
3+
inputs:
4+
target:
5+
description: Rust target triple that will be built on this Windows runner.
6+
required: true
7+
host-arch:
8+
description: Optional Visual Studio host architecture override.
9+
required: false
10+
default: ""
11+
12+
runs:
13+
using: composite
14+
steps:
15+
- name: Expose MSVC SDK environment
16+
shell: pwsh
17+
run: '& "$env:GITHUB_ACTION_PATH/setup-msvc-env.ps1" -Target "${{ inputs.target }}" -HostArch "${{ inputs.host-arch }}"'
Lines changed: 257 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,257 @@
1+
param(
2+
[Parameter(Mandatory = $true)]
3+
[string]$Target,
4+
5+
[string]$HostArch = ""
6+
)
7+
8+
# Cargo can cross-compile the Rust code for Windows ARM64 on a Windows x64
9+
# runner, but rustup alone does not expose the matching MSVC/UCRT include and
10+
# library paths. Ask Visual Studio for the target-specific developer
11+
# environment, then persist the relevant variables through GITHUB_ENV so the
12+
# later Cargo step sees the same environment as a normal VsDevCmd shell.
13+
switch ($Target) {
14+
"x86_64-pc-windows-msvc" {
15+
$TargetArch = "x64"
16+
$RequiredComponent = "Microsoft.VisualStudio.Component.VC.Tools.x86.x64"
17+
}
18+
"aarch64-pc-windows-msvc" {
19+
$TargetArch = "arm64"
20+
$RequiredComponent = "Microsoft.VisualStudio.Component.VC.Tools.ARM64"
21+
}
22+
default {
23+
throw "Unsupported Windows MSVC target: $Target"
24+
}
25+
}
26+
27+
# VsDevCmd needs both sides of the cross compile: the architecture of the
28+
# machine running the tools and the architecture of the binaries being linked.
29+
# Infer the host from the runner unless a caller needs to override it.
30+
if (-not $HostArch) {
31+
$HostArch = if ($env:PROCESSOR_ARCHITEW6432 -eq "ARM64" -or $env:PROCESSOR_ARCHITECTURE -eq "ARM64") {
32+
"arm64"
33+
} else {
34+
"x64"
35+
}
36+
}
37+
38+
$VsWhere = "${env:ProgramFiles(x86)}\Microsoft Visual Studio\Installer\vswhere.exe"
39+
if (-not (Test-Path $VsWhere)) {
40+
throw "vswhere.exe not found"
41+
}
42+
43+
# Require the target VC tools component, not merely any Visual Studio install,
44+
# so an x64 archive producer cannot silently link ARM64 tests with the wrong
45+
# SDK/toolchain layout.
46+
$InstallPath = & $VsWhere -latest -products * -requires $RequiredComponent -property installationPath 2>$null
47+
if (-not $InstallPath) {
48+
throw "Could not locate a Visual Studio installation with component $RequiredComponent"
49+
}
50+
51+
$VsDevCmd = Join-Path $InstallPath "Common7\Tools\VsDevCmd.bat"
52+
if (-not (Test-Path $VsDevCmd)) {
53+
throw "VsDevCmd.bat not found at $VsDevCmd"
54+
}
55+
56+
$VarsToExport = @(
57+
"INCLUDE",
58+
"LIB",
59+
"LIBPATH",
60+
"PATH",
61+
"UCRTVersion",
62+
"UniversalCRTSdkDir",
63+
"VCINSTALLDIR",
64+
"VCToolsInstallDir",
65+
"WindowsLibPath",
66+
"WindowsSdkBinPath",
67+
"WindowsSdkDir",
68+
"WindowsSDKLibVersion",
69+
"WindowsSDKVersion"
70+
)
71+
72+
# Run VsDevCmd inside cmd.exe because it is a batch file, then copy just the
73+
# variables Cargo/rustc need into the GitHub Actions environment file. PowerShell
74+
# cannot mutate the parent composite-action environment directly.
75+
$EnvLines = & cmd.exe /c ('"{0}" -no_logo -arch={1} -host_arch={2} >nul && set' -f $VsDevCmd, $TargetArch, $HostArch)
76+
$VcToolsInstallDir = $null
77+
foreach ($Line in $EnvLines) {
78+
if ($Line -notmatch "^(.*?)=(.*)$") {
79+
continue
80+
}
81+
82+
$Name = $Matches[1]
83+
$Value = $Matches[2]
84+
if ($VarsToExport -contains $Name) {
85+
if ($Name -ieq "Path") {
86+
$Name = "PATH"
87+
}
88+
if ($Name -eq "VCToolsInstallDir") {
89+
$VcToolsInstallDir = $Value
90+
}
91+
"$Name=$Value" | Out-File -FilePath $env:GITHUB_ENV -Encoding utf8 -Append
92+
}
93+
}
94+
95+
if (-not $VcToolsInstallDir) {
96+
throw "VCToolsInstallDir was not exported by VsDevCmd.bat"
97+
}
98+
99+
# Prefer Rust's bundled linker when rustup provides one, then Visual Studio's
100+
# LLVM linker, and finally MSVC link.exe. This keeps the cross-compile path close
101+
# to Rust's normal Windows MSVC behavior while still working on runner images
102+
# where one of those linkers is absent.
103+
$Linker = $null
104+
$Rustc = Get-Command rustc -ErrorAction SilentlyContinue
105+
if ($Rustc) {
106+
$Sysroot = (& rustc --print sysroot 2>$null).Trim()
107+
$RustHost = & rustc -vV 2>$null | Select-String "^host: " | ForEach-Object { $_.Line.Substring(6) }
108+
if ($RustHost) {
109+
$RustHost = $RustHost.Trim()
110+
}
111+
if ($Sysroot -and $RustHost) {
112+
$RustLld = Join-Path $Sysroot "lib\rustlib\$RustHost\bin\rust-lld.exe"
113+
if (Test-Path $RustLld) {
114+
$Linker = $RustLld
115+
}
116+
}
117+
}
118+
if (-not $Linker) {
119+
$Linker = Join-Path $InstallPath "VC\Tools\Llvm\x64\bin\lld-link.exe"
120+
}
121+
if (-not (Test-Path $Linker)) {
122+
$Linker = Join-Path $VcToolsInstallDir "bin\Host${HostArch}\${TargetArch}\link.exe"
123+
}
124+
if (-not (Test-Path $Linker)) {
125+
throw "Windows linker not found at $Linker"
126+
}
127+
128+
# rustc passes `/arm64hazardfree` for ARM64 MSVC links. The lld variants on our
129+
# Windows x64 archive producers reject that flag, including when rustc places it
130+
# inside a response file. Compile a tiny forwarding wrapper that strips only
131+
# that unsupported flag, then delegate every other argument to the real linker.
132+
if ($TargetArch -eq "arm64" -and (Split-Path -Leaf $Linker) -match "lld") {
133+
$WrapperDir = Join-Path $env:RUNNER_TEMP "msvc-lld-wrapper"
134+
New-Item -Path $WrapperDir -ItemType Directory -Force | Out-Null
135+
$WrapperPath = Join-Path $WrapperDir "lld-link-wrapper.exe"
136+
$WrapperSource = @'
137+
using System;
138+
using System.Collections.Generic;
139+
using System.Diagnostics;
140+
using System.IO;
141+
using System.Text;
142+
using System.Text.RegularExpressions;
143+
144+
internal static class Program
145+
{
146+
private static int Main(string[] args)
147+
{
148+
var linker = Environment.GetEnvironmentVariable("MSVC_REAL_LINKER");
149+
if (string.IsNullOrEmpty(linker))
150+
{
151+
Console.Error.WriteLine("MSVC_REAL_LINKER is not set");
152+
return 1;
153+
}
154+
155+
var startInfo = new ProcessStartInfo(linker)
156+
{
157+
UseShellExecute = false,
158+
};
159+
var filteredArgs = new List<string> { "-flavor", "link", "/defaultlib:ucrt", "/nodefaultlib:libucrt" };
160+
foreach (var arg in args)
161+
{
162+
if (!string.Equals(arg, "/arm64hazardfree", StringComparison.OrdinalIgnoreCase))
163+
{
164+
filteredArgs.Add(QuoteArgument(FilterResponseFile(arg)));
165+
}
166+
}
167+
startInfo.Arguments = string.Join(" ", filteredArgs);
168+
169+
using var process = Process.Start(startInfo);
170+
if (process is null)
171+
{
172+
Console.Error.WriteLine($"Failed to start linker: {linker}");
173+
return 1;
174+
}
175+
176+
process.WaitForExit();
177+
return process.ExitCode;
178+
}
179+
180+
private static string FilterResponseFile(string argument)
181+
{
182+
if (argument.Length < 2 || argument[0] != '@')
183+
{
184+
return argument;
185+
}
186+
187+
var responsePath = argument.Substring(1);
188+
if (!File.Exists(responsePath))
189+
{
190+
return argument;
191+
}
192+
193+
var filteredResponsePath = Path.Combine(Path.GetTempPath(), Path.GetRandomFileName() + ".rsp");
194+
var responseContents = Regex.Replace(
195+
File.ReadAllText(responsePath),
196+
"/arm64hazardfree",
197+
string.Empty,
198+
RegexOptions.IgnoreCase);
199+
File.WriteAllText(filteredResponsePath, responseContents);
200+
return "@" + filteredResponsePath;
201+
}
202+
203+
private static string QuoteArgument(string argument)
204+
{
205+
if (argument.Length == 0)
206+
{
207+
return "\"\"";
208+
}
209+
if (argument.IndexOfAny(new[] { ' ', '\t', '"' }) < 0)
210+
{
211+
return argument;
212+
}
213+
214+
var quoted = new StringBuilder("\"");
215+
var backslashes = 0;
216+
foreach (var character in argument)
217+
{
218+
if (character == '\\')
219+
{
220+
backslashes++;
221+
continue;
222+
}
223+
if (character == '"')
224+
{
225+
quoted.Append('\\', (backslashes * 2) + 1);
226+
quoted.Append(character);
227+
backslashes = 0;
228+
continue;
229+
}
230+
231+
quoted.Append('\\', backslashes);
232+
backslashes = 0;
233+
quoted.Append(character);
234+
}
235+
quoted.Append('\\', backslashes * 2);
236+
quoted.Append('"');
237+
return quoted.ToString();
238+
}
239+
}
240+
'@
241+
$WrapperSourcePath = Join-Path $WrapperDir "lld-link-wrapper.cs"
242+
$WrapperSource | Out-File -FilePath $WrapperSourcePath -Encoding utf8
243+
$Csc = Join-Path $InstallPath "MSBuild\Current\Bin\Roslyn\csc.exe"
244+
if (-not (Test-Path $Csc)) {
245+
throw "csc.exe not found at $Csc"
246+
}
247+
& $Csc /nologo /target:exe /out:$WrapperPath $WrapperSourcePath
248+
if ($LASTEXITCODE -ne 0) {
249+
throw "Failed to compile lld-link wrapper"
250+
}
251+
"MSVC_REAL_LINKER=$Linker" | Out-File -FilePath $env:GITHUB_ENV -Encoding utf8 -Append
252+
$Linker = $WrapperPath
253+
}
254+
255+
Write-Output "Using Windows linker: $Linker"
256+
$CargoTarget = $Target.ToUpperInvariant().Replace("-", "_")
257+
"CARGO_TARGET_${CargoTarget}_LINKER=$Linker" | Out-File -FilePath $env:GITHUB_ENV -Encoding utf8 -Append

.github/actions/setup-rusty-v8-musl/action.yml

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -31,16 +31,14 @@ runs:
3131
archive_path="${binding_dir}/librusty_v8_release_${TARGET}.a.gz"
3232
binding_path="${binding_dir}/src_binding_release_${TARGET}.rs"
3333
checksums_path="${binding_dir}/rusty_v8_release_${TARGET}.sha256"
34-
checksums_source="${GITHUB_WORKSPACE}/third_party/v8/rusty_v8_${version//./_}.sha256"
3534
3635
mkdir -p "${binding_dir}"
3736
curl -fsSL "${base_url}/librusty_v8_release_${TARGET}.a.gz" -o "${archive_path}"
3837
curl -fsSL "${base_url}/src_binding_release_${TARGET}.rs" -o "${binding_path}"
39-
grep -E " (librusty_v8_release_${TARGET}[.]a[.]gz|src_binding_release_${TARGET}[.]rs)$" \
40-
"${checksums_source}" > "${checksums_path}"
38+
curl -fsSL "${base_url}/rusty_v8_release_${TARGET}.sha256" -o "${checksums_path}"
4139
4240
if [[ "$(wc -l < "${checksums_path}")" -ne 2 ]]; then
43-
echo "Expected exactly two checksums for ${TARGET} in ${checksums_source}" >&2
41+
echo "Expected exactly two checksums for ${TARGET} in ${checksums_path}" >&2
4442
exit 1
4543
fi
4644

0 commit comments

Comments
 (0)