Skip to content

Commit 649c95e

Browse files
u9gu9g
committed
ci(session-e2e): cross-build Windows artifacts on Linux
The Windows cgo link of ~560 webrtc/portaudio objects overflows the native command-line limit, so build like goreleaser does -- on Linux with zig. Split the Windows arm into cross-build-windows (cross-compiles lk.exe and the e2e test binary on ubuntu with zig + setup-cross.sh) and e2e-windows (downloads them and runs natively). buildLK now honors LK_SESSION_E2E_BIN so the test drives the prebuilt lk instead of rebuilding on the Windows runner.
1 parent a4e4201 commit 649c95e

2 files changed

Lines changed: 130 additions & 46 deletions

File tree

‎.github/workflows/session-e2e.yaml‎

Lines changed: 120 additions & 45 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
11
name: Session E2E
22

33
# Drives the real `lk agent session` start/say/end lifecycle against the minimal
4-
# one-file echo agent in cmd/lk/testdata/echo-agent, on Linux and Windows. This
5-
# exercises the detached daemon, the readiness handshake, the console IPC
6-
# transport, and the model round-trip end to end -- runtime behavior that
7-
# `go test` alone never covers.
4+
# one-file echo agent in cmd/lk/testdata/echo-agent, on Linux, macOS, and
5+
# Windows. This exercises the detached daemon, the readiness handshake, the
6+
# console IPC transport, and the model round-trip end to end -- runtime behavior
7+
# that `go test` alone never covers.
88
#
99
# Runs on manual dispatch and on pushes to any repo branch (forks can't trigger
1010
# `push`, so secrets are only exposed to trusted collaborators). It needs live
@@ -19,6 +19,15 @@ name: Session E2E
1919
# console mode to the TCP console directly to bridge the daemon's
2020
# `python <entry> console --connect-addr` launch.
2121
#
22+
# Windows is split into two jobs: pkg/apm/webrtc's C++ uses MSVC SEH
23+
# (__try/__except) that the runner's mingw GCC can't compile, so we build with
24+
# zig (clang) exactly as .goreleaser.yaml does. zig-as-CC must run on Linux: on
25+
# native Windows the cgo link of ~560 webrtc/portaudio objects overflows the
26+
# ~32KB command-line limit. So `cross-build-windows` cross-compiles lk.exe AND
27+
# the e2e test binary on Linux and uploads them; `e2e-windows` downloads them
28+
# and runs the test natively (LK_SESSION_E2E_BIN points the test at the
29+
# prebuilt lk, so nothing is rebuilt on the Windows runner).
30+
#
2231
# Node is intentionally not in the matrix yet: this branch's session daemon only
2332
# supports Python agents (`detectProject` rejects non-Python), and Node console
2433
# support depends on the brian/agent-session-node-support CLI line (#868/#878)
@@ -34,11 +43,13 @@ concurrency:
3443
cancel-in-progress: true
3544

3645
jobs:
46+
# Linux and macOS build + run natively in one job -- their linkers have no
47+
# command-line-length limit, so `go test` building lk in-process is fine.
3748
e2e:
3849
strategy:
3950
fail-fast: false
4051
matrix:
41-
os: [ubuntu-latest, windows-latest, macos-latest]
52+
os: [ubuntu-latest, macos-latest]
4253

4354
runs-on: ${{ matrix.os }}
4455
name: Agent Session with Python Agent on ${{ matrix.os }}
@@ -58,12 +69,50 @@ jobs:
5869
if: matrix.os == 'ubuntu-latest'
5970
run: sudo apt-get update && sudo apt-get install -y libasound2-dev
6071

61-
# pkg/apm/webrtc's C++ uses MSVC SEH (__try/__except), which the runner's
62-
# default mingw GCC can't compile. The repo only ever builds Windows with
63-
# zig (clang) -- see .goreleaser.yaml -- so use it as the cgo compiler here
64-
# too. Version pinned to match build.yaml's cross-build job.
65-
- name: Set up Zig (Windows)
66-
if: matrix.os == 'windows-latest'
72+
- name: Set up Go
73+
uses: actions/setup-go@4a3601121dd01d1626a1e23e37211e3254c1c06c # v6
74+
with:
75+
go-version-file: go.mod
76+
cache: true
77+
78+
- name: Set up Python
79+
uses: actions/setup-python@v5
80+
with:
81+
python-version: "3.12"
82+
83+
- name: Set up uv
84+
uses: astral-sh/setup-uv@v5
85+
with:
86+
enable-cache: true
87+
88+
- name: Sync echo agent deps
89+
working-directory: cmd/lk/testdata/echo-agent
90+
run: uv sync
91+
92+
- name: Run session e2e
93+
env:
94+
LIVEKIT_API_KEY: ${{ secrets.LIVEKIT_API_KEY }}
95+
LIVEKIT_API_SECRET: ${{ secrets.LIVEKIT_API_SECRET }}
96+
LIVEKIT_URL: ${{ secrets.LIVEKIT_URL }}
97+
run: go test ./cmd/lk -run TestSessionE2E -count=1 -v -timeout 600s
98+
99+
# Cross-compile the Windows artifacts on Linux with zig, mirroring
100+
# .goreleaser.yaml's lk-windows-amd64 build. Produces lk.exe (the binary under
101+
# test) and the compiled e2e test binary, both shipped to e2e-windows.
102+
cross-build-windows:
103+
runs-on: ubuntu-latest
104+
name: Cross-build Windows artifacts (zig)
105+
106+
permissions:
107+
contents: read
108+
109+
steps:
110+
- name: Checkout livekit-cli
111+
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
112+
with:
113+
submodules: true
114+
115+
- name: Set up Zig
67116
uses: mlugg/setup-zig@d1434d08867e3ee9daa34448df10607b98908d29 # v2
68117
with:
69118
version: 0.14.1
@@ -74,30 +123,58 @@ jobs:
74123
go-version-file: go.mod
75124
cache: true
76125

77-
# Use `zig cc`/`zig c++` directly as the cgo toolchain, exactly as
78-
# .goreleaser.yaml's Windows builds do. Go's cgo splits a multi-token
79-
# CC/CXX into argv, so the `-target` flags are carried through to zig.
80-
- name: Set zig as cgo compiler (Windows)
81-
if: matrix.os == 'windows-latest'
82-
shell: bash
83-
run: |
84-
echo 'CC=zig cc -target x86_64-windows-gnu' >> "$GITHUB_ENV"
85-
echo 'CXX=zig c++ -target x86_64-windows-gnu' >> "$GITHUB_ENV"
86-
87-
# Go's compiled objects embed COFF /DEFAULTLIB directives (dbghelp,
88-
# bcrypt, ...) that lld resolves directly, bypassing zig's lazy .def->.a
89-
# import-lib generation, so the link fails with "could not open
90-
# 'libdbghelp.a'". setup-cross.sh pre-generates the full set of MinGW
91-
# import libs into .cross/windows_amd64/mingw_lib (same as the goreleaser
92-
# cross-build), and -extldflags -L points the linker at them. Mixed-path
93-
# (D:/...) so zig's clang driver accepts it. Needs zig on PATH.
94-
- name: Generate MinGW import libs (Windows)
95-
if: matrix.os == 'windows-latest'
96-
shell: bash
126+
# Generate the MinGW import libs lld needs for Go's /DEFAULTLIB references
127+
# (dbghelp, bcrypt, ...), exactly as goreleaser's before-hook does.
128+
- name: Generate MinGW import libs
129+
run: scripts/setup-cross.sh windows/amd64
130+
131+
- name: Cross-compile lk.exe and the e2e test binary
132+
env:
133+
GOOS: windows
134+
GOARCH: amd64
135+
CGO_ENABLED: "1"
136+
# zig cc/c++ as the cgo toolchain, matching .goreleaser.yaml's Windows
137+
# build. -fms-extensions (SEH) isn't on cgo's allowlist; -fno-sanitize
138+
# is a zig UBSan-under-lld workaround -- both copied from goreleaser.
139+
CC: zig cc -target x86_64-windows-gnu
140+
CXX: zig c++ -target x86_64-windows-gnu
141+
CGO_CXXFLAGS_ALLOW: -fms-extensions
142+
CGO_CXXFLAGS: -fno-sanitize=all
97143
run: |
98-
scripts/setup-cross.sh windows/amd64
99-
root="$(cygpath -m "$(pwd)")"
100-
echo "GOTEST_LDFLAGS=-ldflags=-extldflags=-L$root/.cross/windows_amd64/mingw_lib" >> "$GITHUB_ENV"
144+
ldflags="-extldflags=-L$PWD/.cross/windows_amd64/mingw_lib"
145+
mkdir -p dist
146+
go build -ldflags="$ldflags" -o dist/lk.exe ./cmd/lk
147+
go test -c -ldflags="$ldflags" -o dist/session-e2e.test.exe ./cmd/lk
148+
149+
- name: Upload Windows artifacts
150+
uses: actions/upload-artifact@v4
151+
with:
152+
name: windows-e2e
153+
path: dist/*.exe
154+
retention-days: 1
155+
if-no-files-found: error
156+
157+
# Run the prebuilt Windows artifacts natively. No Go/zig/cgo here: the test
158+
# binary just drives lk.exe (via LK_SESSION_E2E_BIN) against the echo agent.
159+
e2e-windows:
160+
needs: cross-build-windows
161+
runs-on: windows-latest
162+
name: Agent Session with Python Agent on windows-latest
163+
164+
permissions:
165+
contents: read
166+
167+
steps:
168+
- name: Checkout livekit-cli
169+
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
170+
# No submodules: the prebuilt binaries already contain the cgo code;
171+
# only the echo-agent fixture (plain repo files) is needed at runtime.
172+
173+
- name: Download Windows artifacts
174+
uses: actions/download-artifact@v4
175+
with:
176+
name: windows-e2e
177+
path: dist
101178

102179
- name: Set up Python
103180
uses: actions/setup-python@v5
@@ -114,19 +191,17 @@ jobs:
114191
run: uv sync
115192

116193
- name: Run session e2e
117-
# bash on every OS so $GOTEST_LDFLAGS expands (it's set only on Windows;
118-
# empty elsewhere -> a plain `go test`). No spaces in the value, so the
119-
# unquoted expansion stays a single argument.
194+
# Run from cmd/lk so the test's default testdata/echo-agent/agent.py
195+
# path resolves; point it at the cross-built lk.exe so nothing rebuilds.
120196
shell: bash
197+
working-directory: cmd/lk
121198
env:
122-
# pkg/apm/webrtc puts -fms-extensions in its #cgo windows CXXFLAGS;
123-
# Go's cgo flag allowlist rejects it unless explicitly permitted (same
124-
# as .goreleaser.yaml's Windows builds). No-op on Linux/macOS.
125-
CGO_CXXFLAGS_ALLOW: -fms-extensions
126-
# zig-0.16 UBSan workaround carried over from .goreleaser.yaml's
127-
# Windows builds; empty (no-op) on Linux/macOS.
128-
CGO_CXXFLAGS: ${{ matrix.os == 'windows-latest' && '-fno-sanitize=all' || '' }}
199+
# Relative to cmd/lk; Go's filepath.Abs resolves it. Forward slashes
200+
# keep bash happy (GITHUB_WORKSPACE would carry Windows backslashes).
201+
LK_SESSION_E2E_BIN: ../../dist/lk.exe
129202
LIVEKIT_API_KEY: ${{ secrets.LIVEKIT_API_KEY }}
130203
LIVEKIT_API_SECRET: ${{ secrets.LIVEKIT_API_SECRET }}
131204
LIVEKIT_URL: ${{ secrets.LIVEKIT_URL }}
132-
run: go test ./cmd/lk -run TestSessionE2E -count=1 -v -timeout 600s $GOTEST_LDFLAGS
205+
run: |
206+
../../dist/session-e2e.test.exe \
207+
-test.run TestSessionE2E -test.count=1 -test.v -test.timeout 600s

‎cmd/lk/session_e2e_test.go‎

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -98,9 +98,18 @@ func TestSessionE2E(t *testing.T) {
9898
}, 10*time.Second, 200*time.Millisecond, "session daemon still listening on port %s after end", port)
9999
}
100100

101-
// buildLK compiles the lk binary into a temp dir and returns its path.
101+
// buildLK returns the path to the lk binary under test. If LK_SESSION_E2E_BIN
102+
// points at a prebuilt binary it's used as-is (the Windows CI arm cross-builds
103+
// lk on Linux and ships it here, so the heavy cgo build never runs on the
104+
// Windows runner); otherwise lk is compiled into a temp dir.
102105
func buildLK(t *testing.T) string {
103106
t.Helper()
107+
if prebuilt := os.Getenv("LK_SESSION_E2E_BIN"); prebuilt != "" {
108+
abs, err := filepath.Abs(prebuilt)
109+
require.NoError(t, err)
110+
require.FileExists(t, abs, "LK_SESSION_E2E_BIN does not point at a binary")
111+
return abs
112+
}
104113
bin := filepath.Join(t.TempDir(), "lk")
105114
if runtime.GOOS == "windows" {
106115
bin += ".exe"

0 commit comments

Comments
 (0)