Skip to content

Commit adc209c

Browse files
ci(audience): run Linux PlayMode under xvfb so tests actually execute (SDK-317)
- Replaces game-ci/unity-test-runner@v4 with docker run of unityci/editor:ubuntu-${unity}-linux-il2cpp-3, wrapped in xvfb-run. - Without an X server every [UnityTest] returned inconclusive (passed=0, failed=0). The action's USE_EXIT_CODE=false hid Unity's exit code 2, dorny/test-reporter does not flag inconclusive, so each Linux cell silently went green for ~3 min without executing a single test. macOS by comparison ran 39 tests in ~25 s with all 39 passing. - xvfb-run gives Unity a software-rendered virtual display via mesa-llvmpipe; no GPU required. - One image tag covers both backends (verified against the Mono cell log from run 25492697422 which pulled the il2cpp tag). - Per-run license activation + return mirrors the dropped self-hosted Linux job (commit 2658686). - Adds a post-step that parses playmode-results.xml via xmllint and fails the cell when inconclusive > 0 or passed+failed == 0; stops any future regression from silently re-greening the matrix. - timeout-minutes: 30 since cells now run ~5-10 min of real work. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent b3a13ba commit adc209c

1 file changed

Lines changed: 120 additions & 10 deletions

File tree

.github/workflows/test-audience-sample-app.yml

Lines changed: 120 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -412,6 +412,11 @@ jobs:
412412
|| github.event_name == 'workflow_dispatch'
413413
name: ${{ matrix.target }} / ${{ matrix.backend }} / Unity ${{ matrix.unity }}
414414
runs-on: ubuntu-latest-8-cores
415+
# Tests now actually execute under xvfb instead of returning instantly
416+
# as inconclusive, so cells take ~5-10 min. The 30 min cap leaves
417+
# headroom for cold caches and the first image pull without leaving
418+
# a stuck job sitting on the runner for the default 6 hours.
419+
timeout-minutes: 30
415420
strategy:
416421
fail-fast: false
417422
matrix:
@@ -433,35 +438,140 @@ jobs:
433438
Library-${{ matrix.backend }}-${{ matrix.target }}-${{ matrix.unity }}-
434439
Library-${{ matrix.backend }}-${{ matrix.target }}-
435440
436-
- uses: game-ci/unity-test-runner@v4
437-
id: playmode
441+
- name: Run PlayMode tests under xvfb
442+
# Manual `docker run` instead of game-ci/unity-test-runner@v4: the
443+
# action hardcodes `-batchmode -nographics` and never starts a
444+
# virtual display, so every [UnityTest] in SampleAppLiveFireTests
445+
# came back "inconclusive" (passed=0, failed=0). NUnit does not
446+
# treat inconclusive as failure and the action's USE_EXIT_CODE=false
447+
# suppresses Unity's exit code 2, so the cells silently went green
448+
# without executing a single test. See SDK-318 for the diagnosis.
449+
shell: bash
438450
env:
439451
UNITY_EMAIL: ${{ secrets.UNITY_EMAIL }}
440452
UNITY_PASSWORD: ${{ secrets.UNITY_PASSWORD }}
441453
UNITY_SERIAL: ${{ secrets.UNITY_SERIAL }}
442454
AUDIENCE_TEST_PUBLISHABLE_KEY: ${{ secrets.AUDIENCE_TEST_PUBLISHABLE_KEY }}
443455
AUDIENCE_SCRIPTING_BACKEND: ${{ matrix.backend }}
444-
with:
445-
unityVersion: ${{ matrix.unity }}
446-
targetPlatform: ${{ matrix.target }}
447-
projectPath: examples/audience
448-
testMode: playmode
449-
githubToken: ${{ secrets.GITHUB_TOKEN }}
456+
UNITY_VERSION: ${{ matrix.unity }}
457+
run: |
458+
set -uo pipefail
459+
mkdir -p artifacts
460+
461+
# The unityci/editor:ubuntu-...-linux-il2cpp-3 image ships both
462+
# the Mono and IL2CPP playback engines plus xvfb, so the same
463+
# tag works for both backends. The previous GameCI runs proved
464+
# this: the StandaloneLinux64/Mono2x cell pulled the il2cpp tag.
465+
image="unityci/editor:ubuntu-${UNITY_VERSION}-linux-il2cpp-3"
466+
467+
docker run --rm \
468+
--workdir /github/workspace \
469+
--env UNITY_EMAIL --env UNITY_PASSWORD --env UNITY_SERIAL \
470+
--env AUDIENCE_TEST_PUBLISHABLE_KEY --env AUDIENCE_SCRIPTING_BACKEND \
471+
--volume "$PWD":/github/workspace:z \
472+
--cpus=8 --memory=30487m \
473+
"$image" \
474+
/bin/bash -c '
475+
set -uo pipefail
476+
477+
# Per-run license activation. Unity occasionally exits non-zero
478+
# on a successful activation (warnings about prior cached state
479+
# in /root), so swallow the rc and assert on the success marker
480+
# in the log instead. Same approach as the dropped self-hosted
481+
# Linux job from SDK-255.
482+
unity-editor -batchmode -nographics -quit \
483+
-username "$UNITY_EMAIL" \
484+
-password "$UNITY_PASSWORD" \
485+
-serial "$UNITY_SERIAL" \
486+
-logFile - 2>&1 | tee /github/workspace/artifacts/activation.log || true
487+
if grep -qE "License activation has failed|\[Licensing::Client\] Error: Code [0-9]+" \
488+
/github/workspace/artifacts/activation.log; then
489+
echo "::error::Unity license activation failed."
490+
exit 1
491+
fi
492+
if ! grep -qE "Successfully activated the entitlement license" \
493+
/github/workspace/artifacts/activation.log; then
494+
echo "::error::Unity license activation: no success marker in log."
495+
exit 1
496+
fi
497+
498+
# xvfb-run gives Unity a virtual X display so PlayMode tests
499+
# that load scenes and exercise UI Toolkit can actually launch
500+
# the player. GLX + render are required for UIElements; the
501+
# image already bundles mesa-llvmpipe for software OpenGL, so
502+
# no GPU is needed. -noreset keeps the X server up across
503+
# Unity client reconnects (the editor opens / closes / reopens
504+
# connections during scene load).
505+
xvfb-run -a --server-args="-screen 0 1280x720x24 -ac +extension GLX +render -noreset" -- \
506+
unity-editor \
507+
-batchmode \
508+
-projectPath /github/workspace/examples/audience \
509+
-runTests \
510+
-testPlatform StandaloneLinux64 \
511+
-testResults /github/workspace/artifacts/playmode-results.xml \
512+
-logFile - 2>&1 | tee /github/workspace/artifacts/playmode.log
513+
test_rc=${PIPESTATUS[0]}
514+
515+
# Always return the seat so reruns and parallel cells do not
516+
# exhaust the activation pool. Tolerate non-zero in case the
517+
# editor process is still mid-shutdown.
518+
unity-editor -batchmode -nographics -quit -returnlicense -logFile - 2>&1 || true
519+
520+
# Unity exits 2 when any test fails or comes back inconclusive.
521+
# Propagating the rc here means the step fails on real failures
522+
# without needing the USE_EXIT_CODE=false hack the GameCI
523+
# action applies.
524+
exit $test_rc
525+
'
526+
527+
- name: Fail when no tests actually executed
528+
# Defense in depth: catches the silent-pass case if a future change
529+
# accidentally re-disables the display, breaks the player launch,
530+
# or restores USE_EXIT_CODE=false on the docker invocation. NUnit
531+
# marks tests "inconclusive" rather than "failed" when the player
532+
# never starts, and dorny/test-reporter does not flag inconclusive
533+
# as failure, so the suite has to be inspected directly.
534+
if: always()
535+
shell: bash
536+
run: |
537+
set -euo pipefail
538+
xml="artifacts/playmode-results.xml"
539+
if [ ! -f "$xml" ]; then
540+
echo "::error::No test-results.xml at $xml. Unity did not produce results."
541+
exit 1
542+
fi
543+
# xmllint is preinstalled on ubuntu-latest via libxml2-utils.
544+
passed=$(xmllint --xpath 'string(/test-run/@passed)' "$xml")
545+
failed=$(xmllint --xpath 'string(/test-run/@failed)' "$xml")
546+
inconclusive=$(xmllint --xpath 'string(/test-run/@inconclusive)' "$xml")
547+
echo "passed=$passed failed=$failed inconclusive=$inconclusive"
548+
if [ "${inconclusive:-0}" -gt 0 ]; then
549+
echo "::error::$inconclusive test(s) came back inconclusive. Unity could not actually execute them. Check that xvfb is running and the player launches."
550+
exit 1
551+
fi
552+
if [ "${passed:-0}" -eq 0 ] && [ "${failed:-0}" -eq 0 ]; then
553+
echo "::error::Zero tests passed and zero failed. The suite did not execute."
554+
exit 1
555+
fi
450556
451557
- name: Publish test report
452558
uses: dorny/test-reporter@v3
453559
if: always()
454560
with:
455561
name: PlayMode (${{ matrix.backend }} / ${{ matrix.target }})
456-
path: ${{ steps.playmode.outputs.artifactsPath }}/playmode-results.xml
562+
path: artifacts/playmode-results.xml
457563
reporter: dotnet-nunit
458564
fail-on-error: true
459565

460566
- uses: actions/upload-artifact@v4
461567
if: always()
462568
with:
463569
name: playmode-${{ matrix.backend }}-${{ matrix.target }}-${{ matrix.unity }}
464-
path: ${{ steps.playmode.outputs.artifactsPath }}
570+
path: |
571+
artifacts/playmode-results.xml
572+
artifacts/playmode.log
573+
artifacts/activation.log
574+
examples/audience/Logs/**
465575
466576
# Mobile IL2CPP build validation — runs on GitHub-hosted Ubuntu via GameCI Docker
467577
# containers so self-hosted macOS/Windows machines are not occupied.

0 commit comments

Comments
 (0)