Skip to content

Commit c2a34de

Browse files
author
wip-management
committed
test: fix mock-tier e2e CI failures — build binary, fix port conflicts, re-tag GPU scenarios
The mock-tier CI job failed all 11 tests due to three issues: 1. No rocm binary: the e2e-mock job depended on build-and-test but never got the compiled binary. Add cargo build and set ROCM_CLI_BINARY in the job. 2. Port conflicts: the mock OpenAI server held ports between tests because ServeLibrary had no teardown. Add ROBOT_LIBRARY_LISTENER with _end_test to stop the mock server, enable SO_REUSEADDR via allow_reuse_address, and call server_close() on shutdown. 3. Wrong tags: scenarios 5-9 were tagged mock but require real GPU serving or the service registry (which can't discover standalone mock servers). Re-tag them as gpu+network so the mock tier only runs tests that work without hardware. After these fixes the mock tier runs 6 tests (baselines 1-2, scenarios 2, 4, 10, 11) — all passing on app-dev. Signed-off-by: wip-management <wip@localhost>
1 parent 19916be commit c2a34de

5 files changed

Lines changed: 28 additions & 6 deletions

File tree

.github/workflows/ci.yml

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -428,6 +428,14 @@ jobs:
428428
steps:
429429
- uses: actions/checkout@v6
430430

431+
- name: Install native build deps
432+
run: sudo apt-get update && sudo apt-get install -y pkg-config libcap-dev
433+
434+
- uses: actions-rust-lang/setup-rust-toolchain@v1
435+
436+
- name: Build rocm binary
437+
run: cargo build --release -p rocm
438+
431439
- uses: actions/setup-python@v5
432440
with:
433441
python-version: '3.12'
@@ -436,6 +444,8 @@ jobs:
436444
run: pip install -r tests/e2e/requirements.txt
437445

438446
- name: Run mock-tier E2E tests
447+
env:
448+
ROCM_CLI_BINARY: ${{ github.workspace }}/target/release/rocm
439449
run: |
440450
python3 -m robot \
441451
--pythonpath tests/e2e \

tests/e2e/libraries/ServeLibrary.py

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,8 @@ class ServeLibrary:
2929
GPU-tier tests interact with real rocm serve endpoints.
3030
"""
3131

32+
ROBOT_LIBRARY_LISTENER = "SELF"
33+
3234
def __init__(self):
3335
self._mock: MockOpenAIServer | None = None
3436
self._serve_result: CliResult | None = None
@@ -38,6 +40,11 @@ def __init__(self):
3840
self._services_result: CliResult | None = None
3941
self._discovered_model: str | None = None
4042

43+
def _end_test(self, name, attrs):
44+
if self._mock:
45+
self._mock.stop()
46+
self._mock = None
47+
4148
# ── Given keywords ───────────────────────────────────────────
4249

4350
@keyword("a mock model server is running on the default port")

tests/e2e/libraries/common/mock_server.py

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,10 @@ def log_message(self, format, *args):
7373
pass
7474

7575

76+
class _ReusableHTTPServer(HTTPServer):
77+
allow_reuse_address = True
78+
79+
7680
class MockOpenAIServer:
7781
"""Start and stop a mock OpenAI-compatible server in a background thread."""
7882

@@ -89,7 +93,7 @@ def __init__(
8993
self._thread: threading.Thread | None = None
9094

9195
def start(self):
92-
self._server = HTTPServer((self.host, self.port), MockOpenAIHandler)
96+
self._server = _ReusableHTTPServer((self.host, self.port), MockOpenAIHandler)
9397
self._server.model_name = self.model_name
9498
self._server.received_models = []
9599
self._thread = threading.Thread(target=self._server.serve_forever, daemon=True)
@@ -98,6 +102,7 @@ def start(self):
98102
def stop(self):
99103
if self._server:
100104
self._server.shutdown()
105+
self._server.server_close()
101106
self._server = None
102107
if self._thread:
103108
self._thread.join(timeout=5)

tests/e2e/suites/chat.robot

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ Suite Teardown Teardown Isolated Environment
1818
# ── Mock tier ────────────────────────────────────────────────────────
1919

2020
Scenario: 9 - Detection finds a served model through the services registry
21-
[Tags] mock chat
21+
[Tags] gpu network chat
2222
Given a mock model server is running
2323
And the server is registered as a managed service
2424
When the dashboard checks for a local engine

tests/e2e/suites/model_serving.robot

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -18,26 +18,26 @@ Suite Teardown Teardown Isolated Environment
1818
# ── Mock tier ────────────────────────────────────────────────────────
1919

2020
Scenario: 5 - Serve plan resolves a model alias to the full identifier
21-
[Tags] mock model-serving
21+
[Tags] gpu network model-serving
2222
Given a managed runtime is active
2323
When the user requests a serve plan for a model alias
2424
Then the serve plan shows the full model identifier
2525

2626
Scenario: 6 - Alias resolution is consistent across engines
27-
[Tags] mock model-serving
27+
[Tags] gpu network model-serving
2828
Given a managed runtime is active
2929
When the user requests a serve plan for the same alias with different engines
3030
Then all serve plans resolve to the same full model identifier
3131

3232
Scenario: 7 - A managed server appears in the services list with its model name
33-
[Tags] mock model-serving
33+
[Tags] gpu network model-serving
3434
Given a mock model server is running on the default port
3535
And the server is registered as a managed service
3636
When the user lists running services
3737
Then the service appears with the correct model name and endpoint
3838

3939
Scenario: 8 - The services registry reports the actual endpoint port
40-
[Tags] mock model-serving
40+
[Tags] gpu network model-serving
4141
Given a mock model server is running on a non-default port
4242
And the server is registered as a managed service
4343
When the user lists running services

0 commit comments

Comments
 (0)