test: tag known-bug scenarios with expected-failure-EAI-* for skiponfailure

Scenarios that exercise filed bugs are now tagged with
expected-failure-EAI-XXXX (e.g. expected-failure-EAI-7219). Robot
Framework's --skiponfailure flag marks these as SKIP instead of FAIL,
keeping CI green while the bugs are open. When a bug is fixed, its
test starts passing and reports PASS — signaling the tag can be
removed.

Tag mapping:
- EAI-7218: scenario 15 (PyTorch dependency resolution)
- EAI-7219: scenarios 5, 6, 12, 16 (alias resolution)
- EAI-7220: scenarios 7, 8, 9, 17 (service registry / detection)
- EAI-7221: scenario 14 (TUI hardcodes model name)
- EAI-7223: scenario 18 (vLLM tool-calling)

Signed-off-by: wip-management <wip@localhost>

Author: wip-management

.github/workflows/ci.yml

@@ -450,6 +450,7 @@ jobs:
           python3 -m robot \
             --pythonpath tests/e2e \
             --outputdir tests/e2e/results \
+            --skiponfailure "expected-failure-*" \
             --include smoke \
             --include mock \
             tests/e2e/suites/
@@ -482,6 +483,7 @@ jobs:
           python3 -m robot \
             --pythonpath tests/e2e \
             --outputdir tests/e2e/results \
+            --skiponfailure "expected-failure-*" \
             --include gpu \
             tests/e2e/suites/
 

tests/e2e/README.md

@@ -204,6 +204,7 @@ def user_serves_model(self, alias: str, engine: str):
 | `runtime` | Runtime setup and management |
 | `model-serving` | Model serving and endpoint behavior |
 | `chat` | TUI chat and endpoint detection |
+| `expected-failure-EAI-*` | Known bug — test is skipped on failure via `--skiponfailure`. Remove the tag when the bug is fixed. |
 
 ## Test design principles
 

tests/e2e/run.sh

@@ -30,4 +30,5 @@ mkdir -p "${RESULTS_DIR}"
 exec python3 -m robot \
     --pythonpath "${E2E_DIR}" \
     --outputdir "${RESULTS_DIR}" \
+    --skiponfailure "expected-failure-*" \
     "${ROBOT_ARGS[@]:-${E2E_DIR}/suites/}"

tests/e2e/suites/chat.robot

@@ -18,7 +18,7 @@ Suite Teardown    Teardown Isolated Environment
 # ── Mock tier ────────────────────────────────────────────────────────
 
 Scenario: 9 - Detection finds a served model through the services registry
-    [Tags]    gpu    network    chat
+    [Tags]    gpu    network    chat    expected-failure-EAI-7220
     Given a mock model server is running
     And the server is registered as a managed service
     When the dashboard checks for a local engine
@@ -40,13 +40,13 @@ Scenario: 11 - The privacy notice is accurate for local endpoints
 
 
 Scenario: 17 - The default serve port is included in engine detection
-    [Tags]    gpu    network    chat
+    [Tags]    gpu    network    chat    expected-failure-EAI-7220
     Given a model is served in the background
     When the user lists running services
     Then the endpoint port matches the default serve port
 
 Scenario: 18 - Chat completion with tool definitions does not fail
-    [Tags]    gpu    network    chat
+    [Tags]    gpu    network    chat    expected-failure-EAI-7223
     Given a managed runtime is active
     And a model is served in the background
     When a chat completion request with tools is sent to the endpoint
@@ -56,7 +56,7 @@ Scenario: 18 - Chat completion with tool definitions does not fail
 # ── GPU tier ─────────────────────────────────────────────────────────
 
 Scenario: 14 - End-to-end chat through a locally served model
-    [Tags]    gpu    network    chat
+    [Tags]    gpu    network    chat    expected-failure-EAI-7221
     Given a managed runtime is active
     And a model is served in the background
     And the dashboard has detected the served model

tests/e2e/suites/model_serving.robot

@@ -18,40 +18,40 @@ Suite Teardown    Teardown Isolated Environment
 # ── Mock tier ────────────────────────────────────────────────────────
 
 Scenario: 5 - Serve plan resolves a model alias to the full identifier
-    [Tags]    gpu    network    model-serving
+    [Tags]    gpu    network    model-serving    expected-failure-EAI-7219
     Given a managed runtime is active
     When the user requests a serve plan for a model alias
     Then the serve plan shows the full model identifier
 
 Scenario: 6 - Alias resolution is consistent across engines
-    [Tags]    gpu    network    model-serving
+    [Tags]    gpu    network    model-serving    expected-failure-EAI-7219
     Given a managed runtime is active
     When the user requests a serve plan for the same alias with different engines
     Then all serve plans resolve to the same full model identifier
 
 Scenario: 7 - A managed server appears in the services list with its model name
-    [Tags]    gpu    network    model-serving
+    [Tags]    gpu    network    model-serving    expected-failure-EAI-7220
     Given a mock model server is running on the default port
     And the server is registered as a managed service
     When the user lists running services
     Then the service appears with the correct model name and endpoint
 
 Scenario: 8 - The services registry reports the actual endpoint port
-    [Tags]    gpu    network    model-serving
+    [Tags]    gpu    network    model-serving    expected-failure-EAI-7220
     Given a mock model server is running on a non-default port
     And the server is registered as a managed service
     When the user lists running services
     Then the endpoint matches the actual server port
 
 
 Scenario: 15 - Serving with the PyTorch engine resolves dependencies correctly
-    [Tags]    gpu    network    model-serving
+    [Tags]    gpu    network    model-serving    expected-failure-EAI-7218
     Given a managed runtime is active
     When the user serves a model using the PyTorch engine
     Then the serve command does not fail with a dependency resolution error
 
 Scenario: 16 - Serve plan shows the full model identifier when given an alias
-    [Tags]    gpu    network    model-serving
+    [Tags]    gpu    network    model-serving    expected-failure-EAI-7219
     Given a managed runtime is active
     When the user requests a serve plan for a model alias
     Then the serve plan shows the full model identifier
@@ -60,7 +60,7 @@ Scenario: 16 - Serve plan shows the full model identifier when given an alias
 # ── GPU tier ─────────────────────────────────────────────────────────
 
 Scenario: 12 - A served model responds to inference requests
-    [Tags]    gpu    network    model-serving
+    [Tags]    gpu    network    model-serving    expected-failure-EAI-7219
     Given a managed runtime is active
     And a model is being served on GPU
     When the user sends a chat completion request to the endpoint