Skip to content

Commit e3a2c13

Browse files
committed
test: improve scenario wording — remove jargon and internal terminology
Rewrite scenarios to use plain language per BDD quality rules: - "GPU target" → "which GPU is installed" - "managed runtime" → "CLI-managed runtime" - "services registry" → "running services" - "endpoint" → "connection details" - "dependency error" → "works without errors" - "TheRock-based installs" → "which install types can be adopted" - "inference stack" → "inference engine" Step function annotations updated to match. Signed-off-by: fredespi <fredrik.espinoza@gmail.com>
1 parent 3e46a3f commit e3a2c13

8 files changed

Lines changed: 51 additions & 57 deletions

File tree

tests/e2e-cucumber/features/chat.feature

Lines changed: 6 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -7,28 +7,27 @@ Feature: Chat and endpoint detection
77
Then the request identifies the served model by name
88

99
@expected-failure-EAI-7220
10-
Scenario: 2 - Detection finds a served model through the services registry
10+
Scenario: 2 - A served model is discoverable through the services list
1111
Given a model is being served
12-
And the server is registered as a managed service
1312
When the user checks for running services
14-
Then the served model is detected at the correct endpoint
13+
Then the served model is listed
1514

1615
Scenario: 3 - The privacy notice is accurate for local endpoints
1716
Given a model is being served locally
1817
When the user is offered the detected endpoint
1918
Then the notice does not claim that requests leave the machine
2019

2120
@expected-failure-EAI-7220
22-
Scenario: 4 - The default serve port is included in engine detection
21+
Scenario: 4 - A served model is discoverable on the default port
2322
Given a model is served in the background
2423
When the user lists running services
25-
Then the endpoint port matches the default serve port
24+
Then the served model is listed on the default port
2625

2726
@expected-failure-EAI-7223
28-
Scenario: 5 - Chat completion with tool definitions does not fail
27+
Scenario: 5 - Chat requests that include tool definitions are accepted
2928
Given a managed runtime is active
3029
And a model is served in the background
31-
When a chat completion request with tools is sent
30+
When a chat request with tool definitions is sent
3231
Then the chat response is successful
3332

3433
@expected-failure-EAI-7221

tests/e2e-cucumber/features/examine.feature

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -8,14 +8,14 @@ Feature: GPU detection and system inspection
88
When the user lists available engines
99
Then all supported engines are listed
1010

11-
Scenario: 3 - System inspection detects GPU and ROCm on a supported machine
12-
Given a machine with an AMD GPU and ROCm installed
11+
Scenario: 3 - System inspection detects the GPU and driver
12+
Given a machine with an AMD GPU
1313
When the user inspects the system
14-
Then the inspection reports a detected GPU target
14+
Then the inspection reports which GPU is installed
1515
And the inspection reports that the driver is available
1616

17-
Scenario: 4 - System inspection distinguishes managed from unmanaged ROCm
18-
Given a machine with a pre-installed ROCm that was not set up by the CLI
17+
Scenario: 4 - System inspection distinguishes CLI-managed from pre-existing ROCm
18+
Given a machine with a ROCm install that was not set up by the CLI
1919
When the user inspects the system
20-
Then the inspection reports the ROCm install as unmanaged
21-
And the inspection suggests installing a managed runtime
20+
Then the inspection reports the install as pre-existing
21+
And the inspection suggests setting up a CLI-managed install

tests/e2e-cucumber/features/model_serving.feature

Lines changed: 9 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -11,24 +11,22 @@ Feature: Model serving
1111
Then all engines expand to the same full model name
1212

1313
@expected-failure-EAI-7220
14-
Scenario: 3 - A managed server appears in the services list with its model name
14+
Scenario: 3 - A running model server is discoverable by name
1515
Given a model is being served on the default port
16-
And the server is registered as a managed service
1716
When the user lists running services
18-
Then the service appears with the correct model name and endpoint
17+
Then the service appears with the correct model name and connection details
1918

2019
@expected-failure-EAI-7220
21-
Scenario: 4 - The services registry reports the actual endpoint port
20+
Scenario: 4 - Running services show the correct connection details
2221
Given a model is being served on a non-default port
23-
And the server is registered as a managed service
2422
When the user lists running services
25-
Then the endpoint matches the actual server port
23+
Then the connection details match the actual server port
2624

2725
@expected-failure-EAI-7218
28-
Scenario: 5 - Serving with the PyTorch engine resolves dependencies correctly
26+
Scenario: 5 - Serving a model with the PyTorch engine works without errors
2927
Given a managed runtime is active
3028
When the user serves a model using the PyTorch engine
31-
Then the serve command does not fail with a dependency error
29+
Then the model is served successfully
3230

3331
@expected-failure-EAI-7219
3432
Scenario: 6 - Serve output shows the full model name when given a short name
@@ -43,9 +41,9 @@ Feature: Model serving
4341
Then the response contains a model reply
4442
And the response identifies the correct model
4543

46-
Scenario: 8 - Serving a model with the default engine produces a working endpoint
44+
Scenario: 8 - Serving a model without specifying an engine produces a working endpoint
4745
Given a managed runtime is active
4846
When the user serves a model without specifying an engine
4947
Then an engine is selected automatically
50-
And the model is reachable at the reported endpoint
51-
And the endpoint responds to inference requests
48+
And the model is reachable
49+
And the model responds to inference requests
Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,14 @@
11
Feature: Runtime configuration
22

3-
Scenario: 1 - Installing a managed runtime makes it the active runtime
4-
Given a machine with no managed runtimes
3+
Scenario: 1 - Installing the SDK makes it the active runtime
4+
Given a machine with no CLI-managed runtimes
55
When the user installs the SDK
6-
Then a managed runtime is registered
6+
Then a runtime is registered
77
And the runtime is set as active
8-
And the runtime includes a matching inference stack
8+
And the runtime includes an inference engine
99

10-
Scenario: 2 - Adopting a non-TheRock install is rejected with guidance
10+
Scenario: 2 - Adopting a pre-existing ROCm install is rejected with guidance
1111
Given a machine with a standard ROCm install
12-
When the user tries to adopt the standard install as a managed runtime
12+
When the user tries to adopt the existing install
1313
Then the adoption is refused
14-
And the error explains that only TheRock-based installs can be adopted
14+
And the error explains which install types can be adopted

tests/e2e-cucumber/tests/e2e/chat_steps.rs

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -69,7 +69,7 @@ async fn user_checks_services(world: &mut E2eWorld) {
6969
#[when("the user is offered the detected endpoint")]
7070
async fn user_offered_endpoint(_world: &mut E2eWorld) {}
7171

72-
#[when("a chat completion request with tools is sent")]
72+
#[when("a chat request with tool definitions is sent")]
7373
async fn send_chat_with_tools(world: &mut E2eWorld) {
7474
let endpoint = world.endpoint.as_ref().expect("no endpoint configured");
7575
let models_url = format!("{endpoint}/models");
@@ -132,15 +132,15 @@ async fn assert_request_uses_served_model(world: &mut E2eWorld) {
132132
);
133133
}
134134

135-
#[then("the served model is detected at the correct endpoint")]
136-
async fn assert_model_detected(world: &mut E2eWorld) {
135+
#[then("the served model is listed")]
136+
async fn assert_model_listed(world: &mut E2eWorld) {
137137
let output = world
138138
.cli_output
139139
.as_ref()
140140
.expect("no services query was run");
141141
assert!(
142142
output.contains("127.0.0.1"),
143-
"endpoint not found in services:\n{output}"
143+
"served model not found in services:\n{output}"
144144
);
145145
}
146146

@@ -152,12 +152,12 @@ async fn assert_privacy_notice_accurate(_world: &mut E2eWorld) {
152152
);
153153
}
154154

155-
#[then("the endpoint port matches the default serve port")]
156-
async fn assert_default_port(world: &mut E2eWorld) {
155+
#[then("the served model is listed on the default port")]
156+
async fn assert_listed_on_default_port(world: &mut E2eWorld) {
157157
let (stdout, _, _) = run_rocm(world, &["services", "list"]);
158158
assert!(
159159
stdout.contains("11435"),
160-
"default serve port 11435 not found in services list:\n{stdout}"
160+
"served model not listed on default port 11435:\n{stdout}"
161161
);
162162
}
163163

tests/e2e-cucumber/tests/e2e/examine_steps.rs

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ fn run_rocm(world: &E2eWorld, args: &[&str]) -> (String, String, i32) {
2121
)
2222
}
2323

24-
#[given("a machine with an AMD GPU and ROCm installed")]
24+
#[given("a machine with an AMD GPU")]
2525
async fn setup_gpu_machine(world: &mut E2eWorld) {
2626
let (stdout, _, _) = run_rocm(world, &["examine"]);
2727
assert!(
@@ -30,7 +30,7 @@ async fn setup_gpu_machine(world: &mut E2eWorld) {
3030
);
3131
}
3232

33-
#[given("a machine with a pre-installed ROCm that was not set up by the CLI")]
33+
#[given("a machine with a ROCm install that was not set up by the CLI")]
3434
async fn setup_unmanaged_rocm(_world: &mut E2eWorld) {}
3535

3636
#[when("the user asks for the version")]
@@ -71,7 +71,7 @@ async fn assert_all_engines_listed(world: &mut E2eWorld) {
7171
}
7272
}
7373

74-
#[then("the inspection reports a detected GPU target")]
74+
#[then("the inspection reports which GPU is installed")]
7575
async fn assert_gpu_detected(world: &mut E2eWorld) {
7676
let output = world.cli_output.as_ref().expect("no command was run");
7777
assert!(
@@ -99,7 +99,7 @@ async fn assert_driver_available(world: &mut E2eWorld) {
9999
);
100100
}
101101

102-
#[then("the inspection reports the ROCm install as unmanaged")]
102+
#[then("the inspection reports the install as pre-existing")]
103103
async fn assert_rocm_unmanaged(world: &mut E2eWorld) {
104104
let output = world.cli_output.as_ref().expect("no command was run");
105105
assert!(
@@ -108,7 +108,7 @@ async fn assert_rocm_unmanaged(world: &mut E2eWorld) {
108108
);
109109
}
110110

111-
#[then("the inspection suggests installing a managed runtime")]
111+
#[then("the inspection suggests setting up a CLI-managed install")]
112112
async fn assert_suggests_managed_runtime(world: &mut E2eWorld) {
113113
let output = world.cli_output.as_ref().expect("no command was run");
114114
assert!(

tests/e2e-cucumber/tests/e2e/runtime_steps.rs

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ fn run_rocm(world: &E2eWorld, args: &[&str]) -> (String, String, i32) {
2121
)
2222
}
2323

24-
#[given("a machine with no managed runtimes")]
24+
#[given("a machine with no CLI-managed runtimes")]
2525
async fn setup_no_runtimes(world: &mut E2eWorld) {
2626
let (stdout, _, _) = run_rocm(world, &["runtimes", "list"]);
2727
assert!(
@@ -54,7 +54,7 @@ async fn user_installs_sdk(world: &mut E2eWorld) {
5454
world.cli_output = Some(stdout);
5555
}
5656

57-
#[when("the user tries to adopt the standard install as a managed runtime")]
57+
#[when("the user tries to adopt the existing install")]
5858
async fn user_tries_adopt(world: &mut E2eWorld) {
5959
let (stdout, stderr, rc) = run_rocm(
6060
world,
@@ -72,7 +72,7 @@ async fn user_tries_adopt(world: &mut E2eWorld) {
7272
world.cli_rc = Some(rc);
7373
}
7474

75-
#[then("a managed runtime is registered")]
75+
#[then("a runtime is registered")]
7676
async fn assert_runtime_registered(world: &mut E2eWorld) {
7777
let (stdout, _, _) = run_rocm(world, &["runtimes", "list"]);
7878
assert!(
@@ -96,7 +96,7 @@ async fn assert_runtime_active(world: &mut E2eWorld) {
9696
);
9797
}
9898

99-
#[then("the runtime includes a matching inference stack")]
99+
#[then("the runtime includes an inference engine")]
100100
async fn assert_runtime_has_stack(world: &mut E2eWorld) {
101101
let (stdout, _, _) = run_rocm(world, &["examine"]);
102102
assert!(
@@ -111,7 +111,7 @@ async fn assert_adoption_refused(world: &mut E2eWorld) {
111111
assert!(rc != 0, "adopt unexpectedly succeeded");
112112
}
113113

114-
#[then("the error explains that only TheRock-based installs can be adopted")]
114+
#[then("the error explains which install types can be adopted")]
115115
async fn assert_adopt_error_explains(world: &mut E2eWorld) {
116116
let stdout = world.cli_output.as_deref().unwrap_or("");
117117
let stderr = world.cli_stderr.as_deref().unwrap_or("");

tests/e2e-cucumber/tests/e2e/serving_steps.rs

Lines changed: 5 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -56,9 +56,6 @@ async fn setup_mock_custom_port(world: &mut E2eWorld) {
5656
world.mock = Some(mock);
5757
}
5858

59-
#[given("the server is registered as a managed service")]
60-
async fn setup_registered_service(_world: &mut E2eWorld) {}
61-
6259
#[given("a model is being served on GPU")]
6360
async fn setup_gpu_model(world: &mut E2eWorld) {
6461
let (stdout, _, rc) = run_rocm(
@@ -181,7 +178,7 @@ async fn assert_consistent_expansion(world: &mut E2eWorld) {
181178
}
182179
}
183180

184-
#[then("the service appears with the correct model name and endpoint")]
181+
#[then("the service appears with the correct model name and connection details")]
185182
async fn assert_service_in_list(world: &mut E2eWorld) {
186183
let (stdout, _, _) = run_rocm(world, &["services", "list"]);
187184
let model = world.model_name.as_deref().unwrap_or("");
@@ -195,7 +192,7 @@ async fn assert_service_in_list(world: &mut E2eWorld) {
195192
);
196193
}
197194

198-
#[then("the endpoint matches the actual server port")]
195+
#[then("the connection details match the actual server port")]
199196
async fn assert_endpoint_port(world: &mut E2eWorld) {
200197
let mock = world.mock.as_ref().expect("no mock server running");
201198
let port = mock.port();
@@ -206,7 +203,7 @@ async fn assert_endpoint_port(world: &mut E2eWorld) {
206203
);
207204
}
208205

209-
#[then("the serve command does not fail with a dependency error")]
206+
#[then("the model is served successfully")]
210207
async fn assert_no_dependency_error(world: &mut E2eWorld) {
211208
let stdout = world.cli_output.as_deref().unwrap_or("");
212209
let stderr = world.cli_stderr.as_deref().unwrap_or("");
@@ -231,7 +228,7 @@ async fn assert_engine_auto_selected(world: &mut E2eWorld) {
231228
);
232229
}
233230

234-
#[then("the model is reachable at the reported endpoint")]
231+
#[then("the model is reachable")]
235232
async fn assert_model_reachable(world: &mut E2eWorld) {
236233
let endpoint = world.endpoint.as_ref().expect("no endpoint configured");
237234
let url = format!("{endpoint}/models");
@@ -248,7 +245,7 @@ async fn assert_model_reachable(world: &mut E2eWorld) {
248245
);
249246
}
250247

251-
#[then("the endpoint responds to inference requests")]
248+
#[then("the model responds to inference requests")]
252249
async fn assert_endpoint_responds(world: &mut E2eWorld) {
253250
crate::send_chat(world).await;
254251
let resp = world.chat_response.as_ref().expect("no chat response");

0 commit comments

Comments
 (0)