Skip to content

Commit e416861

Browse files
committed
LCORE-1253: Address radofuchs review feedback
Rewrite proxy e2e tests to follow project conventions: - Reuse existing step definitions: use "I use query to ask question" from llm_query_response.py and "The status code of the response is" from common_http.py instead of custom query/response steps. - Split service restart into two explicit Given steps: "Llama Stack is restarted" and "Lightspeed Stack is restarted" so restart ordering is visible in the feature file. - Remove local (non-Docker) mode code path. Proxy tests use restart_container() exclusively, consistent with the rest of the e2e test suite. - Check specific status code 500 for error scenarios instead of the broad >= 400 range. - Remove custom send_query, verify_llm_response, and verify_error_response steps that duplicated existing functionality. Net reduction: -183 lines from step definitions.
1 parent cc91b36 commit e416861

2 files changed

Lines changed: 74 additions & 183 deletions

File tree

tests/e2e/features/proxy.feature

Lines changed: 49 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -17,9 +17,13 @@ Feature: Proxy and TLS networking tests for Llama Stack providers
1717
Scenario: LLM traffic is routed through a configured tunnel proxy
1818
Given A tunnel proxy is running on port 8888
1919
And Llama Stack is configured to route inference through the tunnel proxy
20-
And The services are restarted with the modified Llama Stack config
21-
When I send a query "What is 2+2?" to the LLM
22-
Then The LLM responds successfully
20+
And Llama Stack is restarted
21+
And Lightspeed Stack is restarted
22+
When I use "query" to ask question
23+
"""
24+
{"query": "What is 2+2?", "model": "{MODEL}", "provider": "{PROVIDER}"}
25+
"""
26+
Then The status code of the response is 200
2327
And The tunnel proxy handled at least 1 CONNECT request to the LLM provider
2428

2529
# NOTE: no_proxy is defined on Llama Stack's ProxyConfig model but not
@@ -29,9 +33,13 @@ Feature: Proxy and TLS networking tests for Llama Stack providers
2933
@TunnelProxy
3034
Scenario: LLM query fails gracefully when proxy is unreachable
3135
Given Llama Stack is configured to route inference through proxy "http://127.0.0.1:19999"
32-
And The services are restarted with the modified Llama Stack config
33-
When I send a query "What is 2+2?" to the LLM
34-
Then The response indicates a proxy or connection error
36+
And Llama Stack is restarted
37+
And Lightspeed Stack is restarted
38+
When I use "query" to ask question
39+
"""
40+
{"query": "What is 2+2?", "model": "{MODEL}", "provider": "{PROVIDER}"}
41+
"""
42+
Then The status code of the response is 500
3543

3644

3745
# --- AC2: Interception proxy with CA certificate ---
@@ -40,39 +48,59 @@ Feature: Proxy and TLS networking tests for Llama Stack providers
4048
Scenario: LLM traffic works through interception proxy with correct CA
4149
Given An interception proxy with trustme CA is running on port 8889
4250
And Llama Stack is configured to route inference through the interception proxy with CA cert
43-
And The services are restarted with the modified Llama Stack config
44-
When I send a query "What is 2+2?" to the LLM
45-
Then The LLM responds successfully
51+
And Llama Stack is restarted
52+
And Lightspeed Stack is restarted
53+
When I use "query" to ask question
54+
"""
55+
{"query": "What is 2+2?", "model": "{MODEL}", "provider": "{PROVIDER}"}
56+
"""
57+
Then The status code of the response is 200
4658
And The interception proxy intercepted at least 1 connection
4759

4860
@InterceptionProxy
4961
Scenario: LLM query fails when interception proxy CA is not provided
5062
Given An interception proxy with trustme CA is running on port 8890
5163
And Llama Stack is configured to route inference through the interception proxy without CA cert
52-
And The services are restarted with the modified Llama Stack config
53-
When I send a query "What is 2+2?" to the LLM
54-
Then The response indicates a proxy or connection error
64+
And Llama Stack is restarted
65+
And Lightspeed Stack is restarted
66+
When I use "query" to ask question
67+
"""
68+
{"query": "What is 2+2?", "model": "{MODEL}", "provider": "{PROVIDER}"}
69+
"""
70+
Then The status code of the response is 500
5571

5672

5773
# --- AC3: TLS version and cipher configuration ---
5874

5975
@TLSVersion
6076
Scenario: TLS minimum version TLSv1.2 is respected
6177
Given Llama Stack is configured with minimum TLS version "TLSv1.2"
62-
And The services are restarted with the modified Llama Stack config
63-
When I send a query "What is 2+2?" to the LLM
64-
Then The LLM responds successfully
78+
And Llama Stack is restarted
79+
And Lightspeed Stack is restarted
80+
When I use "query" to ask question
81+
"""
82+
{"query": "What is 2+2?", "model": "{MODEL}", "provider": "{PROVIDER}"}
83+
"""
84+
Then The status code of the response is 200
6585

6686
@TLSVersion
6787
Scenario: TLS minimum version TLSv1.3 is respected
6888
Given Llama Stack is configured with minimum TLS version "TLSv1.3"
69-
And The services are restarted with the modified Llama Stack config
70-
When I send a query "What is 2+2?" to the LLM
71-
Then The LLM responds successfully
89+
And Llama Stack is restarted
90+
And Lightspeed Stack is restarted
91+
When I use "query" to ask question
92+
"""
93+
{"query": "What is 2+2?", "model": "{MODEL}", "provider": "{PROVIDER}"}
94+
"""
95+
Then The status code of the response is 200
7296

7397
@TLSCipher
7498
Scenario: Custom cipher suite configuration is respected
7599
Given Llama Stack is configured with ciphers "ECDHE+AESGCM:DHE+AESGCM"
76-
And The services are restarted with the modified Llama Stack config
77-
When I send a query "What is 2+2?" to the LLM
78-
Then The LLM responds successfully
100+
And Llama Stack is restarted
101+
And Lightspeed Stack is restarted
102+
When I use "query" to ask question
103+
"""
104+
{"query": "What is 2+2?", "model": "{MODEL}", "provider": "{PROVIDER}"}
105+
"""
106+
Then The status code of the response is 200

0 commit comments

Comments
 (0)