@@ -17,9 +17,13 @@ Feature: Proxy and TLS networking tests for Llama Stack providers
1717 Scenario : LLM traffic is routed through a configured tunnel proxy
1818 Given A tunnel proxy is running on port 8888
1919 And Llama Stack is configured to route inference through the tunnel proxy
20- And The services are restarted with the modified Llama Stack config
21- When I send a query "What is 2+2?" to the LLM
22- Then The LLM responds successfully
20+ And Llama Stack is restarted
21+ And Lightspeed Stack is restarted
22+ When I use "query" to ask question
23+ """
24+ {"query": "What is 2+2?", "model": "{MODEL}", "provider": "{PROVIDER}"}
25+ """
26+ Then The status code of the response is 200
2327 And The tunnel proxy handled at least 1 CONNECT request to the LLM provider
2428
2529 # NOTE: no_proxy is defined on Llama Stack's ProxyConfig model but not
@@ -29,9 +33,13 @@ Feature: Proxy and TLS networking tests for Llama Stack providers
2933 @TunnelProxy
3034 Scenario : LLM query fails gracefully when proxy is unreachable
3135 Given Llama Stack is configured to route inference through proxy "http://127.0.0.1:19999"
32- And The services are restarted with the modified Llama Stack config
33- When I send a query "What is 2+2?" to the LLM
34- Then The response indicates a proxy or connection error
36+ And Llama Stack is restarted
37+ And Lightspeed Stack is restarted
38+ When I use "query" to ask question
39+ """
40+ {"query": "What is 2+2?", "model": "{MODEL}", "provider": "{PROVIDER}"}
41+ """
42+ Then The status code of the response is 500
3543
3644
3745 # --- AC2: Interception proxy with CA certificate ---
@@ -40,39 +48,59 @@ Feature: Proxy and TLS networking tests for Llama Stack providers
4048 Scenario : LLM traffic works through interception proxy with correct CA
4149 Given An interception proxy with trustme CA is running on port 8889
4250 And Llama Stack is configured to route inference through the interception proxy with CA cert
43- And The services are restarted with the modified Llama Stack config
44- When I send a query "What is 2+2?" to the LLM
45- Then The LLM responds successfully
51+ And Llama Stack is restarted
52+ And Lightspeed Stack is restarted
53+ When I use "query" to ask question
54+ """
55+ {"query": "What is 2+2?", "model": "{MODEL}", "provider": "{PROVIDER}"}
56+ """
57+ Then The status code of the response is 200
4658 And The interception proxy intercepted at least 1 connection
4759
4860 @InterceptionProxy
4961 Scenario : LLM query fails when interception proxy CA is not provided
5062 Given An interception proxy with trustme CA is running on port 8890
5163 And Llama Stack is configured to route inference through the interception proxy without CA cert
52- And The services are restarted with the modified Llama Stack config
53- When I send a query "What is 2+2?" to the LLM
54- Then The response indicates a proxy or connection error
64+ And Llama Stack is restarted
65+ And Lightspeed Stack is restarted
66+ When I use "query" to ask question
67+ """
68+ {"query": "What is 2+2?", "model": "{MODEL}", "provider": "{PROVIDER}"}
69+ """
70+ Then The status code of the response is 500
5571
5672
5773 # --- AC3: TLS version and cipher configuration ---
5874
5975 @TLSVersion
6076 Scenario : TLS minimum version TLSv1.2 is respected
6177 Given Llama Stack is configured with minimum TLS version "TLSv1.2"
62- And The services are restarted with the modified Llama Stack config
63- When I send a query "What is 2+2?" to the LLM
64- Then The LLM responds successfully
78+ And Llama Stack is restarted
79+ And Lightspeed Stack is restarted
80+ When I use "query" to ask question
81+ """
82+ {"query": "What is 2+2?", "model": "{MODEL}", "provider": "{PROVIDER}"}
83+ """
84+ Then The status code of the response is 200
6585
6686 @TLSVersion
6787 Scenario : TLS minimum version TLSv1.3 is respected
6888 Given Llama Stack is configured with minimum TLS version "TLSv1.3"
69- And The services are restarted with the modified Llama Stack config
70- When I send a query "What is 2+2?" to the LLM
71- Then The LLM responds successfully
89+ And Llama Stack is restarted
90+ And Lightspeed Stack is restarted
91+ When I use "query" to ask question
92+ """
93+ {"query": "What is 2+2?", "model": "{MODEL}", "provider": "{PROVIDER}"}
94+ """
95+ Then The status code of the response is 200
7296
7397 @TLSCipher
7498 Scenario : Custom cipher suite configuration is respected
7599 Given Llama Stack is configured with ciphers "ECDHE+AESGCM:DHE+AESGCM"
76- And The services are restarted with the modified Llama Stack config
77- When I send a query "What is 2+2?" to the LLM
78- Then The LLM responds successfully
100+ And Llama Stack is restarted
101+ And Lightspeed Stack is restarted
102+ When I use "query" to ask question
103+ """
104+ {"query": "What is 2+2?", "model": "{MODEL}", "provider": "{PROVIDER}"}
105+ """
106+ Then The status code of the response is 200
0 commit comments