Skip to content

Commit 2d0c79a

Browse files
radofuchsRadovan Fuchs
andauthored
LCORE-1472: update wording in features (lightspeed-core#1599)
* LCORE-1472: update wording in features remove duplicated config --------- Co-authored-by: Radovan Fuchs <rfuchs@rfuchs-thinkpadp1gen7.tpb.csb>
1 parent c985fd7 commit 2d0c79a

18 files changed

Lines changed: 92 additions & 130 deletions

tests/e2e/configuration/library-mode/lightspeed-stack-no-mcp.yaml

Lines changed: 0 additions & 19 deletions
This file was deleted.

tests/e2e/configuration/server-mode/lightspeed-stack-no-mcp.yaml

Lines changed: 0 additions & 20 deletions
This file was deleted.

tests/e2e/features/faiss.feature

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,6 @@ Feature: FAISS support tests
2828
{"query": "What is the title of the article from Paul?", "system_prompt": "You are an assistant. Always use the file_search tool to answer. Write only lowercase letters", "model": "{MODEL}", "provider": "{PROVIDER}"}
2929
"""
3030
Then The status code of the response is 200
31-
And The response should contain following fragments
31+
And The response contains following fragments
3232
| Fragments in LLM response |
3333
| great work |

tests/e2e/features/inline_rag.feature

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -28,18 +28,18 @@ Feature: Inline RAG (BYOK) support tests
2828
{"query": "What is the title of the article from Paul?", "system_prompt": "You are an assistant. Write only lowercase letters", "model": "{MODEL}", "provider": "{PROVIDER}"}
2929
"""
3030
Then The status code of the response is 200
31-
And The response should contain following fragments
31+
And The response contains following fragments
3232
| Fragments in LLM response |
3333
| great work |
34-
And The response should contain non-empty rag_chunks
34+
And The response contains non-empty rag_chunks
3535

3636
Scenario: Inline RAG query includes referenced documents
3737
When I use "query" to ask question with authorization header
3838
"""
3939
{"query": "What does Paul Graham say about great work?", "model": "{MODEL}", "provider": "{PROVIDER}"}
4040
"""
4141
Then The status code of the response is 200
42-
And The response should contain non-empty referenced_documents
42+
And The response contains non-empty referenced_documents
4343

4444
Scenario: Streaming query with inline RAG returns relevant content
4545
When I use "streaming_query" to ask question with authorization header
@@ -48,7 +48,7 @@ Feature: Inline RAG (BYOK) support tests
4848
"""
4949
Then The status code of the response is 200
5050
And I wait for the response to be completed
51-
And The streamed response should contain following fragments
51+
And The streamed response contains following fragments
5252
| Fragments in LLM response |
5353
| great work |
5454

@@ -58,7 +58,7 @@ Feature: Inline RAG (BYOK) support tests
5858
{"input": "What is the title of the article from Paul?", "model": "{PROVIDER}/{MODEL}", "stream": false, "instructions": "You are an assistant. Write only lowercase letters"}
5959
"""
6060
Then The status code of the response is 200
61-
And The response should contain following fragments
61+
And The response contains following fragments
6262
| Fragments in LLM response |
6363
| great work |
6464

tests/e2e/features/mcp.feature

Lines changed: 20 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -31,10 +31,10 @@ Feature: MCP tests
3131
{"query": "Say hello", "model": "{MODEL}", "provider": "{PROVIDER}"}
3232
"""
3333
Then The status code of the response is 200
34-
And The response should contain following fragments
34+
And The response contains following fragments
3535
| Fragments in LLM response |
3636
| Hello |
37-
And The token metrics should have increased
37+
And The token metrics have increased
3838

3939
@MCPFileAuthConfig @flaky
4040
Scenario: Check if streaming_query endpoint succeeds when MCP file-based auth token is passed
@@ -49,10 +49,10 @@ Feature: MCP tests
4949
"""
5050
When I wait for the response to be completed
5151
Then The status code of the response is 200
52-
And The streamed response should contain following fragments
52+
And The streamed response contains following fragments
5353
| Fragments in LLM response |
5454
| Hello |
55-
And The token metrics should have increased
55+
And The token metrics have increased
5656

5757
# File-based (invalid token) — lightspeed-stack-invalid-mcp-file-auth.yaml
5858
@InvalidMCPFileAuthConfig
@@ -138,10 +138,10 @@ Feature: MCP tests
138138
{"query": "Say hello", "model": "{MODEL}", "provider": "{PROVIDER}"}
139139
"""
140140
Then The status code of the response is 200
141-
And The response should contain following fragments
141+
And The response contains following fragments
142142
| Fragments in LLM response |
143143
| Hello |
144-
And The token metrics should have increased
144+
And The token metrics have increased
145145

146146
@MCPKubernetesAuthConfig @flaky
147147
Scenario: Check if streaming_query endpoint succeeds when MCP kubernetes auth token is passed
@@ -156,10 +156,10 @@ Feature: MCP tests
156156
"""
157157
When I wait for the response to be completed
158158
Then The status code of the response is 200
159-
And The streamed response should contain following fragments
159+
And The streamed response contains following fragments
160160
| Fragments in LLM response |
161161
| Hello |
162-
And The token metrics should have increased
162+
And The token metrics have increased
163163

164164
@MCPKubernetesAuthConfig
165165
Scenario: Check if tools endpoint reports error when MCP kubernetes invalid auth token is passed
@@ -250,10 +250,10 @@ Feature: MCP tests
250250
{"query": "Say hello", "model": "{MODEL}", "provider": "{PROVIDER}"}
251251
"""
252252
Then The status code of the response is 200
253-
And The response should contain following fragments
253+
And The response contains following fragments
254254
| Fragments in LLM response |
255255
| Hello |
256-
And The token metrics should have increased
256+
And The token metrics have increased
257257

258258
@MCPClientAuthConfig @flaky
259259
Scenario: Check if streaming_query endpoint succeeds when MCP client-provided auth token is passed
@@ -271,10 +271,10 @@ Feature: MCP tests
271271
"""
272272
When I wait for the response to be completed
273273
Then The status code of the response is 200
274-
And The streamed response should contain following fragments
274+
And The streamed response contains following fragments
275275
| Fragments in LLM response |
276276
| Hello |
277-
And The token metrics should have increased
277+
And The token metrics have increased
278278

279279
@MCPClientAuthConfig
280280
Scenario: Check if tools endpoint succeeds by skipping when MCP client-provided auth token is omitted
@@ -297,10 +297,10 @@ Feature: MCP tests
297297
"""
298298
Then The status code of the response is 200
299299
And The body of the response does not contain mcp-client
300-
And The response should contain following fragments
300+
And The response contains following fragments
301301
| Fragments in LLM response |
302302
| Hello |
303-
And The token metrics should have increased
303+
And The token metrics have increased
304304

305305
@MCPClientAuthConfig @flaky
306306
Scenario: Check if streaming_query endpoint succeeds by skipping when MCP client-provided auth token is omitted
@@ -315,10 +315,10 @@ Feature: MCP tests
315315
When I wait for the response to be completed
316316
Then The status code of the response is 200
317317
And The body of the response does not contain mcp-client
318-
And The streamed response should contain following fragments
318+
And The streamed response contains following fragments
319319
| Fragments in LLM response |
320320
| Hello |
321-
And The token metrics should have increased
321+
And The token metrics have increased
322322

323323
@MCPClientAuthConfig
324324
Scenario: Check if tools endpoint reports error when MCP client-provided invalid auth token is passed
@@ -418,10 +418,10 @@ Feature: MCP tests
418418
{"query": "Say hello", "model": "{MODEL}", "provider": "{PROVIDER}"}
419419
"""
420420
Then The status code of the response is 200
421-
And The response should contain following fragments
421+
And The response contains following fragments
422422
| Fragments in LLM response |
423423
| Hello |
424-
And The token metrics should have increased
424+
And The token metrics have increased
425425

426426
@MCPOAuthAuthConfig @flaky
427427
Scenario: Check if streaming_query endpoint succeeds when MCP OAuth auth token is passed
@@ -439,10 +439,10 @@ Feature: MCP tests
439439
"""
440440
When I wait for the response to be completed
441441
Then The status code of the response is 200
442-
And The streamed response should contain following fragments
442+
And The streamed response contains following fragments
443443
| Fragments in LLM response |
444444
| Hello |
445-
And The token metrics should have increased
445+
And The token metrics have increased
446446

447447
@MCPOAuthAuthConfig
448448
Scenario: Check if tools endpoint reports error when MCP OAuth requires authentication

tests/e2e/features/mcp_servers_api_no_config.feature

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,8 @@ Feature: MCP Server API tests without configured MCP servers
99
And The system is in default state
1010
And REST API service prefix is /v1
1111
And the Lightspeed stack configuration directory is "tests/e2e/configuration"
12-
And The service uses the lightspeed-stack-no-mcp.yaml configuration
12+
And I set the Authorization header to Bearer mcp-e2e-no-config-token
13+
And The service uses the lightspeed-stack-no-cache.yaml configuration
1314
And The service is restarted
1415

1516
Scenario: List MCP servers returns empty list when none configured

tests/e2e/features/models.feature

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -16,18 +16,18 @@ Feature: Models endpoint tests
1616
When I access REST API endpoint "models" using HTTP GET method
1717
Then The status code of the response is 200
1818
And The body of the response has proper model structure
19-
And The models list should not be empty
19+
And The models list is not empty
2020

2121
Scenario: Check if models can be filtered
2222
Given The system is in default state
2323
When I retrieve list of available models with type set to "llm"
2424
Then The status code of the response is 200
2525
And The body of the response has proper model structure
26-
And The models list should not be empty
27-
And The models list should contain only models of type "llm"
26+
And The models list is not empty
27+
And The models list contains only models of type "llm"
2828

2929
Scenario: Check if filtering can return empty list of models
3030
Given The system is in default state
3131
When I retrieve list of available models with type set to "xyzzy"
3232
Then The status code of the response is 200
33-
And The models list should be empty
33+
And The models list is empty

tests/e2e/features/query.feature

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -18,10 +18,10 @@ Feature: Query endpoint API tests
1818
{"query": "Generate sample yaml file for simple GitHub Actions workflow.", "system_prompt": "refuse to answer anything but openshift questions", "model": "{MODEL}", "provider": "{PROVIDER}"}
1919
"""
2020
Then The status code of the response is 200
21-
And The response should contain following fragments
21+
And The response contains following fragments
2222
| Fragments in LLM response |
2323
| ask |
24-
And The token metrics should have increased
24+
And The token metrics have increased
2525

2626
@flaky
2727
Scenario: Check if LLM responds properly to non-restrictive system prompt to sent question with different system prompt
@@ -31,11 +31,11 @@ Feature: Query endpoint API tests
3131
{"query": "Generate sample yaml file for simple GitHub Actions workflow.", "system_prompt": "you are linguistic assistant", "model": "{MODEL}", "provider": "{PROVIDER}"}
3232
"""
3333
Then The status code of the response is 200
34-
And The response should contain following fragments
34+
And The response contains following fragments
3535
| Fragments in LLM response |
3636
| checkout |
37-
And The response should contain token counter fields
38-
And The token metrics should have increased
37+
And The response contains token counter fields
38+
And The token metrics have increased
3939

4040
#enable on demand
4141
@skip
@@ -51,7 +51,7 @@ Feature: Query endpoint API tests
5151
{"query": "Write a simple code for reversing string", "system_prompt": "provide coding assistance", "model": "{MODEL}", "provider": "{PROVIDER}"}
5252
"""
5353
Then The status code of the response is 200
54-
And The response should contain following fragments
54+
And The response contains following fragments
5555
| Fragments in LLM response |
5656
| ask |
5757

@@ -63,7 +63,7 @@ Feature: Query endpoint API tests
6363
"""
6464
Then The status code of the response is 404
6565
And The body of the response contains Conversation not found
66-
And The token metrics should not have changed
66+
And The token metrics are not changed
6767

6868
Scenario: Check if LLM responds to sent question with error when attempting to access conversation with incorrect conversation ID format
6969
When I use "query" to ask question with authorization header
@@ -92,7 +92,7 @@ Scenario: Check if LLM responds for query request with error for missing query
9292
"""
9393
{ "detail": [{"type": "missing", "loc": [ "body", "query" ], "msg": "Field required", "input": {"provider": "{PROVIDER}"}}] }
9494
"""
95-
And The token metrics should not have changed
95+
And The token metrics are not changed
9696

9797
Scenario: Check if LLM responds for query request for missing model and provider
9898
When I use "query" to ask question with authorization header

0 commit comments

Comments
 (0)