Skip to content

Commit 1aac7b4

Browse files
maysunfaisalmaysunfaisal
committed
Address Review comments on Test Use Cases for MCP APIs
Assisted-by: Claude Opus 4.6 Generated-by: Cursor Signed-off-by: Maysun J Faisal <maysunaneek@gmail.com>
1 parent d3fc405 commit 1aac7b4

9 files changed

Lines changed: 171 additions & 8 deletions

File tree

‎docs/config.md‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -374,10 +374,10 @@ Useful resources:
374374
Model context protocol server configuration.
375375

376376
MCP (Model Context Protocol) servers provide tools and capabilities to the
377-
AI agents. These are configured by this structure. Only MCP servers
378-
defined in the lightspeed-stack.yaml configuration are available to the
379-
agents. Tools configured in the llama-stack run.yaml are not accessible to
380-
lightspeed-core agents.
377+
AI agents. These are configured by this structure. MCP servers defined in
378+
lightspeed-stack.yaml and servers registered at runtime via the
379+
`POST /v1/mcp-servers` API are available to agents. Tools configured in
380+
the llama-stack run.yaml are not accessible to lightspeed-core agents.
381381

382382
Useful resources:
383383

‎docs/getting_started.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -261,6 +261,11 @@ After starting the MCP servers and updating `lightspeed-stack.yaml`, test by sen
261261
262262
In addition to static configuration in `lightspeed-stack.yaml`, MCP servers can be registered, listed, and removed at runtime through the REST API. This is useful for development, testing, or scenarios where MCP servers are provisioned dynamically.
263263
264+
When authorization is enabled, callers need the following permissions:
265+
- `REGISTER_MCP_SERVER` for `POST /v1/mcp-servers`
266+
- `LIST_MCP_SERVERS` for `GET /v1/mcp-servers`
267+
- `DELETE_MCP_SERVER` for `DELETE /v1/mcp-servers/{name}`
268+
264269
#### Register an MCP server
265270
266271
```bash

‎tests/e2e/configuration/library-mode/lightspeed-stack-mcp-auth.yaml‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
name: Lightspeed Core Service (LCS)
2+
service:
3+
host: 0.0.0.0
4+
port: 8080
5+
auth_enabled: false
6+
workers: 1
7+
color_log: true
8+
access_log: true
9+
llama_stack:
10+
# Library mode - embeds llama-stack as library
11+
use_as_library_client: true
12+
library_client_config_path: run.yaml
13+
user_data_collection:
14+
feedback_enabled: true
15+
feedback_storage: "/tmp/data/feedback"
16+
transcripts_enabled: true
17+
transcripts_storage: "/tmp/data/transcripts"
18+
authentication:
19+
module: "noop-with-token"
20+
mcp_servers:
21+
- name: "mcp-oauth"
22+
provider_id: "model-context-protocol"
23+
url: "http://mock-mcp:3001"
24+
authorization_headers:
25+
Authorization: "oauth"

‎tests/e2e/configuration/server-mode/lightspeed-stack-mcp-auth.yaml‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
name: Lightspeed Core Service (LCS)
2+
service:
3+
host: 0.0.0.0
4+
port: 8080
5+
auth_enabled: false
6+
workers: 1
7+
color_log: true
8+
access_log: true
9+
llama_stack:
10+
# Server mode - connects to separate llama-stack service
11+
use_as_library_client: false
12+
url: http://llama-stack:8321
13+
api_key: xyzzy
14+
user_data_collection:
15+
feedback_enabled: true
16+
feedback_storage: "/tmp/data/feedback"
17+
transcripts_enabled: true
18+
transcripts_storage: "/tmp/data/transcripts"
19+
authentication:
20+
module: "noop-with-token"
21+
mcp_servers:
22+
- name: "mcp-oauth"
23+
provider_id: "model-context-protocol"
24+
url: "http://mock-mcp:3001"
25+
authorization_headers:
26+
Authorization: "oauth"

‎tests/e2e/features/environment.py‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,10 @@
7676
"tests/e2e/configuration/{mode_dir}/lightspeed-stack-mcp-oauth-auth.yaml",
7777
"tests/e2e-prow/rhoai/configs/lightspeed-stack-mcp-oauth-auth.yaml",
7878
),
79+
"mcp-auth": (
80+
"tests/e2e/configuration/{mode_dir}/lightspeed-stack-mcp-auth.yaml",
81+
"tests/e2e-prow/rhoai/configs/lightspeed-stack-mcp-auth.yaml",
82+
),
7983
}
8084

8185

@@ -441,6 +445,12 @@ def before_feature(context: Context, feature: Feature) -> None:
441445
switch_config(context.feature_config)
442446
restart_container("lightspeed-stack")
443447

448+
if "MCPServerAPIAuth" in feature.tags:
449+
context.feature_config = _get_config_path("mcp-auth", mode_dir)
450+
context.default_config_backup = create_config_backup("lightspeed-stack.yaml")
451+
switch_config(context.feature_config)
452+
restart_container("lightspeed-stack")
453+
444454

445455
def after_feature(context: Context, feature: Feature) -> None:
446456
"""Run after each feature file is exercised.
@@ -478,3 +488,8 @@ def after_feature(context: Context, feature: Feature) -> None:
478488
switch_config(context.default_config_backup)
479489
restart_container("lightspeed-stack")
480490
remove_config_backup(context.default_config_backup)
491+
492+
if "MCPServerAPIAuth" in feature.tags:
493+
switch_config(context.default_config_backup)
494+
restart_container("lightspeed-stack")
495+
remove_config_backup(context.default_config_backup)

‎tests/e2e/features/mcp_servers_api.feature‎

Lines changed: 47 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -52,22 +52,36 @@ Feature: MCP Server Management API tests
5252
Given The system is in default state
5353
When I access REST API endpoint "mcp-servers/non-existent-server" using HTTP DELETE method
5454
Then The status code of the response is 404
55+
And The body of the response contains Mcp Server not found
5556

56-
Scenario: Register MCP server with invalid request body returns 422
57+
Scenario: Register MCP server with missing required fields returns 422
5758
Given The system is in default state
5859
When I access REST API endpoint "mcp-servers" using HTTP POST method
5960
"""
6061
{"url": "http://mock-mcp:3001"}
6162
"""
6263
Then The status code of the response is 422
64+
And The body of the response contains name
6365

6466
Scenario: Register MCP server with invalid URL scheme returns 422
6567
Given The system is in default state
6668
When I access REST API endpoint "mcp-servers" using HTTP POST method
6769
"""
68-
{"name": "bad-url-server", "url": "ftp://mock-mcp:3001"}
70+
{"name": "bad-url-server", "url": "ftp://mock-mcp:3001", "provider_id": "model-context-protocol"}
6971
"""
7072
Then The status code of the response is 422
73+
And The body of the response contains scheme
74+
75+
@skip-in-library-mode
76+
Scenario: Register MCP server returns 503 when Llama Stack is unreachable
77+
Given The system is in default state
78+
And The llama-stack connection is disrupted
79+
When I access REST API endpoint "mcp-servers" using HTTP POST method
80+
"""
81+
{"name": "unreachable-server", "url": "http://mock-mcp:3001", "provider_id": "model-context-protocol"}
82+
"""
83+
Then The status code of the response is 503
84+
And The body of the response contains Llama Stack
7185

7286
@skip-in-library-mode
7387
Scenario: Register and delete MCP server lifecycle
@@ -81,10 +95,39 @@ Feature: MCP Server Management API tests
8195
And The body of the response contains registered successfully
8296
When I access REST API endpoint "mcp-servers" using HTTP GET method
8397
Then The status code of the response is 200
84-
And The body of the response contains e2e-lifecycle-server
85-
And The body of the response contains api
98+
And the body of the response has the following structure
99+
"""
100+
{
101+
"servers": [
102+
{
103+
"name": "mcp-oauth",
104+
"source": "config"
105+
},
106+
{
107+
"name": "e2e-lifecycle-server",
108+
"url": "http://mock-mcp:3001",
109+
"provider_id": "model-context-protocol",
110+
"source": "api"
111+
}
112+
]
113+
}
114+
"""
86115
When I access REST API endpoint "mcp-servers/e2e-lifecycle-server" using HTTP DELETE method
87116
Then The status code of the response is 200
117+
And The body of the response contains e2e-lifecycle-server
88118
And The body of the response contains unregistered successfully
119+
When I access REST API endpoint "mcp-servers/e2e-lifecycle-server" using HTTP DELETE method
120+
Then The status code of the response is 404
89121
When I access REST API endpoint "mcp-servers" using HTTP GET method
90122
Then The status code of the response is 200
123+
And the body of the response has the following structure
124+
"""
125+
{
126+
"servers": [
127+
{
128+
"name": "mcp-oauth",
129+
"source": "config"
130+
}
131+
]
132+
}
133+
"""

‎tests/e2e/features/mcp_servers_api_auth.feature‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
@MCPServerAPIAuth
2+
Feature: MCP Server Management API authentication tests
3+
4+
Tests that the MCP server management endpoints enforce authentication
5+
when authentication is enabled (noop-with-token module).
6+
7+
Background:
8+
Given The service is started locally
9+
And REST API service prefix is /v1
10+
11+
Scenario: List MCP servers returns 401 without auth token
12+
Given The system is in default state
13+
And I remove the auth header
14+
When I access REST API endpoint "mcp-servers" using HTTP GET method
15+
Then The status code of the response is 401
16+
17+
Scenario: Register MCP server returns 401 without auth token
18+
Given The system is in default state
19+
And I remove the auth header
20+
When I access REST API endpoint "mcp-servers" using HTTP POST method
21+
"""
22+
{"name": "auth-test-server", "url": "http://mock-mcp:3001", "provider_id": "model-context-protocol"}
23+
"""
24+
Then The status code of the response is 401
25+
26+
Scenario: Delete MCP server returns 401 without auth token
27+
Given The system is in default state
28+
And I remove the auth header
29+
When I access REST API endpoint "mcp-servers/mcp-oauth" using HTTP DELETE method
30+
Then The status code of the response is 401

‎tests/e2e/features/mcp_servers_api_no_config.feature‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
Feature: MCP Server API tests without configured MCP servers
2+
3+
Tests that the MCP server management endpoints work correctly
4+
when no MCP servers are configured in lightspeed-stack.yaml.
5+
6+
Background:
7+
Given The service is started locally
8+
And REST API service prefix is /v1
9+
10+
Scenario: List MCP servers returns empty list when none configured
11+
Given The system is in default state
12+
When I access REST API endpoint "mcp-servers" using HTTP GET method
13+
Then The status code of the response is 200
14+
And The body of the response is the following
15+
"""
16+
{"servers": []}
17+
"""

‎tests/e2e/test_list.txt‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,4 +17,6 @@ features/streaming_query.feature
1717
features/rest_api.feature
1818
features/mcp.feature
1919
features/mcp_servers_api.feature
20+
features/mcp_servers_api_auth.feature
21+
features/mcp_servers_api_no_config.feature
2022
features/models.feature

0 commit comments

Comments
 (0)