Skip to content

Commit 864411b

Browse files
jrobertboosjrobertboos
committed
Add new MCP authentication configurations and update tests
- Introduced new YAML configuration files for MCP authentication methods: file, Kubernetes, and OAuth. - Updated existing MCP configuration to include new authentication methods. - Enhanced end-to-end tests to cover scenarios for file-based, Kubernetes, and OAuth authentication. - Added utility functions for unregistering MCP toolgroups in the testing environment. - Implemented new step definitions for checking response body content in tests. Add invalid MCP token configuration and update tests - Added a new invalid MCP token file for testing purposes. - Updated the Docker Compose configuration to mount the invalid MCP token. - Introduced a new YAML configuration for testing invalid MCP file authentication. - Enhanced the test scenarios to include checks for invalid MCP file authentication. - Updated feature files to reflect the new authentication configurations. Add additional MCP authentication configurations and update scenario handling - Added new configurations for invalid, Kubernetes, client, and OAuth MCP authentication methods. - Updated scenario handling to reference the new configuration paths for Kubernetes, client, and OAuth authentication. - Enhanced the testing environment to support the new authentication configurations. fixed errored tests skipped failing scenarios and added library mode configs fixed library mode tests error fixed black skipped acidentally missed failing test Refactor Llama Stack utilities and update E2E tests - Renamed `llama_stack_shields.py` to `llama_stack_utils.py` and expanded its functionality to manage both toolgroups and shields. - Removed the deprecated `llama_stack_tools.py` file. - Updated E2E test scenarios to utilize the new utility functions for unregistering toolgroups and clearing Llama Stack storage. - Enhanced feature files to include comments indicating pending fixes for skipped scenarios. Enhance MCP feature tests and streamline Llama Stack storage clearing - Added checks to ensure the response body does not contain 'mcp-client' in MCP feature tests. - Simplified the `clear_llama_stack_storage` function to remove the entire `~/.llama` directory instead of specific files, improving clarity and efficiency. - Updated comments to reflect the changes in storage clearing logic. Remove deprecated MCP file-based authorization feature and update test list addressed code rabbit Add new MCP authentication configurations and update tests - Introduced new YAML configuration files for MCP authentication methods: file, Kubernetes, and OAuth. - Updated existing MCP configuration to include new authentication methods. - Enhanced end-to-end tests to cover scenarios for file-based, Kubernetes, and OAuth authentication. - Added utility functions for unregistering MCP toolgroups in the testing environment. - Implemented new step definitions for checking response body content in tests. fixed errored tests fixed library mode tests error fixed black Refactor Llama Stack utilities and update E2E tests - Renamed `llama_stack_shields.py` to `llama_stack_utils.py` and expanded its functionality to manage both toolgroups and shields. - Removed the deprecated `llama_stack_tools.py` file. - Updated E2E test scenarios to utilize the new utility functions for unregistering toolgroups and clearing Llama Stack storage. - Enhanced feature files to include comments indicating pending fixes for skipped scenarios. Enhance MCP feature tests and streamline Llama Stack storage clearing - Added checks to ensure the response body does not contain 'mcp-client' in MCP feature tests. - Simplified the `clear_llama_stack_storage` function to remove the entire `~/.llama` directory instead of specific files, improving clarity and efficiency. - Updated comments to reflect the changes in storage clearing logic. Remove deprecated MCP file-based authorization feature and update test list addressed code rabbit fixed ruff addressed comments addressed talk with radovan addressed comments fixed black fixed error
1 parent cc31d1c commit 864411b

23 files changed

Lines changed: 770 additions & 55 deletions

‎docker-compose.yaml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -82,6 +82,7 @@ services:
8282
volumes:
8383
- ./lightspeed-stack.yaml:/app-root/lightspeed-stack.yaml:z
8484
- ./tests/e2e/secrets/mcp-token:/tmp/mcp-secret-token:ro
85+
- ./tests/e2e/secrets/invalid-mcp-token:/tmp/invalid-mcp-secret-token:ro
8586
environment:
8687
- OPENAI_API_KEY=${OPENAI_API_KEY}
8788
# Azure Entra ID credentials (AZURE_API_KEY is obtained dynamically)

‎docs/e2e_testing.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,7 @@ tests/e2e/
5858
├── utils/
5959
│ ├── utils.py # restart_container, switch_config, wait_for_container_health, etc.
6060
│ ├── prow_utils.py # Prow/OpenShift helpers (restore_llama_stack_pod, etc.)
61-
│ └── llama_stack_shields.py # Shield unregister/register (server mode, optional)
61+
│ └── llama_stack_utils.py # Toolgroups + shield unregister/register (server mode, optional)
6262
├── mock_mcp_server/ # Mock MCP server for MCP tests
6363
└── rag/ # RAG test data (e.g. for FAISS)
6464
```

‎tests/e2e/configuration/library-mode/lightspeed-stack-invalid-mcp-file-auth.yaml‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
name: Lightspeed Core Service (LCS)
2+
service:
3+
host: 0.0.0.0
4+
port: 8080
5+
auth_enabled: false
6+
workers: 1
7+
color_log: true
8+
access_log: true
9+
llama_stack:
10+
# Library mode - embeds llama-stack as library
11+
use_as_library_client: true
12+
library_client_config_path: run.yaml
13+
user_data_collection:
14+
feedback_enabled: true
15+
feedback_storage: "/tmp/data/feedback"
16+
transcripts_enabled: true
17+
transcripts_storage: "/tmp/data/transcripts"
18+
authentication:
19+
module: "noop"
20+
mcp_servers:
21+
- name: "mcp-file"
22+
url: "http://mock-mcp:3001"
23+
authorization_headers:
24+
Authorization: "/tmp/invalid-mcp-secret-token"

‎tests/e2e/configuration/library-mode/lightspeed-stack-mcp-client-auth.yaml‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
name: Lightspeed Core Service (LCS)
2+
service:
3+
host: 0.0.0.0
4+
port: 8080
5+
auth_enabled: false
6+
workers: 1
7+
color_log: true
8+
access_log: true
9+
llama_stack:
10+
# Library mode - embeds llama-stack as library
11+
use_as_library_client: true
12+
library_client_config_path: run.yaml
13+
user_data_collection:
14+
feedback_enabled: true
15+
feedback_storage: "/tmp/data/feedback"
16+
transcripts_enabled: true
17+
transcripts_storage: "/tmp/data/transcripts"
18+
authentication:
19+
module: "noop"
20+
mcp_servers:
21+
- name: "mcp-client"
22+
url: "http://mock-mcp:3001"
23+
authorization_headers:
24+
Authorization: "client"

‎tests/e2e/configuration/library-mode/lightspeed-stack-mcp-file-auth.yaml‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -18,8 +18,7 @@ user_data_collection:
1818
authentication:
1919
module: "noop"
2020
mcp_servers:
21-
- name: "mcp-file-auth"
22-
provider_id: "model-context-protocol"
21+
- name: "mcp-file"
2322
url: "http://mock-mcp:3001"
2423
authorization_headers:
2524
Authorization: "/tmp/mcp-secret-token"

‎tests/e2e/configuration/library-mode/lightspeed-stack-mcp-kubernetes-auth.yaml‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
name: Lightspeed Core Service (LCS)
2+
service:
3+
host: 0.0.0.0
4+
port: 8080
5+
auth_enabled: false
6+
workers: 1
7+
color_log: true
8+
access_log: true
9+
llama_stack:
10+
# Library mode - embeds llama-stack as library
11+
use_as_library_client: true
12+
library_client_config_path: run.yaml
13+
user_data_collection:
14+
feedback_enabled: true
15+
feedback_storage: "/tmp/data/feedback"
16+
transcripts_enabled: true
17+
transcripts_storage: "/tmp/data/transcripts"
18+
authentication:
19+
module: "noop"
20+
mcp_servers:
21+
- name: "mcp-kubernetes"
22+
url: "http://mock-mcp:3001"
23+
authorization_headers:
24+
Authorization: "kubernetes"

‎tests/e2e/configuration/library-mode/lightspeed-stack-mcp-oauth-auth.yaml‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
name: Lightspeed Core Service (LCS)
2+
service:
3+
host: 0.0.0.0
4+
port: 8080
5+
auth_enabled: false
6+
workers: 1
7+
color_log: true
8+
access_log: true
9+
llama_stack:
10+
# Library mode - embeds llama-stack as library
11+
use_as_library_client: true
12+
library_client_config_path: run.yaml
13+
user_data_collection:
14+
feedback_enabled: true
15+
feedback_storage: "/tmp/data/feedback"
16+
transcripts_enabled: true
17+
transcripts_storage: "/tmp/data/transcripts"
18+
authentication:
19+
module: "noop"
20+
mcp_servers:
21+
- name: "mcp-oauth"
22+
url: "http://mock-mcp:3001"
23+
authorization_headers:
24+
Authorization: "oauth"

‎tests/e2e/configuration/library-mode/lightspeed-stack-mcp.yaml‎

Lines changed: 13 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,18 @@ authentication:
1919
module: "noop"
2020
mcp_servers:
2121
- name: "mcp-oauth"
22-
provider_id: "model-context-protocol"
2322
url: "http://mock-mcp:3001"
2423
authorization_headers:
25-
Authorization: "oauth"
24+
Authorization: "oauth"
25+
- name: "mcp-kubernetes"
26+
url: "http://mock-mcp:3001"
27+
authorization_headers:
28+
Authorization: "kubernetes"
29+
- name: "mcp-file"
30+
url: "http://mock-mcp:3001"
31+
authorization_headers:
32+
Authorization: "/tmp/mcp-secret-token"
33+
- name: "mcp-client"
34+
url: "http://mock-mcp:3001"
35+
authorization_headers:
36+
Authorization: "client"

‎tests/e2e/configuration/server-mode/lightspeed-stack-invalid-mcp-file-auth.yaml‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
name: Lightspeed Core Service (LCS)
2+
service:
3+
host: 0.0.0.0
4+
port: 8080
5+
auth_enabled: false
6+
workers: 1
7+
color_log: true
8+
access_log: true
9+
llama_stack:
10+
# Server mode - connects to separate llama-stack service
11+
use_as_library_client: false
12+
url: http://llama-stack:8321
13+
api_key: xyzzy
14+
user_data_collection:
15+
feedback_enabled: true
16+
feedback_storage: "/tmp/data/feedback"
17+
transcripts_enabled: true
18+
transcripts_storage: "/tmp/data/transcripts"
19+
authentication:
20+
module: "noop"
21+
mcp_servers:
22+
- name: "mcp-file"
23+
url: "http://mock-mcp:3001"
24+
authorization_headers:
25+
Authorization: "/tmp/invalid-mcp-secret-token"

‎tests/e2e/configuration/server-mode/lightspeed-stack-mcp-client-auth.yaml‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
name: Lightspeed Core Service (LCS)
2+
service:
3+
host: 0.0.0.0
4+
port: 8080
5+
auth_enabled: false
6+
workers: 1
7+
color_log: true
8+
access_log: true
9+
llama_stack:
10+
# Server mode - connects to separate llama-stack service
11+
use_as_library_client: false
12+
url: http://llama-stack:8321
13+
api_key: xyzzy
14+
user_data_collection:
15+
feedback_enabled: true
16+
feedback_storage: "/tmp/data/feedback"
17+
transcripts_enabled: true
18+
transcripts_storage: "/tmp/data/transcripts"
19+
authentication:
20+
module: "noop"
21+
mcp_servers:
22+
- name: "mcp-client"
23+
url: "http://mock-mcp:3001"
24+
authorization_headers:
25+
Authorization: "client"

0 commit comments

Comments
 (0)