Skip to content

Commit 36828f5

Browse files
authored
Further enhancements
Key Changes: New Services & Features: * **Prompt Guard Service:** Created `services/prompt_guard_service` to offload prompt injection detection models from the orchestrator and agents. * **CI/CD Pipeline:** Added `.github/workflows/ci.yml` to automate linting (`ruff`), testing (`pytest` with coverage), and security scanning (`bandit`, `pip-audit`). Refactoring & Improvements: * **Vector DB Service:** Removed local `SentenceTransformer` fallback. The service now strictly relies on `EMBEDDING_SERVICE_URL` and uses gRPC (port 6334) for `QdrantClient` performance. * **Prompt Injection Guard:** Refactored `common/prompt_injection/guard.py` to function as a lightweight client for the new `prompt-guard-service`. * **Orchestrator:** optimized `reserve_agent_waiting_if_needed` logic and adjusted wait intervals. * **Dependencies:** Cleaned up `requirements.txt` by moving heavy ML libraries (torch, transformers) to their specific service Dockerfiles. Infrastructure & Configuration: * **Cloud Build:** Updated `cloudbuild.yaml` to include the new prompt-guard service and granular deployment logic. * **Linting:** Added `ruff.toml` for consistent code styling. BREAKING CHANGES: * `PROMPT_GUARD_SERVICE_URL` env var is now required if prompt injection checks are enabled. * `EMBEDDING_SERVICE_URL` env var is now mandatory; local embedding fallback has been removed. * `QDRANT_GRPC_PORT` defaults to 6334.
1 parent f14af95 commit 36828f5

77 files changed

Lines changed: 1579 additions & 1046 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/workflows/ci.yml

Lines changed: 149 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,149 @@
1+
# SPDX-FileCopyrightText: 2025 Taras Paruta (partarstu@gmail.com)
2+
#
3+
# SPDX-License-Identifier: Apache-2.0
4+
5+
name: CI
6+
7+
on:
8+
push:
9+
branches:
10+
- main
11+
- develop
12+
pull_request:
13+
branches:
14+
- main
15+
- develop
16+
workflow_dispatch:
17+
18+
permissions:
19+
contents: read
20+
pull-requests: write
21+
22+
jobs:
23+
lint:
24+
name: Lint
25+
runs-on: ubuntu-latest
26+
steps:
27+
- name: Checkout repository
28+
uses: actions/checkout@v4
29+
30+
- name: Set up Python
31+
uses: actions/setup-python@v5
32+
with:
33+
python-version: "3.14"
34+
cache: "pip"
35+
36+
- name: Install ruff
37+
run: pip install ruff
38+
39+
- name: Run ruff linter
40+
run: ruff check . --output-format=github
41+
42+
test:
43+
name: Test (Python ${{ matrix.python-version }})
44+
runs-on: ubuntu-latest
45+
strategy:
46+
fail-fast: false
47+
matrix:
48+
python-version: ["3.14"]
49+
50+
steps:
51+
- name: Checkout repository
52+
uses: actions/checkout@v4
53+
54+
- name: Set up Python ${{ matrix.python-version }}
55+
uses: actions/setup-python@v5
56+
with:
57+
python-version: ${{ matrix.python-version }}
58+
cache: "pip"
59+
60+
- name: Install system dependencies
61+
run: |
62+
sudo apt-get update
63+
sudo apt-get install -y libmagic1
64+
65+
- name: Install dependencies
66+
run: |
67+
python -m pip install --upgrade pip
68+
pip install -r requirements.txt
69+
pip install ruff
70+
71+
- name: Run tests with coverage
72+
run: |
73+
pytest tests/ \
74+
--cov=. \
75+
--cov-report=xml \
76+
--cov-report=term-missing \
77+
--junitxml=test-results.xml \
78+
-v
79+
80+
- name: Upload coverage report
81+
uses: codecov/codecov-action@v4
82+
if: matrix.python-version == '3.14'
83+
with:
84+
files: ./coverage.xml
85+
fail_ci_if_error: false
86+
verbose: true
87+
continue-on-error: true
88+
89+
- name: Upload test results
90+
uses: actions/upload-artifact@v4
91+
if: always()
92+
with:
93+
name: test-results-${{ matrix.python-version }}
94+
path: |
95+
test-results.xml
96+
coverage.xml
97+
retention-days: 30
98+
99+
security:
100+
name: Security Scan
101+
runs-on: ubuntu-latest
102+
steps:
103+
- name: Checkout repository
104+
uses: actions/checkout@v4
105+
106+
- name: Set up Python
107+
uses: actions/setup-python@v5
108+
with:
109+
python-version: "3.14"
110+
cache: "pip"
111+
112+
- name: Install bandit
113+
run: pip install bandit
114+
115+
- name: Run bandit security scan
116+
run: bandit -r . -x ./tests,./orchestrator/ui,./.venv -f json -o bandit-report.json || true
117+
118+
- name: Upload bandit report
119+
uses: actions/upload-artifact@v4
120+
if: always()
121+
with:
122+
name: bandit-security-report
123+
path: bandit-report.json
124+
retention-days: 30
125+
126+
dependency-check:
127+
name: Dependency Vulnerability Check
128+
runs-on: ubuntu-latest
129+
steps:
130+
- name: Checkout repository
131+
uses: actions/checkout@v4
132+
133+
- name: Set up Python
134+
uses: actions/setup-python@v5
135+
with:
136+
python-version: "3.14"
137+
cache: "pip"
138+
139+
- name: Install pip-audit
140+
run: pip install pip-audit
141+
142+
- name: Install dependencies
143+
run: |
144+
python -m pip install --upgrade pip
145+
pip install -r requirements.txt
146+
147+
- name: Run pip-audit
148+
run: pip-audit --strict --desc || true
149+
continue-on-error: true

Dockerfile.base

Lines changed: 1 addition & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -16,12 +16,5 @@ RUN pip install --no-cache-dir -r requirements.txt
1616
# Create local_models directory for pre-downloaded models
1717
RUN mkdir -p /app/local_models
1818

19-
# Copy and run scripts to pre-download the models
20-
COPY config.py /app/config.py
21-
COPY scripts/download_prompt_guard_model.py /app/scripts/download_prompt_guard_model.py
22-
23-
# Download prompt injection detection model (if enabled)
24-
RUN python /app/scripts/download_prompt_guard_model.py
25-
2619
# Keep the image clean
27-
RUN rm -rf /var/lib/apt/lists/*
20+
RUN rm -rf /var/lib/apt/lists/*

GEMINI.md

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -16,8 +16,13 @@ expertise in working with agentic systems.
1616
solution.
1717
* Every time you work with OS-specific commands, check the OS version and type in order to know which commands are
1818
correct.
19-
* Write code that is clear, readable, and follows the principles of [PEP 8](https://peps.python.org/pep-0008/).
20-
Prioritize clarity over cleverness; avoid overly complex one-liners or list comprehensions.
19+
* Never reformat the code which you haven't modified!
20+
* Before implementing anything, always let the user know what you plan to do and ask the user to confirm it.
21+
* Never duplicate existing functionality. If you've noticed any existing logic or functionality which you need for your implementation,
22+
always reuse it. If reusing it directly can't be done, always extract it so that it's accessible (inheritance or composition) and then
23+
reuse it.
24+
* Never commit changes you've made into git unless explicitly asked by the user.
25+
* Write code that is clear and readable. Prioritize clarity over cleverness; avoid overly complex one-liners or list comprehensions.
2126
* Strictly adhere to PEP 8 naming conventions: `snake_case` for functions, methods, variables, and modules; `PascalCase`
2227
for classes; and `SCREAMING_SNAKE_CASE` for constants.
2328
* Use type hints for all function signatures (arguments and return values) to improve code clarity, enable static
@@ -36,9 +41,7 @@ expertise in working with agentic systems.
3641
code.
3742
* Avoid bare `except:` blocks. Always catch specific exceptions. Never let exceptions pass silently; at a minimum, log
3843
the exception to ensure errors are not ignored.
39-
* Write docstrings for all public modules, classes, and functions, following
40-
the [PEP 257](https://peps.python.org/pep-0257/) conventions. Use comments to explain the *why*, not the *what*, of
41-
non-obvious code.
44+
* Write docstrings for all public modules, classes, and functions, following the PEP 257 conventions. Use comments to explain the *why*, not the *what*, of non-obvious code.
4245
* Use `asyncio` for high-level, I/O-bound tasks, such as network requests or database interactions, to achieve high
4346
concurrency with a single thread.
4447
* Use `threading` for I/O-bound tasks where `asyncio` is not suitable or when integrating with blocking libraries.

README.md

Lines changed: 17 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@ Watch a demo of QuAIA™ in action:
2424
* UI & API Test Execution (separate project)
2525
* Incident Report Creation
2626
* Jira Ticket RAG
27+
* **Dedicated Prompt Guard Service:** A dedicated microservice for detecting prompt injection attacks using the ProtectAI model.
2728
* **Web UI Monitoring Dashboard:** Real-time monitoring interface for:
2829
* Agent status visualization (AVAILABLE, BUSY, BROKEN states)
2930
* Task history with execution details and duration
@@ -101,7 +102,7 @@ For a visual representation of the system's architecture and data flow, please r
101102

102103
### Prerequisites
103104

104-
* Python 3.13+
105+
* Python 3.14+
105106
* Docker
106107
* `pip` (Python package installer)
107108
* `virtualenv` (or `conda` for environment management)
@@ -200,13 +201,14 @@ TEMPERATURE=0.0 # Default: 0.0. Temperature parameter for models.
200201
# Qdrant Vector Database (for RAG and semantic search)
201202
QDRANT_URL=http://localhost # Default: http://localhost. URL of the Qdrant server.
202203
QDRANT_PORT=6333 # Default: 6333. Port of the Qdrant server.
204+
QDRANT_GRPC_PORT=6334 # Default: 6334. gRPC Port of the Qdrant server.
203205
QDRANT_API_KEY= # Optional. API key for Qdrant authentication.
204206
QDRANT_COLLECTION_NAME=jira_issues # Default: jira_issues. Name of the main collection for Jira issues.
205207
QDRANT_METADATA_COLLECTION_NAME=rag_metadata # Default: rag_metadata. Name of the collection for RAG metadata.
206208
RAG_MIN_SIMILARITY_SCORE=0.7 # Default: 0.7. Minimum similarity score for vector search results.
207209
RAG_MAX_RESULTS=5 # Default: 5. Maximum number of results to return from vector search.
208210
RAG_EMBEDDING_MODEL=Qwen/Qwen3-Embedding-0.6B # Default: Qwen/Qwen3-Embedding-0.6B. SentenceTransformer model for embeddings.
209-
EMBEDDING_SERVICE_URL= # Optional. URL of the embedding service for remote embedding generation.
211+
EMBEDDING_SERVICE_URL= # Required for agents using Vector DB. URL of the embedding service for remote embedding generation.
210212
EMBEDDING_SERVICE_TIMEOUT_SECONDS=60.0 # Default: 60.0. Timeout for embedding service requests.
211213
212214
# Incident Creation Agent Configuration
@@ -217,6 +219,7 @@ ISSUE_SEVERITY_FIELD_NAME=customfield_10124 # Default: customfield_10124. Jira c
217219
# Prompt Injection Detection
218220
PROMPT_INJECTION_CHECK_ENABLED=False # Default: False. Set to "True" to enable prompt injection detection.
219221
PROMPT_GUARD_PROVIDER=protect_ai # Default: protect_ai. The provider for prompt injection detection.
222+
PROMPT_GUARD_SERVICE_URL= # Required if PROMPT_INJECTION_CHECK_ENABLED is True. URL of the prompt guard service.
220223
PROMPT_INJECTION_MIN_SCORE=0.8 # Default: 0.8. The minimum score for a prompt to be considered an injection.
221224
PROMPT_INJECTION_MODEL_NAME=ProtectAI/deberta-v3-base-prompt-injection-v2 # Default: ProtectAI/deberta-v3-base-prompt-injection-v2. The name of the model used for prompt injection detection.
222225
@@ -291,7 +294,13 @@ To run the Jira MCP server, you will need Docker installed.
291294
python services/embedding_service/main.py
292295
```
293296

294-
3. **Start Individual Agents:**
297+
3. **Start the Prompt Guard Service (optional):**
298+
Required if prompt injection checks are enabled.
299+
```bash
300+
python services/prompt_guard_service/main.py
301+
```
302+
303+
4. **Start Individual Agents:**
295304
Open separate terminal windows for each agent you want to run:
296305

297306
* **Requirements Review Agent:**
@@ -319,7 +328,7 @@ To run the Jira MCP server, you will need Docker installed.
319328
python agents/jira_rag/main.py
320329
```
321330

322-
4. **Start the Orchestrator:**
331+
5. **Start the Orchestrator:**
323332
```bash
324333
python orchestrator/main.py
325334
```
@@ -418,11 +427,11 @@ you run any of the commands below.
418427
After having all preconditions fulfilled, you can execute the following command:
419428

420429
```bash
421-
gcloud builds submit --config 'path/to/your/cloudbuild.yaml' --substitutions "^;^_BUCKET_NAME=YOUR_GCS_BUCKET_NAME;_ALLURE_REPORTS_BUCKET=YOUR_ALLURE_REPORTS_BUCKET_NAME;_REQUIREMENTS_REVIEW_AGENT_BASE_URL=YOUR_REQUIREMENTS_REVIEW_AGENT_URL;_TEST_CASE_GENERATION_AGENT_BASE_URL=YOUR_TEST_CASE_GENERATION_AGENT_URL;_TEST_CASE_CLASSIFICATION_AGENT_BASE_URL=YOUR_TEST_CASE_CLASSIFICATION_AGENT_URL;_TEST_CASE_REVIEW_AGENT_BASE_URL=YOUR_TEST_CASE_REVIEW_AGENT_URL;_REMOTE_EXECUTION_AGENT_HOSTS=YOUR_COMMA_SEPARATED_AGENT_HOSTS" .
430+
gcloud builds submit --config 'path/to/your/cloudbuild.yaml' --substitutions "^;^_BUCKET_NAME=YOUR_GCS_BUCKET_NAME;_ALLURE_REPORTS_BUCKET=YOUR_ALLURE_REPORTS_BUCKET_NAME;_REQUIREMENTS_REVIEW_AGENT_BASE_URL=YOUR_REQUIREMENTS_REVIEW_AGENT_URL;_TEST_CASE_GENERATION_AGENT_BASE_URL=YOUR_TEST_CASE_GENERATION_AGENT_URL;_TEST_CASE_CLASSIFICATION_AGENT_BASE_URL=YOUR_TEST_CASE_CLASSIFICATION_AGENT_URL;_TEST_CASE_REVIEW_AGENT_BASE_URL=YOUR_TEST_CASE_REVIEW_AGENT_URL;_INCIDENT_CREATION_AGENT_BASE_URL=YOUR_INCIDENT_CREATION_AGENT_URL;_JIRA_RAG_UPDATE_AGENT_BASE_URL=YOUR_JIRA_RAG_UPDATE_AGENT_URL;_REMOTE_EXECUTION_AGENT_HOSTS=YOUR_COMMA_SEPARATED_AGENT_HOSTS;_PROMPT_GUARD_SERVICE_URL=YOUR_PROMPT_GUARD_SERVICE_URL;_DEPLOY_ALL_SERVICES=true" .
422431
```
423432

424433
```powershell
425-
gcloud builds submit --config 'path/to/your/cloudbuild.yaml' --substitutions "`^;`^_BUCKET_NAME=YOUR_GCS_BUCKET_NAME;_ALLURE_REPORTS_BUCKET=YOUR_ALLURE_REPORTS_BUCKET_NAME;_REQUIREMENTS_REVIEW_AGENT_BASE_URL=YOUR_REQUIREMENTS_REVIEW_AGENT_URL;_TEST_CASE_GENERATION_AGENT_BASE_URL=YOUR_TEST_CASE_GENERATION_AGENT_URL;_TEST_CASE_CLASSIFICATION_AGENT_BASE_URL=YOUR_TEST_CASE_CLASSIFICATION_AGENT_URL;_TEST_CASE_REVIEW_AGENT_BASE_URL=YOUR_TEST_CASE_REVIEW_AGENT_URL;_REMOTE_EXECUTION_AGENT_HOSTS=YOUR_COMMA_SEPARATED_AGENT_HOSTS" .
434+
gcloud builds submit --config 'path/to/your/cloudbuild.yaml' --substitutions "`^;`^_BUCKET_NAME=YOUR_GCS_BUCKET_NAME;_ALLURE_REPORTS_BUCKET=YOUR_ALLURE_REPORTS_BUCKET_NAME;_REQUIREMENTS_REVIEW_AGENT_BASE_URL=YOUR_REQUIREMENTS_REVIEW_AGENT_URL;_TEST_CASE_GENERATION_AGENT_BASE_URL=YOUR_TEST_CASE_GENERATION_AGENT_URL;_TEST_CASE_CLASSIFICATION_AGENT_BASE_URL=YOUR_TEST_CASE_CLASSIFICATION_AGENT_URL;_TEST_CASE_REVIEW_AGENT_BASE_URL=YOUR_TEST_CASE_REVIEW_AGENT_URL;_INCIDENT_CREATION_AGENT_BASE_URL=YOUR_INCIDENT_CREATION_AGENT_URL;_JIRA_RAG_UPDATE_AGENT_BASE_URL=YOUR_JIRA_RAG_UPDATE_AGENT_URL;_REMOTE_EXECUTION_AGENT_HOSTS=YOUR_COMMA_SEPARATED_AGENT_HOSTS;_PROMPT_GUARD_SERVICE_URL=YOUR_PROMPT_GUARD_SERVICE_URL;_DEPLOY_ALL_SERVICES=true" .
426435
```
427436

428437
**Substitution Variables:**
@@ -439,6 +448,8 @@ gcloud builds submit --config 'path/to/your/cloudbuild.yaml' --substitutions "`^
439448
* `_JIRA_RAG_UPDATE_AGENT_BASE_URL`: The URL of the deployed Jira RAG Update Agent.
440449
* `_REMOTE_EXECUTION_AGENT_HOSTS`: A comma-separated list of URLs for all deployed agents that the orchestrator will
441450
interact with.
451+
* `_PROMPT_GUARD_SERVICE_URL`: The URL of the deployed Prompt Guard Service.
452+
* `_DEPLOY_ALL_SERVICES`: Set to `true` to deploy all services. Individual service flags (e.g., `_DEPLOY_JIRA_MCP`) are available for granular deployment.
442453

443454
**Important**: Before the initial deployment of the framework into Google Cloud Run it's quite hard to know which URL
444455
will be assigned to each agent and orchestrator. That's why most probably you'll have to run the deployment command

agents/incident_creation/main.py

Lines changed: 7 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -6,22 +6,21 @@
66
import json
77
import os
88
import uuid
9-
from typing import List
109

1110
from a2a.types import FilePart, FileWithBytes
1211
from pydantic_ai import Agent
13-
from qdrant_client import models as qdrant_models
1412
from pydantic_ai.mcp import MCPServerSSE
13+
from qdrant_client import models as qdrant_models
1514

1615
import config
17-
from agents.incident_creation.prompt import IncidentCreationPrompt, DuplicateDetectionPrompt
16+
from agents.incident_creation.prompt import DuplicateDetectionPrompt, IncidentCreationPrompt
1817
from common import utils
1918
from common.agent_base import AgentBase
2019
from common.custom_llm_wrapper import CustomLlmWrapper
2120
from common.models import (
21+
DuplicateDetectionResult,
2222
IncidentCreationInput,
2323
IncidentCreationResult,
24-
DuplicateDetectionResult,
2524
JiraIssue,
2625
)
2726
from common.services.test_management_system_client_provider import get_test_management_client
@@ -130,7 +129,7 @@ async def _search_duplicate_candidates_in_rag(self, incident_description: str) -
130129
return candidates
131130

132131
@staticmethod
133-
async def _get_linked_issues(test_case_key: str) -> List[str]:
132+
async def _get_linked_issues(test_case_key: str) -> list[str]:
134133
"""Fetches all Jira issues linked to the test case.
135134
136135
Args:
@@ -150,7 +149,7 @@ async def _get_linked_issues(test_case_key: str) -> List[str]:
150149

151150
def _save_artifacts(self) -> list[str]:
152151
"""Saves all received file artifacts into the file system and returns their paths.
153-
152+
154153
Files are saved to the local/host path (ATTACHMENTS_LOCAL_DESTINATION_FOLDER_PATH) but
155154
the returned paths use the MCP server's container path (MCP_SERVER_ATTACHMENTS_FOLDER_PATH)
156155
since the Jira MCP server runs in Docker and expects paths relative to its filesystem.
@@ -185,7 +184,7 @@ def _save_artifacts(self) -> list[str]:
185184
saved_paths.append(mcp_file_path)
186185
logger.info(f"Saved artifact '{original_name}' to {local_file_path} (MCP path: {mcp_file_path})")
187186
except Exception:
188-
logger.exception(f"Failed to save artifact.")
187+
logger.exception("Failed to save artifact.")
189188

190189
if saved_paths:
191190
logger.info(f"Saved {len(saved_paths)} artifacts so that they could be used by MCP server.")
@@ -211,7 +210,7 @@ async def _check_if_duplicate(self, input_data: IncidentCreationInput, candidate
211210
@staticmethod
212211
async def _link_issue_to_test_case(test_case_key: str, issue_id: int, link_type: str) -> str:
213212
"""Links a bug issue to the test case using the test management system.
214-
213+
215214
Args:
216215
test_case_key: The key of the test case (e.g., 'PROJ-T123').
217216
issue_id: The numeric ID of the created bug issue (not the key, but the ID).

agents/incident_creation/prompt.py

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,9 +3,10 @@
33
# SPDX-License-Identifier: Apache-2.0
44

55
from pathlib import Path
6-
from common.prompt_base import PromptBase
7-
from common import utils
6+
87
import config
8+
from common import utils
9+
from common.prompt_base import PromptBase
910

1011
logger = utils.get_logger("incident_creation_agent")
1112

0 commit comments

Comments
 (0)