|
| 1 | +""" |
| 2 | +Fuzzer module for performing LLM security scans. |
| 3 | +
|
| 4 | +This module provides the core fuzzing logic for the Agentic Security scanner. |
| 5 | +It supports two scanning modes: |
| 6 | + - **Single-shot scan**: Sends individual prompts from selected datasets to |
| 7 | + probe LLM vulnerabilities (jailbreaks, prompt injection, etc.). |
| 8 | + - **Many-shot scan (MSJ)**: Injects probe prompts within multi-step |
| 9 | + conversations to test context-window attacks and many-shot jailbreaking. |
| 10 | +
|
| 11 | +The module uses Bayesian optimization (via scikit-optimize) to adaptively |
| 12 | +focus scanning effort on high-failure-rate areas and supports early stopping |
| 13 | +based on configurable budget and failure-rate thresholds. |
| 14 | +
|
| 15 | +Key components: |
| 16 | + - ``generate_prompts``: Async generator that yields prompts from lists or |
| 17 | + async sources. |
| 18 | + - ``get_modality_adapter``: Routes requests through image/audio adapters |
| 19 | + based on the LLM's modality. |
| 20 | + - ``process_prompt`` / ``process_prompt_batch``: Core prompt execution and |
| 21 | + response evaluation logic. |
| 22 | + - ``scan_module``: Scans a single prompt module with progress tracking. |
| 23 | + - ``perform_single_shot_scan`` / ``perform_many_shot_scan``: Top-level |
| 24 | + scan orchestrators. |
| 25 | + - ``scan_router``: Entry point that dispatches to the correct scan mode. |
| 26 | +""" |
| 27 | + |
1 | 28 | import asyncio |
2 | 29 | import random |
3 | 30 | import time |
|
19 | 46 | from agentic_security.probe_data import audio_generator, image_generator, msj_data |
20 | 47 | from agentic_security.probe_data.data import prepare_prompts, create_probe_dataset |
21 | 48 |
|
| 49 | +#: Maximum number of characters from a prompt to include in scan results. |
22 | 50 | MAX_PROMPT_LENGTH = settings_var("fuzzer.max_prompt_lenght", 2048) |
| 51 | +#: Multiplier applied to the user-specified budget to derive the internal token limit. |
23 | 52 | BUDGET_MULTIPLIER = settings_var("fuzzer.budget_multiplier", 100000000) |
| 53 | +#: Number of initial random points for the Bayesian optimizer before fitting a model. |
24 | 54 | INITIAL_OPTIMIZER_POINTS = settings_var("fuzzer.initial_optimizer_points", 25) |
| 55 | +#: Minimum number of failure samples required before the optimizer evaluates early stopping. |
25 | 56 | MIN_FAILURE_SAMPLES = settings_var("fuzzer.min_failure_samples", 5) |
| 57 | +#: Failure rate threshold (0–1) above which a module scan is stopped early. |
26 | 58 | FAILURE_RATE_THRESHOLD = settings_var("fuzzer.failure_rate_threshold", 0.5) |
| 59 | +#: File path for exporting failed prompt results as CSV. |
27 | 60 | FAILURES_CSV_PATH = settings_var("fuzzer.failures_csv_path", "failures.csv") |
| 61 | +#: File path for exporting the full scan log as CSV. |
28 | 62 | FULL_LOG_CSV_PATH = settings_var("fuzzer.full_log_csv_path", "full_scan_log.csv") |
| 63 | +#: Maximum number of injection attempts per prompt in many-shot mode. |
29 | 64 | MAX_INJECTION_ATTEMPTS = settings_var("fuzzer.max_injection_attempts", 20) |
30 | 65 |
|
31 | 66 |
|
|
0 commit comments