You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: doc/code/datasets/5_simulated_conversation.ipynb
+1-1Lines changed: 1 addition & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -544,7 +544,7 @@
544
544
"| Parameter | Type | Description |\n",
545
545
"|-----------|------|-------------|\n",
546
546
"| `objective` | `str` | The goal the adversarial chat works toward |\n",
547
-
"| `adversarial_chat` | `PromptChatTarget` | The LLM that generates attack prompts (also plays the simulated target) |\n",
547
+
"| `adversarial_chat` | `PromptTarget` | The LLM that generates attack prompts (also plays the simulated target). Must declare `supports_multi_turn=True` and `supports_editable_history=True`. |\n",
548
548
"| `objective_scorer` | `TrueFalseScorer` | Evaluates whether the final turn achieved the objective |\n",
549
549
"| `num_turns` | `int` | Number of conversation turns to generate (default: 3) |\n",
550
550
"| `adversarial_chat_system_prompt_path` | `str \\| Path` | System prompt for the adversarial chat role |\n",
Copy file name to clipboardExpand all lines: doc/code/datasets/5_simulated_conversation.py
+1-1Lines changed: 1 addition & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -125,7 +125,7 @@
125
125
# | Parameter | Type | Description |
126
126
# |-----------|------|-------------|
127
127
# | `objective` | `str` | The goal the adversarial chat works toward |
128
-
# | `adversarial_chat` | `PromptChatTarget` | The LLM that generates attack prompts (also plays the simulated target) |
128
+
# | `adversarial_chat` | `PromptTarget` | The LLM that generates attack prompts (also plays the simulated target). Must declare `supports_multi_turn=True` and `supports_editable_history=True`. |
129
129
# | `objective_scorer` | `TrueFalseScorer` | Evaluates whether the final turn achieved the objective |
130
130
# | `num_turns` | `int` | Number of conversation turns to generate (default: 3) |
131
131
# | `adversarial_chat_system_prompt_path` | `str \| Path` | System prompt for the adversarial chat role |
Copy file name to clipboardExpand all lines: doc/code/executor/attack/2_red_teaming_attack.ipynb
+1-1Lines changed: 1 addition & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -1456,7 +1456,7 @@
1456
1456
"source": [
1457
1457
"## Other Multi-Turn Attacks\n",
1458
1458
"\n",
1459
-
"The above examples should work using other multi-turn attacks with minimal modification. Check out attacks under `pyrit.executor.attack.multi_turn` for other examples, like Crescendo and Tree of Attacks. These algorithms are always more effective than `RedTeamingAttack`, which is a simple algorithm. However, `RedTeamingAttack` by its nature supports more targets - because it doesn't modify conversation history it can support any `PromptTarget` and not only `PromptChatTargets`."
1459
+
" \"The above examples should work using other multi-turn attacks with minimal modification. Check out attacks under `pyrit.executor.attack.multi_turn` for other examples, like Crescendo and Tree of Attacks. These algorithms are always more effective than `RedTeamingAttack`, which is a simple algorithm. However, `RedTeamingAttack` by its nature supports more targets - because it doesn't modify conversation history it can support any `PromptTarget`, not only chat-style targets that declare `supports_multi_turn=True` and `supports_editable_history=True`.\""
Copy file name to clipboardExpand all lines: doc/code/executor/attack/2_red_teaming_attack.py
+1-1Lines changed: 1 addition & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -314,4 +314,4 @@
314
314
# %% [markdown]
315
315
# ## Other Multi-Turn Attacks
316
316
#
317
-
# The above examples should work using other multi-turn attacks with minimal modification. Check out attacks under `pyrit.executor.attack.multi_turn` for other examples, like Crescendo and Tree of Attacks. These algorithms are always more effective than `RedTeamingAttack`, which is a simple algorithm. However, `RedTeamingAttack` by its nature supports more targets - because it doesn't modify conversation history it can support any `PromptTarget` and not only `PromptChatTargets`.
317
+
# The above examples should work using other multi-turn attacks with minimal modification. Check out attacks under `pyrit.executor.attack.multi_turn` for other examples, like Crescendo and Tree of Attacks. These algorithms are always more effective than `RedTeamingAttack`, which is a simple algorithm. However, `RedTeamingAttack` by its nature supports more targets - because it doesn't modify conversation history it can support any `PromptTarget`, not only chat-style targets that declare `supports_multi_turn=True` and `supports_editable_history=True`.
A `Message` object is a normalized object with all the information a target will need to send a prompt, including a way to get a history for that prompt (in the cases that also needs to be sent). This is discussed in more depth [here](../memory/3_memory_data_types.md).
21
21
22
-
## PromptChatTargets vs PromptTargets
22
+
## Chat-style targets vs general targets
23
23
24
24
A `PromptTarget` is a generic place to send a prompt. With PyRIT, the idea is that it will eventually be consumed by an AI application, but that doesn't have to be immediate. For example, you could have a SharePoint target. Everything you send a prompt to is a `PromptTarget`. Many attacks work generically with any `PromptTarget` including `RedTeamingAttack` and `PromptSendingAttack`.
25
25
26
-
With some algorithms, you want to send a prompt, set a system prompt, and modify conversation history (including PAIR [@chao2023pair], TAP [@mehrotra2023tap], and flip attack [@li2024flipattack]). These often require a `PromptChatTarget`, which implies you can modify a conversation history. `PromptChatTarget` is a subclass of `PromptTarget`.
26
+
With some algorithms, you want to send a prompt, set a system prompt, and modify conversation history (including PAIR [@chao2023pair], TAP [@mehrotra2023tap], and flip attack [@li2024flipattack]). These algorithms require a target whose [`TargetCapabilities`](#target-capabilities) declare both `supports_multi_turn=True` and `supports_editable_history=True` — i.e. you can modify a conversation history. Consumers express this requirement via `CHAT_TARGET_REQUIREMENTS` and validate it against `target.configuration` at construction time. See [Target Capabilities](#target-capabilities) below for the full list of capabilities and how they compose into a `TargetConfiguration`.
27
+
28
+
Note: The previous `PromptChatTarget` class is **deprecated** as of v0.13.0 and will be removed in v0.15.0. Use `PromptTarget` directly with a `TargetConfiguration` declaring `supports_multi_turn=True` and `supports_editable_history=True`. See [Target Capabilities](#target-capabilities) for details.
|**OpenAIChatTarget** (e.g., GPT-4) |**Yes** (`PromptChatTarget`) | Designed for conversational prompts (system messages, conversation history, etc.). |
33
-
|**OpenAIImageTarget**|**No** (not a `PromptChatTarget`) | Used for image generation; does not manage conversation history. |
34
-
|**HTTPTarget**|**No** (not a `PromptChatTarget`) | Generic HTTP target. Some apps might allow conversation history, but this target doesn't handle it. |
35
-
|**AzureBlobStorageTarget**|**No** (not a `PromptChatTarget`) | Used primarily for storage; not for conversation-based AI. |
|**OpenAIChatTarget** (e.g., GPT-4) |**Yes** (multi-turn + editable history) | Designed for conversational prompts (system messages, conversation history, etc.). |
36
+
|**OpenAIImageTarget**|**No**| Used for image generation; does not manage conversation history. |
37
+
|**HTTPTarget**|**No**| Generic HTTP target. Some apps might allow conversation history, but this target doesn't handle it. |
38
+
|**AzureBlobStorageTarget**|**No**| Used primarily for storage; not for conversation-based AI. |
39
+
40
+
## Target Capabilities
41
+
42
+
Every `PromptTarget` exposes a `TargetConfiguration` (via `target.configuration`) that declares what the target natively supports. This lets attacks, converters, and scorers reason about whether a given target is suitable for a given workflow — and, where possible, adapt automatically when a capability is missing.
43
+
44
+
A `TargetConfiguration` composes three concerns:
45
+
46
+
-**`TargetCapabilities`** — an immutable, declarative description of what the target natively supports.
47
+
-**`CapabilityHandlingPolicy`** — for each capability that *can* be adapted, whether to `ADAPT` (apply a normalization step to work around the gap) or `RAISE` (fail immediately).
48
+
-**`ConversationNormalizationPipeline`** — the ordered set of normalizers derived from the gap between the declared capabilities and the policy.
49
+
50
+
### Capabilities
51
+
52
+
`TargetCapabilities` declares the following flags:
|`supports_multi_turn`| The target accepts and uses conversation history (or maintains state externally, e.g. via a WebSocket). |
57
+
|`supports_multi_message_pieces`| The target accepts more than one `MessagePiece` in a single request (e.g. text + image in one message). |
58
+
|`supports_editable_history`| The conversation history can be modified after the fact. Implies `supports_multi_turn`. Required for attacks that rewrite prior turns. |
59
+
|`supports_system_prompt`| The target natively supports a system-role message. |
60
+
|`supports_json_output`| The target supports a "json" response format that guarantees valid JSON output. |
61
+
|`supports_json_schema`| The target supports constraining output to a caller-provided JSON schema. |
62
+
|`input_modalities`| The set of input modality combinations the target accepts (e.g. `{text}`, `{text, image_path}`). |
63
+
|`output_modalities`| The set of output modality combinations the target produces. |
64
+
65
+
Each target class defines defaults; instances can override individual capabilities when they depend on deployment configuration (e.g. `HTTPTarget`, `PlaywrightTarget`).
66
+
67
+
For well-known underlying models, you can look up a profile with `TargetCapabilities.get_known_capabilities(underlying_model="gpt-4o")`.
68
+
69
+
### How consumers use capabilities
70
+
71
+
Components that need a particular capability declare it as a `TargetRequirements` and validate at construction time:
72
+
73
+
```python
74
+
from pyrit.prompt_target importCHAT_TARGET_REQUIREMENTS
75
+
76
+
CHAT_TARGET_REQUIREMENTS.validate(target=target)
77
+
```
78
+
79
+
`TargetRequirements.validate` collects every missing capability and raises a single `ValueError`. For one-off checks against a single capability you can also call `target.configuration.ensure_can_handle(capability=...)` directly.
80
+
81
+
### Adapting vs raising
82
+
83
+
Some capability gaps can be papered over by PyRIT itself. For example, a single-turn target can be made to *appear* multi-turn by flattening the conversation history into a single prompt before sending. The `CapabilityHandlingPolicy` controls this on a per-capability basis:
84
+
85
+
-`UnsupportedCapabilityBehavior.RAISE` — fail at construction time. This is the safe default.
86
+
-`UnsupportedCapabilityBehavior.ADAPT` — run the corresponding normalizer in the conversation pipeline before the target sees the messages.
87
+
88
+
Non-adaptable capabilities (e.g. `supports_editable_history`) are not represented in the policy at all; requesting them on a target that lacks them always raises.
89
+
90
+
### Overriding capabilities per instance
91
+
92
+
For targets whose capabilities depend on deployment (HTTP endpoints, Playwright-driven UIs, custom backends), pass a `TargetConfiguration` to the constructor:
93
+
94
+
```python
95
+
from pyrit.prompt_target.common.target_capabilities import TargetCapabilities
96
+
from pyrit.prompt_target.common.target_configuration import TargetConfiguration
The full implementation lives in [`pyrit/prompt_target/common/target_capabilities.py`](https://github.com/microsoft/PyRIT/blob/main/pyrit/prompt_target/common/target_capabilities.py) and [`pyrit/prompt_target/common/target_configuration.py`](https://github.com/microsoft/PyRIT/blob/main/pyrit/prompt_target/common/target_configuration.py). For runnable examples — inspecting capabilities on a real target, comparing known model profiles, and `ADAPT` vs `RAISE` in action — see [Target Capabilities](./6_1_target_capabilities.ipynb).
Copy file name to clipboardExpand all lines: doc/code/targets/10_3_websocket_copilot_target.ipynb
+1-1Lines changed: 1 addition & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -82,7 +82,7 @@
82
82
"\n",
83
83
"The `WebSocketCopilotTarget` supports multi-turn conversations by leveraging Copilot's server-side conversation management. It automatically generates consistent `session_id` and `conversation_id` values for each PyRIT conversation, enabling Copilot to maintain context across multiple turns.\n",
84
84
"\n",
85
-
"However, this target does not support setting a system prompt nor modifying conversation history. As a result, it cannot be used with attack strategies that require altering prior messages (such as PAIR, TAP, or flip attack) or in contexts where a `PromptChatTarget` is required.\n",
85
+
"\"However, this target does not support setting a system prompt nor modifying conversation history. As a result, it cannot be used with attack strategies that require altering prior messages (such as PAIR, TAP, or flip attack) or in contexts where a chat-style target (one that declares `supports_multi_turn=True` and `supports_editable_history=True`) is required.\\n\",\n",
86
86
"\n",
87
87
"Here is a simple multi-turn conversation example:"
Copy file name to clipboardExpand all lines: doc/code/targets/10_3_websocket_copilot_target.py
+1-1Lines changed: 1 addition & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -45,7 +45,7 @@
45
45
#
46
46
# The `WebSocketCopilotTarget` supports multi-turn conversations by leveraging Copilot's server-side conversation management. It automatically generates consistent `session_id` and `conversation_id` values for each PyRIT conversation, enabling Copilot to maintain context across multiple turns.
47
47
#
48
-
# However, this target does not support setting a system prompt nor modifying conversation history. As a result, it cannot be used with attack strategies that require altering prior messages (such as PAIR, TAP, or flip attack) or in contexts where a `PromptChatTarget` is required.
48
+
# However, this target does not support setting a system prompt nor modifying conversation history. As a result, it cannot be used with attack strategies that require altering prior messages (such as PAIR, TAP, or flip attack) or in contexts where a chat-style target (one that declares `supports_multi_turn=True` and `supports_editable_history=True`) is required.
49
49
#
50
50
# Here is a simple multi-turn conversation example:
0 commit comments