Skip to content

Commit c8418c9

Browse files
rlundeen2Copilot
andauthored
DOC: Architecture Responsibilities (microsoft#2089)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent 9c7e19c commit c8418c9

10 files changed

Lines changed: 332 additions & 71 deletions

.github/copilot-instructions.md

Lines changed: 4 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -4,19 +4,16 @@ PyRIT (Python Risk Identification Tool for generative AI) is an open-source fram
44

55
## Architecture
66

7-
PyRIT uses a modular pluggable-brick design. The main extensibility points are:
7+
PyRIT uses a modular pluggable-brick design.
88

9-
- **Prompt Converters** (`pyrit/prompt_converter/`) — Transform prompts (70+ implementations). Base: `PromptConverter`.
10-
- **Scorers** (`pyrit/score/`) — Evaluate responses. Base: `Scorer`.
11-
- **Prompt Targets** (`pyrit/prompt_target/`) — Send prompts to LLMs/APIs. Base: `PromptTarget`.
12-
- **Executors / Scenarios** (`pyrit/executor/`, `pyrit/scenario/`) — Orchestrate multi-turn attacks.
13-
- **Memory** (`pyrit/memory/`) — `CentralMemory` for prompt/response persistence.
9+
**[`doc/code/framework.md`](../doc/code/framework.md) is the canonical reference for how these pieces fit together.** It defines each component's responsibilities — what it owns and, critically, what it *does not* own — and how scenarios, attack techniques, executors, and the core/shared layers relate. Read it before adding or reviewing components so new code lands in the right place.
1410

1511
## Code Review Guidelines
1612

1713
When performing a code review, be selective. Only leave comments for issues that genuinely matter:
1814

19-
- Bugs, logic errors, or security concerns
15+
- Bugs, correctness, logic errors, or security concerns
16+
- **Component responsibilities** — Each component should do its job and *only* its job, per [`doc/code/framework.md`](../doc/code/framework.md). Flag responsibility bleed: e.g. an executor assembling prepended/system prompts or role-play framing (that's an attack technique), a converter or target making branching decisions (that's an attack/scorer), a scorer acting on its own result (the attack branches), or business logic living in memory/output. If logic belongs in a different brick, say so.
2017
- Unclear code that would benefit from refactoring for readability
2118
- Violations of the critical coding conventions above (async suffix, keyword-only args, type annotations)
2219

.github/instructions/attacks.instructions.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,8 @@ applyTo: "pyrit/executor/attack/**"
66

77
`AttackStrategy` subclasses (single-turn attacks like `PromptSendingAttack`, multi-turn attacks like `RedTeamingAttack`, etc.) are pluggable bricks orchestrated by `AttackExecutor` and the `Scenario` framework. Style rules from `style-guide.instructions.md` (async `_async` suffix, keyword-only args, type hints, enums-over-Literals) still apply and are not repeated here.
88

9+
**Does not own** (see [framework.md](../../doc/code/framework.md)): packaging the attack. Prepended/system prompts, role-play framing, the converter stack, and dataset selection are passed in as configuration by the **attack technique** — an attack must accept them as parameters, not assemble them itself (e.g. `RolePlayAttack` building its own prompt scaffolding is attack-technique work bleeding into the executor). It also must not branch on raw responses (use a scorer), construct its own components (use the registry), or format/persist results itself (output/memory). Flag such bleed in review.
10+
911
## Constructor contract
1012

1113
`AttackStrategy` subclasses MUST follow the keyword-only constructor shape:

.github/instructions/converters.instructions.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,10 @@ applyTo: "pyrit/prompt_converter/**"
44

55
# Prompt Converter Development Guidelines
66

7+
**Responsibility**: A converter transforms a prompt into something else (rephrasing, encoding, translating to a Word document, overlaying text on an image, ...). Converters can be stacked and combined, and any converter may also be a NoOp.
8+
9+
**Does not own** (see [framework.md](../../doc/code/framework.md)): conversation state or attack decisions. A converter transforms input into output (and may call a target to do so); it must not branch on results, score, persist to memory itself, or decide when it runs — the attack/technique configures the stack. Flag such bleed in review.
10+
711
## Base Class Contract
812

913
All converters MUST inherit from `PromptConverter` and implement:

.github/instructions/datasets.instructions.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,10 @@ applyTo: "pyrit/datasets/seed_datasets/**"
44

55
# Seed Dataset Loader Guidelines
66

7+
**Responsibility**: Seed dataset loaders (`SeedDatasetProvider` subclasses) are the single place to manage the prompts/objectives for a source. They load seeds into `CentralMemory`; components then retrieve seeds from memory — components never read from a loader directly.
8+
9+
**Does not own** (see [framework.md](../../doc/code/framework.md)): a loader defines and holds seeds; it must not select or combine which seeds an attack uses (that's a scenario/attack technique) or render/parameterize prompts at send time (converters/normalizers). Flag such bleed in review.
10+
711
These rules apply when adding or modifying loaders under `pyrit/datasets/seed_datasets/`.
812
Style rules from `style-guide.instructions.md` (async `_async` suffix, keyword-only args, type hints, enums-over-Literals) still apply and are not repeated here.
913

.github/instructions/models.instructions.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,8 @@ applyTo: "pyrit/models/**"
44

55
# `pyrit.models` Guidelines
66

7+
**Responsibility**: `pyrit.models` is the lightweight, canonical data layer — the core types shared across components (and preferred in REST) so representations don't drift. It depends only on lightweight Python (the standard library and pydantic) and `pyrit.common`.
8+
79
## Import Boundary
810

911
PyRIT enforces a two-layer rule for its foundational packages. `pyrit.common`

.github/instructions/output.instructions.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,8 @@ For full architecture documentation, usage examples, and extension guides, see [
88

99
This file covers the rules for **writing and reviewing** code in `pyrit/output/`.
1010

11+
**Does not own** (see [framework.md](../../doc/code/framework.md)): deciding *what* to render or *when*. Components hand results to output; format classes only turn data into strings and must never fetch data, touch `CentralMemory`, or call `print()` directly (that's isolated to leaf printer classes). Flag such bleed in review.
12+
1113
## Critical Rules
1214

1315
### Output goes through the sink — never call `print()` directly

.github/instructions/scenarios.instructions.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,8 @@ applyTo: "pyrit/scenario/**"
66

77
Scenarios orchestrate multi-attack security testing campaigns. Each scenario groups `AtomicAttack` instances and executes them sequentially against a target.
88

9+
**Does not own** (see [framework.md](../../doc/code/framework.md)): the per-objective conversation logic. Branching, turn-by-turn adaptation, and scoring-based decisions belong to the attack — a scenario selects and packages existing attack techniques and owns parallelism/resiliency, not new attack algorithms or datasets. Flag such bleed in review.
10+
911
## Base Class Contract
1012

1113
All scenarios inherit from `Scenario` (ABC) and must:

.github/instructions/scorers.instructions.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,8 @@ applyTo: "pyrit/score/**"
66

77
Scorers evaluate model responses against an objective and live under `pyrit/score/`. Style rules from `style-guide.instructions.md` (async `_async` suffix, keyword-only args, type hints, enums-over-Literals) still apply and are not repeated here.
88

9+
**Does not own** (see [framework.md](../../doc/code/framework.md)): acting on its own result. A scorer evaluates a response and returns a score; branching on that score is the attack's job and aggregating scores across runs is analytics'. It may call a target to evaluate, but must not send the attack's objective prompt or manage the conversation. Flag such bleed in review.
10+
911
## Constructor contract
1012

1113
`Scorer` subclasses MUST use the keyword-only constructor shape:

.github/instructions/targets.instructions.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,10 @@ applyTo: "pyrit/prompt_target/**"
44

55
# Prompt Target Development Guidelines
66

7+
**Responsibility**: A prompt target is "the thing we're sending the prompt to" — often an LLM, but it can be any endpoint (e.g. a storage account for cross-domain prompt injection). Targets use `message_normalizer` together with `TargetConfiguration` to transform `Message`s into the format the target supports.
8+
9+
**Does not own** (see [framework.md](../../doc/code/framework.md)): what to send or what to do with the response. A target sends a prepared `Message` and returns a response; it must not convert prompts (converters), score (scorers), or manage the conversation / decide the next turn (attacks). Flag such bleed in review.
10+
711
## Base Class Contract
812

913
All targets MUST inherit from ``PromptTarget`` (or one of its public

0 commit comments

Comments
 (0)