MAINT: Refactor printers into lightweight and flexible output module (#1732)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Author: rlundeen2

.github/instructions/output.instructions.md

@@ -0,0 +1,67 @@
+---
+applyTo: "pyrit/output/**"
+---
+
+# PyRIT Output Module — Coding & Review Guidelines
+
+For full architecture documentation, usage examples, and extension guides, see [doc/code/output/0_output.py](../../../doc/code/output/0_output.py).
+
+This file covers the rules for **writing and reviewing** code in `pyrit/output/`.
+
+## Critical Rules
+
+### Output goes through the sink — never call `print()` directly
+
+All rendering methods return `str`. The inherited `write_async` calls `render_async` then `_write_async(content)`. No bare `print()` calls anywhere in the output module except inside `StdoutSink`.
+
+When reviewing: reject any `print()` call outside `StdoutSink`.
+
+### Data fetching belongs in leaf classes only
+
+Format classes (`PrettyAttackResultPrinter`, `MarkdownAttackResultPrinter`) must not import or reference `CentralMemory`. Only `*MemoryPrinter` leaf classes do data I/O.
+
+When reviewing: reject any `CentralMemory` import in a non-leaf file (`pretty.py`, `markdown.py`, `json.py`).
+
+### Sinks must use async I/O
+
+Sink implementations must not block the event loop. Use `asyncio.to_thread()` or native async libraries for I/O operations. `FileSink` uses an `asyncio.Lock` to prevent concurrent write races.
+
+When reviewing: reject synchronous `open()`, `write()`, or network calls inside a sink's `write_async`.
+
+## Three-Layer Hierarchy
+
+Every domain follows this structure. Do not mix responsibilities across layers.
+
+| Layer | File | Responsibility | May import CentralMemory? |
+|-------|------|---------------|---------------------------|
+| **Base** | `base.py` | Abstract data-fetching methods + abstract `render_async` | No |
+| **Format** | `pretty.py`, `markdown.py`, `json.py` | Implements `render_async`, returns `str` | No |
+| **Leaf** | Same file as format (e.g., `PrettyAttackResultMemoryPrinter`) | Implements data methods via CentralMemory; forwarding `render_async` | Yes |
+
+### File names = output format
+
+- `pretty.py` — ANSI-colored human-readable
+- `markdown.py` — Markdown
+- `json.py` — structured JSON
+
+### Memory leaf classes must work with zero args
+
+```python
+printer = PrettyAttackResultMemoryPrinter()  # defaults: StdoutSink, matching sub-printers
+await printer.write_async(result)
+```
+
+Pass `sink=` to redirect output. Pass sub-printers only to override defaults.
+
+### Convenience functions live in `helpers.py`
+
+Every new domain printer **must** have a corresponding convenience function added to `helpers.py`. This is the primary entry point most callers use.
+
+```python
+from pyrit.output.helpers import output_attack_async
+await output_attack_async(result, format="pretty")
+```
+
+`helpers.py` resolves `format` → printer class, `sink` → Sink, and calls `write_async`.
+
+When reviewing: if a new domain printer is added without a helper function, request one.

doc/code/auxiliary_attacks/0_auxiliary_attacks.ipynb

@@ -64,7 +64,6 @@
    "source": [
     "from pyrit.executor.attack import (\n",
     "    AttackScoringConfig,\n",
-    "    ConsoleAttackResultPrinter,\n",
     "    PromptSendingAttack,\n",
     ")\n",
     "from pyrit.prompt_target import OpenAIChatTarget\n",
@@ -83,8 +82,7 @@
     "attack = PromptSendingAttack(objective_target=target, attack_scoring_config=scoring_config)\n",
     "result = await attack.execute_async(objective=objective)  # type: ignore\n",
     "\n",
-    "printer = ConsoleAttackResultPrinter()\n",
-    "await printer.print_conversation_async(result=result)  # type: ignore"
+    "await output_attack_async(result)"
    ]
   },
   {
@@ -180,7 +178,7 @@
     ")\n",
     "\n",
     "result = await attack.execute_async(objective=objective)  # type: ignore\n",
-    "await printer.print_result_async(result=result)  # type: ignore"
+    "await output_attack_async(result)"
    ]
   }
  ],

doc/code/auxiliary_attacks/0_auxiliary_attacks.py

@@ -8,29 +8,24 @@
 #       format_version: '1.3'
 #       jupytext_version: 1.17.3
 # ---
-
 # %% [markdown]
 # # Auxiliary Attacks
-
 # %% [markdown]
 # Auxiliary attacks cover a variety of techniques that do not fit into the core PyRIT functionality.
 #
 # These attack pipelines may be useful to run before orchestrating other attacks. For example, we provide an Azure Machine Learning (AML) pipeline for generating suffixes using the greedy coordinate gradient (GCG) [@zou2023gcg] algorithm.
-
 # %% [markdown]
 # ## GCG Suffixes
-
 # %% [markdown]
 # The [GCG demo notebook](1_gcg_azure_ml.ipynb) shows how to create an AML environment and submit a job that generates GCG suffixes, which can be appended to a base prompt to jailbreak a language model. In the example below, we compare the response generated by Phi-3-mini with and without a GCG suffix trained on that model.
 #
 # First, we send a harmful prompt to Phi-3-mini without a GCG suffix. If the environment variables `PHI3_MINI_ENDPOINT` and `PHI3_MINI_KEY` are not set in your .env file, the target will default to the model with `AZURE_ML_MANAGED_ENDPOINT` and `AZURE_ML_MANAGED_KEY`.
-
 # %%
 from pyrit.executor.attack import (
     AttackScoringConfig,
-    ConsoleAttackResultPrinter,
     PromptSendingAttack,
 )
+from pyrit.output import output_attack_async
 from pyrit.prompt_target import OpenAIChatTarget
 from pyrit.score import SelfAskRefusalScorer, TrueFalseInverterScorer
 from pyrit.setup import IN_MEMORY, initialize_pyrit_async
@@ -47,8 +42,7 @@
 attack = PromptSendingAttack(objective_target=target, attack_scoring_config=scoring_config)
 result = await attack.execute_async(objective=objective)  # type: ignore
 
-printer = ConsoleAttackResultPrinter()
-await printer.print_conversation_async(result=result)  # type: ignore
+await output_attack_async(result)
 
 # %% [markdown]
 # Next, let's apply a GCG suffix trained on Phi-3-mini to the base prompt using the `SuffixAppendConverter`.
@@ -73,4 +67,4 @@
 )
 
 result = await attack.execute_async(objective=objective)  # type: ignore
-await printer.print_result_async(result=result)  # type: ignore
+await output_attack_async(result)

doc/code/converters/0_converters.ipynb

@@ -279,7 +279,6 @@
    "source": [
     "from pyrit.executor.attack import (\n",
     "    AttackConverterConfig,\n",
-    "    ConsoleAttackResultPrinter,\n",
     "    PromptSendingAttack,\n",
     ")\n",
     "from pyrit.prompt_converter import StringJoinConverter, VariationConverter\n",
@@ -306,8 +305,7 @@
     "\n",
     "result = await attack.execute_async(objective=objective)  # type: ignore\n",
     "\n",
-    "printer = ConsoleAttackResultPrinter()\n",
-    "await printer.print_conversation_async(result=result)  # type: ignore"
+    "await output_attack_async(result)"
    ]
   },
   {
@@ -370,7 +368,6 @@
    "source": [
     "from pyrit.executor.attack import (\n",
     "    AttackConverterConfig,\n",
-    "    ConsoleAttackResultPrinter,\n",
     "    PromptSendingAttack,\n",
     ")\n",
     "from pyrit.prompt_converter import TranslationConverter\n",
@@ -407,8 +404,7 @@
     "result = await attack.execute_async(objective=objective)  # type: ignore\n",
     "\n",
     "# Print the conversation showing both original and converted values\n",
-    "printer = ConsoleAttackResultPrinter()\n",
-    "await printer.print_conversation_async(result=result)  # type: ignore"
+    "await output_attack_async(result)"
    ]
   }
  ],

doc/code/converters/0_converters.py

@@ -8,10 +8,8 @@
 #       format_version: '1.3'
 #       jupytext_version: 1.19.1
 # ---
-
 # %% [markdown]
 # # Converters
-
 # %% [markdown]
 # Converters are used to transform prompts before sending them to the target.
 #
@@ -24,10 +22,10 @@
 # ## Converter Modality Reference Table
 #
 # The following table shows all available converters organized by their input and output modalities:
-
 # %%
 import pandas as pd
 
+from pyrit.output import output_attack_async
 from pyrit.prompt_converter import get_converter_modalities
 from pyrit.setup import IN_MEMORY, initialize_pyrit_async
 
@@ -102,7 +100,6 @@
 # %%
 from pyrit.executor.attack import (
     AttackConverterConfig,
-    ConsoleAttackResultPrinter,
     PromptSendingAttack,
 )
 from pyrit.prompt_converter import StringJoinConverter, VariationConverter
@@ -129,8 +126,7 @@
 
 result = await attack.execute_async(objective=objective)  # type: ignore
 
-printer = ConsoleAttackResultPrinter()
-await printer.print_conversation_async(result=result)  # type: ignore
+await output_attack_async(result)
 
 # %% [markdown]
 # ## Response Converters
@@ -154,7 +150,6 @@
 # %%
 from pyrit.executor.attack import (
     AttackConverterConfig,
-    ConsoleAttackResultPrinter,
     PromptSendingAttack,
 )
 from pyrit.prompt_converter import TranslationConverter
@@ -191,5 +186,4 @@
 result = await attack.execute_async(objective=objective)  # type: ignore
 
 # Print the conversation showing both original and converted values
-printer = ConsoleAttackResultPrinter()
-await printer.print_conversation_async(result=result)  # type: ignore
+await output_attack_async(result)

doc/code/converters/5_file_converters.ipynb

@@ -90,7 +90,6 @@
     "from pyrit.common.path import CONVERTER_SEED_PROMPT_PATH\n",
     "from pyrit.executor.attack import (\n",
     "    AttackConverterConfig,\n",
-    "    ConsoleAttackResultPrinter,\n",
     "    PromptSendingAttack,\n",
     ")\n",
     "from pyrit.models import SeedPrompt\n",
@@ -147,7 +146,7 @@
     ")\n",
     "\n",
     "result = await attack.execute_async(objective=prompt)  # type: ignore\n",
-    "await ConsoleAttackResultPrinter().print_conversation_async(result=result)  # type: ignore"
+    "await output_attack_async(result)"
    ]
   },
   {
@@ -226,7 +225,7 @@
     ")\n",
     "\n",
     "result = await attack.execute_async(objective=prompt)  # type: ignore\n",
-    "await ConsoleAttackResultPrinter().print_conversation_async(result=result)  # type: ignore"
+    "await output_attack_async(result)"
    ]
   },
   {
@@ -363,7 +362,7 @@
     ")\n",
     "\n",
     "result = await attack.execute_async(objective=\"Modify existing PDF\")  # type: ignore\n",
-    "await ConsoleAttackResultPrinter().print_conversation_async(result=result)  # type: ignore"
+    "await output_attack_async(result)"
    ]
   },
   {
@@ -445,7 +444,7 @@
     ")\n",
     "\n",
     "result = await attack.execute_async(objective=\"This is a simple test string for Word document generation.\")  # type: ignore\n",
-    "await ConsoleAttackResultPrinter().print_conversation_async(result=result)  # type: ignore"
+    "await output_attack_async(result)"
    ]
   },
   {
@@ -529,7 +528,7 @@
     ")\n",
     "\n",
     "result = await attack.execute_async(objective=\"AI Red Team Engineer\")  # type: ignore\n",
-    "await ConsoleAttackResultPrinter().print_conversation_async(result=result)  # type: ignore"
+    "await output_attack_async(result)"
    ]
   }
  ],

doc/code/converters/5_file_converters.py

@@ -8,7 +8,6 @@
 #       format_version: '1.3'
 #       jupytext_version: 1.17.3
 # ---
-
 # %% [markdown]
 # # 5. File Converters
 #
@@ -20,7 +19,6 @@
 #
 # - **PDFConverter**: Convert text to PDF documents with templates or direct generation
 # - **WordDocConverter**: Convert text to Word (.docx) documents with optional placeholder injection
-
 # %% [markdown]
 # ## PDFConverter
 #
@@ -29,22 +27,20 @@
 # 1. **Template-Based PDF Generation**: Use YAML templates to render dynamic content into PDFs
 # 2. **Direct Prompt PDF Generation**: Convert plain text strings into PDFs without templates
 # 3. **Modify Existing PDFs**: Inject text into existing PDF documents
-
 # %% [markdown]
 # ### Template-Based PDF Generation
 #
 # This mode populates placeholders in a YAML-based template and converts the rendered content into a PDF.
-
 # %%
 import pathlib
 
 from pyrit.common.path import CONVERTER_SEED_PROMPT_PATH
 from pyrit.executor.attack import (
     AttackConverterConfig,
-    ConsoleAttackResultPrinter,
     PromptSendingAttack,
 )
 from pyrit.models import SeedPrompt
+from pyrit.output import output_attack_async
 from pyrit.prompt_converter import PDFConverter
 from pyrit.prompt_normalizer import PromptConverterConfiguration
 from pyrit.prompt_target import TextTarget
@@ -98,7 +94,7 @@
 )
 
 result = await attack.execute_async(objective=prompt)  # type: ignore
-await ConsoleAttackResultPrinter().print_conversation_async(result=result)  # type: ignore
+await output_attack_async(result)
 
 # %% [markdown]
 # ### Direct Prompt PDF Generation (No Template)
@@ -133,7 +129,7 @@
 )
 
 result = await attack.execute_async(objective=prompt)  # type: ignore
-await ConsoleAttackResultPrinter().print_conversation_async(result=result)  # type: ignore
+await output_attack_async(result)
 
 # %% [markdown]
 # ### Modifying Existing PDFs with Injection Items
@@ -205,7 +201,7 @@
 )
 
 result = await attack.execute_async(objective="Modify existing PDF")  # type: ignore
-await ConsoleAttackResultPrinter().print_conversation_async(result=result)  # type: ignore
+await output_attack_async(result)
 
 # %% [markdown]
 # ## WordDocConverter
@@ -238,7 +234,7 @@
 )
 
 result = await attack.execute_async(objective="This is a simple test string for Word document generation.")  # type: ignore
-await ConsoleAttackResultPrinter().print_conversation_async(result=result)  # type: ignore
+await output_attack_async(result)
 
 # %% [markdown]
 # ### Placeholder-Based Injection into Existing Word Documents
@@ -278,4 +274,4 @@
 )
 
 result = await attack.execute_async(objective="AI Red Team Engineer")  # type: ignore
-await ConsoleAttackResultPrinter().print_conversation_async(result=result)  # type: ignore
+await output_attack_async(result)