Skip to content

README.md

Latest commit

 

History

History
110 lines (71 loc) · 2.63 KB

README.md

File metadata and controls

110 lines (71 loc) · 2.63 KB
Raw

Cloud mode (PoC)

Run multiple attacks as a batch job. Instead of calling runner/run.py once per attack manually, you write a YAML with a list of attacks and one model config, and the cloud layer dispatches them to a queue, runs workers, and collects results.

Works with: any black-box attack (continuation, promptinject, snowball, …)
Does not work with: white-box attacks (gcg, beast, autodan) or multi-turn red-team attacks (fitd, crescendo)

How it works

batch.yaml → dispatcher → queue → worker(s) → collector → report.json

Two queue backends:

  • InMemory (default) — everything in one process, no extra deps
  • Redis — workers can run on separate machines/pods

Quickstart — single command (no Redis needed)

python -m cloud.cli dispatch cloud/example_batch_hf.yaml --workers 2

Runs dispatch + 2 workers + collect all at once. Done.

Multi-terminal test with Redis

This is how to verify the full distributed flow on one machine.

Step 1 — start Redis (one time):

redis-server --daemonize yes
redis-cli ping  # should print PONG

Step 2 — dispatch (terminal 1):

python -m cloud.cli dispatch cloud/example_batch_hf.yaml --queue redis

Prints something like dispatched 2 work unit(s) and exits.

Step 3 — worker (terminal 2):

python -m cloud.cli worker --queue redis --worker-id pod-0

Pulls units from the queue, runs the attacks, saves JSON results to results/example_campaign_hf/. Exits when queue is empty.

Want more parallelism? Open more terminals and run with --worker-id pod-1, pod-2, …

Step 4 — collect (terminal 1, after worker finishes):

python -m cloud.cli collect results/example_campaign_hf/

Merges all result JSONs into results/example_campaign_hf/report.json.

Batch YAML format

job_id: my_campaign

attacks:
  - name: continuation
    config: # optional — override any attack config field
      prompt_cap: 3
  - name: promptinject
    config:
      prompt_cap: 2
      generations_per_prompt: 1

model:
  loader: hf # "hf" or "api"
  model_path: Qwen/Qwen2.5-0.5B-Instruct
  conv_template_name: chatml

  # loader: api only:
  # api_base_url: http://127.0.0.1:8000/v1
  # api_key_env: OPENAI_API_KEY

output_dir: results/my_campaign/

See cloud/example_batch_hf.yaml (HF model) and cloud/example_batch.yaml (API/vLLM).

Tests

No model or server needed — run_attack() is patched with a fake.

python -m pytest tests/test_cloud_units.py tests/test_cloud_integration.py -v

Live Redis tests are auto-skipped if Redis isn't running.