Skip to content

404SecNotFound/LLM-Pentest-Reliability

Repository files navigation

LLM Pentest Reliability

Does an independent verifier make an LLM penetration-testing agent more reliable? Sometimes. It depends on the model.

This repository holds the code, data, and analysis for an MSc dissertation that measured run-to-run variance in an autonomous LLM penetration-testing agent, then tested whether an external consistency checker reduces that variance.

License Python Agent Benchmark Design Runs


TL;DR

An LLM agent asked to solve the same penetration-testing task ten times will solve it ten different ways. Some runs finish in six steps, some in twenty-five, some fail. This project measured that inconsistency as a formal metric, which prior work had not done, then added a separate service that verifies the agent's claims against the real target after every action.

On the strongest test case, the checker cut step-count variance by 58% with no loss of success. On a weaker model, the same checker made things worse. The deciding factor was the capability of the underlying model, not the design of the checker.


Research question

How does independent verification affect the reliability of LLM-based penetration-testing agents?

"Reliability" here means run-to-run consistency on identical tasks under identical settings. The effectiveness metric is task success rate. The consistency metric is the standard deviation of step counts across repeated runs.


Key result

Task WS0 (path traversal on a PHP app), model gpt-4.1, ten runs per condition:

Condition Runs Success Mean steps Step SD
Baseline (no checker) 10 100% 6.6 1.26
With checker 10 100% 7.5 0.53

Step-count standard deviation fell from 1.26 to 0.53, a 58% reduction. Success held at 100% in both conditions. Mann-Whitney U on the step counts returned p = 0.008 with Cohen's d = 0.93.

The checker did not make the agent solve more. It made a capable agent solve the same task far more predictably. Predictability is what matters for cost and time estimation in a real engagement, where each step is an API call and a chunk of wall-clock time.


The capability-dependent finding

The checker was tested against two models across the same five tasks. Success rates across all four conditions:

Task 4o-mini base 4o-mini + checker gpt-4.1 base gpt-4.1 + checker
AC0 (sudo brute-force) 100% 90% 100% 90%
AC2 (cron root) 0% 0% 0% 0%
AC4 (SETUID env) 0% 0% 30% 30%
NS0 (SSH standard port) 80% 20% 90% 70%
WS0 (path traversal) 0% 20% 100% 100%
Aggregate 18/50 (36%) 13/50 (26%) 32/50 (64%) 29/50 (58%)

On gpt-4o-mini, adding the checker dropped aggregate success from 36% to 26%. The weaker model could not act on correction prompts. Each failed check consumed a step of the budget without producing progress. The stronger model used the same corrections productively.

This is the central contribution. Verification effectiveness is conditional on model capability. Wrapping a weak model in a checker does not rescue it. That finding could not have come from a single-model experiment, which is why the design crosses two models against the checker.

The 36% figure is what analysis/analyse_all.py computes from the committed run logs (18 of 50 baseline runs succeeded). It is reproducible from this repository.


Variance detail (gpt-4.1, successful runs only)

Standard deviation of step counts, baseline versus checker:

Task SD base SD + checker Change Mann-Whitney U p
WS0 1.26 0.53 -58% 0.008 (significant)
NS0 6.51 2.37 -64% 0.064 (not significant)
AC4 1.53 0.58 -62% 0.383 (n = 3)
AC0 0.00 0.67 baseline already deterministic 0.0002

The direction is consistent on the tasks with real variance to reduce. WS0 is the only comparison with enough successful runs on both sides to reach significance. NS0 and AC4 point the same way but are underpowered. On gpt-4o-mini the effect reversed, with step-count SD rising instead of falling (AC0 +453%, NS0 +94%).


How it works

The agent proposes an action. The adapter runs it against the target. Before the agent is allowed to build on a claim, the checker confirms that claim against live target state. A failed check returns a correction prompt instead of accepting the claim.

flowchart TD
    A["PentestGPT agent<br/>(gpt-4.1 or gpt-4o-mini)"] -->|"proposed action + claim"| B["Adapter<br/>runner/adapter.py"]
    B -->|"run command"| T["AutoPenBench target<br/>(Docker, in-vitro task)"]
    T -->|"observation"| B
    B -->|"verify claim"| C["Consistency Checker<br/>FastAPI, 127.0.0.1:8100"]
    C -.->|"/verify/port : TCP connect"| T
    C -.->|"/verify/ssh : real SSH login"| T
    C -->|"/verify/cmd : output sanity"| B
    C -->|"pass, continue"| B
    C -->|"fail, correction prompt, retry up to 2"| A
    B -->|"log every step"| L[("Per-run JSONL logs<br/>data/")]
Loading

The checker is a standalone FastAPI service. It shares no memory or state with the agent. It only reports what the real target shows. Three endpoints do the work:

  • POST /verify/port opens a TCP connection to confirm a service is actually listening at the claimed host and port. This catches hallucinated services and wrong target addresses.
  • POST /verify/ssh performs a real SSH login with paramiko to confirm claimed credentials work. This catches invented usernames and passwords.
  • POST /verify/cmd inspects command output for timeouts, empty responses, and expected content. This catches unresponsive shells and fabricated command results.

Because the checker is decoupled from the agent, it ports to other agent frameworks without modification.


Experimental design

A 2x2 factorial design crosses two models against two checker conditions, run over five AutoPenBench in-vitro tasks, with ten repeats each. That is 200 runs total, plus a 10-run feasibility pilot on gpt-4.1.

Factor Levels
Model gpt-4o-mini, gpt-4.1
Checker off (baseline), on
Tasks AC0, AC2, AC4, NS0, WS0 (AutoPenBench in-vitro)
Repeats 10 per cell
Temperature 0.2
Step limit 30
Agent PentestGPT, pinned at commit 6e84be8

Each run started from a clean target with no leftover state. Statistics use Mann-Whitney U (two-tailed) rather than a t-test, because ten runs per cell is too few to assume normality and step counts are discrete integers. Cohen's d accompanies every comparison as an effect-size measure.


Repository structure

agents/pentestgpt/     PentestGPT under test, pinned at commit 6e84be8 (see COMMIT.txt)
runner/
  adapter.py           Bridge between the model and the AutoPenBench driver; calls the checker
  config.yaml          Model, temperature, step limit, and the five task definitions
  run_baseline.sh      Batch driver for 50 baseline runs (5 tasks x 10 repeats)
  prompts/             Pentest system-prompt template
checker/
  app.py               FastAPI consistency checker (/verify/port, /verify/ssh, /verify/cmd, /health)
  rules.yaml           Proof rules for step verification
benchmarks/
  autopenbench/        Download helper and target hashes (no VM images committed)
  hostmaps/            Host-to-IP maps
data/
  gpt41_baseline/      gpt-4.1, checker off
  gpt41_checker/       gpt-4.1, checker on
  intervention_v2/     gpt-4o-mini, checker on (final, after the false-correction fix below)
  intervention/        gpt-4o-mini, checker on (superseded first pass, kept for the record)
  pilot_gpt41_baseline/  10-run feasibility pilot
  summary/             Summary CSV headers
analysis/
  analyse_all.py       Computes success rates, step and token stats, Mann-Whitney U, Cohen's d
  full_results.txt     Committed output of the analysis
  figs/                Figure output directory

Note on runners. runner/adapter.py is the working entry point that produced the data, driven per task or in batch by runner/run_baseline.sh. runner/run_experiment.py and the make baseline / make checker targets are scaffolding placeholders, not the path used for the experiments.

Note on data. The gpt-4o-mini baseline logs sit under data/raw/, which is excluded by .gitignore for size. The other three conditions are committed in full. Re-running analysis/analyse_all.py against a fresh clone will need those baseline logs regenerated.


Reproduce the analysis

python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
python analysis/analyse_all.py     # prints the tables in analysis/full_results.txt

Reproduce a run

AutoPenBench and its Docker targets are installed separately. With a target up:

# Terminal 1: start the checker
uvicorn checker.app:app --host 127.0.0.1 --port 8100

# Terminal 2: run one task (add --checker to enable verification)
cd <autopenbench-dir>
python3 -m adapter --category web_security --vm 0 --run-id 1 --checker

Batch all 50 baseline runs with bash runner/run_baseline.sh.


Engineering note: a false correction is worse than no correction

During early testing the checker carried a bug. When the agent scanned a subnet such as 192.168.0.0/16, the checker took the network address 192.168.0.0, treated it as a host, and tried to connect to port 22. That address is not a live host, so the check always failed. The checker kept telling the agent its scan had failed when the scan was fine. Success on that task collapsed from around 80% to 0%.

The fix was one line. The lesson is larger. A false correction actively derails the agent, worse than staying silent. Any verification layer has to tell "the agent is confused" apart from "my check is confused." The intervention and intervention_v2 data folders hold the runs from before and after this fix.


Limitations

  • Ten runs per task per condition. Rare successes such as AC4 at 3 of 10 produce wide confidence intervals.
  • Five tasks from a single benchmark. Cryptography and multi-host scenarios were out of scope.
  • No Bonferroni correction was applied. The WS0 result survives a corrected 0.01 threshold, but that correction was not formally run.
  • Temperature was 0.2 rather than 0, so some variance is sampling noise by design.
  • CHECKER_MAX_RETRIES is defined but was not enforced as a hard per-action cap.

Contribution

  1. The first formal measurement of run-to-run variance in an LLM penetration-testing agent, across 200 repeated runs on a standardised benchmark.
  2. A lightweight consistency checker that verifies agent claims against live target state, decoupled from the agent and portable to other frameworks.
  3. Evidence that external verification helps or hurts depending on the capability of the underlying model, established through a two-model design.

Author and citation

Imran Hafeez MSc Cyber Security, Edinburgh Napier University, School of Computing Supervisor: Sana Ullah Jan Dissertation submitted April 2026. Viva passed May 2026.

@mastersthesis{hafeez2026verification,
  author = {Hafeez, Imran},
  title  = {How Does Independent Verification Affect the Reliability of LLM-Based Penetration Testing Agents},
  school = {Edinburgh Napier University},
  year   = {2026}
}

Agent under test: PentestGPT, pinned at commit 6e84be8. Benchmark: AutoPenBench in-vitro tasks.

License

Apache-2.0. See LICENSE.

About

MSc Dissertation Work

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors