|
| 1 | +# UAT Report — 2026-03-21 |
| 2 | + |
| 3 | +## Session Summary |
| 4 | + |
| 5 | +- **Range**: Acme Corp multi-segment network (perimeter → office → datacenter) |
| 6 | +- **Wallhack version**: 0.14.1+c2c73af.20260321T004320.release |
| 7 | +- **Persona**: CTF Player (primary), shifting to Pentester critique mid-session |
| 8 | +- **Objective**: Pivot through the range using Wallhack MCP, reach internal services, capture flags |
| 9 | +- **Outcome**: Successfully pivoted to office network (10.99.2.0/24), captured 1 flag, enumerated internal services and credentials. **Blocked from reaching datacenter (10.99.3.0/24)** — wallhack binary not deployed to office-network or most perimeter VMs, preventing multi-hop pivoting. Dynamic connect/listen not implemented, further limiting relay setup options. |
| 10 | + |
| 11 | +## Pontoon MCP — Findings |
| 12 | + |
| 13 | +### Tool Completeness |
| 14 | + |
| 15 | +Tools were sufficient for basic VM interaction. `vm_exec`, `vm_exec_bg`, `vm_ps`, `vm_logs` cover the main use cases. `network_tcpdump` is a great addition for debugging. |
| 16 | + |
| 17 | +**Missing:** There is no tool to list VMs or show the range topology. A user starting cold has to guess VM names/IPs or rely on ping sweeps. A `vm_list` or `range_status` tool would be very helpful. |
| 18 | + |
| 19 | +### UX & Discoverability |
| 20 | + |
| 21 | +Tool names are clear and self-explanatory. `vm_exec` vs `vm_exec_bg` distinction makes sense. The `vm` parameter accepts both IP and hostname, which is convenient. |
| 22 | + |
| 23 | +The restriction model (only `wallhack` commands allowed on target VMs) is enforced well, with clear error messages when violations are attempted: `"command 'which' not allowed on this VM (allowed: wallhack)"`. |
| 24 | + |
| 25 | +### Error Messages & Feedback |
| 26 | + |
| 27 | +- **Good:** Command-not-allowed errors are clear and list what IS allowed |
| 28 | +- **Good:** Timeout errors include the VM name and duration |
| 29 | +- **Issue:** `vm_exec` on a command that doesn't exist (wallhack binary missing) returns no output and no error — silent failure. A user has no way to distinguish "command succeeded with no output" from "binary not found". |
| 30 | + |
| 31 | +### Specific Issues |
| 32 | + |
| 33 | +| Severity | Finding | Suggested Fix | |
| 34 | +| --- | --- | --- | |
| 35 | +| 🔴 Critical | Wallhack binary only deployed to `attacker` and `gateway-perimeter`. All other VMs (office network hosts, perimeter hosts .51/.60) have no wallhack binary — `wallhack --version` returns no output. This blocks all multi-hop pivoting. | Deploy wallhack to all VMs via the range layer/initrd build | |
| 36 | +| 🟡 Minor | No `vm_list` or `range_status` tool — user can't discover what VMs exist without scanning | Add a tool that lists VMs with their names and network assignments | |
| 37 | +| 🟡 Minor | `vm_exec` returns no output and no error when the target binary doesn't exist on a VM. Indistinguishable from a command that succeeds silently. | Return stderr/exit code, or detect command-not-found and report it | |
| 38 | +| 🟡 Minor | `vm_ps` and `vm_restart` blocked on non-attacker VMs with "not allowed on this VM" — this is by design for CTF, but the error doesn't explain WHY. A pentester might think it's a bug. | Add context to error: "restricted to attacker VM only" or similar | |
| 39 | +| 🟢 Suggestion | `vm_exec_bg` timed out the same way as `vm_exec` when starting wallhack on gateway-perimeter (5s timeout). Expected `vm_exec_bg` to return immediately after backgrounding. | Ensure `vm_exec_bg` returns a confirmation promptly without waiting for the command to produce output | |
| 40 | + |
| 41 | +## Wallhack MCP — Findings |
| 42 | + |
| 43 | +### Tool Completeness |
| 44 | + |
| 45 | +The MCP exposes a solid set of tools for basic pivoting: `info`, `peers`, `routes`, `connect`, `disconnect`, `listen`, `role`, `route_add`, `route_del`, `stats`, `logs`. The core workflow (check info → connect peer → check routes → scan through tunnel) works well for single-hop scenarios. |
| 46 | + |
| 47 | +**Critical gap:** `connect` and `listen` tools exist but return "dynamic connect/listen not yet implemented — specify --connect at startup". These tools appear in the tool list but cannot be used. This is confusing and misleading — a user discovers them, tries them, and gets told they don't work. |
| 48 | + |
| 49 | +**Missing for multi-hop:** There's no way to orchestrate relay setup through the MCP. The only control plane is for the attacker daemon. To set up multi-hop, you'd need to control gateway-perimeter's daemon too (e.g., tell it to also listen), which isn't possible through the MCP. |
| 50 | + |
| 51 | +### Logging Quality |
| 52 | + |
| 53 | +Log output is clean and well-structured: |
| 54 | +- Info lines are concise and informative |
| 55 | +- The PSK warning ("No authentication configured") is excellent — clear, actionable |
| 56 | +- Role negotiation is logged clearly: "Role resolved: peer=gateway-perimeter local_role=entry peer_role=exit" |
| 57 | +- Route installation is logged: "Installed 2 auto route(s) advertised by gateway-perimeter" |
| 58 | + |
| 59 | +**No issues with verbosity** — the default level is appropriate for a CTF/pentest user. |
| 60 | + |
| 61 | +### Terminology Consistency |
| 62 | + |
| 63 | +Terminology is consistent throughout the session. "peer", "route", "role", "entry", "exit" used consistently across info, peers, routes, logs, and error messages. No conflicting terms observed. |
| 64 | + |
| 65 | +The `--help` output uses "auto-negotiated mode" which is slightly unusual but well-explained. No Ligolo-style "agent/proxy" confusion. |
| 66 | + |
| 67 | +### UX & Workflow |
| 68 | + |
| 69 | +**What works well:** |
| 70 | +- Auto-route discovery is seamless — connect a peer and routes appear automatically with "(auto)" label |
| 71 | +- The `info` command gives a complete snapshot: name, role, listen addr, tun, version, uptime |
| 72 | +- Prefix matching on peer names for `disconnect` is convenient |
| 73 | +- Error messages consistently include valid options (e.g., role error lists all valid roles) |
| 74 | +- Manual route_add warns when peer doesn't advertise the route — accepts but informs |
| 75 | + |
| 76 | +**What's confusing:** |
| 77 | +- `tun: true` in info output — what does this mean operationally? Is it on? Configured? Available? |
| 78 | +- `stats` shows "connections: 0, flows: 0" even after dozens of TCP connections through the tunnel — these counters appear broken or measure something different than a user would expect |
| 79 | +- After `disconnect`, routes remain and traffic continues to flow (see bug below) |
| 80 | + |
| 81 | +### Error Messages |
| 82 | + |
| 83 | +Error messages are consistently excellent: |
| 84 | + |
| 85 | +- `"invalid CIDR: invalid-cidr"` — clear |
| 86 | +- `"peer not found: nonexistent"` — clear |
| 87 | +- `"invalid role 'bogus' (expected: auto, entry, exit, relay)"` — enumerates valid values, perfect |
| 88 | +- `"route not found: 192.168.0.0/16"` — clear |
| 89 | +- `"dynamic connect not yet implemented — specify --connect at startup"` — honest and explains workaround |
| 90 | + |
| 91 | +### Specific Issues |
| 92 | + |
| 93 | +| Severity | Finding | Suggested Fix | |
| 94 | +| --- | --- | --- | |
| 95 | +| 🔴 Critical | `connect` and `listen` MCP tools are listed but non-functional ("not yet implemented"). Users discover tools that don't work. This also blocks dynamic relay setup for multi-hop. | Either implement dynamic connect/listen, or don't expose the tools until they work. If keeping them, mark them clearly (e.g., tool description says "NOT YET IMPLEMENTED"). | |
| 96 | +| 🔴 Critical | After `disconnect`, auto-routes remain in the routing table AND traffic continues to flow to the disconnected peer's networks. `peers` shows "No connected peers" but `routes` still lists routes and curl through the tunnel succeeds. The disconnect operation does not clean up TUN/routing state. | On disconnect: remove auto-routes for the peer, tear down the TUN interface (or remove kernel routes). Traffic should stop. | |
| 97 | +| 🟡 Minor | `stats` counters for `connections` and `flows` are always 0 despite active TCP traffic through the tunnel (nmap scans, curl requests). `packets in/out` also 0. Only `bytes in/out` increments. | Fix counters or clarify what they measure. If they track something different from TCP connections, document it. | |
| 98 | +| 🟡 Minor | `tun: true` in `info` output is ambiguous — does it mean TUN is available, active, or configured? For a peer, `tun=false` is shown — is that because the peer doesn't have a TUN, or it's not being used for this connection? | Use clearer labels: "tun: active" / "tun: available" / "tun: disabled", or explain in tool description | |
| 99 | +| 🟡 Minor | `wallhack --version` on the attacker (via `vm_exec`) only outputs the version string with no "wallhack" prefix — just `0.14.1+c2c73af...`. Other tools (e.g., `info`) show the full context. Minor inconsistency. | Prefix with binary name: `wallhack 0.14.1+...` | |
| 100 | +| 🟢 Suggestion | The route warning on `route_add` ("peer does not advertise a route covering X") is great UX. Consider also suggesting which peer DOES advertise that route, if any. | "...traffic may not reach the destination. No peer currently advertises this range." or "...try peer X instead." | |
| 101 | +| 🟢 Suggestion | `peers` output shows `tun=false listen=false connect=true` — these boolean flags would benefit from brief explanations or a legend somewhere. A CTF player wouldn't immediately know what `tun=false` means for a peer. | Add a brief explanation to the `peers` tool description, or include a header row. | |
| 102 | +| 🟢 Suggestion | No `scan` or `discover` tool to find hosts on routed networks. Users must shell out to nmap via `vm_exec`. A lightweight host discovery built into wallhack would streamline the workflow. | Consider a `wallhack scan <cidr>` feature for basic ping sweep / port scan through the tunnel. | |
| 103 | + |
| 104 | +## Cross-Cutting Observations |
| 105 | + |
| 106 | +1. **Multi-hop is the critical gap.** The single-hop experience (attacker → gateway → office network) is polished and works well. But the multi-hop story is completely blocked: wallhack isn't deployed to most VMs, `connect`/`listen` aren't dynamic, and there's no way to control remote wallhack instances through the MCP. This limits the range to a single-pivot CTF, which undersells wallhack's capabilities. |
| 107 | + |
| 108 | +2. **The Pontoon + Wallhack integration is smooth when it works.** Using `vm_exec` to start wallhack on targets, then checking `peers`/`routes` via MCP, then scanning through the tunnel with `vm_exec` on the attacker — this workflow feels natural and powerful. The tools complement each other well. |
| 109 | + |
| 110 | +3. **Silent failures are the worst UX pattern across both MCPs.** Pontoon's `vm_exec` returns no output when a binary doesn't exist. Wallhack's `disconnect` succeeds but doesn't actually stop traffic. These silent failures erode user trust — you can't tell if something worked or not. |
| 111 | + |
| 112 | +4. **Error messages are a bright spot.** Both MCPs consistently produce clear, actionable error messages with specific details. The wallhack role error that enumerates valid values is the gold standard. This quality should be maintained. |
| 113 | + |
| 114 | +5. **The `connect`/`listen` tools should not be exposed until implemented.** Having tools that look functional but return "not yet implemented" is worse than not having them at all. It wastes user time and creates false expectations about the tool's capabilities. |
| 115 | + |
| 116 | +## Session Transcript (Condensed) |
| 117 | + |
| 118 | +▶ Reviewed Pontoon MCP tools (13 tools) and Wallhack MCP tools (11 tools) |
| 119 | + → Names are intuitive for both. No tool to list VMs in Pontoon. |
| 120 | + 💬 First impression: clean tool set, wallhack tools are well-scoped. |
| 121 | + |
| 122 | +▶ `wallhack info` — version 0.14.1, role=entry, listening on [::]:443 |
| 123 | + → Clean output with all key details. TUN=true is ambiguous. |
| 124 | + |
| 125 | +▶ Ping sweep 10.99.1.0/24 — found 7 hosts |
| 126 | + → Port scan: .21=FTP, .50=proxy(3128), .80=HTTP, .10/.51/.60=no ports |
| 127 | + 💬 Standard recon through the range. |
| 128 | + |
| 129 | +▶ Found staging creds `admin:admin123` in HTML comment on web server |
| 130 | + → FTP has anonymous access, `salaries.txt` file (empty). Proxy is Tinyproxy. |
| 131 | + |
| 132 | +▶ Started wallhack on gateway-perimeter (10.99.1.10) connecting to attacker |
| 133 | + → `vm_exec` timeout = wallhack started successfully. Peer appeared in `peers`. |
| 134 | + → Auto-routes installed: 10.99.1.0/24 and 10.99.2.0/24 |
| 135 | + |
| 136 | +▶ Scanned 10.99.2.0/24 through tunnel — 8 hosts found |
| 137 | + → .22=SSH, .31=loot(5000), .80=intranet(80), .100=SMB, .200=printer(5000) |
| 138 | + → **Flag captured:** `flag{ec30a91f722ace520fbff8e494a29816}` from loot server |
| 139 | + → Intranet leaked DB host (10.99.3.20), creds, and API server reference (10.99.3.80) |
| 140 | + |
| 141 | +▶ Attempted multi-hop to reach 10.99.3.0/24 — **BLOCKED** |
| 142 | + → `wallhack --version` returns no output on office VMs — binary not installed |
| 143 | + → Tried relay approach: second wallhack instance on gateway-perimeter listening on :4433 |
| 144 | + → `connect`/`listen` MCP tools return "not yet implemented" |
| 145 | + → Could not set up multi-hop pivoting through any method |
| 146 | + |
| 147 | +▶ Tested error handling — all error messages clear and actionable |
| 148 | + → Invalid CIDR, nonexistent peer, invalid role — all excellent |
| 149 | + → Route add with warning about unmatched peer — great UX |
| 150 | + |
| 151 | +▶ Tested disconnect — routes remain, traffic continues flowing |
| 152 | + → **Bug:** disconnect doesn't clean up routing/TUN state |
| 153 | + |
| 154 | +▶ Stats show connections=0, flows=0 despite active traffic |
| 155 | + → Only bytes in/out increments correctly |
0 commit comments