|
| 1 | +# UAT Report — 2026-03-21 (Session 2) |
| 2 | + |
| 3 | +## Session Summary |
| 4 | + |
| 5 | +- **Wallhack Version**: 0.14.1+c2c73af.20260321T004320.release |
| 6 | +- **Pontoon Version**: 0.2.0 |
| 7 | +- **Range**: 27-VM cyber range (perimeter → office → datacenter topology) |
| 8 | +- **Persona**: CTF Player → Professional Pentester (natural progression) |
| 9 | +- **Objective**: Pivot through the range using Wallhack — reach internal services, discover flags, test multi-hop pivoting |
| 10 | +- **Outcome**: Single-hop pivoting works excellently. Multi-hop pivoting is completely blocked by a combination of role model limitations and a TUN resource leak bug. Found 1 flag, discovered 3 network segments, reached the office network successfully through single-hop exit. |
| 11 | + |
| 12 | +## Pontoon MCP — Findings |
| 13 | + |
| 14 | +### Tool Completeness |
| 15 | + |
| 16 | +Tools are sufficient for range operation, exec, background tasks, and debugging. The `vm_tail` / `vm_exec_bg` pair works well for long-running processes. `vm_ps` was useful for diagnosing the init-respawned daemon. `vm_restart` cleanly handles stuck VMs. |
| 17 | + |
| 18 | +Missing: no tool to list all VMs or see the topology. With 27 VMs, discovery relies entirely on scanning. A `vm_list` or `range_status` tool showing VM names/IPs/networks would help operators debug topology issues without scanning. |
| 19 | + |
| 20 | +### UX & Discoverability |
| 21 | + |
| 22 | +- Tool names are mostly intuitive. `vm_exec` vs `vm_exec_bg` is a clear split. |
| 23 | +- `helo` is cute but non-standard — `status` or `ping` would be more discoverable. Might trip up users searching for "how to check if pontoon is running". |
| 24 | +- `network_tcpdump` is clear and specific — good. |
| 25 | +- `vm_pkill` has per-VM restrictions that aren't documented in the tool description. The error message ("tool 'vm_pkill' not allowed on this VM") is clear, but a user would wonder *why* and *which* VMs allow it. |
| 26 | + |
| 27 | +### Error Messages & Feedback |
| 28 | + |
| 29 | +- Command restrictions on target VMs are well-enforced with clear error messages: `command 'echo' not allowed on this VM (allowed: wallhack)`. |
| 30 | +- `vm_helo` timeout on gateway-perimeter was confusing — the VM was network-reachable via SSH but pontoon's exec channel was dead. The error message didn't distinguish between "VM is down" and "exec channel is broken". |
| 31 | +- `vm_exec` timeout error exposes the internal VM name (e.g. "gateway-perimeter") which is useful for debugging but could be considered information leakage in a strict CTF setting. |
| 32 | + |
| 33 | +### Specific Issues |
| 34 | + |
| 35 | +| Severity | Finding | Suggested Fix | |
| 36 | +| --- | --- | --- | |
| 37 | +| 🔴 Critical | `vm_helo` and `vm_exec` permanently broken on `gateway-perimeter` VM — exec channel dead while VM is network-reachable | Investigate why the 9p/exec channel fails on this specific VM. Add healthcheck that distinguishes "VM down" from "exec channel broken" | |
| 38 | +| 🟡 Minor | `vm_exec_bg` on `gateway-perimeter` returns the same timeout error as `vm_exec` — background exec should return immediately regardless of exec channel state | `vm_exec_bg` should fork before attempting exec, or return a different error | |
| 39 | +| 🟡 Minor | No `vm_list` or `range_status` tool to see all VMs and their networks | Add a tool that returns VM names, IPs, and network memberships | |
| 40 | +| 🟢 Suggestion | `helo` → rename to `status` or `ping` for discoverability | | |
| 41 | +| 🟢 Suggestion | `vm_pkill` restrictions should be documented in the tool description or list which VMs allow it | | |
| 42 | + |
| 43 | +## Wallhack MCP — Findings |
| 44 | + |
| 45 | +### Tool Completeness |
| 46 | + |
| 47 | +Core tunneling (single-hop exit) works well. Auto-routes are excellent UX — a peer connects and its networks just appear. The tool set covers info, peers, routes, stats, logs, role, connect/disconnect, and route management. |
| 48 | + |
| 49 | +**Major gaps:** |
| 50 | +- `connect` and `listen` tools exist in the MCP but return "dynamic connect/listen not yet implemented". These should be removed from the tool listing until functional — advertising unusable tools wastes user time and creates false expectations. |
| 51 | +- No way to control remote peer daemons. In multi-hop scenarios, you need to tell a relay to start listening or change its role. The MCP only controls the local daemon. |
| 52 | +- No tool to inspect TUN device state (which TUN is allocated, is it busy, what's using it). |
| 53 | + |
| 54 | +### Logging Quality |
| 55 | + |
| 56 | +**Good:** |
| 57 | +- Startup banner with version, capabilities, eligible roles, listen address — excellent orientation |
| 58 | +- "No authentication configured" warning is exactly right for a security tool |
| 59 | +- "Peer connected" and "Installed N auto route(s)" — clear and actionable |
| 60 | +- Error messages for stream handler failures include the target address and OS error — useful for debugging |
| 61 | + |
| 62 | +**Needs improvement:** |
| 63 | +- "Route update listener started for peer X on tun Y" — too internal/developer-facing for end users |
| 64 | +- No log when traffic actually flows through a tunnel. A first-packet log ("First flow: attacker → 10.99.2.80:80 via gateway") would help users confirm the tunnel is working |
| 65 | +- During the TUN busy reconnection loop, the same error repeats every ~2 seconds with no backoff cap and no summary. After 30+ repetitions the log buffer fills with identical lines. Should cap backoff and/or deduplicate ("TUN busy, retried 15 times, giving up") |
| 66 | +- `warn: sendmsg error: Os { code: 101 ... }` exposes raw Rust OS error — should be "Network unreachable: cannot connect to 10.99.1.100:443" for a user-facing tool |
| 67 | + |
| 68 | +### Terminology Consistency |
| 69 | + |
| 70 | +No major inconsistencies found in this session. "entry/exit/relay" is used consistently. "peer" is always "peer". Good. |
| 71 | + |
| 72 | +### UX & Workflow |
| 73 | + |
| 74 | +**Single-hop pivoting is excellent.** Deploy wallhack with `--connect`, peer appears with auto-routes, curl through the tunnel just works. This is genuinely better than Ligolo-ng for the simple case — no manual route setup, no confusing "session" concept. |
| 75 | + |
| 76 | +**Multi-hop pivoting is completely broken:** |
| 77 | +1. A pivot host needs to be both exit (route local traffic) and relay (accept connections from deeper peers). Wallhack's single-role model doesn't support this. |
| 78 | +2. `--connect` + `--listen` auto-negotiates as relay. `--role exit` is silently ignored — it doesn't override auto-negotiation. |
| 79 | +3. Manual routes through a relay (`route_add` to a relay peer) accept OK but traffic doesn't flow. |
| 80 | +4. There's no documented or discoverable workflow for multi-hop. A CTF player would hit this wall and give up. |
| 81 | + |
| 82 | +**TUN resource leak is a showstopper.** Once a TUN device becomes "busy" (from a disconnected or failed peer session), it never recovers. The only fix is restarting the VM. This happened twice during the session and required full VM restarts each time. The error message ("TUN subsystem error: tun error: Resource busy") gives no recovery guidance. |
| 83 | + |
| 84 | +**`role` command is non-functional.** Setting any role via the MCP returns OK but `info` never reflects the change and behavior doesn't change. Either it's not implemented, it only affects future negotiations (undocumented), or it's a bug. |
| 85 | + |
| 86 | +### Error Messages |
| 87 | + |
| 88 | +**Good:** |
| 89 | +- `invalid CIDR: garbage` — clear validation |
| 90 | +- `peer not found: fakepeer` — helpful |
| 91 | +- `invalid role 'banana' (expected: auto, entry, exit, relay)` — lists valid options, excellent |
| 92 | + |
| 93 | +**Bad:** |
| 94 | +- `dynamic connect not yet implemented` — the tool shouldn't be listed if not implemented |
| 95 | +- `TUN subsystem error: tun error: Resource busy (os error 16)` — raw OS error, no recovery guidance |
| 96 | +- `sendmsg error: Os { code: 101, kind: NetworkUnreachable ... }` — raw Rust error struct exposed to user |
| 97 | + |
| 98 | +### Specific Issues |
| 99 | + |
| 100 | +| Severity | Finding | Suggested Fix | |
| 101 | +| --- | --- | --- | |
| 102 | +| 🔴 Critical | TUN resource leak: disconnecting a peer never releases the TUN device. All subsequent peer sessions fail with "Resource busy" until daemon restart | TUN cleanup must happen on peer disconnect. Consider reference counting or explicit teardown | |
| 103 | +| 🔴 Critical | Multi-hop pivoting impossible: no way for a node to be both exit (route traffic) and relay (accept peer connections) simultaneously | Support dual-role (exit+relay) or automatic exit capability on relay nodes for their local networks | |
| 104 | +| 🔴 Critical | `--role exit` flag silently ignored when both `--connect` and `--listen` are specified — auto-negotiation always wins | Explicit `--role` should override auto-negotiation. If it can't, emit a warning explaining why | |
| 105 | +| 🟡 Minor | `connect` and `listen` MCP tools advertised but not implemented | Remove from tool listing until implemented, or implement them | |
| 106 | +| 🟡 Minor | `role` MCP tool returns OK but has no visible effect on daemon state | Either implement role changes on a running daemon or return an error explaining the limitation | |
| 107 | +| 🟡 Minor | `stats` shows bytes in/out but packets=0, connections=0, flows=0 despite active TCP traffic | Fix packet/connection/flow counters | |
| 108 | +| 🟡 Minor | TUN busy reconnection loop fills log buffer with identical warnings (no backoff cap, no dedup) | Cap exponential backoff and deduplicate repeated errors ("TUN busy, retried N times") | |
| 109 | +| 🟡 Minor | Raw Rust error structs in user-facing log messages (`Os { code: 101, kind: NetworkUnreachable, message: ... }`) | Format as human-readable: "Network unreachable: 10.99.1.100:443" | |
| 110 | +| 🟢 Suggestion | Add a first-flow log message when traffic first traverses a tunnel to a new destination | Helps users confirm tunnel is working without needing `stats` | |
| 111 | +| 🟢 Suggestion | Add TUN status to `info` output (device name, state, allocated/free) | Would help diagnose TUN busy issues | |
| 112 | +| 🟢 Suggestion | CLI `--help` could show short examples for common workflows (single-hop, multi-hop) | Especially valuable for CTF players who learn by example | |
| 113 | +| 🟢 Suggestion | Version string format (0.14.1+c2c73af.20260321T004320.release) is thorough but the timestamp with seconds is unusual — consider just date or commit hash | Minor readability nit | |
| 114 | + |
| 115 | +## Cross-Cutting Observations |
| 116 | + |
| 117 | +1. **The single-hop experience is genuinely good.** Auto-routes, clean peer management, clear startup logs. For the simple case, Wallhack is already better than Ligolo. The gap is in multi-hop, which is the advanced-but-essential CTF scenario. |
| 118 | + |
| 119 | +2. **MCP tool parity with daemon capabilities is incomplete.** The MCP advertises `connect`, `listen`, and `role` tools that don't work as expected. This creates a trust gap — after hitting "not yet implemented" twice, a user starts doubting whether other tools work. |
| 120 | + |
| 121 | +3. **TUN lifecycle management is the #1 technical debt.** The resource leak blocked the session twice and required full VM restarts. This affects both single-hop (if you ever disconnect and reconnect) and multi-hop (relay session failures poison the TUN). |
| 122 | + |
| 123 | +4. **Pontoon + Wallhack MCP together are usable** for single-hop scenarios. The main friction is multi-hop topology, which requires both tools to cooperate in ways that aren't yet supported. |
| 124 | + |
| 125 | +5. **Range design is excellent.** The breadcrumb trail (FTP anonymous → SSH key → gateway → intranet with DB creds → deeper networks) is well-paced and realistic. A CTF player would enjoy the discovery process. |
| 126 | + |
| 127 | +## Session Transcript (Condensed) |
| 128 | + |
| 129 | +``` |
| 130 | +▶ Oriented with wallhack info + pontoon helo |
| 131 | + → Wallhack v0.14.1, entry role, listening on :443. Pontoon v0.2.0, 27/27 VMs. |
| 132 | + 💬 Tool names clear. No topology overview tool in Pontoon. |
| 133 | +
|
| 134 | +▶ Scanned perimeter network (10.99.1.0/24) |
| 135 | + → 7 hosts: .10 (SSH), .21 (FTP), .50 (Squid), .51 (SOCKS), .60 (nothing), .80 (HTTP) |
| 136 | + 💬 Good attack surface variety. |
| 137 | +
|
| 138 | +▶ FTP anonymous access → found backup/id_ed25519 + README pointing to deploy@10.99.1.10 |
| 139 | + → SSH key for the perimeter gateway. Classic CTF breadcrumb. |
| 140 | +
|
| 141 | +▶ SSH into gateway → confirmed dual-homed: 10.99.1.10/24 + 10.99.2.2/24 |
| 142 | + → Gateway bridges perimeter to office network. |
| 143 | +
|
| 144 | +▶ Deployed wallhack on gateway via SSH (--connect 10.99.1.100:443) |
| 145 | + → Peer appeared as exit, auto-routes for both networks. TCP forwarding worked immediately. |
| 146 | + 💬 Auto-routes are killer UX. Curl to internal intranet just worked. |
| 147 | +
|
| 148 | +▶ Discovered intranet at 10.99.2.80 — DB creds, deeper network references (10.99.3.x) |
| 149 | + → Flag found on loot server at 10.99.2.31: flag{9ce7617e615ca4ec79e5380412f2ce86} |
| 150 | +
|
| 151 | +▶ Attempted multi-hop: office-gateway (10.99.2.10) → can't reach attacker directly |
| 152 | + → Network unreachable. Need relay on perimeter gateway. |
| 153 | +
|
| 154 | +▶ Restarted gateway as relay (--connect + --listen 10.99.2.2:4433) |
| 155 | + → Connected as relay, but TUN "Resource busy" error. Could not recover without VM restart. |
| 156 | +
|
| 157 | +▶ First VM restart. Tried relay again from scratch. |
| 158 | + → Relay connected OK, but manual routes through relay didn't forward traffic. Relay-only nodes don't exit. |
| 159 | +
|
| 160 | +▶ Tried --role exit with --listen — auto-negotiation overrode to relay. |
| 161 | + → TUN busy again from role flip. Second VM restart required. |
| 162 | +
|
| 163 | +▶ Gave up on multi-hop. Verified single-hop TCP forwarding (confirmed working). |
| 164 | + → Tested error handling: good messages for invalid CIDR, bad peer names, invalid roles. |
| 165 | + → Found: connect/listen MCP tools not implemented, role tool non-functional, stats counters broken. |
| 166 | +``` |
0 commit comments