|
| 1 | +--- |
| 2 | +name: http-toolkit-intercept |
| 3 | +description: Intercept and debug HTTP traffic from a Bun- or Node-based CLI using HTTP Toolkit. Use when you need to inspect LLM API calls, backend requests, auth flows, or debug network-level issues. |
| 4 | +--- |
| 5 | + |
| 6 | +# HTTP Toolkit Intercept |
| 7 | + |
| 8 | +Use this skill when you need authoritative evidence of what your CLI sent to a remote API and what it received back while verifying a code change. |
| 9 | + |
| 10 | +The reliable pattern is: |
| 11 | + |
| 12 | +1. Start HTTP Toolkit correctly. |
| 13 | +2. Run the CLI through the proxy in a mode that produces a machine-readable log (e.g. `--output-format json`). |
| 14 | +3. Export outbound HTTP requests from HTTP Toolkit. |
| 15 | +4. Pair the outbound HTTP export with the inbound CLI session log. |
| 16 | + |
| 17 | +Do not rely on TUI screenshots alone when the question is about request payloads, auth headers, or wire-level behavior. |
| 18 | + |
| 19 | +## Prerequisites / Known-Good Launch |
| 20 | + |
| 21 | +### Start HTTP Toolkit |
| 22 | + |
| 23 | +On Linux/headless environments, plain `httptoolkit` often fails due to sandbox/X11 issues. Prefer: |
| 24 | + |
| 25 | +```bash |
| 26 | +xvfb-run --auto-servernum httptoolkit --no-sandbox |
| 27 | +``` |
| 28 | + |
| 29 | +If a stale server is already running on ports `45456/45457`, stop it first: |
| 30 | + |
| 31 | +```bash |
| 32 | +pkill -f "HTTP Toolkit Server|httptoolkit|xvfb-run --auto-servernum httptoolkit" || true |
| 33 | +``` |
| 34 | + |
| 35 | +### Verify the proxy is reachable |
| 36 | + |
| 37 | +```bash |
| 38 | +# HTTP Toolkit's admin API lives on port 45456/45457 by default. |
| 39 | +# Treat 200/401/403 as "reachable"; only connection failure means the server is dead. |
| 40 | +curl -s -o /dev/null -w "%{http_code}\n" http://127.0.0.1:45456/config |
| 41 | +``` |
| 42 | + |
| 43 | +## Quick Start |
| 44 | + |
| 45 | +### 1. Launch your CLI with the proxy env vars set |
| 46 | + |
| 47 | +The canonical Bun-friendly pattern: |
| 48 | + |
| 49 | +```bash |
| 50 | +BUN_CONFIG_PROXY="http://127.0.0.1:8000" \ |
| 51 | +HTTP_PROXY="http://127.0.0.1:8000" \ |
| 52 | +HTTPS_PROXY="http://127.0.0.1:8000" \ |
| 53 | +ALL_PROXY="http://127.0.0.1:8000" \ |
| 54 | +NODE_TLS_REJECT_UNAUTHORIZED=0 \ |
| 55 | +<your-cli> <args> |
| 56 | +``` |
| 57 | + |
| 58 | +> `NODE_TLS_REJECT_UNAUTHORIZED=0` is only appropriate for controlled local debugging. See "TLS Safety" below. |
| 59 | +
|
| 60 | +### 2. Capture the inbound session log |
| 61 | + |
| 62 | +If your CLI supports a machine-readable output mode (e.g. `--output-format stream-json`, `--json`, `--log-level debug`), pipe it to a file: |
| 63 | + |
| 64 | +```bash |
| 65 | +<your-cli> exec --output-format stream-json "your prompt" \ |
| 66 | + > /tmp/cli-intercept-stdout.log 2> /tmp/cli-intercept-stderr.log |
| 67 | +``` |
| 68 | + |
| 69 | +### 3. Export outbound HTTP from HTTP Toolkit |
| 70 | + |
| 71 | +Either: |
| 72 | + |
| 73 | +- Use the HTTP Toolkit GUI export (File → Export → JSON), or |
| 74 | +- Hit the admin API directly. The exact endpoint depends on your HTTP Toolkit version; inspect DevTools in the HTTP Toolkit UI to see the requests it makes. |
| 75 | + |
| 76 | +### 4. Cross-reference the two streams |
| 77 | + |
| 78 | +- **Outbound HTTP** (from HTTP Toolkit): authoritative for request bodies, headers, auth tokens, retry timing. |
| 79 | +- **Inbound session log** (from the CLI): authoritative for how your code reacted to the responses. |
| 80 | + |
| 81 | +Together they answer: "what did we send?" and "what did we do with the response?" |
| 82 | + |
| 83 | +## What finally worked for payload verification |
| 84 | + |
| 85 | +The critical correct pathways that proved reliable were: |
| 86 | + |
| 87 | +1. **Use a non-interactive / exec mode, not the TUI, when verifying payloads** |
| 88 | + - Interactive TUIs are slower and much harder to analyze. |
| 89 | + - Use whatever your CLI has for scripting (`--output-format`, `--json`, `--headless`). |
| 90 | + |
| 91 | +2. **Treat outbound and inbound as separate evidence sources** |
| 92 | + - HTTP Toolkit gives outbound HTTP requests. |
| 93 | + - The CLI's session log gives inbound assistant/tool behavior. |
| 94 | + - You need both to answer: "what did the model receive?" and "what did it actually do?" |
| 95 | + |
| 96 | +3. **Use Bun's proxy env var explicitly** |
| 97 | + - `BUN_CONFIG_PROXY` is the important one. |
| 98 | + - Setting only `HTTP_PROXY`/`HTTPS_PROXY` is not enough for Bun. |
| 99 | + |
| 100 | +4. **Disable TLS verification only for controlled local debugging when needed** |
| 101 | + - If the HTTP Toolkit CA is not trusted locally, use `NODE_TLS_REJECT_UNAUTHORIZED=0` as an escape hatch. |
| 102 | + - Prefer trusting the CA in your OS / Node trust store instead. |
| 103 | + - Never disable TLS in production repros. |
| 104 | + |
| 105 | +5. **Keep runs bounded** |
| 106 | + - Long tool-heavy prompts can take time. |
| 107 | + - If you only need to prove request shape, export after the LLM request is observed — you do not always need to wait for full completion. |
| 108 | + |
| 109 | +## Key Facts |
| 110 | + |
| 111 | +- **Bun requires `BUN_CONFIG_PROXY`** — treat it as mandatory |
| 112 | +- **HTTP_PROXY / HTTPS_PROXY alone are silently ignored by Bun** |
| 113 | +- **HTTP Toolkit admin API is request-oriented** — outbound HTTP comes from HTTP Toolkit, inbound behavior comes from the CLI log |
| 114 | +- **TLS is verified by default** — `NODE_TLS_REJECT_UNAUTHORIZED=0` is a local-dev escape hatch only |
| 115 | + |
| 116 | +## Bun Proxy Gotchas |
| 117 | + |
| 118 | +| Approach | Works? | Notes | |
| 119 | +|----------|--------|-------| |
| 120 | +| `BUN_CONFIG_PROXY` | Yes | Bun's official proxy env var | |
| 121 | +| `HTTP_PROXY` / `HTTPS_PROXY` | No (Bun) | Silently ignored by bun but respected by some libraries used via Bun | |
| 122 | +| `SSL_CERT_FILE` with combined CA | Unreliable | Can break on some Bun builds; verify in your local Bun version | |
| 123 | +| `NODE_TLS_REJECT_UNAUTHORIZED=0` | Local dev only | Disables TLS verification | |
| 124 | +| Running via `tsx` / Node.js | Depends on your CLI | Some CLIs have native deps (`.dylib`) that require Bun | |
| 125 | + |
| 126 | +### TLS Safety Guardrails |
| 127 | + |
| 128 | +- Keep TLS verification enabled whenever possible. |
| 129 | +- Prefer trusting the HTTP Toolkit CA in your local trust store instead of disabling verification. |
| 130 | +- Use `NODE_TLS_REJECT_UNAUTHORIZED=0` only for controlled local debugging in development. |
| 131 | +- Never disable TLS when intercepting production traffic. |
| 132 | + |
| 133 | +## Inspecting Captured Logs |
| 134 | + |
| 135 | +### Filter to LLM-call events only |
| 136 | + |
| 137 | +If your CLI emits newline-delimited JSON, use `jq`: |
| 138 | + |
| 139 | +```bash |
| 140 | +jq -c 'select(.type == "tool_call" or .type == "message")' /tmp/cli-intercept-stdout.log |
| 141 | +``` |
| 142 | + |
| 143 | +### Match outbound HTTP to inbound events |
| 144 | + |
| 145 | +Sort both streams by timestamp, then interleave them. The sequence usually is: |
| 146 | + |
| 147 | +``` |
| 148 | +outbound POST /chat/completions (from HTTP Toolkit) |
| 149 | +inbound message role=assistant (from CLI log) |
| 150 | +inbound tool_call tool=foo (from CLI log) |
| 151 | +outbound POST /chat/completions (next turn) |
| 152 | +``` |
| 153 | + |
| 154 | +If a CLI log shows an outbound request that HTTP Toolkit didn't capture, that's a proxy-config bug. |
| 155 | + |
| 156 | +## Troubleshooting |
| 157 | + |
| 158 | +| Problem | Cause | Fix | |
| 159 | +|---------|-------|-----| |
| 160 | +| CLI hangs, no events after startup | Proxy env vars not reaching the process, or TLS verification blocking | Re-run with `BUN_CONFIG_PROXY` explicitly set; if necessary set `NODE_TLS_REJECT_UNAUTHORIZED=0` in dev | |
| 161 | +| `ECONNRESET` on every request | Using `HTTP_PROXY` not `BUN_CONFIG_PROXY` with Bun | Switch to `BUN_CONFIG_PROXY` | |
| 162 | +| TLS cert errors via proxy | MITM CA not trusted | Trust HTTP Toolkit CA locally, or use `NODE_TLS_REJECT_UNAUTHORIZED=0` in dev only | |
| 163 | +| `ERR_UNKNOWN_FILE_EXTENSION .dylib` | Running with tsx/Node on a CLI that needs Bun | Run the CLI with `bun` directly | |
| 164 | +| HTTP Toolkit API 403s on `/config` | Auth-gated config endpoint | Treat 200/401/403 as reachable; only connection failure means the server is dead | |
| 165 | +| Export has outbound data but no matching inbound events | Didn't capture the CLI log | Add `> /tmp/cli.log` redirection to the CLI launch | |
| 166 | +| HTTP Toolkit misses the first request | Started capturing after the process launched | Start HTTP Toolkit first, THEN launch the CLI | |
0 commit comments