|
1 | | -## reflex — HTTPS referrer emulation for analytics testing |
| 1 | +## reflex — Local HTTPS referrer emulator ⚡️ |
2 | 2 |
|
3 | | -**Reflex** is a lightweight CLI tool helps growth, marketing, and analytics teams reliably emulate inbound referrals (e.g., from `https://news.google.com`) in modern browsers. It does this by: |
| 3 | +Emulate real inbound referrers in modern browsers — safely and repeatably. |
4 | 4 |
|
5 | | -- Mapping a chosen referrer hostname to your machine via the system hosts file |
6 | | -- Serving a locally trusted HTTPS site for that hostname (certs via mkcert) |
7 | | -- Issuing a redirect (302/meta/JS) to your target URL so the browser sends the correct `Referer` |
| 5 | +✨ What it does |
| 6 | +- 🧭 Maps a referrer host to your machine (`/etc/hosts`) |
| 7 | +- 🔒 Serves a locally trusted HTTPS site (certs via mkcert) |
| 8 | +- 🚀 Redirects to your target so the browser sends a real `Referer` |
8 | 9 |
|
9 | | -This enables safe, repeatable end-to-end tests of attribution, A/B splits, and analytics pipelines without fragile header spoofing extensions. |
| 10 | +Perfect for attribution tests, analytics pipelines, and E2E growth flows. |
10 | 11 |
|
11 | | -Important: This tool modifies your hosts file and runs a local HTTPS server. It restores cleanly on exit, and you can run `reflex cleanup` at any time. |
| 12 | +⚠️ Requires sudo/admin (modifies `/etc/hosts`, binds 443) |
12 | 13 |
|
13 | | -How it works |
| 14 | +### 🧩 Visual Flow |
14 | 15 |
|
15 | | -- Host spoof: Adds a tagged line like `127.0.0.1 news.google.com # reflex-managed` to your hosts file |
16 | | -- HTTPS: Generates a locally trusted certificate for the hostname with `mkcert` |
17 | | -- Redirect: Serves a small HTTPS site on that host which redirects to your target; the browser sends a real `Referer` header per current referrer policies |
| 16 | +```text |
| 17 | + 🧑💻 You click the referrer URL |
| 18 | + │ |
| 19 | + ▼ |
| 20 | + /etc/hosts ➜ 127.0.0.1 (spoofs referrer host) |
| 21 | + │ |
| 22 | + ▼ |
| 23 | + 🔒 Reflex HTTPS server (mkcert‑trusted) |
| 24 | + │ 302 / <meta> / JS → https://your-app.example |
| 25 | + ▼ |
| 26 | + 🎯 Target site receives Referer: https://news.google.com/ |
| 27 | +``` |
18 | 28 |
|
19 | | -Install |
| 29 | +### 🚀 Quick Start |
20 | 30 |
|
21 | | -- Requires Go 1.21+ for building |
22 | | -- Requires `mkcert` (https://github.com/FiloSottile/mkcert) |
23 | | -- Requires sudo/admin privileges (the CLI runs with sudo) |
| 31 | +1) One‑time (Linux only) — share CA between root and your user |
24 | 32 |
|
25 | | -Build from source |
| 33 | +```bash |
| 34 | +sudo mkcert -install # system trust store |
| 35 | +mkcert -install # your user’s Firefox/Chromium trust |
| 36 | +sudo mkdir -p /etc/mkcert |
| 37 | +sudo cp -a "$(mkcert -CAROOT)/." /etc/mkcert/ |
| 38 | +sudo chmod 755 /etc/mkcert && sudo chmod 644 /etc/mkcert/* |
| 39 | +``` |
26 | 40 |
|
27 | | -- `go build ./cmd/reflex` |
28 | | -- Or: `go run ./cmd/reflex --help` |
| 41 | +• macOS: `brew install mkcert nss && sudo mkcert -install` |
| 42 | +• Windows: `choco install mkcert` then `mkcert -install` (admin PowerShell) |
29 | 43 |
|
30 | | -One-time Setup (Linux) |
| 44 | +2) Run a referrer → target flow |
31 | 45 |
|
32 | | -To avoid the common "no Firefox/Chrome security databases found" issue when running under sudo, pin a shared CAROOT used by both root and your user. |
| 46 | +```bash |
| 47 | +sudo reflex run \ |
| 48 | + --referrer https://news.google.com \ |
| 49 | + --target https://your-app.example |
| 50 | +``` |
33 | 51 |
|
34 | | -1. Install the local CA into the system trust store and your user’s browser trust store: |
| 52 | +Reflex opens your default browser (as your normal user) in a private window and serves a small referrer page using HTTPS with a trusted local cert. |
35 | 53 |
|
36 | | -- `sudo mkcert -install` # system trust store |
37 | | -- `mkcert -install` # your user’s Firefox/Chromium trust |
| 54 | +### 🔧 Install / Build |
38 | 55 |
|
39 | | -2. Pin a shared CAROOT so root and user use the same CA: |
| 56 | +- 🦫 Go 1.21+ |
| 57 | +- 🔑 `mkcert` in PATH |
| 58 | +- 🏗️ Build: `go build ./cmd/reflex` |
| 59 | +- 📖 Help: `go run ./cmd/reflex --help` |
40 | 60 |
|
41 | | -- `sudo mkdir -p /etc/mkcert` |
42 | | -- `sudo cp -a "$(mkcert -CAROOT)/." /etc/mkcert/` |
43 | | -- `sudo chmod 755 /etc/mkcert && sudo chmod 644 /etc/mkcert/*` |
| 61 | +### 🕹️ Commands |
44 | 62 |
|
45 | | -After this, reflex (which runs with sudo) uses `/etc/mkcert` automatically to generate certs that match your user’s trusted CA. |
| 63 | +- ▶️ `reflex run` Start HTTPS server, spoof host, open browser |
| 64 | +- 🧹 `reflex cleanup` Remove hosts entry and generated certs (add `--all` to wipe everything) |
| 65 | +- 🔍 `reflex status` Show current state for a referrer |
46 | 66 |
|
47 | | -Notes for other platforms: |
| 67 | +### 🎛️ Flags you’ll actually use |
48 | 68 |
|
49 | | -- macOS: `brew install mkcert nss && sudo mkcert -install` (no pinned CAROOT needed) |
50 | | -- Windows: `choco install mkcert` then `mkcert -install` in an elevated shell |
| 69 | +- 🔗 `--referrer` Referrer URL or host (required) |
| 70 | +- 🎯 `--target` Target URL to navigate to (required) |
| 71 | +- 🔁 `--method` Redirect: meta (default), 302, js |
| 72 | +- 🛡️ `--referrer-policy` `origin-when-cross-origin` (default) or `unsafe-url` for full URL |
| 73 | +- 🕶️ `--private` Open browser in incognito/private mode (default true) |
| 74 | +- 🚫 `--no-browser` Don’t auto‑open a browser |
51 | 75 |
|
52 | | -Run with sudo |
| 76 | +More: |
| 77 | +- ⏱️ `--delay` (meta/js, ms), 🔌 `--port` (default 443, falls back to 8443), 🗂️ `--keep-certs`, 🧪 `--no-hosts`, 🧹 `--force-unlock` |
53 | 78 |
|
54 | | -- reflex requires elevated privileges to modify `/etc/hosts` and bind to port 443. |
55 | | -- Always run commands with sudo, e.g.: `sudo reflex run --referrer ... --target ...` |
| 79 | +### 🩹 Troubleshooting (fast answers) |
56 | 80 |
|
57 | | -Quick start |
| 81 | +- 🥚 Empty `document.referrer`? |
| 82 | + - Use `--method meta` (default) or `--method js` |
| 83 | + - Try `--referrer-policy unsafe-url` for full URL referrers |
| 84 | +- 🧪 Linux mkcert warning under sudo (“no Firefox/Chromium DBs”)? |
| 85 | + - Complete the one‑time setup above (shared CAROOT in `/etc/mkcert`) |
| 86 | +- 🖥️ Browser didn’t open? |
| 87 | + - Reflex launches the browser as your non‑root user. If DBus/XDG is missing (headless), copy the printed URL and open manually |
| 88 | +- 🌐 Hosts entry not taking effect? |
| 89 | + - Check VPNs/enterprise DNS overrides. `sudo reflex status --referrer <host>` helps debug |
58 | 90 |
|
59 | | -- Spoof Google News and redirect to your site: |
60 | | - - `reflex run --referrer https://news.google.com --target https://your-app.example` |
| 91 | +### 🧼 Safety and cleanup |
61 | 92 |
|
62 | | -Commands |
| 93 | +- 🏷️ Hosts entries are tagged (`# reflex-managed`) for safe removal |
| 94 | +- 🕰️ A timestamped hosts backup is written before first modification |
| 95 | +- 🧽 `reflex cleanup --referrer <host>` removes the entry and temp certs (unless `--keep-certs`) |
63 | 96 |
|
64 | | -- `reflex run`: Start HTTPS server, spoof host, and open browser |
65 | | -- `reflex cleanup`: Remove hosts entry and generated certificates for a host; add `--all` to remove all reflex-managed entries, all certs, and the lock |
66 | | -- `reflex status`: Inspect whether hosts entry and certs exist |
| 97 | +### 🧰 Dev notes |
67 | 98 |
|
68 | | -Run options |
| 99 | +Code map: |
| 100 | +- 🧩 `cmd/reflex` CLI |
| 101 | +- 🗂️ `internal/hosts` Hosts manager |
| 102 | +- 🔑 `internal/certs` mkcert bridge (Linux uses `/etc/mkcert`) |
| 103 | +- 🔒 `internal/server` HTTPS redirector |
| 104 | +- 🌐 `internal/browser` Browser opener (drops sudo → user, incognito) |
| 105 | +- 🛠️ `internal/util` Port/lock/helpers |
69 | 106 |
|
70 | | -- `--referrer`: Referrer URL or hostname (required) |
71 | | -- `--target`: Target URL to navigate to (required) |
72 | | -- `--method`: Redirect strategy: meta (default), 302, js |
73 | | -- `--delay`: Delay for meta/js methods in ms (default 1500) |
74 | | -- `--port`: TLS port (default 443). Falls back to 8443 if unavailable |
75 | | -- `--no-browser`: Do not open the browser automatically |
76 | | -- `--private`: Open browser in incognito/private mode (default true) |
77 | | -- `--keep-certs`: Keep generated certs after exit |
78 | | -- `--no-hosts`: Do not modify hosts (advanced) |
79 | | -- `--force-unlock`: Forcefully clear a stale lock before starting |
80 | | -- `--referrer-policy`: Explicit Referrer-Policy for the referrer page (default `origin-when-cross-origin`). Use `unsafe-url` to send the full URL to cross-origin targets. |
| 107 | +🧪 Tests: `go test ./...` (unit tests generate self‑signed certs; no mkcert required) |
81 | 108 |
|
82 | | -Notes |
| 109 | +### 🗺️ Roadmap |
83 | 110 |
|
84 | | -- Elevated privileges: Binding to 443 and editing the hosts file typically require admin/sudo |
85 | | -- If 443 is busy or you lack privileges, the server falls back to 8443 and opens `https://<host>:8443` |
86 | | -- Modern default referrer policy is `strict-origin-when-cross-origin`, so your target receives at least the origin |
87 | | -- HSTS domains: A valid, trusted certificate is required. `mkcert` provides a locally trusted CA for this purpose. Reflex now runs `mkcert -install` for you (idempotent) before generating certs; if it fails, you’ll see a clear instruction. |
88 | | - |
89 | | -Cleanup and safety |
90 | | - |
91 | | -- Tagged hosts entries let reflex remove only what it added |
92 | | -- A timestamped backup of the hosts file is created adjacent to the file on first write |
93 | | -- On exit or `reflex cleanup --referrer <host>`, reflex removes the tagged line and generated certs (unless `--keep-certs`) |
94 | | - |
95 | | -Troubleshooting |
96 | | - |
97 | | -- Browser blocks the page: Ensure `mkcert` is installed and its root CA is trusted |
98 | | -- Empty `document.referrer`: Set `--referrer-policy origin-when-cross-origin` (default) for origin referrers, or `--referrer-policy unsafe-url` to send the full path. You can also try `--method meta` or `--method js` if your stack handles those better. |
99 | | -- Linux note: If you see "no Firefox and/or Chrome/Chromium security databases found" while using sudo, complete the One-time Setup above to pin a shared CAROOT at `/etc/mkcert`. |
100 | | -- Wrong host is opened: Verify the URL shown by the tool and that your browser didn’t cache an older page |
101 | | -- Hosts entry doesn’t apply: Some VPN/enterprise setups override name resolution; try disabling the VPN temporarily |
102 | | - |
103 | | -Development |
104 | | - |
105 | | -- Code layout: |
106 | | - - `cmd/reflex`: CLI entrypoint and subcommands |
107 | | - - `internal/hosts`: Safe hosts file management with tagged lines |
108 | | - - `internal/certs`: mkcert integration and cert lifecycle |
109 | | - - `internal/server`: HTTPS server and redirect strategies |
110 | | - - `internal/browser`: Cross-platform browser opener |
111 | | - - `internal/util`: Utilities (hostname parsing, port probing, locking) |
112 | | -- Tests: `internal/hosts` and `internal/util` include basic unit tests you can run with `go test ./...` |
113 | | - |
114 | | -Roadmap |
115 | | - |
116 | | -- Optional DNS-based spoofing mode (no hosts edits) |
117 | | -- CAP_NET_BIND_SERVICE support on Linux to bind 443 without root |
118 | | -- UI control panel and session recorder |
119 | | -- Native installers and signed binaries |
| 111 | +- 🧭 Optional DNS spoofing mode (no hosts edits) |
| 112 | +- 🧷 CAP_NET_BIND_SERVICE on Linux to bind 443 without sudo |
| 113 | +- 🖱️ Simple UI control panel / recorder |
| 114 | +- 📦 Packages / signed binaries |
0 commit comments