Skip to content

Commit 651595d

Browse files
committed
docs: Add Request Logging page with BigQuery example
- Document --request-log flag and log format - Show example output with allowed/blocked indicators - Add BigQuery streaming integration example using Line Processor - Demonstrate combining local logging with cloud analytics
1 parent 210eadc commit 651595d

4 files changed

Lines changed: 100 additions & 33 deletions

File tree

docs/guide/configuration.md

Lines changed: 12 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,6 @@ httpjail follows a simple configuration hierarchy:
88

99
1. **Command-line options** - Highest priority, override everything
1010
2. **Environment variables** - Set by httpjail for the jailed process
11-
3. **Default behavior** - Deny all requests unless explicitly allowed
1211

1312
## Key Configuration Areas
1413

@@ -76,22 +75,22 @@ httpjail --proc ./rate-limiter.py \
7675

7776
These are automatically set in the jailed process:
7877

79-
| Variable | Description | Example |
80-
|----------|-------------|---------|
81-
| `HTTP_PROXY` | HTTP proxy address | `http://127.0.0.1:34567` |
82-
| `HTTPS_PROXY` | HTTPS proxy address | `http://127.0.0.1:34567` |
83-
| `SSL_CERT_FILE` | CA certificate path | `/tmp/httpjail-ca.pem` |
84-
| `SSL_CERT_DIR` | CA certificate directory | `/tmp/httpjail-certs/` |
85-
| `NO_PROXY` | Bypass proxy for these hosts | `localhost,127.0.0.1` |
78+
| Variable | Description | Example |
79+
| --------------- | ---------------------------- | ------------------------ |
80+
| `HTTP_PROXY` | HTTP proxy address | `http://127.0.0.1:34567` |
81+
| `HTTPS_PROXY` | HTTPS proxy address | `http://127.0.0.1:34567` |
82+
| `SSL_CERT_FILE` | CA certificate path | `/tmp/httpjail-ca.pem` |
83+
| `SSL_CERT_DIR` | CA certificate directory | `/tmp/httpjail-certs/` |
84+
| `NO_PROXY` | Bypass proxy for these hosts | `localhost,127.0.0.1` |
8685

8786
### Controlling httpjail
8887

8988
These affect httpjail's behavior:
9089

91-
| Variable | Description | Example |
92-
|----------|-------------|---------|
93-
| `RUST_LOG` | Logging level | `debug`, `info`, `warn`, `error` |
94-
| `HTTPJAIL_CA_CERT` | Custom CA certificate path | `/etc/pki/custom-ca.pem` |
90+
| Variable | Description | Example |
91+
| ------------------ | -------------------------- | -------------------------------- |
92+
| `RUST_LOG` | Logging level | `debug`, `info`, `warn`, `error` |
93+
| `HTTPJAIL_CA_CERT` | Custom CA certificate path | `/etc/pki/custom-ca.pem` |
9594

9695
## Platform-Specific Configuration
9796

@@ -111,14 +110,6 @@ These affect httpjail's behavior:
111110

112111
See [Platform Support](./platform-support.md) for detailed information.
113112

114-
## Configuration Best Practices
115-
116-
1. **Start simple**: Begin with basic JavaScript rules
117-
2. **Log everything in dev**: Use `--request-log /dev/stdout` during development
118-
3. **Test isolation**: Verify no requests leak through
119-
4. **Monitor performance**: Watch for slow rule evaluation
120-
5. **Document rules**: Keep rule logic clear and maintainable
121-
122113
## Troubleshooting Configuration
123114

124115
### Rules not matching
@@ -155,4 +146,4 @@ openssl x509 -in ~/.config/httpjail/ca-cert.pem -text -noout
155146

156147
- [Rule Engines](./rule-engines/index.md) - Choose the right evaluation method
157148
- [Request Logging](./request-logging.md) - Monitor and audit requests
158-
- [Platform Support](./platform-support.md) - Platform-specific details
149+
- [Platform Support](./platform-support.md) - Platform-specific details

docs/guide/installation.md

Lines changed: 17 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -2,11 +2,23 @@
22

33
httpjail can be installed in several ways depending on your needs and platform.
44

5+
## Cargo
6+
7+
If you have Rust installed, you can install httpjail using Cargo:
8+
9+
```bash
10+
# optional: brew install rust
11+
cargo install httpjail
12+
```
13+
14+
This will compile httpjail from source and install it to your Cargo bin directory (usually `~/.cargo/bin/`).
15+
516
## Pre-built Binaries
617

718
The easiest way to install httpjail is to download a pre-built binary from the [releases page](https://github.com/coder/httpjail/releases).
819

920
### Linux
21+
1022
```bash
1123
# Download the latest release (example for Linux x86_64)
1224
curl -L https://github.com/coder/httpjail/releases/latest/download/httpjail-linux-amd64 -o httpjail
@@ -15,23 +27,14 @@ sudo mv httpjail /usr/local/bin/
1527
```
1628

1729
### macOS
30+
1831
```bash
1932
# Download the latest release (example for macOS arm64)
2033
curl -L https://github.com/coder/httpjail/releases/latest/download/httpjail-darwin-arm64 -o httpjail
2134
chmod +x httpjail
2235
sudo mv httpjail /usr/local/bin/
2336
```
2437

25-
## Install via Cargo
26-
27-
If you have Rust installed, you can install httpjail using Cargo:
28-
29-
```bash
30-
cargo install httpjail
31-
```
32-
33-
This will compile httpjail from source and install it to your Cargo bin directory (usually `~/.cargo/bin/`).
34-
3538
## Build from Source
3639

3740
For development or to get the latest unreleased features:
@@ -61,11 +64,13 @@ This profile provides reasonable performance with significantly faster build tim
6164
## System Requirements
6265

6366
### Linux
67+
6468
- Linux kernel 3.8+ (for network namespaces)
6569
- Root privileges (for network namespace creation)
6670
- iptables (for traffic redirection)
6771

68-
### macOS
72+
### macOS
73+
6974
- macOS 10.15+ (Catalina or later)
7075
- No special privileges required (uses weak mode)
7176

@@ -98,4 +103,4 @@ This is especially important on macOS for applications that use the system keych
98103

99104
## Next Steps
100105

101-
Now that you have httpjail installed, check out the [Quick Start](./quick-start.md) guide to learn how to use it.
106+
Now that you have httpjail installed, check out the [Quick Start](./quick-start.md) guide to learn how to use it.

docs/guide/platform-support.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,7 @@ Full network isolation using namespaces and nftables.
2929
┌─────────────────────────────────────────────────┐
3030
│ Target Process │
3131
│ • Isolated in network namespace │
32+
│ • User dropped to $SUDO_USER │
3233
│ • All HTTP/HTTPS → local proxy │
3334
│ • All DNS queries → dummy resolver (6.6.6.6) │
3435
│ • CA cert trusted via env vars │

docs/guide/request-logging.md

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1 +1,71 @@
11
# Request Logging
2+
3+
Log all HTTP/HTTPS requests to a file for auditing, debugging, or analysis.
4+
5+
## Basic Usage
6+
7+
```bash
8+
httpjail --request-log requests.log --js "true" -- npm install
9+
```
10+
11+
## Log Format
12+
13+
Each request is logged on a single line:
14+
```
15+
<timestamp> <+/-> <METHOD> <URL>
16+
```
17+
18+
- `+` indicates allowed requests
19+
- `-` indicates blocked requests
20+
21+
## Example Output
22+
23+
```
24+
2025-09-22T14:23:45.123Z + GET https://registry.npmjs.org/react
25+
2025-09-22T14:23:45.234Z + GET https://registry.npmjs.org/react-dom
26+
2025-09-22T14:23:45.345Z - POST https://analytics.example.com/track
27+
```
28+
29+
## BigQuery Integration
30+
31+
Stream request logs to BigQuery using Line Processor mode:
32+
33+
```bash
34+
#!/bin/bash
35+
# log-to-bigquery.sh
36+
37+
# Configure BigQuery
38+
PROJECT="my-project"
39+
DATASET="httpjail_logs"
40+
TABLE="requests"
41+
42+
# Process requests and log to BigQuery
43+
while read -r line; do
44+
# Parse the request JSON
45+
request=$(echo "$line" | jq -c '{
46+
timestamp: now | todate,
47+
url: .url,
48+
method: .method,
49+
host: .host,
50+
path: .path
51+
}')
52+
53+
# Log to BigQuery (streaming insert)
54+
echo "$request" | bq insert --project_id="$PROJECT" \
55+
"$DATASET.$TABLE"
56+
57+
# Allow all requests
58+
echo "true"
59+
done
60+
```
61+
62+
Usage:
63+
```bash
64+
httpjail --proc ./log-to-bigquery.sh --request-log local-backup.log -- your-app
65+
```
66+
67+
This approach:
68+
- Streams requests to BigQuery in real-time
69+
- Maintains a local backup in `local-backup.log`
70+
- Allows custom processing and enrichment
71+
- Scales to high-volume applications

0 commit comments

Comments
 (0)