Skip to content

Commit fcc1781

Browse files
committed
fix(relay): complete stream timeout configuration
Preserve the new body-stream idle timeout across every config-facing path that can rewrite configuration. Desktop Save now round-trips stream_timeout_secs alongside request_timeout_secs, preventing hand-edited large-download timeout settings from being silently dropped. Document the split timeout model in the TOML examples and user guides: request_timeout_secs bounds relay connection and response-header arrival, while stream_timeout_secs bounds idle time between body chunks after headers have arrived. This keeps dead destinations bounded without aborting slow range downloads mid-stream. Update the guide's implementation matrix to reflect the active range-aware download path and its remaining quota cost. Large range-capable responses can stream in chunks and resume cleanly with Range: bytes=N-, but each chunk remains an Apps Script UrlFetchApp call. Add regression coverage for TOML parsing and defaults, JSON migration round-trip preservation, Range header start parsing, and validation of non-zero-offset resume probes.
1 parent bfdd0d6 commit fcc1781

7 files changed

Lines changed: 117 additions & 11 deletions

File tree

config.example.toml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,11 @@
22
mode = "apps_script"
33
script_id = "YOUR_APPS_SCRIPT_DEPLOYMENT_ID"
44
auth_key = "CHANGE_ME_TO_A_STRONG_SECRET"
5+
# Response header/connect timeout for each relay request.
6+
request_timeout_secs = 30
7+
# Per-body-chunk idle timeout after headers arrive. Keep this larger than
8+
# request_timeout_secs so slow large downloads can continue streaming.
9+
stream_timeout_secs = 300
510

611
[network]
712
google_ip = "216.239.38.120"

config.full.example.toml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,11 @@
22
mode = "full"
33
script_id = "YOUR_APPS_SCRIPT_DEPLOYMENT_ID"
44
auth_key = "CHANGE_ME_TO_A_STRONG_SECRET"
5+
# Response header/connect timeout for each relay request.
6+
request_timeout_secs = 30
7+
# Per-body-chunk idle timeout after headers arrive. Keep this larger than
8+
# request_timeout_secs so slow large downloads can continue streaming.
9+
stream_timeout_secs = 300
510

611
[network]
712
google_ip = "216.239.38.120"

docs/guide.fa.md

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -230,7 +230,8 @@ HTTP / HTTPS مثل قبل از Apps Script می‌رود (تغییری نمی
230230
**محافظ‌های منابع:**
231231
- **حداکثر ۵۰ op** در هر بَچ — اگر سشن‌های فعال بیشتر باشند، مالتی‌پلکسر چند بَچ می‌فرستد
232232
- **سقف payload ۴ مگابایت** در هر بَچ — خیلی کمتر از ۵۰ مگابایت Apps Script
233-
- **timeout ۳۰ ثانیه** هر بَچ — مقصد کند / مرده نمی‌تواند سایر سشن‌ها را گیر بیاندازد
233+
- **timeout ۳۰ ثانیه برای اتصال و هدرها** در هر بَچ — مقصد کند / مرده نمی‌تواند سایر سشن‌ها را گیر بیاندازد
234+
- **timeout ۳۰۰ ثانیه برای هر chunk بدنه** بعد از رسیدن هدرها — دانلودهای بزرگ و کند با بودجهٔ کوتاه هدر قطع نمی‌شوند
234235

235236
### راه‌اندازی سریع حالت full
236237

@@ -253,6 +254,8 @@ HTTP / HTTPS مثل قبل از Apps Script می‌رود (تغییری نمی
253254
mode = "full"
254255
script_id = ["id1", "id2", "id3", "id4", "id5", "id6"]
255256
auth_key = "your-secret"
257+
request_timeout_secs = 30
258+
stream_timeout_secs = 300
256259
```
257260

258261
## Exit node
@@ -368,14 +371,15 @@ sni_hosts = ["www.google.com", "drive.google.com", "docs.google.com"]
368371
| بیلد musl | OpenWRT / Alpine / محیط‌های بدون libc — باینری استاتیک، با procd init |
369372
| **Exit node** | برای سایت‌های پشت Cloudflare (v1.9.4+) |
370373
| **Unwrap goog.script.init** | دفاع‌در‌عمق در مقابل Deploymentهایی که پاسخ HtmlService-wrapped می‌فرستند (v1.9.6+) |
374+
| دانلود بزرگ Range-aware | استریم chunk شده و resume برای درخواست‌های `Range: bytes=N-` |
375+
| timeout جدا برای رله | `request_timeout_secs` برای اتصال/هدر و `stream_timeout_secs` برای idle هر chunk بدنه |
371376

372377
### عمداً پیاده نشده
373378

374379
| ویژگی | چرا نه |
375380
|---|---|
376381
| HTTP/2 multiplexing | state machine کریت `h2` (stream IDs، flow control، GOAWAY) موارد hang ظریف زیادی دارد؛ coalescing + pool ۲۰-conn بیشتر فایده را می‌گیرد |
377382
| Batch (`q:[...]` در apps_script) | connection pool + tokio async از قبل خوب موازی‌سازی می‌کند؛ batch ~۲۰۰ خط مدیریت state اضافه می‌کند با سود نامشخص |
378-
| Range-based parallel download | edge case‌های واقعی (سرورهای بدون Range، chunked وسط stream)؛ ویدیوی یوتیوب از قبل با تونل بازنویسی SNI، Apps Script را دور می‌زند |
379383
| حالت‌های `domain_fronting` / `google_fronting` / `custom_domain` | Cloudflare در ۲۰۲۴ domain fronting عمومی را کشت؛ Cloud Run پلن پولی می‌خواهد |
380384

381385
## محدودیت‌های شناخته‌شده
@@ -394,6 +398,10 @@ HTML یوتیوب سریع می‌آید (از تونل بازنویسی SNI)،
394398

395399
برای مرور متنی خوب است، برای ۱۰۸۰p دردناک. چند `script_id` بچرخان برای هد روم بیشتر، یا VPN واقعی برای ویدیو.
396400

401+
### دانلودهای بزرگ هنوز سهمیه مصرف می‌کنند
402+
403+
رله می‌تواند پاسخ‌های Range-capable را chunk شده استریم کند و وقتی کلاینت با `Range: bytes=N-` ادامه می‌دهد، resume تمیز داشته باشد. اما هر chunk هنوز یک اجرای `UrlFetchApp` است. `request_timeout_secs` فقط اتصال و رسیدن هدرها را کنترل می‌کند؛ `stream_timeout_secs` سکوت بین chunkهای بدنه را.
404+
397405
### Brotli حذف می‌شود
398406

399407
از هدر `Accept-Encoding``br` حذف می‌شود. Apps Script gzip را decompress می‌کند ولی Brotli نه؛ forward کردن `br` پاسخ را خراب می‌کند. سربار حجمی جزئی.

docs/guide.md

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -230,7 +230,8 @@ More deployments = more total concurrency = lower per-session latency. Each batc
230230
**Resource guards:**
231231
- **50 ops max** per batch — if more sessions are active, the mux splits into multiple batches
232232
- **4 MB payload cap** per batch — well under Apps Script's 50 MB limit
233-
- **30 s timeout** per batch — slow / dead targets can't block other sessions forever
233+
- **30 s header/connect timeout** per batch — slow / dead targets can't block other sessions forever
234+
- **300 s per-chunk stream timeout** after headers arrive — slow large downloads can keep moving without being killed by the shorter header budget
234235

235236
### Full mode quick start
236237

@@ -253,6 +254,8 @@ More deployments = more total concurrency = lower per-session latency. Each batc
253254
mode = "full"
254255
script_id = ["id1", "id2", "id3", "id4", "id5", "id6"]
255256
auth_key = "your-secret"
257+
request_timeout_secs = 30
258+
stream_timeout_secs = 300
256259
```
257260

258261
## Exit node
@@ -364,12 +367,13 @@ This port focuses on the **`apps_script` mode** — the only one that reliably w
364367
- [x] OpenWRT / Alpine / musl builds — static binaries, procd init script included
365368
- [x] **Exit node** support for Cloudflare-fronted sites (v1.9.4+)
366369
- [x] **Goog.script.init iframe unwrap** — defense-in-depth against deployments that return HtmlService-wrapped responses (v1.9.6+)
370+
- [x] Range-aware large download streaming with resume support for `Range: bytes=N-` requests
371+
- [x] Separate relay header/connect timeout and per-chunk body idle timeout
367372

368373
Intentionally **not** implemented:
369374

370375
- **HTTP/2 multiplexing**`h2` crate state machine has too many subtle hang cases; coalescing + 20-conn pool gets most of the benefit
371376
- **Request batching (`q:[...]` mode in apps_script mode)** — connection pool + tokio async already parallelizes well; batching adds ~200 lines of state for unclear gain
372-
- **Range-based parallel download** — edge cases real (non-Range servers, chunked mid-stream); YouTube already bypasses Apps Script via SNI-rewrite tunnel
373377
- **Other modes** (`domain_fronting`, `google_fronting`, `custom_domain`) — Cloudflare killed generic domain fronting in 2024; Cloud Run needs a paid plan
374378

375379
## Known limitations
@@ -378,6 +382,7 @@ These are inherent to the Apps Script + domain-fronting approach, not bugs in th
378382

379383
- **User-Agent fixed to `Google-Apps-Script`** for traffic through the relay. `UrlFetchApp.fetch()` doesn't allow override. Sites that detect bots (Google search, some CAPTCHAs) serve degraded / no-JS pages. Workaround: add the affected domain to the `hosts` map so it's routed through the SNI-rewrite tunnel with your real browser's UA. `google.com`, `youtube.com`, `fonts.googleapis.com` are already there.
380384
- **Video playback slow and quota-limited** for anything through the relay. YouTube HTML loads fast (SNI-rewrite tunnel), but `googlevideo.com` chunks go through Apps Script. Free tier: ~20k `UrlFetchApp` calls / day, 50 MB body cap per fetch. Fine for text browsing, painful for 1080p. Rotate multiple `script_id`s for headroom, or use a real VPN for video.
385+
- **Large downloads still consume Apps Script calls.** The relay can stream range-capable responses in chunks and resume cleanly when clients retry with `Range: bytes=N-`, but every chunk is still a `UrlFetchApp` invocation. `request_timeout_secs` controls connection/headers; `stream_timeout_secs` controls idle time between body chunks.
381386
- **Brotli stripped** from forwarded `Accept-Encoding`. Apps Script can decompress gzip but not `br`; forwarding `br` would garble responses. Minor size overhead.
382387
- **WebSockets don't work** through the relay — it's request / response JSON. Sites that upgrade to WS fail (ChatGPT streaming, Discord voice, etc.).
383388
- **HSTS-preloaded / hard-pinned sites** reject the MITM cert. Most sites are fine; a handful aren't.

src/bin/ui.rs

Lines changed: 11 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -286,14 +286,16 @@ struct FormState {
286286
/// there is no UI editor for these yet, only file-edited config.
287287
/// See config.rs `fronting_groups`.
288288
fronting_groups: Vec<FrontingGroup>,
289-
/// Auto-blacklist tuning + per-batch timeout. Config-only knobs (no UI
289+
/// Auto-blacklist tuning + relay timeouts. Config-only knobs (no UI
290290
/// fields yet — power-user file edit). Round-tripped through FormState
291291
/// so Save preserves the user's hand-edited values. See config.rs
292-
/// `auto_blacklist_*` and `request_timeout_secs`.
292+
/// `auto_blacklist_*`, `request_timeout_secs`, and
293+
/// `stream_timeout_secs`.
293294
auto_blacklist_strikes: u32,
294295
auto_blacklist_window_secs: u64,
295296
auto_blacklist_cooldown_secs: u64,
296297
request_timeout_secs: u64,
298+
stream_timeout_secs: u64,
297299
/// Optional second-hop exit node for CF-anti-bot bypass (chatgpt.com /
298300
/// claude.ai / grok.com / x.com). Config-only — no UI editor yet.
299301
/// See `assets/exit_node/` for the generic exit-node handler.
@@ -391,6 +393,7 @@ fn load_form() -> (FormState, Option<String>) {
391393
auto_blacklist_window_secs: c.auto_blacklist_window_secs,
392394
auto_blacklist_cooldown_secs: c.auto_blacklist_cooldown_secs,
393395
request_timeout_secs: c.request_timeout_secs,
396+
stream_timeout_secs: c.stream_timeout_secs,
394397
exit_node: c.exit_node.clone(),
395398
}
396399
} else {
@@ -427,12 +430,14 @@ fn load_form() -> (FormState, Option<String>) {
427430
bypass_doh_hosts: Vec::new(),
428431
block_doh: true,
429432
fronting_groups: Vec::new(),
430-
// Defaults match `default_auto_blacklist_*` and
431-
// `default_request_timeout_secs` in src/config.rs.
433+
// Defaults match `default_auto_blacklist_*`,
434+
// `default_request_timeout_secs`, and
435+
// `default_stream_timeout_secs` in src/config.rs.
432436
auto_blacklist_strikes: 3,
433437
auto_blacklist_window_secs: 30,
434438
auto_blacklist_cooldown_secs: 120,
435439
request_timeout_secs: 30,
440+
stream_timeout_secs: 300,
436441
exit_node: mhrv_rs::config::ExitNodeConfig::default(),
437442
}
438443
};
@@ -610,14 +615,15 @@ impl FormState {
610615
// batch alongside the system-proxy toggle (#432).
611616
coalesce_step_ms: 0,
612617
coalesce_max_ms: 0,
613-
// Auto-blacklist + batch timeout: config-only knobs (#391,
618+
// Auto-blacklist + relay timeouts: config-only knobs (#391,
614619
// #444, #430). Round-trip through FormState so Save doesn't
615620
// drop hand-edited values. UI editor planned alongside the
616621
// v1.8.x desktop UI batch.
617622
auto_blacklist_strikes: self.auto_blacklist_strikes,
618623
auto_blacklist_window_secs: self.auto_blacklist_window_secs,
619624
auto_blacklist_cooldown_secs: self.auto_blacklist_cooldown_secs,
620625
request_timeout_secs: self.request_timeout_secs,
626+
stream_timeout_secs: self.stream_timeout_secs,
621627
// Exit-node config (CF-anti-bot bypass for chatgpt.com / claude.ai
622628
// / grok.com / x.com). Round-trip through FormState — config-only
623629
// editing for now, UI editor planned for v1.9.x desktop UI batch.

src/config.rs

Lines changed: 38 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1232,7 +1232,9 @@ mod rt_tests {
12321232
"fetch_ips_from_api": true,
12331233
"max_ips_to_scan": 50,
12341234
"scan_batch_size": 100,
1235-
"google_ip_validation": true
1235+
"google_ip_validation": true,
1236+
"request_timeout_secs": 45,
1237+
"stream_timeout_secs": 600
12361238
}"#;
12371239
let tmp = std::env::temp_dir().join("mhrv-rt-test.json");
12381240
std::fs::write(&tmp, json).unwrap();
@@ -1242,6 +1244,8 @@ mod rt_tests {
12421244
assert_eq!(cfg.listen_port, 8085);
12431245
assert_eq!(cfg.upstream_socks5.as_deref(), Some("127.0.0.1:50529"));
12441246
assert_eq!(cfg.parallel_relay, 2);
1247+
assert_eq!(cfg.request_timeout_secs, 45);
1248+
assert_eq!(cfg.stream_timeout_secs, 600);
12451249
assert_eq!(
12461250
cfg.sni_hosts.as_ref().unwrap(),
12471251
&vec!["www.google.com".to_string(), "drive.google.com".to_string()]
@@ -1367,6 +1371,38 @@ hosts = ["claude.ai", "chatgpt.com"]
13671371
assert_eq!(cfg.exit_node.hosts, vec!["claude.ai", "chatgpt.com"]);
13681372
}
13691373

1374+
#[test]
1375+
fn toml_parses_separate_header_and_stream_timeouts() {
1376+
let s = r#"
1377+
[relay]
1378+
mode = "apps_script"
1379+
auth_key = "SECRET"
1380+
script_id = "X"
1381+
request_timeout_secs = 45
1382+
stream_timeout_secs = 900
1383+
"#;
1384+
let toml_cfg: TomlConfig = toml::from_str(s).unwrap();
1385+
let cfg = Config::from(toml_cfg);
1386+
assert_eq!(cfg.request_timeout_secs, 45);
1387+
assert_eq!(cfg.stream_timeout_secs, 900);
1388+
cfg.validate().unwrap();
1389+
}
1390+
1391+
#[test]
1392+
fn toml_defaults_stream_timeout_when_omitted() {
1393+
let s = r#"
1394+
[relay]
1395+
mode = "apps_script"
1396+
auth_key = "SECRET"
1397+
script_id = "X"
1398+
"#;
1399+
let toml_cfg: TomlConfig = toml::from_str(s).unwrap();
1400+
let cfg = Config::from(toml_cfg);
1401+
assert_eq!(cfg.request_timeout_secs, 30);
1402+
assert_eq!(cfg.stream_timeout_secs, 300);
1403+
cfg.validate().unwrap();
1404+
}
1405+
13701406
#[test]
13711407
fn toml_parses_fronting_groups_array_of_tables() {
13721408
let s = r#"
@@ -1471,4 +1507,4 @@ script_id = "ABCDEF"
14711507
let _ = std::fs::remove_file(&json_path);
14721508
let _ = std::fs::remove_file(&toml_path);
14731509
}
1474-
}
1510+
}

src/domain_fronter.rs

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5803,6 +5803,47 @@ Content-Length: 45812\r\n\r\n"
58035803
assert!(validate_probe_range(206, &headers, b"hey", 4).is_none());
58045804
}
58055805

5806+
#[test]
5807+
fn parse_range_start_accepts_resume_ranges() {
5808+
assert_eq!(parse_range_start("bytes=123-"), Some(123));
5809+
assert_eq!(parse_range_start("bytes=123-456"), Some(123));
5810+
assert_eq!(parse_range_start(" bytes=0-999 "), Some(0));
5811+
assert_eq!(parse_range_start("items=123-456"), None);
5812+
assert_eq!(parse_range_start("bytes=-500"), None);
5813+
}
5814+
5815+
#[test]
5816+
fn validate_probe_range_at_offset_accepts_exact_resume_probe() {
5817+
let headers = vec![(
5818+
"Content-Range".to_string(),
5819+
"bytes 262144-524287/1048576".to_string(),
5820+
)];
5821+
let body = vec![0u8; 262144];
5822+
let range = validate_probe_range_at_offset(206, &headers, &body, 262144, 524287)
5823+
.expect("exact resume probe must validate");
5824+
5825+
assert_eq!(
5826+
range,
5827+
ContentRange {
5828+
start: 262144,
5829+
end: 524287,
5830+
total: 1048576,
5831+
}
5832+
);
5833+
}
5834+
5835+
#[test]
5836+
fn validate_probe_range_at_offset_rejects_wrong_start_or_body_len() {
5837+
let headers = vec![(
5838+
"Content-Range".to_string(),
5839+
"bytes 262144-524287/1048576".to_string(),
5840+
)];
5841+
let body = vec![0u8; 262144];
5842+
5843+
assert!(validate_probe_range_at_offset(206, &headers, &body, 0, 524287).is_none());
5844+
assert!(validate_probe_range_at_offset(206, &headers, b"short", 262144, 524287).is_none());
5845+
}
5846+
58065847
#[test]
58075848
fn extract_exact_range_body_rejects_body_length_mismatch() {
58085849
let raw = b"HTTP/1.1 206 Partial Content\r\n\

0 commit comments

Comments
 (0)