Skip to content

Commit 0b7f26d

Browse files
committed
docs: document multi-url batch output and concurrency
1 parent 57ea3a5 commit 0b7f26d

3 files changed

Lines changed: 52 additions & 7 deletions

File tree

README.md

Lines changed: 16 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@ If you use tools like Codex or Claude Code, note that they may already include b
2323
- `browser`: always uses headless browser
2424
- `raw`: send `Accept: text/markdown`, then print that single HTTP response body as-is (no fallback/conversion)
2525
- `--meta` (default `true`): control whether non-`raw` outputs include front matter (`title`/`description`). For `auto`/`static` direct markdown responses, it may do one extra HTML request to collect metadata.
26+
- One or more URL arguments are supported. For multiple URLs, requests run concurrently (configurable via `--concurrency`) and output is emitted in input order.
2627

2728
## Runtime dependency
2829

@@ -79,7 +80,7 @@ It includes installation guidance and usage instructions for `agent-fetch`.
7980
## Usage
8081

8182
```bash
82-
agent-fetch <url>
83+
agent-fetch <url> [url ...]
8384
```
8485

8586
Common flags:
@@ -89,10 +90,23 @@ agent-fetch --mode auto --timeout 20s --browser-timeout 30s https://example.com
8990
agent-fetch --mode browser --wait-selector 'article' https://example.com
9091
agent-fetch --mode static --meta=false https://example.com
9192
agent-fetch --mode raw https://example.com
93+
agent-fetch --mode static --concurrency 4 https://example.com https://example.org
9294
agent-fetch --header 'Authorization: Bearer <token>' https://example.com
9395
```
9496

95-
Fetched content is printed to `stdout` (`raw` mode prints the single HTTP response body unprocessed). Errors are printed to `stderr`.
97+
Fetched content is printed to `stdout` (`raw` mode prints the single HTTP response body unprocessed).
98+
For multiple URLs, output uses task markers so each result maps back to its input URL:
99+
100+
```text
101+
<!-- count: 3, succeeded: 2, failed: 1 -->
102+
<!-- task[1]: https://example.com/hello -->
103+
...markdown...
104+
<!-- /task[1] -->
105+
<!-- task[2](failed): https://abc.com -->
106+
<!-- error[2]: ... -->
107+
```
108+
109+
Exit code is `0` when all tasks succeed, `1` when any task fails, and `2` for argument/usage errors.
96110

97111
## Build
98112

README.zh.md

Lines changed: 16 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@
2323
- `browser`:始终使用无头浏览器
2424
- `raw`:带 `Accept: text/markdown` 发起一次 HTTP 请求,并将该响应体原样输出(不做回退/转换)
2525
- `--meta`(默认 `true`):控制非 `raw` 输出是否附带 front matter(`title`/`description`)。对于 `auto`/`static` 的直返 markdown,可能会额外发起一次 HTML 请求用于补齐元信息。
26+
- 支持一个或多个 URL 参数。传入多个 URL 时会并发请求(可用 `--concurrency` 调整),并按输入顺序输出结果。
2627

2728
## 运行时依赖
2829

@@ -79,7 +80,7 @@ Skill 位置:[`skills/agent-fetch`](./skills/agent-fetch/SKILL.md)。
7980
## 使用方式
8081

8182
```bash
82-
agent-fetch <url>
83+
agent-fetch <url> [url ...]
8384
```
8485

8586
常用参数示例:
@@ -89,10 +90,23 @@ agent-fetch --mode auto --timeout 20s --browser-timeout 30s https://example.com
8990
agent-fetch --mode browser --wait-selector 'article' https://example.com
9091
agent-fetch --mode static --meta=false https://example.com
9192
agent-fetch --mode raw https://example.com
93+
agent-fetch --mode static --concurrency 4 https://example.com https://example.org
9294
agent-fetch --header 'Authorization: Bearer <token>' https://example.com
9395
```
9496

95-
抓取成功内容输出到 `stdout``raw` 模式为单次 HTTP 响应体原样输出);错误信息输出到 `stderr`
97+
抓取成功内容输出到 `stdout``raw` 模式为单次 HTTP 响应体原样输出)。
98+
多个 URL 时,输出包含任务标记,便于与输入 URL 对齐:
99+
100+
```text
101+
<!-- count: 3, succeeded: 2, failed: 1 -->
102+
<!-- task[1]: https://example.com/hello -->
103+
...markdown...
104+
<!-- /task[1] -->
105+
<!-- task[2](failed): https://abc.com -->
106+
<!-- error[2]: ... -->
107+
```
108+
109+
退出码:全部成功为 `0`,部分或全部失败为 `1`,参数/用法错误为 `2`
96110

97111
## 构建
98112

skills/agent-fetch/SKILL.md

Lines changed: 20 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,8 @@ Browser runtime requirement:
5555
4. Use `static` mode when browser execution is not desired.
5656
5. Use `raw` mode when the exact HTTP response body is needed.
5757
6. Add repeated `--header` flags for auth/session requirements.
58-
7. Tune `--timeout`, `--browser-timeout`, `--network-idle`, and `--max-body-bytes` for slow or large pages.
58+
7. Tune `--timeout`, `--browser-timeout`, `--network-idle`, `--max-body-bytes`, and `--concurrency` for slow or large batches.
59+
8. For multiple URLs, rely on task markers in output to map each result back to its input URL.
5960

6061
## Command patterns
6162

@@ -65,6 +66,12 @@ Default:
6566
agent-fetch https://example.com
6667
```
6768

69+
Multiple URLs with shared flags:
70+
71+
```bash
72+
agent-fetch --mode static --concurrency 4 https://example.com https://example.org
73+
```
74+
6875
Static-only:
6976

7077
```bash
@@ -92,5 +99,15 @@ agent-fetch --header "Authorization: Bearer <token>" https://example.com
9299
## Output contract
93100

94101
- Read fetched content from `stdout`.
95-
- Read errors from `stderr`.
96-
- Treat non-zero exit status as fetch failure.
102+
- For multiple URLs, parse output sections in this shape:
103+
104+
```text
105+
<!-- count: N, succeeded: X, failed: Y -->
106+
<!-- task[1]: <input-url> -->
107+
...markdown...
108+
<!-- /task[1] -->
109+
<!-- task[2](failed): <input-url> -->
110+
<!-- error[2]: <error text> -->
111+
```
112+
113+
- Treat exit code `0` as all successful, `1` as partial/complete task failure, and `2` as argument/usage error.

0 commit comments

Comments
 (0)