Skip to content

Commit 7869bdb

Browse files
authored
feat(browser): polish adapter author verify workflow
1 parent 2e93ac6 commit 7869bdb

13 files changed

Lines changed: 330 additions & 24 deletions

README.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -97,7 +97,7 @@ If you want to add your own commands, start with the [Extending OpenCLI guide](.
9797
|------|------------------|
9898
| Keep personal website commands in your own Git repo | `opencli plugin create` + `opencli plugin install file://...` |
9999
| Quickly draft a private local adapter | `opencli browser init <site>/<command>` in `~/.opencli/clis/` |
100-
| Modify an official adapter locally | `opencli adapter eject/status/reset` |
100+
| Modify an official adapter locally | `opencli adapter eject <site>` + `opencli adapter reset <site>` |
101101
| Publish or install third-party commands | `opencli plugin install github:user/repo` |
102102
| Wrap an existing local binary | `opencli external register <name>` |
103103

@@ -174,7 +174,7 @@ When the site you need is not yet covered, use the `opencli-adapter-author` skil
174174
2. Discover the right endpoint — network inspection, initial state, bundle search, token trace, or interceptor fallback.
175175
3. Decide the auth strategy — `PUBLIC` / `COOKIE` / `HEADER` / `INTERCEPT`.
176176
4. Decode response fields and design output columns.
177-
5. `opencli browser init <site>/<name>` → write adapter → `opencli browser verify <site>/<name>`.
177+
5. `opencli browser analyze <url>` for one-shot recon, then `opencli browser init <site>/<name>` → write adapter → `opencli browser verify <site>/<name>`.
178178
6. Persist site knowledge to `~/.opencli/sites/<site>/` so the next adapter for the same site is faster.
179179

180180
### CLI Hub and desktop adapters
@@ -407,7 +407,7 @@ Before writing any adapter code, read the [`opencli-adapter-author` skill](./ski
407407
- Recon the site and pick a pattern (SPA / SSR / JSONP / Token / Streaming).
408408
- Discover the right endpoint via `opencli browser network`, `eval`, or the interceptor fallback.
409409
- Decide auth strategy (`PUBLIC` / `COOKIE` / `HEADER` / `INTERCEPT`).
410-
- Decode response fields, design columns, scaffold with `opencli browser init`.
410+
- Run `opencli browser analyze <url>` for one-shot recon, decode response fields, design columns, scaffold with `opencli browser init`.
411411
- Verify with `opencli browser verify <site>/<name>` before shipping.
412412

413413
For long-lived personal commands that should live in your own Git repo, use a local plugin instead; see [Extending OpenCLI](./docs/guide/extending-opencli.md). Quick private adapters can still live at `~/.opencli/clis/<site>/<name>.js`. Site knowledge (endpoints, field maps, fixtures) accumulates in `~/.opencli/sites/<site>/` so the next adapter for the same site starts from context instead of zero.

README.zh-CN.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -81,7 +81,7 @@ opencli bilibili hot --limit 5
8181
|------|----------|
8282
| 把个人网站命令放在自己的 Git repo | `opencli plugin create` + `opencli plugin install file://...` |
8383
| 快速写一个本机私人 adapter | `opencli browser init <site>/<command>`,放在 `~/.opencli/clis/` |
84-
| 本地修改官方 adapter | `opencli adapter eject/status/reset` |
84+
| 本地修改官方 adapter | `opencli adapter eject <site>` + `opencli adapter reset <site>` |
8585
| 发布或安装第三方命令 | `opencli plugin install github:user/repo` |
8686
| 包装已有本机 binary | `opencli external register <name>` |
8787

@@ -158,7 +158,7 @@ Agent 在内部自动处理所有 `opencli browser` 命令——你只需用自
158158
2. 发现目标 endpoint——network 精读、initial state、bundle 搜索、token 溯源,或 interceptor 兜底
159159
3. 定认证策略——`PUBLIC` / `COOKIE` / `HEADER` / `INTERCEPT`
160160
4. 字段解码 + 设计输出列
161-
5. `opencli browser init <site>/<name>` → 写适配器 → `opencli browser verify <site>/<name>`
161+
5. `opencli browser analyze <url>` 一步侦察,再 `opencli browser init <site>/<name>` → 写适配器 → `opencli browser verify <site>/<name>`
162162
6. 把站点知识沉到 `~/.opencli/sites/<site>/`,下次写同站点的其他命令直接吃缓存
163163

164164
### CLI 枢纽与桌面端适配器
@@ -505,7 +505,7 @@ opencli plugin uninstall my-tool # 卸载
505505
- 侦察站点,选定 pattern(SPA / SSR / JSONP / Token / Streaming)
506506
-`opencli browser network``eval`、interceptor 等找到目标 endpoint
507507
- 定认证策略(`PUBLIC` / `COOKIE` / `HEADER` / `INTERCEPT`
508-
- 字段解码、设计 columns、`opencli browser init` 生成骨架
508+
- 先用 `opencli browser analyze <url>` 一步侦察,再字段解码、设计 columns、`opencli browser init` 生成骨架
509509
- 交付前用 `opencli browser verify <site>/<name>` 验证
510510

511511
在仓库外写的私有适配器放到 `~/.opencli/clis/<site>/<name>.js`;每个站点的 endpoint、字段映射、抓包样本会累积在 `~/.opencli/sites/<site>/`,下次写同站点的其他命令可以直接复用。

docs/developer/ai-workflow.md

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -11,10 +11,11 @@ From a new site URL to a passing `opencli browser verify` — one skill, one set
1111
# skills/opencli-adapter-author/SKILL.md
1212

1313
# 2. Reconnaissance
14-
opencli browser open https://example.com
15-
opencli browser wait time 3
16-
opencli browser network # inspect XHR / fetch calls
17-
opencli browser state # extract __INITIAL_STATE__ / __NEXT_DATA__
14+
opencli browser analyze https://example.com
15+
# Fallback primitives when analyze says deeper inspection is needed:
16+
# opencli browser open https://example.com
17+
# opencli browser network # inspect XHR / fetch calls
18+
# opencli browser state # extract __INITIAL_STATE__ / __NEXT_DATA__
1819

1920
# 3. Scaffold + verify
2021
opencli browser init <site>/<name>
@@ -30,6 +31,7 @@ See [skills/opencli-adapter-author/SKILL.md](https://github.com/jackwener/opencl
3031
| Command | Purpose |
3132
|---------|---------|
3233
| `opencli doctor` | Sanity check: bridge, Chrome, signals |
34+
| `opencli browser analyze <url>` | One-shot site recon: anti-bot, pattern, nearest adapter, next step |
3335
| `opencli browser open <url>` | Open a tab in the Chrome session |
3436
| `opencli browser network` | List recent XHR / fetch calls |
3537
| `opencli browser state` | Page state: URL, title, interactive elements |

docs/guide/extending-opencli.md

Lines changed: 14 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -45,18 +45,30 @@ User adapters are loaded from:
4545

4646
This path is convenient for quick local automation. For code you want to version, review, or share, prefer a plugin.
4747

48+
If the command takes required positional args and no fixture exists yet, seed the first verify run explicitly:
49+
50+
```bash
51+
opencli browser verify instagram/collection-create --write-fixture --seed-args opencli-verify
52+
opencli browser verify example/detail --write-fixture --seed-args '["https://example.com/item/1", "--limit", 3]'
53+
```
54+
55+
`--seed-args` is only used when the fixture has no `args`. Once the fixture is written, `opencli browser verify` reads args from `~/.opencli/sites/<site>/verify/<command>.json`.
56+
4857
## Local overrides for official adapters
4958

5059
Use `adapter eject` when you want to customize an existing official adapter.
5160

5261
```bash
5362
opencli adapter eject twitter
54-
opencli adapter status
5563
# edit ~/.opencli/clis/twitter/*.js
5664
opencli adapter reset twitter
5765
```
5866

59-
Ejected adapters override the packaged official adapter on this machine. `adapter reset` removes the local override and returns to the packaged version.
67+
Files in `~/.opencli/clis/<site>/<command>.js` override packaged adapters with the same `site/command` on this machine. `opencli browser verify <site>/<command>` also runs the local override, so a passing local verify does not prove that the packaged adapter was changed.
68+
69+
The packaged `cli-manifest.json` only describes bundled adapters. User adapters are discovered at runtime and do not need manifest entries.
70+
71+
After copying a local fix into the repository for a PR, remove the local copy or run `opencli adapter reset <site>` after merge. Otherwise the local file keeps shadowing future package updates. `opencli doctor` warns when it detects this shadowing.
6072

6173
## Plugins for sharing commands
6274

docs/zh/guide/extending-opencli.md

Lines changed: 14 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -45,18 +45,30 @@ User adapter 加载路径是:
4545

4646
这条路径适合快速本地自动化。需要版本管理、review、共享的代码推荐做成 plugin。
4747

48+
如果命令有 required positional args,而且 fixture 还没创建,第一次 verify 时直接传 seed:
49+
50+
```bash
51+
opencli browser verify instagram/collection-create --write-fixture --seed-args opencli-verify
52+
opencli browser verify example/detail --write-fixture --seed-args '["https://example.com/item/1", "--limit", 3]'
53+
```
54+
55+
`--seed-args` 只在 fixture 没有 `args` 时生效。fixture 写出后,`opencli browser verify` 会从 `~/.opencli/sites/<site>/verify/<command>.json` 读取 args。
56+
4857
## 本地覆盖官方 adapter
4958

5059
如果你想改一个已有官方 adapter,用 `adapter eject`
5160

5261
```bash
5362
opencli adapter eject twitter
54-
opencli adapter status
5563
# edit ~/.opencli/clis/twitter/*.js
5664
opencli adapter reset twitter
5765
```
5866

59-
Ejected adapter 会在本机覆盖 package 里的官方 adapter。`adapter reset` 会移除本地覆盖,恢复到 package 版本。
67+
`~/.opencli/clis/<site>/<command>.js` 会在本机覆盖同名 package adapter。`opencli browser verify <site>/<command>` 也会跑本地覆盖版本,所以本地 verify 通过不代表 package 里的 adapter 已经改好。
68+
69+
Package 里的 `cli-manifest.json` 只描述 bundled adapter。User adapter 是运行时发现的,不需要写 manifest。
70+
71+
把本地修复复制到仓库发 PR 后,merge 后要删除本地副本,或运行 `opencli adapter reset <site>`。否则本地文件会继续 shadow 后续 package 更新。`opencli doctor` 会在发现这种 shadowing 时给出 warning。
6072

6173
## Plugin:共享命令
6274

src/adapter-shadow.test.ts

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
import { describe, expect, it } from 'vitest';
2+
import * as fs from 'node:fs';
3+
import * as os from 'node:os';
4+
import * as path from 'node:path';
5+
import { findShadowedUserAdapters, formatAdapterShadowIssue } from './adapter-shadow.js';
6+
7+
describe('adapter shadow detection', () => {
8+
it('reports user adapters that shadow packaged manifest commands', () => {
9+
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'opencli-adapter-shadow-'));
10+
try {
11+
const userClisDir = path.join(root, 'user-clis');
12+
const builtinRoot = path.join(root, 'pkg');
13+
const builtinClisDir = path.join(builtinRoot, 'clis');
14+
fs.mkdirSync(path.join(userClisDir, 'instagram'), { recursive: true });
15+
fs.mkdirSync(path.join(userClisDir, 'twitter'), { recursive: true });
16+
fs.mkdirSync(path.join(builtinClisDir, 'instagram'), { recursive: true });
17+
fs.mkdirSync(path.join(builtinClisDir, 'twitter'), { recursive: true });
18+
19+
fs.writeFileSync(path.join(userClisDir, 'instagram', 'saved.js'), '', 'utf-8');
20+
fs.writeFileSync(path.join(userClisDir, 'instagram', 'utils.js'), '', 'utf-8');
21+
fs.writeFileSync(path.join(userClisDir, 'twitter', 'search.js'), '', 'utf-8');
22+
fs.writeFileSync(path.join(builtinClisDir, 'instagram', 'saved.js'), '', 'utf-8');
23+
fs.writeFileSync(path.join(builtinClisDir, 'instagram', 'utils.js'), '', 'utf-8');
24+
fs.writeFileSync(path.join(builtinClisDir, 'twitter', 'search.js'), '', 'utf-8');
25+
fs.writeFileSync(path.join(builtinRoot, 'cli-manifest.json'), `${JSON.stringify([
26+
{ site: 'instagram', name: 'saved', sourceFile: 'instagram/saved.js' },
27+
])}\n`, 'utf-8');
28+
29+
expect(findShadowedUserAdapters({ userClisDir, builtinClisDir })).toEqual([
30+
{
31+
name: 'instagram/saved',
32+
userPath: path.join(userClisDir, 'instagram', 'saved.js'),
33+
builtinPath: path.join(builtinClisDir, 'instagram', 'saved.js'),
34+
},
35+
]);
36+
} finally {
37+
fs.rmSync(root, { recursive: true, force: true });
38+
}
39+
});
40+
41+
it('formats a concise doctor issue', () => {
42+
const issue = formatAdapterShadowIssue([
43+
{
44+
name: 'instagram/saved',
45+
userPath: '/home/me/.opencli/clis/instagram/saved.js',
46+
builtinPath: '/pkg/clis/instagram/saved.js',
47+
},
48+
]);
49+
50+
expect(issue).toContain('instagram/saved');
51+
expect(issue).toContain('opencli adapter reset <site>');
52+
});
53+
});

src/adapter-shadow.ts

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
import * as fs from 'node:fs';
2+
import * as os from 'node:os';
3+
import * as path from 'node:path';
4+
import { fileURLToPath } from 'node:url';
5+
import type { ManifestEntry } from './manifest-types.js';
6+
import { findPackageRoot, getCliManifestPath } from './package-paths.js';
7+
8+
export type AdapterShadow = {
9+
name: string;
10+
userPath: string;
11+
builtinPath: string;
12+
};
13+
14+
export type AdapterShadowOptions = {
15+
userClisDir?: string;
16+
builtinClisDir?: string;
17+
};
18+
19+
function defaultBuiltinClisDir(): string {
20+
return path.join(findPackageRoot(fileURLToPath(import.meta.url)), 'clis');
21+
}
22+
23+
function safeReaddir(dir: string): fs.Dirent[] {
24+
try {
25+
return fs.readdirSync(dir, { withFileTypes: true });
26+
} catch {
27+
return [];
28+
}
29+
}
30+
31+
function loadBuiltinCommandFiles(builtinClisDir: string): Set<string> {
32+
try {
33+
const raw = fs.readFileSync(getCliManifestPath(builtinClisDir), 'utf-8');
34+
const entries = JSON.parse(raw) as ManifestEntry[];
35+
const files = new Set<string>();
36+
for (const entry of entries) {
37+
const rel = entry.sourceFile ?? entry.modulePath;
38+
if (rel) files.add(path.resolve(builtinClisDir, rel));
39+
}
40+
return files;
41+
} catch {
42+
return new Set();
43+
}
44+
}
45+
46+
export function findShadowedUserAdapters(opts: AdapterShadowOptions = {}): AdapterShadow[] {
47+
const userClisDir = opts.userClisDir ?? path.join(os.homedir(), '.opencli', 'clis');
48+
const builtinClisDir = opts.builtinClisDir ?? defaultBuiltinClisDir();
49+
const builtinCommandFiles = loadBuiltinCommandFiles(builtinClisDir);
50+
const shadows: AdapterShadow[] = [];
51+
52+
for (const siteEntry of safeReaddir(userClisDir)) {
53+
if (!siteEntry.isDirectory()) continue;
54+
const site = siteEntry.name;
55+
const userSiteDir = path.join(userClisDir, site);
56+
const builtinSiteDir = path.join(builtinClisDir, site);
57+
58+
for (const commandEntry of safeReaddir(userSiteDir)) {
59+
if (!commandEntry.isFile() || !commandEntry.name.endsWith('.js')) continue;
60+
const userPath = path.join(userSiteDir, commandEntry.name);
61+
const builtinPath = path.join(builtinSiteDir, commandEntry.name);
62+
const builtinResolved = path.resolve(builtinPath);
63+
if (!builtinCommandFiles.has(builtinResolved)) continue;
64+
65+
shadows.push({
66+
name: `${site}/${commandEntry.name.replace(/\.js$/, '')}`,
67+
userPath,
68+
builtinPath,
69+
});
70+
}
71+
}
72+
73+
return shadows.sort((a, b) => a.name.localeCompare(b.name));
74+
}
75+
76+
export function formatAdapterShadowIssue(shadows: AdapterShadow[]): string {
77+
const visible = shadows.slice(0, 10);
78+
const lines = ['Local adapter overrides shadow packaged adapters:'];
79+
for (const shadow of visible) {
80+
lines.push(` ${shadow.name}: ${shadow.userPath} overrides ${shadow.builtinPath}`);
81+
}
82+
if (shadows.length > visible.length) {
83+
lines.push(` ... and ${shadows.length - visible.length} more`);
84+
}
85+
lines.push('Remove the local ~/.opencli/clis copy, or run opencli adapter reset <site>, when you want packaged updates.');
86+
return lines.join('\n');
87+
}

src/browser/verify-fixture.test.ts

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
import { describe, expect, it } from 'vitest';
2-
import { deriveFixture, expandFixtureArgs, validateRows, type Fixture } from './verify-fixture.js';
2+
import { deriveFixture, expandFixtureArgs, parseSeedArgs, validateRows, type Fixture } from './verify-fixture.js';
33

44
describe('validateRows', () => {
55
it('passes when rows meet all expectations', () => {
@@ -221,3 +221,22 @@ describe('expandFixtureArgs', () => {
221221
]);
222222
});
223223
});
224+
225+
describe('parseSeedArgs', () => {
226+
it('treats plain text as one positional arg', () => {
227+
expect(parseSeedArgs('opencli-verify')).toEqual(['opencli-verify']);
228+
});
229+
230+
it('accepts JSON array seed args', () => {
231+
expect(parseSeedArgs('["subject", "--limit", 3]')).toEqual(['subject', '--limit', 3]);
232+
});
233+
234+
it('accepts JSON object seed args', () => {
235+
expect(parseSeedArgs('{"limit":3,"sort":"hot"}')).toEqual({ limit: 3, sort: 'hot' });
236+
});
237+
238+
it('ignores empty input', () => {
239+
expect(parseSeedArgs(undefined)).toBeUndefined();
240+
expect(parseSeedArgs(' ')).toBeUndefined();
241+
});
242+
});

src/browser/verify-fixture.ts

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -239,6 +239,21 @@ export function expandFixtureArgs(args: FixtureArgs | undefined): string[] {
239239
return out;
240240
}
241241

242+
export function parseSeedArgs(raw: string | undefined): FixtureArgs | undefined {
243+
if (raw === undefined) return undefined;
244+
const trimmed = raw.trim();
245+
if (!trimmed) return undefined;
246+
247+
try {
248+
const parsed = JSON.parse(trimmed) as unknown;
249+
if (Array.isArray(parsed)) return parsed;
250+
if (parsed !== null && typeof parsed === 'object') return parsed as Record<string, unknown>;
251+
return [parsed];
252+
} catch {
253+
return [raw];
254+
}
255+
}
256+
242257
function jsType(v: unknown): string {
243258
if (v === null) return 'null';
244259
if (Array.isArray(v)) return 'array';

0 commit comments

Comments
 (0)