Skip to content

Commit 084b0b7

Browse files
lidge-junlidge-jun
committed
docs(devlog): archive finished plan units and record codex-rs research
Add a source-backed codex-rs realtime/subagent research unit from the 2026-07-23 local pull, inventory every open plan folder from live evidence, and move only proven-finished units into _fin.
1 parent 50ffb78 commit 084b0b7

875 files changed

Lines changed: 66579 additions & 0 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎devlog/_fin/140_remaining-provider-ports/00_overview.md‎

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
# 140.00 — Overview: Remaining jawcode Provider Ports into opencodex
2+
3+
## What this phase is
4+
5+
After prior opencodex port cycles, jawcode still ships **five live providers** that opencodex
6+
does not route today. This phase surveys jawcode's wire/auth/streaming implementations, maps
7+
each provider onto opencodex's **five-adapter** architecture, and produces a sequenced port
8+
plan. It is a **foundation cycle** (research + decision) — three docs (00–02) only.
9+
10+
**No production code changes in this cycle.** Implementation is a later, approval-gated step
11+
(mirror phases `110/00_overview.md:13-14` and `120/00_overview.md:10-11`).
12+
13+
## TL;DR
14+
15+
1. opencodex resolves upstream calls through **exactly five adapters** today
16+
(`server.ts:72-86`: `openai-chat`, `anthropic`, `openai-responses`, `google`, `azure-openai`).
17+
Every remaining port either **extends** one of those adapters (auth/URL/body variants) or
18+
requires a **new adapter id** plus `resolveAdapter()` wiring — there is no sixth slot yet.
19+
2. Of the five un-ported providers, **two are Gemini-family with reuse paths**:
20+
`google-vertex` (Vertex `streamGenerateContent` + GCP ADC) and `google-antigravity` (Cloud
21+
Code Assist OAuth envelope over Gemini-shaped SSE). Both are **MEDIUM** effort if the
22+
existing `google` adapter gains pluggable auth/URL/body hooks (`adapters/google.ts:89-117`).
23+
3. **Three are structurally non-OpenAI-compatible**: `cursor` (HTTP/2 Connect+protobuf agent
24+
protocol), `amazon-bedrock` (SigV4 + `bedrock-converse-stream` eventstream), `kiro`
25+
(CodeWhisperer streaming + Bearer + eventstream). Each needs a **new adapter** — **HARD**.
26+
4. Recommended sequencing: **vertex → antigravity → bedrock → kiro → cursor** (easiest auth/wire
27+
reuse first; cursor last because of bidirectional exec/MCP and conversation state).
28+
5. Model surface (jawcode `models.json` + static lists): **119** bedrock, **145** cursor,
29+
**15** antigravity, **13** vertex; **kiro** has **8** static models in
30+
`special.ts:82-91` (none in `models.json`).
31+
32+
## The five-adapter constraint
33+
34+
| Adapter | jawcode APIs it can cover (partially) | Auth modes opencodex supports today |
35+
|---------|--------------------------------------|-------------------------------------|
36+
| `openai-chat` | OpenAI-compatible chat/completions | `key`, `oauth` (`oauth/index.ts:19-64`) |
37+
| `anthropic` | Anthropic Messages | `key`, `oauth` |
38+
| `openai-responses` | Codex/Responses passthrough | `forward`, `key` |
39+
| `google` | `google-generative-ai` (AI Studio) | `key` (`x-goog-api-key`, `google.ts:113`) |
40+
| `azure-openai` | Azure Responses passthrough | `key` |
41+
42+
The `ProviderAdapter` contract (`adapters/base.ts:8-20`) is small: `buildRequest`,
43+
`parseStream`, optional `parseResponse`. Ports must implement that surface (then the existing
44+
bridge in `server.ts` re-encodes to Responses SSE).
45+
46+
## The five un-ported providers (scope of 140)
47+
48+
| Provider | jawcode `Api` | Alive? | Port axis |
49+
|----------|---------------|--------|-----------|
50+
| `google-antigravity` | `google-gemini-cli` (shared wire) | **Yes** — active product | Extend `google` + OAuth |
51+
| `google-vertex` | `google-vertex` | Yes | Extend `google` + GCP auth |
52+
| `cursor` | `cursor-agent` | Yes | **New adapter** + Cursor OAuth |
53+
| `amazon-bedrock` | `bedrock-converse-stream` | Yes | **New adapter** + AWS SigV4 |
54+
| `kiro` | `kiro-streaming` | Yes | **New adapter** + Kiro token import |
55+
56+
**Explicitly out of this list:** `openai-codex` — already covered by opencodex native `openai`
57+
forward + `openai-apikey` options. **`google-gemini-cli`** — excluded as dead per product
58+
direction (Antigravity supersedes it for Cloud Code Assist OAuth).
59+
60+
## Why some ports are hard
61+
62+
| Factor | vertex / antigravity | bedrock / kiro | cursor |
63+
|--------|---------------------|----------------|--------|
64+
| Wire shape | Gemini SSE (`alt=sse`) | Amazon eventstream JSON | Connect+proto over HTTP/2 |
65+
| OpenAI-compat | No (Gemini body) | No | No — agent RPC, not chat |
66+
| Auth | GCP OAuth / ADC | AWS keys / Bearer | Cursor OAuth poll |
67+
| Streaming quirks | CCA wrapper vs Vertex URL | Block-indexed converse events | Bidirectional exec + KV blobs |
68+
| Tool loop | functionCall in SSE | toolUse blocks | Server-driven exec messages |
69+
70+
Cursor is the outlier: jawcode's `cursor.ts` is ~2.6k lines because the upstream is an **agent
71+
runtime** (`AgentService/Run` at `cursor.ts:357-368`), not a completion endpoint. opencodex
72+
would need either a minimal exec stub or a deliberate "text-only, no server tools" subset.
73+
74+
## Relationship to prior phases
75+
76+
| Phase | Axis | Relevance to 140 |
77+
|-------|------|------------------|
78+
| 100 | catalog/policy/error parity (HTTP) | Routed catalog entries need provider config + models |
79+
| 110 | SSE lifecycle reliability | New adapters inherit bridge RC1–RC3 obligations |
80+
| 120 | WebSocket transport parity | Independent; new providers start HTTP/SSE only |
81+
| **140** | **Last jawcode provider ports** | this phase — planning only |
82+
83+
## Scope & baseline
84+
85+
- **In scope:** per-provider survey (`01_`), port design + sequencing + risk table (`02_`).
86+
- **Out of scope (this cycle):** adapter implementation, OAuth CLI/GUI flows, catalog sync
87+
changes, tests, or config defaults. Those ship only after explicit approval.
88+
- **Evidence baseline:** jawcode at `packages/ai/src/providers/*.ts`, `types.ts:35-61`,
89+
`descriptors.ts:287-304`; opencodex at `src/adapters/`, `src/oauth/`, `src/server.ts:72-86`.
90+
91+
## Documents
92+
93+
- `01_provider-survey.md` — wire/auth/streaming/quirks per provider with jawcode file:line evidence
94+
- `02_port-plan.md` — adapter reuse vs new, auth plan, difficulty table, sequencing, risks

‎devlog/_fin/140_remaining-provider-ports/01_provider-survey.md‎

Lines changed: 261 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,261 @@
1+
# 140.01 — Provider Survey (jawcode source of truth)
2+
3+
Evidence from jawcode implementations and model catalogs. Model counts are rows in
4+
`packages/ai/src/models.json` unless noted as static.
5+
6+
## Summary table
7+
8+
| Provider | Models | jawcode `Api` | Primary implementation | Auth mechanism |
9+
|----------|--------|---------------|------------------------|----------------|
10+
| `google-antigravity` | **15** | `google-gemini-cli` | `google-gemini-cli.ts` (shared) | OAuth JSON credential + Bearer |
11+
| `google-vertex` | **13** | `google-vertex` | `google-vertex.ts` + `google-shared` | API key **or** GCP ADC Bearer |
12+
| `amazon-bedrock` | **119** | `bedrock-converse-stream` | `amazon-bedrock.ts` | AWS SigV4 (profile/env/ADC chain) |
13+
| `kiro` | **8** static | `kiro-streaming` | `kiro.ts` | Bearer (+ kiro-cli SQLite / refresh) |
14+
| `cursor` | **145** | `cursor-agent` | `cursor.ts` (+ `cursor/gen/agent_pb`) | Cursor OAuth access token |
15+
16+
Descriptor defaults (`descriptors.ts:301-304`, `291`): bedrock
17+
`us.anthropic.claude-opus-4-6-v1`, antigravity `gemini-3-pro-high`, vertex
18+
`gemini-3-pro-preview`, cursor `claude-sonnet-4-6`, kiro `kiro-auto`.
19+
20+
---
21+
22+
## 1. `google-antigravity`
23+
24+
### Status
25+
26+
**Alive.** Shares the Cloud Code Assist stream implementation with `google-gemini-cli`; provider
27+
slug is switched at runtime (`google-gemini-cli.ts:1-5`, `291`).
28+
29+
### Wire protocol
30+
31+
- **Endpoint:** Antigravity daily/sandbox fallbacks
32+
(`google-gemini-cli.ts:69-72`, `310`): `daily-cloudcode-pa.googleapis.com`,
33+
`daily-cloudcode-pa.sandbox.googleapis.com`.
34+
- **Path:** `POST …/v1internal:streamGenerateContent?alt=sse` (`338`).
35+
- **Body envelope:** `CloudCodeAssistRequest` — top-level `{ project, model, request, requestType?, userAgent?, requestId? }`
36+
(`195-222`, `785-796`). Antigravity adds `requestType: "agent"`, `userAgent: "antigravity"`,
37+
`requestId: "agent-{uuid}"` (`789-794`).
38+
- **Inner request:** Gemini-shaped `contents`, `systemInstruction`, `generationConfig`,
39+
`tools`, `toolConfig` (`727-756`).
40+
- **Streaming:** SSE JSON chunks with nested `response.candidates[].content.parts`
41+
(`394-501`) — text, `thought`/`thoughtSignature`, `functionCall`.
42+
43+
### Auth
44+
45+
- Requires OAuth credential JSON in `options.apiKey` (`286-288`).
46+
- Parsed via `parseGeminiCliCredentials` → `{ accessToken, projectId, refreshToken?, expiresAt? }`
47+
(`143-179`).
48+
- `Authorization: Bearer ${accessToken}` (`320`).
49+
- Login/onboard: `utils/oauth/google-antigravity.ts` — Google OAuth + `loadCodeAssist` /
50+
`onboardUser` project provisioning (`116-172`), stores project id in credential blob.
51+
- Refresh skew: 60s for antigravity (`89`, `182-192`).
52+
53+
### Streaming / fidelity quirks
54+
55+
| Quirk | Evidence |
56+
|-------|----------|
57+
| Antigravity session id derived from first user text hash | `653-659`, `731-733` |
58+
| Claude on Antigravity: `anthropic-beta` header, `VALIDATED` tool mode | `95-97`, `324`, `765-770` |
59+
| System instruction injection for Claude/Gemini-3 | `773-783` |
60+
| Tool schema: `parametersJsonSchema` → `parameters` via `normalizeSchemaForCCA` | `662-678` |
61+
| Empty-stream retry (up to 2) | `513-557` |
62+
| Endpoint failover on retry attempts | `337-338` |
63+
| Deletes `maxOutputTokens` for non-Claude antigravity models | `758-763` |
64+
65+
### Model discovery
66+
67+
Dynamic via `fetchAntigravityDiscoveryModels` when OAuth token present
68+
(`google.ts:47-66`, `markUnlistedOutsideDynamic: true`).
69+
70+
### opencodex gap
71+
72+
Existing `google` adapter targets AI Studio URL
73+
(`google.ts:108-110`: `/v1beta/models/{id}:streamGenerateContent`) with flat Gemini body and
74+
`x-goog-api-key` — **not** the CCA envelope or Bearer auth.
75+
76+
---
77+
78+
## 2. `google-vertex`
79+
80+
### Wire protocol
81+
82+
- Delegates to `streamGoogleGenAI` with `api: "google-vertex"` (`google-vertex.ts:24-28`).
83+
- **Two URL modes:**
84+
- API key: `https://aiplatform.googleapis.com/v1/publishers/google/models/{id}:streamGenerateContent?alt=sse`
85+
with `x-goog-api-key` (`37-44`).
86+
- ADC: `https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}/publishers/google/models/{id}:streamGenerateContent?alt=sse`
87+
with `Authorization: Bearer` (`47-57`).
88+
- **Body:** standard `buildGoogleGenerateContentParams` (Gemini generateContent shape via
89+
`google-shared`).
90+
- **Streaming:** same SSE JSON as AI Studio path inside `streamGoogleGenAI`.
91+
92+
### Auth
93+
94+
- API key: `GOOGLE_CLOUD_API_KEY` or real-looking `options.apiKey` (`61-67`).
95+
- ADC: `getVertexAccessToken` from `google-auth.ts:1-13` — service account JWT, authorized_user
96+
refresh, or GCE metadata (`6-9`).
97+
- Project/location required for ADC path (`69-87`).
98+
99+
### Quirks
100+
101+
- `retainTextSignature: true` for vertex (`28`) — thought signatures preserved for tool/thinking
102+
continuity.
103+
- No dynamic model discovery yet (`google.ts:36-44` comment).
104+
105+
### opencodex gap
106+
107+
`google` adapter hardcodes AI Studio base URL and API key header. Vertex needs configurable
108+
base host, path prefix (`projects/…/locations/…`), and Bearer injection from ADC refresh.
109+
110+
---
111+
112+
## 3. `amazon-bedrock`
113+
114+
### Wire protocol
115+
116+
- **API:** Bedrock Runtime `ConverseStream` — `POST /model/{modelId}/converse-stream`
117+
(`amazon-bedrock.ts:217-218`).
118+
- **Accept:** `application/vnd.amazon.eventstream` (`244`).
119+
- **Body:** `ConverseStreamRequest` — `messages`, optional `system`, `inferenceConfig`,
120+
`toolConfig`, `additionalModelRequestFields` for thinking (`204-214`, `763-810`).
121+
- **Streaming:** AWS eventstream frames decoded by `decodeEventStream` (`276`); event types
122+
`messageStart`, `contentBlockStart/Delta/Stop`, `messageStop`, `metadata` (`298-334`).
123+
124+
### Auth
125+
126+
- `resolveAwsCredentials` with optional profile (`233-237`); test bypass
127+
`AWS_BEDROCK_SKIP_AUTH` (`230-231`).
128+
- Request signed via custom `signRequest` (SigV4, no AWS SDK) (`246-255`).
129+
130+
### Message / tool mapping
131+
132+
- Messages converted to Bedrock wire roles with cache points for Anthropic models
133+
(`570-714`).
134+
- Tool results batched into single user message (`662-696`).
135+
- Thinking: `additionalModelRequestFields.thinking` with adaptive vs budget modes
136+
(`763-810`); `thinkingDisplay` default `"summarized"` (`65`, `779`).
137+
138+
### Model catalog
139+
140+
- **119** models in `models.json`.
141+
- Discovery via models.dev key `amazon-bedrock` (`openai-compat.ts:2170-2204`) with cross-region
142+
id transform and EU variant duplication for Claude (`2184-2202`).
143+
144+
### opencodex gap
145+
146+
No Bedrock adapter, no SigV4 signer, no eventstream parser in opencodex. Cannot reuse
147+
`anthropic` adapter — request/response shapes differ despite similar message semantics.
148+
149+
---
150+
151+
## 4. `kiro`
152+
153+
### Wire protocol
154+
155+
- **Host:** `https://runtime.{region}.kiro.dev/` (`kiro.ts:49-50`, `545-546`).
156+
- **Headers:** AWS-style `x-amz-target:
157+
AmazonCodeWhispererStreamingService.GenerateAssistantResponse` (`50`, `92`).
158+
- **Content-Type:** `application/x-amz-json-1.0` (`90`).
159+
- **Accept:** `application/vnd.amazon.eventstream` (`91`).
160+
- **Body:** `conversationState` with `history`, `currentMessage.userInputMessage`, optional
161+
`tools` / `toolResults` in context (`145-240`).
162+
- **Streaming:** shared `decodeEventStream` (`591`); payload JSON heuristics in
163+
`parseKiroPayload` (`256-305`) — `content`, tool `{name,input,toolUseId}`, `{stop:true}`,
164+
`{usage}`.
165+
166+
### Auth
167+
168+
- Multi-source resolver `resolveKiroAuth` (`392-487`): explicit token, `aoa*` apiKey prefix,
169+
`KIRO_ACCESS_TOKEN`, cache + refresh, prokiro `auth.json`, kiro-cli SQLite
170+
(`328-368`, `462-484`).
171+
- Refresh: `POST https://prod.{region}.auth.desktop.kiro.dev/refreshToken` (`312`, `371-389`).
172+
- **Anti-detection:** machine fingerprint in User-Agent (`72-83`, `85-98`).
173+
174+
### Quirks
175+
176+
| Quirk | Evidence |
177+
|-------|----------|
178+
| Model id mapping table (`kiro-auto` → `auto`, etc.) | `728-749` |
179+
| Stable `conversationId` hash from messages | `751-765` |
180+
| Tool name truncated to 64 chars | `121` |
181+
| No thinking stream — text + tools only | `602-671` |
182+
| 401 → refresh + single retry | `562-579` |
183+
184+
### Models
185+
186+
**8** static entries in `special.ts:82-91` (not in `models.json`). Default `kiro-auto`
187+
(`descriptors.ts:222`).
188+
189+
### opencodex gap
190+
191+
Proprietary payload + IDE impersonation headers. Eventstream decoder could be **shared** with
192+
Bedrock port, but request builder and auth import path are Kiro-specific.
193+
194+
---
195+
196+
## 5. `cursor`
197+
198+
### Wire protocol
199+
200+
- **Not an LLM REST API.** HTTP/2 Connect to `AgentService/Run` (`cursor.ts:133-134`, `355-368`).
201+
- **Content-Type:** `application/connect+proto` (`360`).
202+
- **Framing:** 5-byte Connect frames (`175-180`, `412-421`); protobuf payloads
203+
(`AgentClientMessage` / `AgentServerMessage`).
204+
- **Streaming:** `interactionUpdate` cases — `textDelta`, `thinkingDelta`, `toolCallStarted`,
205+
`toolCallDelta`, `toolCallCompleted`, `turnEnded` (`1955-2098`).
206+
- **Bidirectional:** server sends `execServerMessage`, `kvServerMessage`; client must respond
207+
(`611-622`, `968-1203`).
208+
209+
### Auth
210+
211+
- Bearer access token required (`337-340`).
212+
- OAuth: PKCE + browser login + poll (`utils/oauth/cursor.ts:4-76`); refresh via
213+
`exchange_user_api_key` (`98+`).
214+
- Descriptor: `oauthProvider: "cursor"` (`descriptors.ts:291`).
215+
216+
### Conversation / prompt state
217+
218+
- `buildGrpcRequest` maintains `conversationState`, blob store, `rootPromptMessagesJson`
219+
(`2509-2640`) — critical for multi-turn (`2273-2276`).
220+
- Host override system prompt so model doesn't assume Cursor IDE (`2293-2297`).
221+
- Tools advertised via `requestContext` exec handshake, not in initial Run request
222+
(`2616-2617`, `977-998`).
223+
224+
### Exec / tools (major complexity)
225+
226+
- Shell, read, write, grep, ls, mcp, etc. handlers (`1005-1167`).
227+
- Without `execHandlers`, tools return "Tool not available" / "Not implemented"
228+
(`1256-1257`, `1108`, `1135`).
229+
- Heartbeat every 5s (`468-479`).
230+
231+
### Models
232+
233+
**145** entries in `models.json`. Dynamic discovery when API key present
234+
(`special.ts:46-58` → `fetchCursorUsableModels`).
235+
236+
### opencodex gap
237+
238+
Requires new adapter with HTTP/2 + protobuf + optional exec bridge. Cannot map to any existing
239+
five adapters. Highest risk: Codex tool calls vs Cursor native tool loop mismatch.
240+
241+
---
242+
243+
## Cross-reference: jawcode type system
244+
245+
`types.ts:35-61` registers all five APIs in `KnownApi` / `ApiOptionsMap`. Provider slugs
246+
(`types.ts:101-149`) include all five survey subjects. Stream dispatch:
247+
`register-builtins.ts:358-371` (gemini-cli/antigravity), `366-371` (vertex), plus bedrock,
248+
cursor, kiro lazy loaders in the same file.
249+
250+
## opencodex adapter interface (target shape)
251+
252+
Every port must satisfy (`adapters/base.ts:8-20`):
253+
254+
```typescript
255+
buildRequest(parsed, incoming?) → { url, method, headers, body }
256+
parseStream(response) → AsyncGenerator<AdapterEvent>
257+
parseResponse?(response) → Promise<AdapterEvent[]>
258+
```
259+
260+
OAuth pattern reference: `oauth/index.ts:11-17` (`login`, `refresh`, `providerConfig`,
261+
`defaultModel`) — exemplar flow in `oauth/xai.ts:1-71` (PKCE, discovery, token refresh).

0 commit comments

Comments
 (0)