Skip to content

Commit 604b8b2

Browse files
Merge pull request #109 from shravan-lambdatest/docs/api-calls
docs: document making API calls directly in objectives
2 parents 9c413a2 + 34e486b commit 604b8b2

10 files changed

Lines changed: 277 additions & 11 deletions

File tree

.agents/skills/kane-cli/SKILL.md

Lines changed: 14 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -199,15 +199,16 @@ kane-cli run "Go to https://app.example.com and login with {{username}} and {{pa
199199

200200
## 4. Writing objectives
201201

202-
How you phrase the objective string determines what the agent does. Three patterns:
202+
How you phrase the objective string determines what the agent does. Four patterns:
203203

204-
> For the full catalog — every action verb, every assertion analyze method (Visual / Textual-DOM / URL / Title / DevTools→Network/Console/Performance/Cookies/localStorage/Clipboard), operators, chaining, conditional/negative patterns, and worked examples — Read `references/objectives-cookbook.md`. Same grammar applies to one-shot `kane-cli run` objectives and `_test.md` step bodies.
204+
> For the full catalog — every action verb, every assertion analyze method (Visual / Textual-DOM / URL / Title / DevTools→Network/Console/Performance/Cookies/localStorage/Clipboard), direct API calls, operators, chaining, conditional/negative patterns, and worked examples — Read `references/objectives-cookbook.md`. Same grammar applies to one-shot `kane-cli run` objectives and `_test.md` step bodies.
205205
206206
| Pattern | Trigger words | Behavior |
207207
|---|---|---|
208208
| 🎯 **Action** | "go to", "click", "type", "search", "fill" | Performs browser actions |
209209
|**Assertion** | "assert", "verify", "confirm", "check that" | Pass/fail check on a condition |
210210
| 📦 **Extraction** | "store X as 'name'" | Persists a value into `run_end.final_state` |
211+
| 🔌 **API call** | "call", "POST/GET a URL", a pasted `curl` | The agent makes the HTTP request itself; "save the response as X", then assert/reference `{{X.status}}` / `{{X.response_body…}}` |
211212

212213
### The "store as" rule (critical for extraction)
213214

@@ -218,6 +219,17 @@ Vague phrasing like "read", "tell me", "report" does NOT reliably extract data
218219

219220
Stored values appear in `run_end.final_state` and become the second results table per §1.4.
220221

222+
### Calling APIs directly
223+
224+
The agent can make API calls itself — not just observe the page's traffic. Phrase an explicit call and name the response:
225+
226+
```text
227+
"Call POST https://api.example.com/login with body {...}, save the response as login,
228+
assert {{login.status}} is 200"
229+
```
230+
231+
Reference the saved response as `{{login.status}}`, `{{login.response_body}}`, or `{{login.response_body.<field>}}`; a pasted `curl` works too. Full grammar in `references/objectives-cookbook.md` §3.5.
232+
221233
### Chaining
222234

223235
Action → extraction → assertion in one objective:

.agents/skills/kane-cli/references/objectives-cookbook.md

Lines changed: 52 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
<!-- Read this when authoring objective strings — for either `kane-cli run "..."` or step bodies inside a `_test.md`. Owns the full catalog of action verbs, assertion patterns, checkpoint analyze methods (Visual, Textual/DOM, URL, Title, DevTools→Network/Console/Performance/Cookies/localStorage), operators, extraction patterns, chaining, variables, and common pitfalls. The same objective grammar applies to one-shot runs and testmd steps. -->
1+
<!-- Read this when authoring objective strings — for either `kane-cli run "..."` or step bodies inside a `_test.md`. Owns the full catalog of action verbs, assertion patterns, checkpoint analyze methods (Visual, Textual/DOM, URL, Title, DevTools→Network/Console/Performance/Cookies/localStorage), operators, extraction patterns, direct API calls, chaining, variables, and common pitfalls. The same objective grammar applies to one-shot runs and testmd steps. -->
22

33
# Writing Kane-CLI Objectives — Pattern Cookbook
44

@@ -81,6 +81,8 @@ Five subdomains. Each one is the right choice when the data you care about lives
8181

8282
The agent captures every HTTP request/response per step. **Resets each step** — assert on traffic in the same step it happens (or extract and carry forward).
8383

84+
These assertions also cover API calls **you** make directly in an objective (§3.5), not just the requests the page makes on its own — the agent records its own calls and you query them with the same fields below.
85+
8486
Queryable fields: `method`, `url`, `domain`, `path`, `query_params`, `resource_type`, `request_headers`, `request_body`, `response_status`, `response_headers`, `response_body`, `timing.duration_ms`, `timing.ttfb_ms`, `failed`, `failure_reason`.
8587

8688
```text
@@ -217,6 +219,43 @@ Assertions support these comparisons. Phrase them naturally — the agent maps t
217219

218220
If you're not sure which method, default to **Visual** — that's what the agent does too.
219221

222+
### 3.5 Calling an API directly
223+
224+
Distinct from observing the page's network traffic (§3.2): you can have the agent **make an HTTP request itself** as part of an objective — to seed data, hit a backend, or check a service — then assert on or reuse the response.
225+
226+
Phrase an explicit call and name the response with "save the response as …":
227+
228+
```text
229+
Call POST https://api.example.com/orders with body {"item": "sku_42", "qty": 1}, save the response as order
230+
Hit GET https://api.example.com/orders/123, save the response as fetched
231+
```
232+
233+
A pasted `curl` works too and is kept verbatim (method, headers, body, auth):
234+
235+
```text
236+
curl -X POST https://api.example.com/login -H 'Content-Type: application/json' -d '{"u":"a","p":"b"}', save the response as login
237+
```
238+
239+
Once saved, reference the response by name:
240+
241+
| Reference | Resolves to |
242+
|---|---|
243+
| `{{order.status}}` | the HTTP status code (e.g. `201`) |
244+
| `{{order.response_body}}` | the whole response body |
245+
| `{{order.response_body.<field>}}` | a field from the JSON response body |
246+
247+
Then assert or chain on it — API calls and browser actions mix freely in one objective:
248+
249+
```text
250+
Call POST https://api.example.com/login with body {"u": "{{user}}", "p": "{{password}}"}, save the response as login,
251+
assert {{login.status}} is 200,
252+
then open https://app.example.com and verify the dashboard loads
253+
```
254+
255+
**Direct call vs. DevTools/Network (§3.2):** use a **direct call** when *you* want the agent to make the request (seed a record, call a backend, set up state). Use **DevTools/Network** when you want to **observe the requests the page itself makes** during a UI flow. They compose: seed via a direct call, drive the UI, then assert on the page's traffic.
256+
257+
**Tokens:** put any credential/token in a variable marked `secret: true` (§6) so it's masked and never logged.
258+
220259
---
221260

222261
## 4. Extraction — the "store as" rule
@@ -394,3 +433,15 @@ Store the order number as 'order_id'.
394433
```
395434

396435
The step body is exactly the same grammar as `kane-cli run`. Everything in this cookbook applies.
436+
437+
### Example D — Seed via API, then verify in the UI
438+
439+
```text
440+
"Call POST https://api.example.com/orders with body {"item": "sku_42", "qty": 1},
441+
save the response as order,
442+
assert {{order.status}} is 201,
443+
then open https://app.example.com/orders,
444+
assert an order for 'sku_42' is visible"
445+
```
446+
447+
The agent makes the API call directly, captures the response, asserts on its status, then drives the UI to confirm the seeded data appears. (Direct call → UI verify — contrast with §3.2, where you observe the page's own traffic.)

.claude/skills/kane-cli/SKILL.md

Lines changed: 14 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -199,15 +199,16 @@ kane-cli run "Go to https://app.example.com and login with {{username}} and {{pa
199199

200200
## 4. Writing objectives
201201

202-
How you phrase the objective string determines what the agent does. Three patterns:
202+
How you phrase the objective string determines what the agent does. Four patterns:
203203

204-
> For the full catalog — every action verb, every assertion analyze method (Visual / Textual-DOM / URL / Title / DevTools→Network/Console/Performance/Cookies/localStorage/Clipboard), operators, chaining, conditional/negative patterns, and worked examples — Read `references/objectives-cookbook.md`. Same grammar applies to one-shot `kane-cli run` objectives and `_test.md` step bodies.
204+
> For the full catalog — every action verb, every assertion analyze method (Visual / Textual-DOM / URL / Title / DevTools→Network/Console/Performance/Cookies/localStorage/Clipboard), direct API calls, operators, chaining, conditional/negative patterns, and worked examples — Read `references/objectives-cookbook.md`. Same grammar applies to one-shot `kane-cli run` objectives and `_test.md` step bodies.
205205
206206
| Pattern | Trigger words | Behavior |
207207
|---|---|---|
208208
| 🎯 **Action** | "go to", "click", "type", "search", "fill" | Performs browser actions |
209209
|**Assertion** | "assert", "verify", "confirm", "check that" | Pass/fail check on a condition |
210210
| 📦 **Extraction** | "store X as 'name'" | Persists a value into `run_end.final_state` |
211+
| 🔌 **API call** | "call", "POST/GET a URL", a pasted `curl` | The agent makes the HTTP request itself; "save the response as X", then assert/reference `{{X.status}}` / `{{X.response_body…}}` |
211212

212213
### The "store as" rule (critical for extraction)
213214

@@ -218,6 +219,17 @@ Vague phrasing like "read", "tell me", "report" does NOT reliably extract data
218219

219220
Stored values appear in `run_end.final_state` and become the second results table per §1.4.
220221

222+
### Calling APIs directly
223+
224+
The agent can make API calls itself — not just observe the page's traffic. Phrase an explicit call and name the response:
225+
226+
```text
227+
"Call POST https://api.example.com/login with body {...}, save the response as login,
228+
assert {{login.status}} is 200"
229+
```
230+
231+
Reference the saved response as `{{login.status}}`, `{{login.response_body}}`, or `{{login.response_body.<field>}}`; a pasted `curl` works too. Full grammar in `references/objectives-cookbook.md` §3.5.
232+
221233
### Chaining
222234

223235
Action → extraction → assertion in one objective:

.claude/skills/kane-cli/references/objectives-cookbook.md

Lines changed: 52 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
<!-- Read this when authoring objective strings — for either `kane-cli run "..."` or step bodies inside a `_test.md`. Owns the full catalog of action verbs, assertion patterns, checkpoint analyze methods (Visual, Textual/DOM, URL, Title, DevTools→Network/Console/Performance/Cookies/localStorage), operators, extraction patterns, chaining, variables, and common pitfalls. The same objective grammar applies to one-shot runs and testmd steps. -->
1+
<!-- Read this when authoring objective strings — for either `kane-cli run "..."` or step bodies inside a `_test.md`. Owns the full catalog of action verbs, assertion patterns, checkpoint analyze methods (Visual, Textual/DOM, URL, Title, DevTools→Network/Console/Performance/Cookies/localStorage), operators, extraction patterns, direct API calls, chaining, variables, and common pitfalls. The same objective grammar applies to one-shot runs and testmd steps. -->
22

33
# Writing Kane-CLI Objectives — Pattern Cookbook
44

@@ -81,6 +81,8 @@ Five subdomains. Each one is the right choice when the data you care about lives
8181

8282
The agent captures every HTTP request/response per step. **Resets each step** — assert on traffic in the same step it happens (or extract and carry forward).
8383

84+
These assertions also cover API calls **you** make directly in an objective (§3.5), not just the requests the page makes on its own — the agent records its own calls and you query them with the same fields below.
85+
8486
Queryable fields: `method`, `url`, `domain`, `path`, `query_params`, `resource_type`, `request_headers`, `request_body`, `response_status`, `response_headers`, `response_body`, `timing.duration_ms`, `timing.ttfb_ms`, `failed`, `failure_reason`.
8587

8688
```text
@@ -217,6 +219,43 @@ Assertions support these comparisons. Phrase them naturally — the agent maps t
217219

218220
If you're not sure which method, default to **Visual** — that's what the agent does too.
219221

222+
### 3.5 Calling an API directly
223+
224+
Distinct from observing the page's network traffic (§3.2): you can have the agent **make an HTTP request itself** as part of an objective — to seed data, hit a backend, or check a service — then assert on or reuse the response.
225+
226+
Phrase an explicit call and name the response with "save the response as …":
227+
228+
```text
229+
Call POST https://api.example.com/orders with body {"item": "sku_42", "qty": 1}, save the response as order
230+
Hit GET https://api.example.com/orders/123, save the response as fetched
231+
```
232+
233+
A pasted `curl` works too and is kept verbatim (method, headers, body, auth):
234+
235+
```text
236+
curl -X POST https://api.example.com/login -H 'Content-Type: application/json' -d '{"u":"a","p":"b"}', save the response as login
237+
```
238+
239+
Once saved, reference the response by name:
240+
241+
| Reference | Resolves to |
242+
|---|---|
243+
| `{{order.status}}` | the HTTP status code (e.g. `201`) |
244+
| `{{order.response_body}}` | the whole response body |
245+
| `{{order.response_body.<field>}}` | a field from the JSON response body |
246+
247+
Then assert or chain on it — API calls and browser actions mix freely in one objective:
248+
249+
```text
250+
Call POST https://api.example.com/login with body {"u": "{{user}}", "p": "{{password}}"}, save the response as login,
251+
assert {{login.status}} is 200,
252+
then open https://app.example.com and verify the dashboard loads
253+
```
254+
255+
**Direct call vs. DevTools/Network (§3.2):** use a **direct call** when *you* want the agent to make the request (seed a record, call a backend, set up state). Use **DevTools/Network** when you want to **observe the requests the page itself makes** during a UI flow. They compose: seed via a direct call, drive the UI, then assert on the page's traffic.
256+
257+
**Tokens:** put any credential/token in a variable marked `secret: true` (§6) so it's masked and never logged.
258+
220259
---
221260

222261
## 4. Extraction — the "store as" rule
@@ -394,3 +433,15 @@ Store the order number as 'order_id'.
394433
```
395434

396435
The step body is exactly the same grammar as `kane-cli run`. Everything in this cookbook applies.
436+
437+
### Example D — Seed via API, then verify in the UI
438+
439+
```text
440+
"Call POST https://api.example.com/orders with body {"item": "sku_42", "qty": 1},
441+
save the response as order,
442+
assert {{order.status}} is 201,
443+
then open https://app.example.com/orders,
444+
assert an order for 'sku_42' is visible"
445+
```
446+
447+
The agent makes the API call directly, captures the response, asserts on its status, then drives the UI to confirm the seeded data appears. (Direct call → UI verify — contrast with §3.2, where you observe the page's own traffic.)
Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
# API Calls
2+
3+
Objectives can have the agent **make an API call directly** — not just observe the requests a page makes. This is useful for seeding data before a flow, hitting a backend to set up state, or checking a service, then asserting on or reusing the response.
4+
5+
## Making a call
6+
7+
Phrase an explicit HTTP request and name its response with "save the response as …":
8+
9+
```
10+
Call POST https://api.example.com/orders with body {"item": "sku_42", "qty": 1}, save the response as order
11+
Hit GET https://api.example.com/orders/123, save the response as fetched
12+
Call DELETE https://api.example.com/orders/123
13+
```
14+
15+
A pasted `curl` works too and is kept exactly as written — method, headers, body, and auth:
16+
17+
```
18+
curl -X POST https://api.example.com/login -H 'Content-Type: application/json' -d '{"u":"a","p":"b"}', save the response as login
19+
```
20+
21+
## Using the response
22+
23+
Once you've saved a response under a name, reference it elsewhere in the objective:
24+
25+
| Reference | Resolves to |
26+
|-----------|-------------|
27+
| `{{order.status}}` | the HTTP status code (e.g. `201`) |
28+
| `{{order.response_body}}` | the whole response body |
29+
| `{{order.response_body.<field>}}` | a field from the JSON response body |
30+
31+
Assert on it, or feed it into later actions — API calls and browser actions mix freely in one objective:
32+
33+
```
34+
Call POST https://api.example.com/login with body {"u": "{{user}}", "p": "{{password}}"}, save the response as login,
35+
assert {{login.status}} is 200,
36+
then open https://app.example.com and verify the dashboard loads
37+
```
38+
39+
```
40+
Call POST https://api.example.com/orders with body {"item": "sku_42", "qty": 1}, save the response as order,
41+
assert {{order.status}} is 201,
42+
then open https://app.example.com/orders and verify an order for "sku_42" is visible
43+
```
44+
45+
## Tokens and secrets
46+
47+
Put any API token or credential in a variable marked `secret: true` (see [Variables and Context](../variables-and-context.md)) so it is masked in logs and never stored in plain text:
48+
49+
```
50+
curl -X DELETE https://api.example.com/records/42 -H "Authorization: Bearer {{api_token}}"
51+
```
52+
53+
## Make a call vs. observe page traffic
54+
55+
These are two different things:
56+
57+
- **Make a call (this page)***you* tell the agent to send a request: seed a record, call a backend, set up state.
58+
- **[Network assertions](./checkpoints/devtools/network.md)***observe* the requests the page itself makes during a UI flow (status codes, bodies, timing).
59+
60+
They compose: seed state with a direct call, drive the UI, then assert on the page's own network traffic.

docs/user-guide/features/checkpoints/devtools/network.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@
22

33
Network assertions let you verify HTTP traffic — API responses, status codes, headers, response bodies, and request timing.
44

5+
> These network assertions also cover API calls **you** make directly in an objective — not just the requests the page makes on its own. The agent records its own calls and you query them with the same fields below. To *make* a call (seed data, hit a backend), see [API Calls](../../api-calls.md).
6+
57
## How Capture Works
68

79
KaneAI captures all HTTP network traffic automatically during each test step:

docs/user-guide/running-tests.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,9 @@ Open https://app.example.com, set a cookie named session with value
3939
Objectives can also set, delete, and clear cookies, localStorage, and the
4040
test clipboard directly — see [Browser State Actions](./features/browser-state.md).
4141

42+
Objectives can also make API calls directly — POST/GET a URL or paste a `curl`,
43+
save the response, and assert on it — see [API Calls](./features/api-calls.md).
44+
4245
### Bad to better
4346

4447
| | Objective |

0 commit comments

Comments
 (0)