Skip to content

Commit 17e247e

Browse files
committed
docs: canonicalize trajectory assertion examples
1 parent 4553280 commit 17e247e

1 file changed

Lines changed: 21 additions & 8 deletions

File tree

apps/web/src/content/docs/docs/next/graders/trajectory-assertions.mdx

Lines changed: 21 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -122,12 +122,13 @@ assert:
122122
description: Validate research agent tool usage
123123
target: codex_agent
124124
125+
prompts:
126+
- "{{ task }}"
127+
125128
tests:
126129
- id: comprehensive-research
127-
criteria: Agent thoroughly researches the topic
128-
129-
input: Research machine learning frameworks
130-
130+
vars:
131+
task: Research machine learning frameworks
131132
assert:
132133
# Check minimum tool usage
133134
- name: coverage
@@ -153,17 +154,26 @@ tests:
153154
- webSearch
154155
- documentRead
155156
- summarize
157+
158+
# Check final answer quality
159+
- name: answer-quality
160+
type: llm-rubric
161+
value: The answer synthesizes the research into a useful framework comparison.
156162
```
157163

158164
### Multi-Step Pipeline
159165

160166
```yaml
161-
tests:
162-
- id: data-pipeline
163-
criteria: Process data through complete pipeline
167+
description: Validate a strict data pipeline
168+
target: pipeline_agent
164169
165-
input: Process the customer dataset
170+
prompts:
171+
- "{{ task }}"
166172
173+
tests:
174+
- id: data-pipeline
175+
vars:
176+
task: Process the customer dataset
167177
assert:
168178
- name: pipeline-check
169179
type: trajectory:tool-sequence
@@ -174,6 +184,9 @@ tests:
174184
- validate
175185
- transform
176186
- export
187+
- name: result-quality
188+
type: llm-rubric
189+
value: The final response explains that the dataset was loaded, validated, transformed, and exported.
177190
```
178191

179192
## Best Practices

0 commit comments

Comments
 (0)