Skip to content

Commit da98718

Browse files
docs: increase validation timeout to 60 minutes and add failed claims table
- Increase Bash tool timeout from 900000ms (~15 min) to 3600000ms (~60 min) - Add failed claims markdown table to validation summary output - Table includes: claim ID, trace ID, claim text, claimed value, source tool ID, actual text, actual data, error %, and root cause
1 parent 22f1fee commit da98718

File tree

1 file changed

+13
-2
lines changed
  • packages/opencode/src/skill/validate

1 file changed

+13
-2
lines changed

packages/opencode/src/skill/validate/SKILL.md

Lines changed: 13 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -47,13 +47,13 @@ Parse `$ARGUMENTS` to determine the mode and construct the command:
4747
- If it contains `--from` → date range mode: `uv run --with python-dotenv --with requests python "$VALIDATE_SCRIPT" --project-root "$PROJECT_ROOT" --from-time "<from>" --to-time "<to>" --user-id "<user_id>"`
4848
- Otherwise → single trace ID: `uv run --with python-dotenv --with requests python "$VALIDATE_SCRIPT" --project-root "$PROJECT_ROOT" --trace-ids "$ARGUMENTS"`
4949

50-
Run the command using the Bash tool with `timeout: 900000` (milliseconds) to allow up to ~15 minutes for long-running validations:
50+
Run the command using the Bash tool with `timeout: 3600000` (milliseconds) to allow up to ~60 minutes for long-running validations:
5151

5252
```bash
5353
uv run --with python-dotenv --with requests python "$VALIDATE_SCRIPT" --project-root "$PROJECT_ROOT" <appropriate_args>
5454
```
5555

56-
**IMPORTANT**: Always pass `timeout: 900000` to the Bash tool when running this command. The default 2-minute bash timeout is too short for validation jobs.
56+
**IMPORTANT**: Always pass `timeout: 3600000` to the Bash tool when running this command. The default 2-minute bash timeout is too short for validation jobs.
5757

5858
The script will:
5959
- Call the Altimate backend directly (no Langfuse dependency)
@@ -241,7 +241,18 @@ For EACH category:
241241
- **Common Strengths**: Patterns of success observed across traces
242242
- **Common Weaknesses**: Recurring issues found across traces
243243
- **Recommendations**: Actionable improvements based on the analysis
244+
245+
Finally generate all the failed claims in the below markdown format from all the traces
246+
247+
| # | Trace ID |Claim | Claimed | Source Tool ID | Actual Text | Actual Data | Error | Root Cause |
248+
|---|---|---|---|---|---|---|---|---|
249+
| <claim_id> | <trace_id>| <claim_text> | <claimed_value> | <source_tool_id> | <source_data> | <Input data> | <error %> | <reasoning> |
250+
251+
REMEMBER that no claim should be truncated. ALL THE VALUES MUST BE COMPLETE.
252+
244253
```
245254

255+
256+
246257
After writing the file, tell the user:
247258
> Summary written to `<report_dir>/SUMMARY.md`

0 commit comments

Comments
 (0)