You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
description: Writing the viewer/inspector/trajectory viewer
3
+
globs:
4
+
alwaysApply: false
5
+
---
6
+
7
+
You are building a trajectory viewer for an AI agent benchmark that pits
8
+
two AI agent in a round-based game against each other.
9
+
10
+
## Framework notes
11
+
12
+
* Write a python server using flask
13
+
* As much output as possible should be formatted uses jinja templates
14
+
* JS should be kept to a minimum
15
+
* Your input is a game folder (described below)
16
+
* Use clean modern CSS, but don't go too crazy, simplicity is key
17
+
* This is a single page application
18
+
19
+
## Input folder
20
+
21
+
The input folders are all subfolders of `logs/`
22
+
23
+
The input folder has the following files
24
+
25
+
* results.json
26
+
* main.log
27
+
* round_$i.log, where $i is the round. Infer the number of rounds from how many of these files we have
28
+
* .*_p$i_r$j.traj.log, where $i$ is the player (there's always 1 or 2 players and $j is the round number
29
+
* Trajectory files do not exist for the last round
30
+
31
+
Here's how a `.traj.log` file looks:
32
+
33
+
34
+
```
35
+
{
36
+
"info": {
37
+
"exit_status": null,
38
+
"submission": null,
39
+
"memory": null,
40
+
"model_stats": {
41
+
"instance_cost": 0.0661002,
42
+
"api_calls": 11
43
+
}
44
+
},
45
+
"messages": [
46
+
{
47
+
"role": "system",
48
+
"content": "You are a helpful assistant that can interact with a computer.\n\nYour response must contain exactly ONE bash code block with ONE command (or commands connected with && or ||).\nInclude a THOUGHT section before your command where you explain your reasoning process.\nFormat your response as shown in <format_example>.\n\n<format_example>\nYour reasoning and analysis here. Explain why you want to perform the action.\n\n```bash\nyour_command_here\n```\n</format_example>\n\nFailure to follow these rules will cause your response to be rejected.\nTo finish, issue the following command: `echo MINI_SWE_AGENT_FINAL_OUTPUT`\nwithout any other command."
49
+
},
50
+
{
51
+
"role": "user",
52
+
"content": [
53
+
{
54
+
"type": "text",
55
+
"text": "Please solve this issue: in gitconfig, add a new alias ldc which copies the last diff ..."
56
+
}
57
+
]
58
+
},
59
+
{
60
+
"role": "assistant",
61
+
"content": "THOUGHT: I need to first look at the current directory to see the gitconfig file and understand its structure. Then I'll examine the existing aliases to understand the pattern and add the new 'ldc' alias that combines 'ld' (last diff) with clipboard functionality.\n\n```bash\nls -la\n```"
"content": "THOUGHT: I can see there's a `gitconfig.sh` file in the current directory. Let me examine its contents to understand the current structure and find the existing aliases, particularly looking for the 'ld' alias and any clipboard-related aliases that I need to combine.\n\n```bash\ncat gitconfig.sh\n```"
75
+
},
76
+
"trajectory_format": "mini-swe-agent-1"
77
+
}
78
+
```
79
+
80
+
## Page design
81
+
82
+
### Header
83
+
84
+
* Here's some control elements to choose the input folder or visualization settings
85
+
* There's a dark/light mode switch
86
+
87
+
### Main body
88
+
89
+
#### Overall results
90
+
91
+
* At the top, we show the content of `metadata.json`
92
+
* There's a foldout that displays main.log (hidden by default)
93
+
94
+
#### Rounds
95
+
96
+
First, show a foldout of `round_1.log`.
97
+
98
+
Then, show a foldout with the trajectory view of player 1 round 1 (see below for trajectory view)
99
+
Then, show a foldout with the content of trajectory player 1 round 1 ["submission"]
100
+
Then, show a foldout with the content of trajectory player 1 round 1 ["memory"]
101
+
Then, show a foldout with the trajectory view of player 2 round 1 (see below for trajectory view).
102
+
Then, show a foldout with the content of trajectory player 2 round 1 ["submission"]
103
+
Then, show a foldout with the content of trajectory player 2 round 1 ["memory"]
104
+
105
+
Then, show a foldout of `round_2.log`.
106
+
107
+
Then, show a foldout with the trajectory view of player 1 round 2 (see below for trajectory view)
108
+
Then, show a foldout with the trajectory view of player 2 round 2 (see below for trajectory view).
109
+
Then, show a foldout with the content of trajectory player 2 round 2 ["submission"]
110
+
Then, show a foldout with the content of trajectory player 2 round 2 ["memory"]
111
+
112
+
etc.
113
+
114
+
Note that he last round only has the `round_n.log`, but there's no more trajectories for the last one.
115
+
116
+
#### Trajectory view
117
+
118
+
This should be its own jinja template.
119
+
120
+
First show the following properties from the `info` section
121
+
122
+
* Number of LM calls (api calls)
123
+
* Cost (instance cost)
124
+
125
+
Then, show all the messages. Each message should be its own block together with the role.
126
+
Because the content can be long, make sure to have a foldout for the content.
0 commit comments