Skip to content

Commit 53d0b22

Browse files
abrichrclaude
andauthored
feat(agent): add closed-loop demo-conditioned controller (#92)
Add VLM-based step verification (plan_verify.py), demo-conditioned controller state machine (demo_controller.py), and plan progress tracking in the CU agent. Enables the agent to verify each step's expected outcome via screenshot, override premature "done" signals, and retry/replan failed steps. Key additions: - plan_verify.py: verify_step(), verify_plan_progress(), verify_goal_completion() - demo_controller.py: DemoController state machine with step-by-step execution - claude_computer_use_agent.py: plan parsing, progress injection, done override - CLI --controller flag for both openadapt-evals and run_dc_eval.py - 120 tests (31 plan_verify + 36 demo_controller + 53 agent) Validated offline: - Level 1: 91% accuracy on real eval screenshots (10/11 correct) - Level 2: Done-override correctly prevents premature quit Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 5907827 commit 53d0b22

9 files changed

Lines changed: 3776 additions & 18 deletions
Lines changed: 125 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,125 @@
1+
GOAL: In a new sheet with 4 headers "Year", "CA changes", "FA changes", and "OA changes", calculate the annual changes for the Current Assets, Fixed Assets, and Other Assets columns
2+
3+
PLAN:
4+
1. Create a new sheet for calculating and displaying annual changes in asset categories
5+
2. Create a header row for organizing annual asset changes
6+
3. Begin entering annual data by specifying the first year in the "Year" column
7+
4. Calculate the annual percentage change for Current Assets using data from Sheet1
8+
5. Confirm and apply the formula entered in cell B2 for calculating annual changes in Current Assets
9+
6. Copy the formula in B2 down the column so that annual changes for "CA changes" are calculated for each year
10+
7. Calculate the annual change for Fixed Assets as a percentage using data from Sheet1
11+
8. Confirm or complete the entry of a formula or value in the spreadsheet
12+
9. Apply the formula in C2 to the rest of the rows in the "CA changes" column for consistent annual change calculations
13+
10. Calculate the annual percentage change for the "OA changes" column using data from Sheet1
14+
11. Compute the annual percentage change for Other Assets in the new sheet
15+
12. Copy the formula in D2 down the "OA changes" column for all relevant rows
16+
13. Select the range of calculated annual changes and their corresponding years for further formatting or analysis
17+
18+
REFERENCE TRAJECTORY (for disambiguation -- adapt actions to your actual screen):
19+
20+
Step 1:
21+
Think: I need to create a new sheet for calculating and displaying annual changes in asset categories.
22+
Action: Right-click on the "Sheet1" tab at the bottom and select "Insert Sheet" or "New Sheet".
23+
Expect: A new, blank sheet named "Sheet2" should be added to the workbook and displayed.
24+
25+
Step 2:
26+
Think: I need to create a header row for organizing annual asset changes.
27+
Action: Click cell A1 and type "Year"
28+
Expect: The text "Year" should appear in cell A1.
29+
30+
Step 3:
31+
Think: I need to add the next header, "CA changes," as part of setting up the required columns for annual asset changes.
32+
Action: Press Tab and type "CA changes"
33+
Expect: The header "CA changes" should be entered in cell B1, and the sheet should display two headers: "Year" and "CA changes."
34+
35+
Step 4:
36+
Think: I need to continue setting up the required headers for calculating annual changes in asset categories.
37+
Action: Press Tab and type "FA changes".
38+
Expect: The header "FA changes" should be added to cell C1.
39+
40+
Step 5:
41+
Think: I need to complete the set of required headers by adding "OA changes" as the fourth column.
42+
Action: Press Tab and type "OA changes".
43+
Expect: The "OA changes" header should be added in cell D1, completing the four required headers in the first row.
44+
45+
Step 6:
46+
Think: I need to begin entering annual data by specifying the first year in the "Year" column.
47+
Action: Click cell A2 and type "2015".
48+
Expect: The value "2015" appears in cell A2 under the "Year" header.
49+
50+
Step 7:
51+
Think: I need to begin entering the years for which asset changes will be calculated.
52+
Action: Press Enter and type "2016" in cell A2.
53+
Expect: The value "2016" appears in cell A2, starting the list of years under the "Year" header.
54+
55+
Step 8:
56+
Think: I need to begin listing the years for which annual changes will be calculated.
57+
Action: Press Enter and type "2017".
58+
Expect: "2017" should be entered into cell A3, directly below "2016" in the "Year" column.
59+
60+
Step 9:
61+
Think: I need to continue populating the "Year" column with the next chronological year for annual change calculations.
62+
Action: Press Enter and type "2018".
63+
Expect: The value "2018" appears in the next row of the "Year" column.
64+
65+
Step 10:
66+
Think: I need to add the next year in the sequence to the "Year" column in preparation for calculating annual changes.
67+
Action: Press Enter and type "2019".
68+
Expect: The value "2019" appears in the next row under the "Year" column.
69+
70+
Step 11:
71+
Think: I need to calculate the annual percentage change for Current Assets using data from Sheet1.
72+
Action: Click cell B2 and type "=(Sheet1.B3-Sheet1.B2)/Sheet1.B2".
73+
Expect: Cell B2 should contain a formula to compute the percentage change in Current Assets between two years from Sheet1.
74+
75+
Step 12:
76+
Think: I need to confirm and apply the formula entered in cell B2 for calculating annual changes in Current Assets.
77+
Action: Press Enter
78+
Expect: The formula in cell B2 should be executed, displaying the calculated percentage change for "CA changes" in 2016.
79+
80+
Step 13:
81+
Think: I need to copy the formula in B2 down the column so that annual changes for "CA changes" are calculated for each year.
82+
Action: Click cell B2, then drag the fill handle down to B6.
83+
Expect: The formula from B2 is filled down to B6, calculating "CA changes" for all listed years.
84+
85+
Step 14:
86+
Think: I need to calculate the annual change for Fixed Assets as a percentage using data from Sheet1.
87+
Action: Click cell C2 and type "=(Sheet1.C3-Sheet1.C2)/Sheet1.C2"
88+
Expect: Cell C2 should contain the formula to compute the percentage change in Fixed Assets between two years from Sheet1.
89+
90+
Step 15:
91+
Think: I need to confirm or complete the entry of a formula or value in the spreadsheet.
92+
Action: Press Enter.
93+
Expect: The cell selection moves from the current cell (D2) to the next cell below (D3), confirming the entry in D2.
94+
95+
Step 16:
96+
Think: I need to apply the formula in C2 to the rest of the rows in the "CA changes" column for consistent annual change calculations.
97+
Action: Click cell C2, then drag the fill handle down to C6.
98+
Expect: The formula from C2 is copied down to C6, populating the "CA changes" column with calculated values for each year.
99+
100+
Step 17:
101+
Think: I need to calculate the annual percentage change for the "OA changes" column using data from Sheet1.
102+
Action: Click cell D2 and type "=(Sheet1.D3-Sheet1.D2)/Sheet1.D2"
103+
Expect: Cell D2 should be populated with the formula to compute the percentage change for "OA changes" between two years.
104+
105+
Step 18:
106+
Think: I need to compute the annual percentage change for Other Assets in the new sheet.
107+
Action: Press Enter
108+
Expect: The formula should be executed in cell D3, displaying the calculated percentage change for Other Assets.
109+
110+
Step 19:
111+
Think: I need to copy the formula in D2 down the "OA changes" column for all relevant rows.
112+
Action: Click cell D2, then drag the fill handle down to D6.
113+
Expect: The formula from D2 is filled down through D6, calculating values for each row in the "OA changes" column.
114+
115+
Step 20:
116+
Think: I need to select the range of calculated annual changes and their corresponding years for further formatting or analysis.
117+
Action: Click and drag to select cells B2:D6.
118+
Expect: Cells B2:D6 should be highlighted, indicating they should be selected for the next operation.
119+
120+
Step 21:
121+
Think: I need to format the annual change values as percentages for better readability.
122+
Action: Click the % button in the toolbar (or press Ctrl+Shift+5).
123+
Expect: The values in columns B, C, and D should now be displayed as percentages.
124+
125+
If your screen doesn't match what's expected, re-evaluate based on the PLAN and decide the best next action.

0 commit comments

Comments
 (0)