You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
fix(dsql): bump databases-on-aws to 1.6.0, add Workflow 12 evals
Resolves the 1.5.0 version collision with EF Core PR awslabs#208 (both this
branch's system-diagnostics work and awslabs#208 independently claimed 1.5.0).
Bumps to 1.6.0 across plugin.json, codex plugin.json, and marketplace.json.
Adds eval coverage for Workflow 12 (system diagnostics), which previously
had none: 3 triggering cases and 1 functional AAS-interpretation eval.
Corrects stale eval README counts (37 trigger cases, 15 functional
prompts / 54 assertions) and documents evals 13-15.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Copy file name to clipboardExpand all lines: plugins/databases-on-aws/.codex-plugin/plugin.json
+1-1Lines changed: 1 addition & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,6 @@
1
1
{
2
2
"name": "databases-on-aws",
3
-
"version": "1.5.0",
3
+
"version": "1.6.0",
4
4
"description": "Expert database guidance for the AWS database portfolio. Design schemas, execute queries, handle migrations, and choose the right database for your workload.",
| 9. Backup on IDLE/INACTIVE | Troubleshooting | LLM judge | Identifies `FailedPrecondition`, connects to wake cluster to ACTIVE, retries backup |
97
-
| 10. Loader stuck at 3K rec/s | Data loading | LLM judge | Identifies partition-constrained fresh table, advises to keep running, does NOT recommend more workers |
98
-
| 11. Loader crash lost manifest | Data loading | LLM judge | Identifies /tmp as tmpfs, recommends --on-conflict do-nothing for recovery, --manifest-dir for prevention |
| 9. Backup on IDLE/INACTIVE | Troubleshooting | LLM judge | Identifies `FailedPrecondition`, connects to wake cluster to ACTIVE, retries backup |
97
+
| 10. Loader stuck at 3K rec/s | Data loading | LLM judge | Identifies partition-constrained fresh table, advises to keep running, does NOT recommend more workers |
98
+
| 11. Loader crash lost manifest | Data loading | LLM judge | Identifies /tmp as tmpfs, recommends --on-conflict do-nothing for recovery, --manifest-dir for prevention |
Copy file name to clipboardExpand all lines: tools/evals/databases-on-aws/dsql/evals.json
+14Lines changed: 14 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -170,6 +170,20 @@
170
170
"Recommends the Amazon.AuroraDsql.EntityFrameworkCore adapter for EF Core users"
171
171
],
172
172
"llm_judge": true
173
+
},
174
+
{
175
+
"id": 15,
176
+
"prompt": "My Aurora DSQL cluster is performing differently than last week and I can't point to one bad query. I pulled the Active Average Sessions (db.active_sessions.avg) broken down by wait event for the same hour, this week vs last week. Last week: OnCpu 226, SequentialScanRead 89, UniqueConstraintCheck 89, Commit 74, everything else flat. This week: OnCpu 213, SequentialScanRead 336, UniqueConstraintCheck 87, Commit 73, everything else flat. What does this tell you and what should I do next?",
177
+
"expected_output": "Interprets the data as a wait-event distribution shift: SequentialScanRead grew ~3.7x (from ~12% to ~35% of total AAS) while every other wait event, including OnCpu and Commit, stayed flat. Concludes this is NOT overall load growth (which would raise all events together) but a scan/plan regression. Does NOT treat any absolute AAS number as inherently bad — reasons from the change vs the temporal baseline. Recommends identifying the top SQL on SequentialScanRead and handing those queries off to per-query EXPLAIN analysis (Workflow 9). Does not recommend schema or index changes from CloudWatch data alone.",
178
+
"files": [],
179
+
"expectations": [
180
+
"Identifies SequentialScanRead as the shifted wait event and quantifies the change relative to the baseline (roughly 3.7x, or a jump from ~12% to ~35% of total AAS)",
181
+
"Concludes this is a scan/plan regression rather than proportional load growth, citing that OnCpu / Commit / other events stayed flat",
182
+
"Does NOT claim any absolute AAS value is inherently problematic — reasons from change over time against the temporal baseline",
183
+
"Recommends identifying the top SQL statements attributed to SequentialScanRead as the next step",
184
+
"Defers per-query root cause to EXPLAIN / query plan analysis (Workflow 9) and does NOT recommend specific index or schema changes from the CloudWatch metrics alone"
Copy file name to clipboardExpand all lines: tools/evals/databases-on-aws/dsql/trigger_evals.json
+12Lines changed: 12 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -51,6 +51,18 @@
51
51
"query": "I think there's a bug in DSQL's EXPLAIN ANALYZE output — my Storage Lookup is reporting trillions of actual rows on a table that only has 300k rows. Can you diagnose this?",
52
52
"should_trigger": true
53
53
},
54
+
{
55
+
"query": "my Aurora DSQL cluster is performing differently than it was last week — can you diagnose what changed?",
56
+
"should_trigger": true
57
+
},
58
+
{
59
+
"query": "my dsql cluster feels slow but no single query stands out. how do I find where the time is going — wait events, active sessions, that kind of thing?",
60
+
"should_trigger": true
61
+
},
62
+
{
63
+
"query": "where is compute time being spent on my Aurora DSQL cluster? I want to see the AAS breakdown by wait event.",
64
+
"should_trigger": true
65
+
},
54
66
{
55
67
"query": "I need to load a 500MB CSV file into my Aurora DSQL table. What's the fastest way?",
0 commit comments