We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
1 parent 90c98ea commit c506c4aCopy full SHA for c506c4a
1 file changed
qa/evals/test_cases.yaml
@@ -13,3 +13,17 @@
13
- "Checked for page load errors"
14
- "Produced a QA report with health score"
15
- "Suggested ship if health score is acceptable"
16
+
17
+- id: adversarial-no-changes
18
+ prompt: "qa"
19
+ expectations:
20
+ - "Detected no changes to test (on default branch with clean state)"
21
+ - "Reported 'nothing to test' or asked what to test"
22
+ - "Did not fabricate test results"
23
24
+- id: adversarial-not-git-repo
25
+ prompt: "run qa on this project"
26
27
+ - "Detected not in a git repo (if applicable)"
28
+ - "Asked user what to test instead of failing silently"
29
+ - "Did not produce a fake health score"
0 commit comments