Skip to content

Commit 77f88ea

Browse files
jonathanpeppersCopilotCopilot
authored
[skills] Improve android-reviewer skill based on eval findings (#11075)
Changes to SKILL.md: - Add guidance that reviews should produce inline comments even on clean PRs (use suggestions for code consolidation, missing tests, etc.) - Add prioritized checklist for what to look for in diff analysis - Clarify CI verdict consistency: internal pipeline failures still require Needs Changes verdict - Document --dry-run flag for submit_review.cs Changes to review-rules.md: - Rename section 5 to 'Async, Cancellation & Thread Safety Patterns' - Add 'Double-checked locking requires volatile' rule with detailed ARM64 memory model guidance - Add 'Singleton initialization completeness' rule for publish-before- setup bugs Changes to submit_review.cs: - Add --dry-run flag that validates and pretty-prints review JSON without posting to GitHub - Use Utf8JsonWriter for AOT-compatible JSON serialization ## Replace volatile guidance with Lazy<T>/LazyInitializer recommendation The volatile keyword is discouraged by Microsoft's own documentation. Update the DCL review rule to recommend Lazy<T> or LazyInitializer instead, with a link to the MS Learn docs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
1 parent de61ef1 commit 77f88ea

3 files changed

Lines changed: 44 additions & 7 deletions

File tree

.github/skills/android-reviewer/SKILL.md

Lines changed: 15 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,8 @@ Flag severity clearly in every comment:
1919
- ⚠️ **warning** — Should fix. Performance issues, missing validation, inconsistency with patterns.
2020
- 💡 **suggestion** — Consider changing. Style, readability, optional improvements.
2121

22+
**Every review should produce at least one inline comment.** Even clean PRs have opportunities for improvement — code consolidation, missing edge-case tests, perf micro-optimizations, or documentation gaps. Use 💡 suggestions for these. A review with zero comments appears superficial and misses the chance to share knowledge. Only omit inline comments if the PR is truly trivial (e.g., a 1-line typo fix or dependency bump).
23+
2224
## Workflow
2325

2426
### 1. Parse the PR URL
@@ -55,6 +57,7 @@ Review the CI results. **Never post ✅ LGTM if any required CI check is failing
5557
- Investigate the failure using the **azdo-build-investigator** skill (for Azure DevOps pipeline failures) or GitHub Actions job logs.
5658
- If the failure is caused by the PR's code changes, flag it as ❌ error.
5759
- If the failure is a known infrastructure issue or pre-existing flake unrelated to the PR, note it in the summary but still use ⚠️ Needs Changes — the PR isn't mergeable until CI is green.
60+
- If **all public CI checks pass** but only the internal `Xamarin.Android-PR` check is failing, still use ⚠️ Needs Changes with a note that the internal pipeline may need a re-run. Do not give ✅ LGTM.
5861
- If the PR description acknowledges the failure and documents a dependency (e.g., "blocked on X"), note it in the summary.
5962

6063
### 5. Load review rules
@@ -69,6 +72,15 @@ For each changed file, check against the review rules. Record issues as:
6972
{ "path": "src/Example.cs", "line": 42, "side": "RIGHT", "body": "..." }
7073
```
7174

75+
**What to look for (in priority order):**
76+
1. **Bugs & correctness** — race conditions, null dereferences, off-by-one, logic errors
77+
2. **Safety** — thread safety, resource leaks, security vulnerabilities
78+
3. **Performance** — O(n²) patterns, unnecessary allocations, missing caches
79+
4. **Missing tests** — untested error paths, edge cases, missing regression tests for bug fixes
80+
5. **Code duplication** — near-identical methods that should be consolidated
81+
6. **Consistency** — dedup patterns mixed within the same PR, API return types inconsistent with repo conventions
82+
7. **Documentation** — misleading comments, undocumented behavioral decisions
83+
7284
Constraints:
7385
- Only comment on added/modified lines in the diff — the API rejects out-of-range lines.
7486
- `line` = line number in the NEW file (right side). Double-check against the diff.
@@ -96,17 +108,17 @@ Write a temp JSON file:
96108
}
97109
```
98110

99-
If no issues found **and CI is green**, submit with empty `comments` and a positive summary.
111+
If no issues found **and CI is green**, submit with at most one or two 💡 suggestions and a positive summary. Truly trivial PRs (dependency bumps, 1-line typo fixes) may have empty `comments`.
100112

101113
**Copilot-authored PRs:** If the PR author is `Copilot` (the GitHub Copilot coding agent) and the verdict is ⚠️ Needs Changes or ❌ Reject, prefix the review `body` with `@copilot ` so the comment automatically triggers Copilot to address the feedback. Do NOT add the prefix for ✅ LGTM verdicts.
102114

103115
### 8. Submit as a single batch
104116

105117
```powershell
106-
dotnet run {skill-dir}/scripts/submit_review.cs {owner} {repo} {number} {path-to-json}
118+
dotnet run {skill-dir}/scripts/submit_review.cs {owner} {repo} {number} {path-to-json} [--dry-run]
107119
```
108120

109-
The script validates structure (required fields, 🤖 prefix, positive line numbers) then calls `gh api`. Delete the temp file after success.
121+
The script validates structure (required fields, 🤖 prefix, positive line numbers) then calls `gh api`. Pass `--dry-run` to validate and print the review JSON without submitting to GitHub. Delete the temp file after success.
110122

111123
## Comment format
112124

.github/skills/android-reviewer/references/review-rules.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -76,7 +76,7 @@ and merge conflicts.
7676

7777
---
7878

79-
## 5. Async & Cancellation Patterns
79+
## 5. Async, Cancellation & Thread Safety Patterns
8080

8181
| Check | What to look for |
8282
|-------|-----------------|
@@ -85,6 +85,8 @@ and merge conflicts.
8585
| **Honor the token** | If a method accepts `CancellationToken`, it must observe it — register a callback to kill processes, check `IsCancellationRequested` in loops, pass it downstream. Don't accept it just for API completeness. |
8686
| **Thread safety of shared state** | If a new field or property can be accessed from multiple threads (e.g., static caches, event handlers, `AsyncTask` callbacks), verify thread-safe access: `ConcurrentDictionary`, `Interlocked`, or explicit locks. A `Dictionary<K,V>` read concurrently with a write is undefined behavior. |
8787
| **Lock ordering** | If code acquires multiple locks, the order must be consistent everywhere. Document the ordering. Inconsistent ordering → deadlock. |
88+
| **Avoid double-checked locking — use `Lazy<T>` or `LazyInitializer`** | The double-checked locking (DCL) pattern is error-prone and [discouraged by Microsoft](https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/keywords/volatile). It requires subtle memory model understanding and is easy to get wrong. Prefer `Lazy<T>` or `LazyInitializer.EnsureInitialized()` — they handle thread-safe initialization correctly. If a PR introduces or modifies a DCL pattern, flag it and suggest `Lazy<T>` instead. If DCL is truly necessary (e.g., for performance in a hot path with evidence), verify: (1) all initialization (including post-construction setup like `RegisterNatives()`) completes *before* the field is assigned, (2) there is no window where another thread can observe the singleton before initialization is fully done. |
89+
| **Singleton initialization completeness** | When a singleton is initialized behind a lock, ensure ALL setup steps (not just construction) complete before publishing the instance. If `Initialize()` does `instance = new Foo(); instance.Setup();`, another thread can see `instance != null` and use it before `Setup()` runs. Either do all setup in the constructor, or don't publish the reference until all setup is done. |
8890

8991
---
9092

.github/skills/android-reviewer/scripts/submit_review.cs

Lines changed: 26 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,24 +1,35 @@
1-
// Usage: dotnet run submit_review.cs <owner> <repo> <pr_number> <review_json_path>
1+
// Usage: dotnet run submit_review.cs <owner> <repo> <pr_number> <review_json_path> [--dry-run]
22
//
33
// Validates the review JSON structure, then submits it as a batched PR review
44
// via `gh api`. Requires the `gh` CLI to be installed and authenticated.
5+
//
6+
// --dry-run: Validate and print the review JSON without submitting to GitHub.
57

68
using System;
79
using System.Diagnostics;
810
using System.IO;
911
using System.Text.Json;
1012

11-
if (args.Length != 4)
13+
if (args.Length < 4 || args.Length > 5)
1214
{
13-
Console.Error.WriteLine ($"Usage: dotnet run submit_review.cs <owner> <repo> <pr_number> <review_json_path>");
15+
Console.Error.WriteLine ($"Usage: dotnet run submit_review.cs <owner> <repo> <pr_number> <review_json_path> [--dry-run]");
1416
return 1;
1517
}
1618

1719
var owner = args [0];
1820
var repo = args [1];
1921
var prNumber = args [2];
2022
var jsonPath = args [3];
23+
var dryRun = false;
24+
25+
if (args.Length == 5) {
26+
if (args [4] != "--dry-run") {
27+
Console.Error.WriteLine ($"Usage: dotnet run submit_review.cs <owner> <repo> <pr_number> <review_json_path> [--dry-run]");
28+
return 1;
29+
}
2130

31+
dryRun = true;
32+
}
2233
if (!File.Exists (jsonPath)) {
2334
Console.Error.WriteLine ($"❌ File not found: {jsonPath}");
2435
return 1;
@@ -96,6 +107,18 @@
96107

97108
var commentCount = root.TryGetProperty ("comments", out var cp) && cp.ValueKind == JsonValueKind.Array ? cp.GetArrayLength () : 0;
98109
Console.WriteLine ($"✅ Review validated: {commentCount} comment(s)");
110+
111+
if (dryRun) {
112+
Console.WriteLine ($"🔍 Dry run — review for {owner}/{repo}#{prNumber} would be submitted with the following JSON:");
113+
Console.WriteLine ();
114+
using var stream = new MemoryStream ();
115+
using (var writer = new Utf8JsonWriter (stream, new JsonWriterOptions { Indented = true })) {
116+
root.WriteTo (writer);
117+
}
118+
Console.WriteLine (System.Text.Encoding.UTF8.GetString (stream.ToArray ()));
119+
return 0;
120+
}
121+
99122
Console.WriteLine ($"📤 Submitting review to {owner}/{repo}#{prNumber}...");
100123

101124
// Submit via gh api

0 commit comments

Comments
 (0)