Skip to content

Commit a2e08b7

Browse files
feat(review): add structured category and severity to findings (#311)
Add two structured fields, category and severity, to every review finding so CI integrations can sort, group, filter, or gate builds without re-parsing natural-language comment text. - Tool schema (tools.json): add category/severity as enum-constrained, required properties of code_comment. severity is limited to critical/high/medium/low (info dropped, since LLMs struggle to distinguish low from info). - System prompt (task_template.json) is intentionally left untouched to avoid the review-quality regression observed on the benchmark suite; the tool schema alone drives field population. - JSON output: category/severity are flat siblings of content/start_line, omitted entirely when empty (backward compatible). - CLI output: render an inline [category - severity] badge before the comment, colored by severity. - Sync docs across all five README locales.
1 parent 39eb0f3 commit a2e08b7

11 files changed

Lines changed: 304 additions & 3 deletions

File tree

README.ja-JP.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -404,6 +404,15 @@ ocr review \
404404

405405
`--format json`フラグは、CIスクリプトでのパースに適した機械可読な結果を出力します。
406406

407+
各指摘には2つの構造化フィールドが付与され、CI統合はコメント本文を再パースせずに並べ替え・グループ化・フィルタリング・ビルドのゲート判定を行えます:
408+
409+
| フィールド | 許可される値 | 説明 |
410+
|-----------|-------------|------|
411+
| `category` | `bug``security``performance``maintainability``test``style``documentation``other` | 指摘が属するカテゴリ。 |
412+
| `severity` | `critical``high``medium``low` | 指摘の重要度。 |
413+
414+
JSON出力ではこの2つのフィールドは`content``start_line`などと同じ階層に並びます。ターミナルでは、コメントの前にインラインの`[category · severity]`バッジとして表示され、重要度に応じて色分けされます。
415+
407416
統合例は[`examples/`](./examples/)ディレクトリを参照してください:
408417

409418
- [`github_actions/`](./examples/github_actions/) — GitHub Actions統合の例

README.ko-KR.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -404,6 +404,15 @@ ocr review \
404404

405405
`--format json` flag는 CI script에서 파싱하기 좋은 machine-readable 결과를 출력합니다.
406406

407+
각 finding에는 두 개의 구조화된 field가 포함되어, CI 통합에서 comment 텍스트를 다시 파싱하지 않고도 정렬·그룹화·필터링하거나 build를 gate할 수 있습니다:
408+
409+
| Field | 허용 값 | 설명 |
410+
|-------|--------|------|
411+
| `category` | `bug`, `security`, `performance`, `maintainability`, `test`, `style`, `documentation`, `other` | 이슈가 속한 카테고리. |
412+
| `severity` | `critical`, `high`, `medium`, `low` | 이슈의 중요도. |
413+
414+
JSON 출력에서 두 field는 `content`, `start_line` 등과 같은 수준의 sibling으로 나타납니다. 터미널에서는 comment 앞에 인라인 `[category · severity]` badge로 표시되며 severity에 따라 색상이 지정됩니다.
415+
407416
통합 예시는 [`examples/`](./examples/) 디렉터리를 참고하세요.
408417

409418
- [`github_actions/`](./examples/github_actions/): GitHub Actions 통합 예시

README.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -406,6 +406,15 @@ The `--from` flag accepts a branch ref (e.g., `origin/main`) or commit SHA as th
406406

407407
The `--format json` flag outputs machine-readable results suitable for parsing in CI scripts.
408408

409+
Each finding carries two structured fields so CI integrations can sort, group, filter, or gate builds without re-parsing comment text:
410+
411+
| Field | Allowed values | Notes |
412+
|-------|----------------|-------|
413+
| `category` | `bug`, `security`, `performance`, `maintainability`, `test`, `style`, `documentation`, `other` | The category the issue belongs to. |
414+
| `severity` | `critical`, `high`, `medium`, `low` | The importance of the issue. |
415+
416+
In JSON output the two fields appear as siblings alongside `content`, `start_line`, etc. In the terminal, they render as an inline `[category · severity]` badge before the comment, colored by severity.
417+
409418
See the [`examples/`](./examples/) directory for integration examples:
410419

411420
- [`github_actions/`](./examples/github_actions/) — GitHub Actions integration example

README.ru-RU.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -406,6 +406,15 @@ ocr review \
406406

407407
Флаг `--format json` выводит машиночитаемый результат, удобный для разбора в CI-скриптах.
408408

409+
Каждое замечание содержит два структурированных поля, чтобы CI-интеграции могли сортировать, группировать, фильтровать замечания или блокировать сборку без повторного разбора текста комментария:
410+
411+
| Поле | Допустимые значения | Примечание |
412+
|------|---------------------|------------|
413+
| `category` | `bug`, `security`, `performance`, `maintainability`, `test`, `style`, `documentation`, `other` | Категория, к которой относится замечание. |
414+
| `severity` | `critical`, `high`, `medium`, `low` | Важность замечания. |
415+
416+
В JSON-выводе эти два поля располагаются рядом с `content`, `start_line` и др. В терминале они отображаются перед комментарием как встроенный бейдж `[category · severity]`, цвет которого определяется важностью.
417+
409418
Примеры интеграции — в каталоге [`examples/`](./examples/):
410419

411420
- [`github_actions/`](./examples/github_actions/) — пример интеграции с GitHub Actions

README.zh-CN.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -404,6 +404,15 @@ ocr review \
404404

405405
`--format json` 参数输出适合 CI 脚本解析的机器可读结果。
406406

407+
每条评审结果都带有两个结构化字段,便于 CI 集成在无需解析评论文本的情况下排序、分组、过滤或卡点构建:
408+
409+
| 字段 | 允许的取值 | 说明 |
410+
|------|-----------|------|
411+
| `category` | `bug``security``performance``maintainability``test``style``documentation``other` | 问题所属的类别。 |
412+
| `severity` | `critical``high``medium``low` | 问题的严重程度。 |
413+
414+
在 JSON 输出中,这两个字段与 `content``start_line` 等平级;在终端中,它们会以内联的 `[category · severity]` 徽章形式显示在评论前,并按严重程度着色。
415+
407416
集成示例请参见 [`examples/`](./examples/) 目录:
408417

409418
- [`github_actions/`](./examples/github_actions/) — GitHub Actions 集成示例

cmd/opencodereview/output.go

Lines changed: 48 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -61,7 +61,19 @@ func renderComment(comment model.LlmComment) {
6161
fmt.Printf("\n\033[2m─── %s:%d-%d ───\033[0m\n", sanitizeTerminal(comment.Path), comment.StartLine, comment.EndLine)
6262

6363
if comment.Content != "" {
64-
for _, ln := range wrapByRunes(sanitizeTerminal(comment.Content), 100) {
64+
badge := buildBadge(comment)
65+
content := sanitizeTerminal(comment.Content)
66+
if badge != "" {
67+
// Prepend the plain badge text to the content so it wraps inline with
68+
// the first line, then colorize just the badge prefix after wrapping.
69+
content = badge + " " + content
70+
}
71+
lines := wrapByRunes(content, 100)
72+
for i, ln := range lines {
73+
if i == 0 && badge != "" && strings.HasPrefix(ln, badge) {
74+
color := severityColor(comment.Severity)
75+
ln = color + badge + "\033[0m" + ln[len(badge):]
76+
}
6577
fmt.Printf("%s\n", ln)
6678
}
6779
fmt.Println()
@@ -83,6 +95,41 @@ func renderComment(comment model.LlmComment) {
8395
fmt.Println()
8496
}
8597

98+
// buildBadge renders a compact "[category · severity]" tag for a finding. It returns
99+
// an empty string when neither structured field is present, so text output for findings
100+
// without metadata is unchanged.
101+
func buildBadge(comment model.LlmComment) string {
102+
category := sanitizeTerminal(comment.Category)
103+
severity := sanitizeTerminal(comment.Severity)
104+
switch {
105+
case category != "" && severity != "":
106+
return fmt.Sprintf("[%s · %s]", category, severity)
107+
case category != "":
108+
return fmt.Sprintf("[%s]", category)
109+
case severity != "":
110+
return fmt.Sprintf("[%s]", severity)
111+
default:
112+
return ""
113+
}
114+
}
115+
116+
// severityColor maps a finding severity to an ANSI color used for its badge.
117+
// Unknown or empty severities fall back to dim.
118+
func severityColor(severity string) string {
119+
switch severity {
120+
case "critical":
121+
return "\033[1;91m" // bold bright red
122+
case "high":
123+
return "\033[91m" // bright red
124+
case "medium":
125+
return "\033[93m" // bright yellow
126+
case "low":
127+
return "\033[94m" // bright blue
128+
default:
129+
return "\033[2m" // dim
130+
}
131+
}
132+
86133
// printDiffLine renders a single diff line with colored prefix and background on content.
87134
func printDiffLine(prefix, content, fgColor, bgColor string) {
88135
fmt.Printf("%s%s%s %s%s\033[0m\n", fgColor+bgColor, prefix, "\033[0m"+bgColor, content, "\033[0m")

cmd/opencodereview/output_test.go

Lines changed: 73 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,78 @@
11
package main
22

3-
import "testing"
3+
import (
4+
"strings"
5+
"testing"
6+
7+
"github.com/open-code-review/open-code-review/internal/model"
8+
)
9+
10+
func TestBuildBadge(t *testing.T) {
11+
tests := []struct {
12+
name string
13+
comment model.LlmComment
14+
want string
15+
}{
16+
{"both fields", model.LlmComment{Category: "security", Severity: "high"}, "[security · high]"},
17+
{"category only", model.LlmComment{Category: "bug"}, "[bug]"},
18+
{"severity only", model.LlmComment{Severity: "low"}, "[low]"},
19+
{"neither", model.LlmComment{}, ""},
20+
{"strips control chars", model.LlmComment{Category: "bug\x1b[0m", Severity: "high"}, "[bug[0m · high]"},
21+
}
22+
for _, tt := range tests {
23+
t.Run(tt.name, func(t *testing.T) {
24+
if got := buildBadge(tt.comment); got != tt.want {
25+
t.Errorf("buildBadge() = %q, want %q", got, tt.want)
26+
}
27+
})
28+
}
29+
}
30+
31+
func TestSeverityColor(t *testing.T) {
32+
// Each known severity must map to a distinct color; unknown falls back to dim.
33+
seen := map[string]string{}
34+
for _, sev := range []string{"critical", "high", "medium", "low"} {
35+
c := severityColor(sev)
36+
if c == "" {
37+
t.Errorf("severityColor(%q) is empty", sev)
38+
}
39+
if prev, ok := seen[c]; ok {
40+
t.Errorf("severityColor(%q) shares color with %q", sev, prev)
41+
}
42+
seen[c] = sev
43+
}
44+
if got := severityColor("bogus"); got != "\033[2m" {
45+
t.Errorf("severityColor(unknown) = %q, want dim", got)
46+
}
47+
if got := severityColor(""); got != "\033[2m" {
48+
t.Errorf("severityColor(empty) = %q, want dim", got)
49+
}
50+
}
51+
52+
// TestRenderComment_BadgeInline verifies the badge is colorized and rendered inline
53+
// with the first line of the comment content.
54+
func TestRenderComment_BadgeInline(t *testing.T) {
55+
out := captureStdout(t, func() {
56+
renderComment(model.LlmComment{
57+
Path: "internal/mcp/client.go",
58+
StartLine: 27,
59+
EndLine: 27,
60+
Content: "Potential environment variable leak.",
61+
Category: "security",
62+
Severity: "high",
63+
})
64+
})
65+
if !strings.Contains(out, "[security · high]") {
66+
t.Errorf("expected badge in output, got:\n%s", out)
67+
}
68+
// severity high → bright red; the badge must be wrapped in the color + reset.
69+
if !strings.Contains(out, "\033[91m[security · high]\033[0m") {
70+
t.Errorf("expected colorized badge, got:\n%q", out)
71+
}
72+
if !strings.Contains(out, "Potential environment variable leak.") {
73+
t.Errorf("expected content in output, got:\n%s", out)
74+
}
75+
}
476

577
func TestSanitizeTerminal(t *testing.T) {
678
tests := []struct {

internal/config/toolsconfig/tools.json

Lines changed: 27 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -51,11 +51,37 @@
5151
"suggestion_code": {
5252
"type": "string",
5353
"description": "Corresponding suggested code snippet, maintaining consistent code style."
54+
},
55+
"category": {
56+
"type": "string",
57+
"enum": [
58+
"bug",
59+
"security",
60+
"performance",
61+
"maintainability",
62+
"test",
63+
"style",
64+
"documentation",
65+
"other"
66+
],
67+
"description": "The category the issue belongs to."
68+
},
69+
"severity": {
70+
"type": "string",
71+
"enum": [
72+
"critical",
73+
"high",
74+
"medium",
75+
"low"
76+
],
77+
"description": "The severity of the issue."
5478
}
5579
},
5680
"required": [
5781
"content",
58-
"existing_code"
82+
"existing_code",
83+
"category",
84+
"severity"
5985
]
6086
}
6187
}

internal/model/review.go

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,12 @@ type LlmComment struct {
99
StartLine int `json:"start_line"`
1010
EndLine int `json:"end_line"`
1111
Thinking string `json:"thinking,omitempty"`
12+
// Category classifies the finding. One of:
13+
// bug, security, performance, maintainability, test, style, documentation, other.
14+
Category string `json:"category,omitempty"`
15+
// Severity indicates the importance of the finding. One of:
16+
// critical, high, medium, low.
17+
Severity string `json:"severity,omitempty"`
1218
}
1319

1420
// CodeReviewResult holds raw LLM-generated review suggestion for a code segment.

internal/tool/code_comment.go

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -68,6 +68,12 @@ func ParseComments(args map[string]any) ([]model.LlmComment, string) {
6868
if thinking, ok := obj["thinking"].(string); ok {
6969
cm.Thinking = thinking
7070
}
71+
if category, ok := obj["category"].(string); ok {
72+
cm.Category = category
73+
}
74+
if severity, ok := obj["severity"].(string); ok {
75+
cm.Severity = severity
76+
}
7177
if path, ok := args["path"].(string); ok {
7278
cm.Path = path
7379
}

0 commit comments

Comments
 (0)