Skip to content

Commit 6a48684

Browse files
alerizzoclaude
andcommitted
fix: stop issues --overview noise suggestions on non-noisy repos
Gate the "Suggested actions to reduce noise" section behind two absolute floors, AND-ed with the existing relative rules, so it stops firing on repositories that aren't actually noisy: - NOISE_MIN_TOTAL (200): suppress the whole section unless the repo has a substantial issue volume. Kept above the per-pattern floor so it does independent work rather than being subsumed by it. - NOISE_MIN_PATTERN (100): a pattern must produce enough issues on its own. Without it, a long tail of tiny patterns drags the median to ~3, making a 9-issue pattern look "3x the median" and get flagged as noise. Also harden the relative rules: the >=10% share rule now only applies with >=8 distinct patterns (with fewer, an even split already exceeds 10% each), and the >=3x multiple rule measures against the median (new medianOf) instead of the mean, so one huge pattern can't inflate the baseline and mask smaller disproportionate patterns. Co-Authored-By: Claude <noreply@anthropic.com>
1 parent bf903e4 commit 6a48684

6 files changed

Lines changed: 260 additions & 18 deletions

File tree

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
---
2+
"@codacy/codacy-cloud-cli": patch
3+
---
4+
5+
Stop `issues --overview` from suggesting noise reduction on repositories that aren't
6+
actually noisy. The "Suggested actions to reduce noise" section now requires two absolute
7+
floors before anything is suggested: the repository must have at least 200 issues in total,
8+
and an individual pattern must produce at least 100 issues on its own. The per-pattern floor
9+
matters because a repository with a long tail of tiny patterns pulls the median issues-per-
10+
pattern very low, which previously made a pattern with only a handful of issues look
11+
disproportionate — now a rule has to genuinely flood the repo before it's flagged. On top of
12+
those floors, a pattern must still show a relative signal: the "dominant share" rule (≥10% of
13+
all issues) only applies when there are at least 8 distinct patterns (with fewer, an even
14+
split already exceeds 10% each), and the "disproportionate count" rule now compares each
15+
pattern against the **median** issues-per-pattern instead of the mean, so a single huge
16+
pattern can no longer inflate the baseline and hide smaller-but-still-disproportionate ones.

SPECS/README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,3 +74,4 @@ _No pending tasks._ All commands implemented.
7474
| 2026-06-24 | `findings` and `finding` now surface the vulnerable dependency's import chain from the new `dependencyChains` field: Direct (`Update <pkg> to <fixed>`) vs Transitive (`<chain> (Fixed in <fixed>)`), with the middle collapsed to `... N more ...` for 4+ packages. List shows the first chain + `... and X more`; detail shows all chains aligned under a single label. New helpers in `utils/formatting.ts` (`formatDependencyChain`, `formatDependencyChainsLine`, `formatDependencyChainsBlock`); `dependencyChains` added to both JSON projections (17 new tests, 390 total) |
7575
| 2026-06-30 | npm-style "update available" notice via `update-notifier@5`: one-time stderr hint when a newer version is published, gated to `--output table` (suppressed for `json`, when piped, in CI, under `npx`). Non-blocking daily background check; never auto-updates. New `src/version.ts` (single source of name/version) + `src/utils/update-check.ts` (`maybeNotifyUpdate`); `preAction` hook + `--no-update-notifier` flag wired in `index.ts`. Opt-outs: `CODACY_DISABLE_UPDATE_CHECK`, `NO_UPDATE_NOTIFIER`, `--no-update-notifier`. `package.json` `overrides` pin transitive `got@^11.8.6`/`package-json@^7` (CVE-2022-33987) (7 new tests, 409 total) |
7676
| 2026-07-07 | `ls` and `directories` commands: browse a repository's folders/files with quality metrics (Grade/Issues/Complexity/Duplication/Coverage). Auto-detect provider/org/repo and the cwd-relative path; `--path`/`--branch` options; `directories --plus-children` shows one extra level as a `└─` tree (header adds `, M subdirectories`). `--sort`/`--direction` (server-side; in `ls`, directories and files sorted independently), and `ls --search <term>` (files only; folds the path into the search as `<path>/%<term>` and shows full paths). Duplication uses `numberOfClones`; Complexity uses `complexity` (hotspots). Both fetch **all** pages (no pagination warning) via new `src/utils/repo-tree.ts` (path resolution + `resolveSort`/`resolveDirection` + `fetchAllDirectories`/`fetchAllFiles`). Row markers `` folder / dim `·` file (no emojis). Promoted `formatGrade` to `utils/formatting.ts` (now also colors E red) and added `formatCountCell`/`formatCoverageCell` (48 new tests, 457 total) |
77+
| 2026-07-08 | `issues --overview` noise suggestions tuned to stop firing on low-volume repos: added a `NOISE_MIN_TOTAL` (200) floor on the repo's total issues that suppresses the whole "reduce noise" section below it, and a `NOISE_MIN_PATTERN` (100) absolute floor on each pattern's own count (AND-gated with the relative rules) so a long tail of tiny patterns can't drag the median down and make a ~9-issue pattern look noisy — the total floor is kept above the per-pattern floor so it isn't dead code; the ≥10% share rule now only applies with ≥8 distinct patterns (`NOISE_MIN_PATTERNS_FOR_SHARE`, since with fewer an even split already exceeds 10%); and the ≥3× multiple rule now measures against the **median** (via new `medianOf()`) instead of the mean, so a single huge pattern can no longer inflate the baseline and mask smaller disproportionate patterns (4 new tests, 464 total) |

SPECS/commands/issues.md

Lines changed: 22 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -78,11 +78,28 @@ False Positive" (the bucket is keyed on FP probability vs. the configured
7878
threshold, so at/above threshold = a potential false positive).
7979

8080
After the tables, a **"Suggested actions to reduce noise"** section lists patterns
81-
worth disabling. A pattern is "noisy" when it accounts for **≥10% of all issues**
82-
shown, **or** has **≥3× the average** issues-per-pattern. The owning tool is
83-
resolved by matching the pattern ID against each tool's `prefix` (longest match
84-
wins); patterns whose tool can't be resolved (no/unknown prefix) are dropped
85-
silently. The list is capped at 10 with a "… (N more)" note.
81+
worth disabling. A pattern must clear two absolute floors **and** show a relative
82+
signal:
83+
84+
- **Total floor** (`NOISE_MIN_TOTAL`, 200): the section is suppressed entirely
85+
unless the repo has ≥200 issues in total — on low-volume repos, disabling a rule
86+
to shave a handful of issues isn't real noise reduction. Kept above the
87+
per-pattern floor so it does independent work (were they equal, any pattern that
88+
clears the per-pattern floor would already push the repo past an equal total
89+
floor, making it dead code).
90+
- **Per-pattern floor** (`NOISE_MIN_PATTERN`, 100): the individual pattern must
91+
have ≥100 issues on its own. Without it, a long tail of tiny patterns drags the
92+
median down (e.g. to 3) so far that a pattern with only ~9 issues clears the
93+
relative bar below — yet 9 issues is nothing worth disabling a rule over.
94+
- **Relative signal** (either one): accounts for **≥10% of all issues** shown
95+
(only applied when there are **≥8 distinct patterns**, since with fewer an even
96+
split already exceeds 10% each), **or** has **≥3× the median** issues-per-pattern.
97+
The median (not the mean) is used so a single huge pattern can't inflate the
98+
baseline and mask smaller-but-still-disproportionate patterns.
99+
100+
The owning tool is resolved by matching the pattern ID against each tool's
101+
`prefix` (longest match wins); patterns whose tool can't be resolved (no/unknown
102+
prefix) are dropped silently. The list is capped at 10 with a "… (N more)" note.
86103

87104
The suggested step depends on **how the pattern is managed**, since not every
88105
pattern can be disabled through the CLI:

src/commands/AGENTS.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -259,7 +259,7 @@ Keeps the two command handlers thin: they only supply the API-specific callbacks
259259
- **List mode** (default): card-style format sorted by severity (Error > High > Warning > Info)
260260
- **Overview mode** (`-O, --overview`): seven count tables — Category, Severity, Language, Tag, Pattern, Author, False Positives
261261
- The False Positives table relabels the API's raw bucket names via `FALSE_POSITIVE_LABELS`: `belowThreshold` → "Not a False Positive", `equalOrAboveThreshold` → "Potential False Positive" (the threshold is on FP probability, so at/above = potential FP — matching `printIssueCard`)
262-
- **Noise suggestions**: after the tables, `detectNoisyPatterns()` flags patterns that account for ≥10% of all issues (`NOISE_SHARE`) **or** have ≥3× the average issues-per-pattern (`NOISE_AVG_MULTIPLE`), sorted by count desc. For each, `resolvePatternTool()` maps the pattern ID to its owning tool by matching `Tool.prefix` (e.g. `Bandit_B101` → prefix `Bandit_`; longest match wins) against the global tool list (`fetchAllTools()` / `ToolsService.listTools`). Resolved ones print a `> codacy pattern <tool> <patternId> --disable` line under "Suggested actions to reduce noise". Patterns whose tool can't be resolved (no/unmatched prefix) are **silently discarded**. The tools fetch only happens when noisy patterns exist; JSON output is unaffected (raw counts only)
262+
- **Noise suggestions**: after the tables, `detectNoisyPatterns()` flags noisy patterns, sorted by count desc. A pattern must clear **two absolute floors** and show a **relative signal**: (1) the repo must have at least `NOISE_MIN_TOTAL` (200) issues total, else the whole section is suppressed — kept above `NOISE_MIN_PATTERN` so it does independent work (were they equal, any pattern clearing the per-pattern floor would already push the repo past an equal total floor, making it dead code); (2) the individual pattern must have at least `NOISE_MIN_PATTERN` (100) issues — this AND-gate is what stops a long tail of tiny patterns (which drags the median to ~3) from making a 9-issue pattern look "3× the median"; (3) *and* it must either account for ≥`NOISE_SHARE` (10%) of all issues **or** have ≥`NOISE_AVG_MULTIPLE` (3×) the **median** issues-per-pattern. The share rule only applies when there are at least `NOISE_MIN_PATTERNS_FOR_SHARE` (8) distinct patterns — with fewer patterns an even split already exceeds 10% each, so the signal is meaningless. The median (not the mean, via `medianOf()`) is deliberate: a single huge pattern would inflate a mean baseline and mask smaller-but-still-disproportionate patterns. For each noisy pattern, `resolvePatternTool()` maps the pattern ID to its owning tool by matching `Tool.prefix` (e.g. `Bandit_B101` → prefix `Bandit_`; longest match wins) against the global tool list (`fetchAllTools()` / `ToolsService.listTools`). Resolved ones print a `> codacy pattern <tool> <patternId> --disable` line under "Suggested actions to reduce noise". Patterns whose tool can't be resolved (no/unmatched prefix) are **silently discarded**. The tools fetch only happens when noisy patterns exist; JSON output is unaffected (raw counts only)
263263
- **Filters**: `--branch`, `--patterns`, `--tools`, `--severities`, `--categories`, `--languages`, `--tags`, `--authors`, `--limit`
264264
- **`--false-positives [value]`** (`-F`): tri-state filter — `true` (default when flag present) sends `onlyPotentialFalsePositives: true`, `false` sends `onlyPotentialFalsePositives: false`, omitted sends nothing
265265
- **`--ignore` mode** (`-I`): fetches all issues matching current filters (all pages), then calls `AnalysisService.bulkIgnoreIssues` in batches of 100

src/commands/issues.test.ts

Lines changed: 166 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -376,8 +376,9 @@ describe("issues command", () => {
376376
});
377377

378378
it("shows no suggestions and skips the tools fetch when nothing is noisy", async () => {
379-
// Twelve evenly-sized patterns: each is ~8.3% of total and equal to the
380-
// average, so none crosses the >=10% or >=3x-average thresholds.
379+
// Twelve evenly-sized patterns: each is ~8.3% of total (below the >=10%
380+
// share floor of 120) and equal to the median (below the >=3x-median floor
381+
// of 300), so none crosses either threshold.
381382
const patterns = Array.from({ length: 12 }, (_, i) => ({
382383
id: `Tool_${i}`,
383384
title: `Pattern ${i}`,
@@ -406,6 +407,169 @@ describe("issues command", () => {
406407
expect(output).not.toContain("Suggested actions to reduce noise");
407408
expect(ToolsService.listTools).not.toHaveBeenCalled();
408409
});
410+
411+
it("makes no suggestions below the absolute total floor", async () => {
412+
// A single pattern of 120 that clears the per-pattern floor (100) and is
413+
// disproportionate (median 6, 3x = 18; 80% share), so everything except the
414+
// total floor would flag it. But the repo has only 150 issues total, below
415+
// NOISE_MIN_TOTAL (200) — so the whole section is suppressed regardless.
416+
const patterns = [
417+
{ id: "Bandit_B101", title: "Use of assert detected", total: 120 },
418+
...Array.from({ length: 5 }, (_, i) => ({
419+
id: `Other_${i}`,
420+
title: `Pattern ${i}`,
421+
total: 6,
422+
})),
423+
];
424+
vi.mocked(AnalysisService.issuesOverview).mockResolvedValue({
425+
data: {
426+
counts: {
427+
categories: [],
428+
levels: [{ name: "Warning", total: 150 }],
429+
languages: [],
430+
tags: [],
431+
patterns,
432+
authors: [],
433+
potentialFalsePositives: [],
434+
},
435+
},
436+
} as any);
437+
438+
const program = createProgram();
439+
await program.parseAsync([
440+
"node", "test", "issues", "gh", "test-org", "test-repo", "--overview",
441+
]);
442+
443+
const output = getAllOutput();
444+
expect(output).not.toContain("Suggested actions to reduce noise");
445+
// Nothing is noisy, so the tools lookup is skipped entirely.
446+
expect(ToolsService.listTools).not.toHaveBeenCalled();
447+
});
448+
449+
it("does not apply the share rule when there are too few patterns", async () => {
450+
// Five patterns of comparable size: {120, 80, 80, 80, 80}, total 440.
451+
// The 120 is 27% of the total, but with only 5 patterns an even split is
452+
// already 20% each, so the share rule is disabled. And 120 < 3x median(80)
453+
// = 240, so the multiple rule doesn't fire either → nothing is noisy.
454+
const patterns = [
455+
{ id: "Bandit_B101", title: "Use of assert detected", total: 120 },
456+
...Array.from({ length: 4 }, (_, i) => ({
457+
id: `Other_${i}`,
458+
title: `Pattern ${i}`,
459+
total: 80,
460+
})),
461+
];
462+
vi.mocked(AnalysisService.issuesOverview).mockResolvedValue({
463+
data: {
464+
counts: {
465+
categories: [],
466+
levels: [{ name: "Warning", total: 440 }],
467+
languages: [],
468+
tags: [],
469+
patterns,
470+
authors: [],
471+
potentialFalsePositives: [],
472+
},
473+
},
474+
} as any);
475+
476+
const program = createProgram();
477+
await program.parseAsync([
478+
"node", "test", "issues", "gh", "test-org", "test-repo", "--overview",
479+
]);
480+
481+
const output = getAllOutput();
482+
expect(output).not.toContain("Suggested actions to reduce noise");
483+
expect(ToolsService.listTools).not.toHaveBeenCalled();
484+
});
485+
486+
it("flags a secondary pattern a mean baseline would have masked", async () => {
487+
// A mega-outlier (5000) drags the *mean* to 512, so the old 3x-mean floor
488+
// of 1536 would have hidden the genuinely-disproportionate 120-issue pattern
489+
// (24x the typical 5). The median floor (3x median 5 = 15) catches both, and
490+
// 120 clears the absolute per-pattern floor of 100.
491+
const patterns = [
492+
{ id: "Bandit_B608", title: "Possible SQL injection", total: 5000 },
493+
{ id: "Bandit_B101", title: "Use of assert detected", total: 120 },
494+
...Array.from({ length: 8 }, (_, i) => ({
495+
id: `Other_${i}`,
496+
title: `Pattern ${i}`,
497+
total: 5,
498+
})),
499+
];
500+
vi.mocked(AnalysisService.issuesOverview).mockResolvedValue({
501+
data: {
502+
counts: {
503+
categories: [],
504+
levels: [{ name: "Warning", total: 5160 }],
505+
languages: [],
506+
tags: [],
507+
patterns,
508+
authors: [],
509+
potentialFalsePositives: [],
510+
},
511+
},
512+
} as any);
513+
vi.mocked(ToolsService.listTools).mockResolvedValue(banditTool as any);
514+
vi.mocked(AnalysisService.listRepositoryTools).mockResolvedValue(
515+
banditRepoTool() as any,
516+
);
517+
vi.mocked(AnalysisService.listRepositoryToolPatterns).mockResolvedValue(
518+
banditPattern() as any,
519+
);
520+
521+
const program = createProgram();
522+
await program.parseAsync([
523+
"node", "test", "issues", "gh", "test-org", "test-repo", "--overview",
524+
]);
525+
526+
const output = getAllOutput().replace(/\x1b\[[0-9;]*m/g, "");
527+
expect(output).toContain("Suggested actions to reduce noise");
528+
// The mega-outlier and the secondary pattern are both suggested.
529+
expect(output).toContain("codacy pattern Bandit Bandit_B608 --disable");
530+
expect(output).toContain("codacy pattern Bandit Bandit_B101 --disable");
531+
// The typical 5-issue patterns are not.
532+
expect(output).not.toContain('Disable "Pattern 0"');
533+
});
534+
535+
it("does not flag a disproportionate pattern below the per-pattern floor", async () => {
536+
// A repo of 210 issues (above NOISE_MIN_TOTAL) where one pattern of 90 is
537+
// dominant — 43% share, and well above 3x the median of 8 (24). It clears
538+
// every gate except the per-pattern floor: 90 < NOISE_MIN_PATTERN (100). A
539+
// rule producing 90 issues isn't a flood worth disabling, so despite being
540+
// disproportionate it must not be suggested.
541+
const patterns = [
542+
{ id: "Bandit_B101", title: "Use of assert detected", total: 90 },
543+
...Array.from({ length: 15 }, (_, i) => ({
544+
id: `Other_${i}`,
545+
title: `Pattern ${i}`,
546+
total: 8,
547+
})),
548+
];
549+
vi.mocked(AnalysisService.issuesOverview).mockResolvedValue({
550+
data: {
551+
counts: {
552+
categories: [],
553+
levels: [{ name: "Warning", total: 210 }],
554+
languages: [],
555+
tags: [],
556+
patterns,
557+
authors: [],
558+
potentialFalsePositives: [],
559+
},
560+
},
561+
} as any);
562+
563+
const program = createProgram();
564+
await program.parseAsync([
565+
"node", "test", "issues", "gh", "test-org", "test-repo", "--overview",
566+
]);
567+
568+
const output = getAllOutput();
569+
expect(output).not.toContain("Suggested actions to reduce noise");
570+
// Nothing clears the absolute floor, so the tools lookup is skipped.
571+
expect(ToolsService.listTools).not.toHaveBeenCalled();
572+
});
409573
});
410574

411575
it("should pass filter options to the API body", async () => {

0 commit comments

Comments
 (0)