Skip to content

Commit d164aab

Browse files
alerizzoclaude
andcommitted
fix: address PR review on noise-suggestion thresholds
- Raise NOISE_MIN_PATTERNS_FOR_SHARE 8 -> 11 (Codacy): an even split of N patterns gives each 1/N share, which only drops below the 10% threshold once N > 10, so at 8-10 patterns a perfectly balanced repo would flag every pattern. Add a regression test for the 10-evenly-distributed-patterns case. - Rename NOISE_AVG_MULTIPLE -> NOISE_MEDIAN_MULTIPLE (Gemini): the constant now multiplies the median, not the mean, so the old name was misleading. Renamed across the declaration, its usage, and all docs. Co-Authored-By: Claude <noreply@anthropic.com>
1 parent 6a48684 commit d164aab

6 files changed

Lines changed: 58 additions & 15 deletions

File tree

.changeset/tune-issues-overview-noise-suggestions.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,8 @@ matters because a repository with a long tail of tiny patterns pulls the median
1010
pattern very low, which previously made a pattern with only a handful of issues look
1111
disproportionate — now a rule has to genuinely flood the repo before it's flagged. On top of
1212
those floors, a pattern must still show a relative signal: the "dominant share" rule (≥10% of
13-
all issues) only applies when there are at least 8 distinct patterns (with fewer, an even
14-
split already exceeds 10% each), and the "disproportionate count" rule now compares each
13+
all issues) only applies when there are at least 11 distinct patterns (an even split of N
14+
patterns only drops below 10% once N is above 10, so 8-10 balanced patterns would otherwise
15+
all be flagged), and the "disproportionate count" rule now compares each
1516
pattern against the **median** issues-per-pattern instead of the mean, so a single huge
1617
pattern can no longer inflate the baseline and hide smaller-but-still-disproportionate ones.

SPECS/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -74,4 +74,4 @@ _No pending tasks._ All commands implemented.
7474
| 2026-06-24 | `findings` and `finding` now surface the vulnerable dependency's import chain from the new `dependencyChains` field: Direct (`Update <pkg> to <fixed>`) vs Transitive (`<chain> (Fixed in <fixed>)`), with the middle collapsed to `... N more ...` for 4+ packages. List shows the first chain + `... and X more`; detail shows all chains aligned under a single label. New helpers in `utils/formatting.ts` (`formatDependencyChain`, `formatDependencyChainsLine`, `formatDependencyChainsBlock`); `dependencyChains` added to both JSON projections (17 new tests, 390 total) |
7575
| 2026-06-30 | npm-style "update available" notice via `update-notifier@5`: one-time stderr hint when a newer version is published, gated to `--output table` (suppressed for `json`, when piped, in CI, under `npx`). Non-blocking daily background check; never auto-updates. New `src/version.ts` (single source of name/version) + `src/utils/update-check.ts` (`maybeNotifyUpdate`); `preAction` hook + `--no-update-notifier` flag wired in `index.ts`. Opt-outs: `CODACY_DISABLE_UPDATE_CHECK`, `NO_UPDATE_NOTIFIER`, `--no-update-notifier`. `package.json` `overrides` pin transitive `got@^11.8.6`/`package-json@^7` (CVE-2022-33987) (7 new tests, 409 total) |
7676
| 2026-07-07 | `ls` and `directories` commands: browse a repository's folders/files with quality metrics (Grade/Issues/Complexity/Duplication/Coverage). Auto-detect provider/org/repo and the cwd-relative path; `--path`/`--branch` options; `directories --plus-children` shows one extra level as a `└─` tree (header adds `, M subdirectories`). `--sort`/`--direction` (server-side; in `ls`, directories and files sorted independently), and `ls --search <term>` (files only; folds the path into the search as `<path>/%<term>` and shows full paths). Duplication uses `numberOfClones`; Complexity uses `complexity` (hotspots). Both fetch **all** pages (no pagination warning) via new `src/utils/repo-tree.ts` (path resolution + `resolveSort`/`resolveDirection` + `fetchAllDirectories`/`fetchAllFiles`). Row markers `` folder / dim `·` file (no emojis). Promoted `formatGrade` to `utils/formatting.ts` (now also colors E red) and added `formatCountCell`/`formatCoverageCell` (48 new tests, 457 total) |
77-
| 2026-07-08 | `issues --overview` noise suggestions tuned to stop firing on low-volume repos: added a `NOISE_MIN_TOTAL` (200) floor on the repo's total issues that suppresses the whole "reduce noise" section below it, and a `NOISE_MIN_PATTERN` (100) absolute floor on each pattern's own count (AND-gated with the relative rules) so a long tail of tiny patterns can't drag the median down and make a ~9-issue pattern look noisy — the total floor is kept above the per-pattern floor so it isn't dead code; the ≥10% share rule now only applies with ≥8 distinct patterns (`NOISE_MIN_PATTERNS_FOR_SHARE`, since with fewer an even split already exceeds 10%); and the ≥3× multiple rule now measures against the **median** (via new `medianOf()`) instead of the mean, so a single huge pattern can no longer inflate the baseline and mask smaller disproportionate patterns (4 new tests, 464 total) |
77+
| 2026-07-08 | `issues --overview` noise suggestions tuned to stop firing on low-volume repos: added a `NOISE_MIN_TOTAL` (200) floor on the repo's total issues that suppresses the whole "reduce noise" section below it, and a `NOISE_MIN_PATTERN` (100) absolute floor on each pattern's own count (AND-gated with the relative rules) so a long tail of tiny patterns can't drag the median down and make a ~9-issue pattern look noisy — the total floor is kept above the per-pattern floor so it isn't dead code; the ≥10% share rule now only applies with ≥11 distinct patterns (`NOISE_MIN_PATTERNS_FOR_SHARE`an even split only drops below 10% once N > 10, so 8–10 balanced patterns would otherwise all be flagged); and the ≥3× multiple rule (`NOISE_MEDIAN_MULTIPLE`) now measures against the **median** (via new `medianOf()`) instead of the mean, so a single huge pattern can no longer inflate the baseline and mask smaller disproportionate patterns (5 new tests, 465 total) |

SPECS/commands/issues.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -92,8 +92,10 @@ signal:
9292
median down (e.g. to 3) so far that a pattern with only ~9 issues clears the
9393
relative bar below — yet 9 issues is nothing worth disabling a rule over.
9494
- **Relative signal** (either one): accounts for **≥10% of all issues** shown
95-
(only applied when there are **≥8 distinct patterns**, since with fewer an even
96-
split already exceeds 10% each), **or** has **≥3× the median** issues-per-pattern.
95+
(only applied when there are **≥11 distinct patterns** — an even split of N
96+
patterns gives each `1/N`, which only drops below 10% once N > 10, so with 8–10
97+
patterns a balanced repo would otherwise flag every one), **or** has **≥3× the
98+
median** issues-per-pattern.
9799
The median (not the mean) is used so a single huge pattern can't inflate the
98100
baseline and mask smaller-but-still-disproportionate patterns.
99101

src/commands/AGENTS.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -259,7 +259,7 @@ Keeps the two command handlers thin: they only supply the API-specific callbacks
259259
- **List mode** (default): card-style format sorted by severity (Error > High > Warning > Info)
260260
- **Overview mode** (`-O, --overview`): seven count tables — Category, Severity, Language, Tag, Pattern, Author, False Positives
261261
- The False Positives table relabels the API's raw bucket names via `FALSE_POSITIVE_LABELS`: `belowThreshold` → "Not a False Positive", `equalOrAboveThreshold` → "Potential False Positive" (the threshold is on FP probability, so at/above = potential FP — matching `printIssueCard`)
262-
- **Noise suggestions**: after the tables, `detectNoisyPatterns()` flags noisy patterns, sorted by count desc. A pattern must clear **two absolute floors** and show a **relative signal**: (1) the repo must have at least `NOISE_MIN_TOTAL` (200) issues total, else the whole section is suppressed — kept above `NOISE_MIN_PATTERN` so it does independent work (were they equal, any pattern clearing the per-pattern floor would already push the repo past an equal total floor, making it dead code); (2) the individual pattern must have at least `NOISE_MIN_PATTERN` (100) issues — this AND-gate is what stops a long tail of tiny patterns (which drags the median to ~3) from making a 9-issue pattern look "3× the median"; (3) *and* it must either account for ≥`NOISE_SHARE` (10%) of all issues **or** have ≥`NOISE_AVG_MULTIPLE` (3×) the **median** issues-per-pattern. The share rule only applies when there are at least `NOISE_MIN_PATTERNS_FOR_SHARE` (8) distinct patterns — with fewer patterns an even split already exceeds 10% each, so the signal is meaningless. The median (not the mean, via `medianOf()`) is deliberate: a single huge pattern would inflate a mean baseline and mask smaller-but-still-disproportionate patterns. For each noisy pattern, `resolvePatternTool()` maps the pattern ID to its owning tool by matching `Tool.prefix` (e.g. `Bandit_B101` → prefix `Bandit_`; longest match wins) against the global tool list (`fetchAllTools()` / `ToolsService.listTools`). Resolved ones print a `> codacy pattern <tool> <patternId> --disable` line under "Suggested actions to reduce noise". Patterns whose tool can't be resolved (no/unmatched prefix) are **silently discarded**. The tools fetch only happens when noisy patterns exist; JSON output is unaffected (raw counts only)
262+
- **Noise suggestions**: after the tables, `detectNoisyPatterns()` flags noisy patterns, sorted by count desc. A pattern must clear **two absolute floors** and show a **relative signal**: (1) the repo must have at least `NOISE_MIN_TOTAL` (200) issues total, else the whole section is suppressed — kept above `NOISE_MIN_PATTERN` so it does independent work (were they equal, any pattern clearing the per-pattern floor would already push the repo past an equal total floor, making it dead code); (2) the individual pattern must have at least `NOISE_MIN_PATTERN` (100) issues — this AND-gate is what stops a long tail of tiny patterns (which drags the median to ~3) from making a 9-issue pattern look "3× the median"; (3) *and* it must either account for ≥`NOISE_SHARE` (10%) of all issues **or** have ≥`NOISE_MEDIAN_MULTIPLE` (3×) the **median** issues-per-pattern. The share rule only applies when there are at least `NOISE_MIN_PATTERNS_FOR_SHARE` (11) distinct patterns — an even split of N patterns gives each `1/N`, which only drops below 10% once N > 10, so with 8–10 patterns a perfectly balanced repo would otherwise flag every pattern. The median (not the mean, via `medianOf()`) is deliberate: a single huge pattern would inflate a mean baseline and mask smaller-but-still-disproportionate patterns. For each noisy pattern, `resolvePatternTool()` maps the pattern ID to its owning tool by matching `Tool.prefix` (e.g. `Bandit_B101` → prefix `Bandit_`; longest match wins) against the global tool list (`fetchAllTools()` / `ToolsService.listTools`). Resolved ones print a `> codacy pattern <tool> <patternId> --disable` line under "Suggested actions to reduce noise". Patterns whose tool can't be resolved (no/unmatched prefix) are **silently discarded**. The tools fetch only happens when noisy patterns exist; JSON output is unaffected (raw counts only)
263263
- **Filters**: `--branch`, `--patterns`, `--tools`, `--severities`, `--categories`, `--languages`, `--tags`, `--authors`, `--limit`
264264
- **`--false-positives [value]`** (`-F`): tri-state filter — `true` (default when flag present) sends `onlyPotentialFalsePositives: true`, `false` sends `onlyPotentialFalsePositives: false`, omitted sends nothing
265265
- **`--ignore` mode** (`-I`): fetches all issues matching current filters (all pages), then calls `AnalysisService.bulkIgnoreIssues` in batches of 100

src/commands/issues.test.ts

Lines changed: 39 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -209,8 +209,10 @@ describe("issues command", () => {
209209
});
210210

211211
describe("overview noise suggestions", () => {
212-
// One dominant pattern plus nine small ones: total 2950, avg 295.
213-
// Bandit_B101 (2500) is both >=10% of total and >=3x average → noisy.
212+
// One dominant pattern plus nine small ones: total 2950, median 50.
213+
// Bandit_B101 (2500) clears the per-pattern floor (100) and is >=3x the
214+
// median (150) → noisy. (With only 10 patterns the share rule is off, so the
215+
// median rule is what flags it here.)
214216
function noisyOverview() {
215217
const patterns = [
216218
{ id: "Bandit_B101", title: "Use of assert detected", total: 2500 },
@@ -408,6 +410,41 @@ describe("issues command", () => {
408410
expect(ToolsService.listTools).not.toHaveBeenCalled();
409411
});
410412

413+
it("does not flag every pattern in a perfectly balanced repo", async () => {
414+
// Ten evenly-distributed patterns of 100 (total 1000): each is exactly 10%
415+
// of the total and clears the per-pattern floor, so a share threshold of 10
416+
// would flag ALL of them. NOISE_MIN_PATTERNS_FOR_SHARE (11) disables the
417+
// share rule here (10 < 11), and the median rule can't fire on a flat
418+
// distribution (100 < 3x median 100), so nothing is flagged.
419+
const patterns = Array.from({ length: 10 }, (_, i) => ({
420+
id: `Tool_${i}`,
421+
title: `Pattern ${i}`,
422+
total: 100,
423+
}));
424+
vi.mocked(AnalysisService.issuesOverview).mockResolvedValue({
425+
data: {
426+
counts: {
427+
categories: [],
428+
levels: [{ name: "Warning", total: 1000 }],
429+
languages: [],
430+
tags: [],
431+
patterns,
432+
authors: [],
433+
potentialFalsePositives: [],
434+
},
435+
},
436+
} as any);
437+
438+
const program = createProgram();
439+
await program.parseAsync([
440+
"node", "test", "issues", "gh", "test-org", "test-repo", "--overview",
441+
]);
442+
443+
const output = getAllOutput();
444+
expect(output).not.toContain("Suggested actions to reduce noise");
445+
expect(ToolsService.listTools).not.toHaveBeenCalled();
446+
});
447+
411448
it("makes no suggestions below the absolute total floor", async () => {
412449
// A single pattern of 120 that clears the per-pattern floor (100) and is
413450
// disproportionate (median 6, 3x = 18; 80% share), so everything except the

src/commands/issues.ts

Lines changed: 10 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -51,16 +51,19 @@ const BULK_BATCH_SIZE = 100;
5151
// enough:
5252
// - share: it alone accounts for at least NOISE_SHARE of all issues. Only applied
5353
// when there are at least NOISE_MIN_PATTERNS_FOR_SHARE distinct patterns — below
54-
// that, an even split already puts every pattern above the threshold (e.g. with
55-
// 5 patterns the average share is 20%), so the signal is meaningless.
56-
// - multiple: it has at least NOISE_AVG_MULTIPLE times the *median* issues-per-
54+
// that, an even split already puts every pattern at or above the threshold, so
55+
// the signal is meaningless. The floor is 11 because an even split of N patterns
56+
// gives each a 1/N share, and 1/N only drops below NOISE_SHARE (10%) once N > 10
57+
// — with 8, 9, or 10 patterns a perfectly balanced repo (12.5% / 11.1% / 10%
58+
// each) would otherwise flag every pattern.
59+
// - multiple: it has at least NOISE_MEDIAN_MULTIPLE times the *median* issues-per-
5760
// pattern. Using the median (not the mean) keeps one huge pattern from inflating
5861
// the baseline and masking smaller-but-still-disproportionate patterns.
5962
const NOISE_MIN_TOTAL = 200;
6063
const NOISE_MIN_PATTERN = 100;
6164
const NOISE_SHARE = 0.1;
62-
const NOISE_MIN_PATTERNS_FOR_SHARE = 8;
63-
const NOISE_AVG_MULTIPLE = 3;
65+
const NOISE_MIN_PATTERNS_FOR_SHARE = 11;
66+
const NOISE_MEDIAN_MULTIPLE = 3;
6467
// Cap how many disable suggestions we print, to keep the section actionable.
6568
const MAX_NOISE_SUGGESTIONS = 10;
6669

@@ -236,15 +239,15 @@ function medianOf(values: number[]): number {
236239
* when it clears the absolute per-pattern floor (NOISE_MIN_PATTERN) AND shows a
237240
* relative signal: it accounts for at least NOISE_SHARE of all issues (share
238241
* rule — only when there are at least NOISE_MIN_PATTERNS_FOR_SHARE patterns) or
239-
* has at least NOISE_AVG_MULTIPLE times the median issues-per-pattern (multiple
242+
* has at least NOISE_MEDIAN_MULTIPLE times the median issues-per-pattern (multiple
240243
* rule). Sorted by count desc. See the NOISE_* constants for the rationale.
241244
*/
242245
function detectNoisyPatterns(patterns: PatternsCount[]): PatternsCount[] {
243246
if (patterns.length === 0) return [];
244247
const total = patterns.reduce((sum, p) => sum + p.total, 0);
245248
if (total < NOISE_MIN_TOTAL) return [];
246249
const shareFloor = NOISE_SHARE * total;
247-
const medianFloor = NOISE_AVG_MULTIPLE * medianOf(patterns.map((p) => p.total));
250+
const medianFloor = NOISE_MEDIAN_MULTIPLE * medianOf(patterns.map((p) => p.total));
248251
const shareApplies = patterns.length >= NOISE_MIN_PATTERNS_FOR_SHARE;
249252
return patterns
250253
.filter(

0 commit comments

Comments
 (0)