Skip to content

Commit d190563

Browse files
github-actions[bot]Copilot
github-actions[bot]
and
Copilot
authored
Add Duplicate Code Detector workflow (#168)
This workflow automatically identifies duplicate code patterns across the codebase and suggests refactoring opportunities. It analyzes recent changes to catch new duplication early and creates focused, actionable issues. Source: https://github.com/github/gh-aw/blob/v0.45.5/.github/workflows/duplicate-code-detector.md Merge rate: 79% (76 merged PRs out of 96 proposed) Key features: - Semantic pattern matching beyond text similarity - Focuses on recently changed files - Creates max 3 issues per run to avoid overwhelm - Auto-assigns issues to @copilot for remediation - Language-agnostic approach Phase 1: No consistency issues found Phase 2: Proposing Duplicate Code Detector (79% merge rate) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent ebf240f commit d190563

3 files changed

Lines changed: 330 additions & 0 deletions

File tree

‎README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,7 @@ You can use the "/plan" agent to turn the reports into actionable issues which c
5252
- [📖 Glossary Maintainer](docs/glossary-maintainer.md) - Automatically maintain project glossary based on codebase changes
5353
- [🔗 Link Checker](docs/link-checker.md) - Daily automated link checker that finds and fixes broken links in documentation
5454
- [✨ Code Simplifier](docs/code-simplifier.md) - Automatically simplify recently modified code for improved clarity and maintainability
55+
- [🔍 Duplicate Code Detector](docs/duplicate-code-detector.md) - Identify duplicate code patterns and suggest refactoring opportunities
5556
- [⚡ Daily Progress](docs/daily-progress.md) - Automated daily feature development following a structured roadmap
5657
- [🧪 Daily Test Coverage Improver](docs/daily-test-improver.md) - Improve test coverage by adding meaningful tests to under-tested areas
5758
- [⚡ Daily Performance Improver](docs/daily-perf-improver.md) - Analyze and improve code performance through benchmarking and optimization

‎docs/duplicate-code-detector.md‎

Lines changed: 106 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,106 @@
1+
# 🔍 Duplicate Code Detector
2+
3+
The **Duplicate Code Detector** workflow automatically identifies duplicate code patterns across your codebase and suggests refactoring opportunities to improve maintainability.
4+
5+
## What It Does
6+
7+
This workflow runs daily to analyze recent code changes and detect duplicate patterns:
8+
9+
- **Analyzes Recent Changes**: Focuses on files modified in recent commits to catch new duplication early
10+
- **Semantic Analysis**: Uses intelligent pattern matching to find duplicates beyond simple text matching
11+
- **Creates Actionable Issues**: Generates focused issues (max 3 per run) for significant duplication patterns
12+
- **Assigns to Copilot**: Issues are automatically assigned to @copilot for potential automated remediation
13+
14+
## Why It's Valuable
15+
16+
Code duplication is a common source of technical debt that:
17+
18+
- Makes maintenance harder (fixes must be applied in multiple places)
19+
- Increases bug risk (inconsistent fixes across duplicated code)
20+
- Bloats the codebase unnecessarily
21+
- Reduces code clarity and understandability
22+
23+
The Duplicate Code Detector helps by:
24+
25+
- **Catching duplication early** before it spreads
26+
- **Providing specific recommendations** for refactoring
27+
- **Enabling automated remediation** through Copilot assignment
28+
- **Improving code quality** continuously over time
29+
30+
## How It Works
31+
32+
1. **Daily Schedule**: Runs automatically every day
33+
2. **Change Analysis**: Examines files modified in recent commits
34+
3. **Pattern Detection**: Searches for:
35+
- Identical or nearly identical functions
36+
- Repeated logic blocks
37+
- Similar implementations of the same functionality
38+
- Copy-pasted code with minor variations
39+
4. **Reporting**: Creates separate issues for each significant pattern found (threshold: >10 lines or 3+ instances)
40+
5. **Assignment**: Issues are assigned to @copilot for automated or manual remediation
41+
42+
## What Gets Detected
43+
44+
### Reported Patterns
45+
46+
- Identical or nearly identical functions in different files
47+
- Repeated code blocks that could be extracted to utilities
48+
- Similar classes or modules with overlapping functionality
49+
- Copy-pasted code with minor modifications
50+
- Duplicated business logic across components
51+
52+
### Excluded Patterns
53+
54+
- Standard boilerplate code (imports, package declarations)
55+
- Test setup/teardown code (acceptable duplication in tests)
56+
- Configuration files with similar structure
57+
- Language-specific patterns (constructors, getters/setters)
58+
- Small code snippets (<5 lines) unless highly repetitive
59+
60+
## Example Output
61+
62+
When duplication is detected, the workflow creates issues like:
63+
64+
````markdown
65+
# 🔍 Duplicate Code Detected: Error Handling Pattern in Parser Module
66+
67+
**Severity**: High
68+
**Occurrences**: 4 instances
69+
**Locations**:
70+
- `src/parser/json.ts` (lines 45-58)
71+
- `src/parser/xml.ts` (lines 67-80)
72+
- `src/parser/yaml.ts` (lines 34-47)
73+
- `src/parser/csv.ts` (lines 89-102)
74+
75+
## Refactoring Recommendations
76+
77+
1. **Extract Common Error Handler**
78+
- Create utility: `src/parser/error-handler.ts`
79+
- Estimated effort: 2-3 hours
80+
- Benefits: Single source of truth, easier to maintain
81+
````
82+
83+
## Merge Rate
84+
85+
Based on usage in the gh-aw repository, this workflow has demonstrated **79% merge rate** (76 merged PRs out of 96 proposed), indicating high practical value and acceptance by maintainers.
86+
87+
## Configuration
88+
89+
The workflow is configured to:
90+
91+
- Run daily on a schedule
92+
- Create max 3 issues per run to avoid overwhelm
93+
- Auto-expire issues after 2 days if not addressed
94+
- Skip test files, generated code, and workflow files
95+
- Focus on source code files only
96+
97+
## Getting Started
98+
99+
This workflow requires no additional setup - it works out of the box with any repository. Simply enable it and it will start analyzing your codebase for duplication patterns.
100+
101+
The workflow uses standard code exploration tools and git commands, making it applicable to any programming language and project structure.
102+
103+
## Learn More
104+
105+
- [GitHub Agentic Workflows Documentation](https://github.github.io/gh-aw/)
106+
- [Blog: Continuous Simplicity Workflows](https://github.github.io/gh-aw/blog/2026-01-13-meet-the-workflows-continuous-simplicity/)

‎workflows/duplicate-code-detector.md‎

Lines changed: 223 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,223 @@
1+
---
2+
name: Duplicate Code Detector
3+
description: Identifies duplicate code patterns across the codebase and suggests refactoring opportunities
4+
on:
5+
workflow_dispatch:
6+
schedule: daily
7+
permissions:
8+
contents: read
9+
issues: read
10+
pull-requests: read
11+
engine: codex
12+
safe-outputs:
13+
create-issue:
14+
expires: 2d
15+
title-prefix: "[duplicate-code] "
16+
labels: [code-quality, automated-analysis]
17+
assignees: copilot
18+
group: true
19+
max: 3
20+
timeout-minutes: 15
21+
strict: true
22+
---
23+
24+
# Duplicate Code Detection
25+
26+
Analyze code to identify duplicated patterns using semantic analysis. Report significant findings that require refactoring.
27+
28+
## Task
29+
30+
Detect and report code duplication by:
31+
32+
1. **Analyzing Recent Commits**: Review changes in the latest commits
33+
2. **Detecting Duplicated Code**: Identify similar or duplicated code patterns using semantic analysis
34+
3. **Reporting Findings**: Create a detailed issue if significant duplication is detected (threshold: >10 lines or 3+ similar patterns)
35+
36+
## Context
37+
38+
- **Repository**: ${{ github.repository }}
39+
- **Commit ID**: ${{ github.event.head_commit.id }}
40+
- **Triggered by**: @${{ github.actor }}
41+
42+
## Analysis Workflow
43+
44+
### 1. Changed Files Analysis
45+
46+
Identify and analyze modified files:
47+
- Determine files changed in the recent commits using `git log` and `git diff`
48+
- Focus on source code files (programming language files)
49+
- **Exclude test files** from analysis (files matching patterns: `*_test.*`, `*.test.*`, `*.spec.*`, `test_*.*`, or located in directories named `test`, `tests`, `__tests__`, or `spec`)
50+
- **Exclude generated files** and build artifacts
51+
- **Exclude workflow files** from analysis (files under `.github/workflows/*`)
52+
- Use code exploration tools to understand file structure
53+
- Read modified file contents to examine changes
54+
55+
### 2. Duplicate Detection
56+
57+
Apply analysis to find duplicates:
58+
59+
**Pattern Search**:
60+
- Search for duplication indicators using grep and code search:
61+
- Similar function signatures
62+
- Repeated logic blocks
63+
- Similar variable naming patterns
64+
- Near-identical code blocks
65+
- Look for functions with similar names across different files
66+
- Identify structural similarities in code organization
67+
68+
**Semantic Analysis**:
69+
- Compare code blocks for logical similarity beyond textual matching
70+
- Identify different implementations of the same functionality
71+
- Look for copy-paste patterns with minor variations
72+
73+
### 3. Duplication Evaluation
74+
75+
Assess findings to identify true code duplication:
76+
77+
**Duplication Types**:
78+
- **Exact Duplication**: Identical code blocks in multiple locations
79+
- **Structural Duplication**: Same logic with minor variations (different variable names, etc.)
80+
- **Functional Duplication**: Different implementations of the same functionality
81+
- **Copy-Paste Programming**: Similar code blocks that could be extracted into shared utilities
82+
83+
**Assessment Criteria**:
84+
- **Severity**: Amount of duplicated code (lines of code, number of occurrences)
85+
- **Impact**: Where duplication occurs (critical paths, frequently called code)
86+
- **Maintainability**: How duplication affects code maintainability
87+
- **Refactoring Opportunity**: Whether duplication can be easily refactored
88+
89+
### 4. Issue Reporting
90+
91+
Create separate issues for each distinct duplication pattern found (maximum 3 patterns per run). Each pattern should get its own issue to enable focused remediation.
92+
93+
**When to Create Issues**:
94+
- Only create issues if significant duplication is found (threshold: >10 lines of duplicated code OR 3+ instances of similar patterns)
95+
- **Create one issue per distinct duplication pattern** - do NOT bundle multiple patterns in a single issue
96+
- Limit to the top 3 most significant patterns if more are found
97+
- Use the `create_issue` tool from safe-outputs MCP **once for each pattern**
98+
99+
**Issue Contents for Each Pattern**:
100+
- **Executive Summary**: Brief description of this specific duplication pattern
101+
- **Duplication Details**: Specific locations and code blocks for this pattern only
102+
- **Severity Assessment**: Impact and maintainability concerns for this pattern
103+
- **Refactoring Recommendations**: Suggested approaches to eliminate this pattern
104+
- **Code Examples**: Concrete examples with file paths and line numbers for this pattern
105+
106+
## Detection Scope
107+
108+
### Report These Issues
109+
110+
- Identical or nearly identical functions in different files
111+
- Repeated code blocks that could be extracted to utilities
112+
- Similar classes or modules with overlapping functionality
113+
- Copy-pasted code with minor modifications
114+
- Duplicated business logic across components
115+
116+
### Skip These Patterns
117+
118+
- Standard boilerplate code (imports, exports, package declarations)
119+
- Test setup/teardown code (acceptable duplication in tests)
120+
- **All test files** (files matching: `*_test.*`, `*.test.*`, `*.spec.*`, `test_*.*`, or in `test/`, `tests/`, `__tests__/`, `spec/` directories)
121+
- **All workflow files** (files under `.github/workflows/*`)
122+
- Configuration files with similar structure
123+
- Language-specific patterns (constructors, getters/setters)
124+
- Small code snippets (<5 lines) unless highly repetitive
125+
- Generated code or vendored dependencies
126+
127+
### Analysis Depth
128+
129+
- **Primary Focus**: Files changed in recent commits (excluding test files and workflow files)
130+
- **Secondary Analysis**: Check for duplication with existing codebase
131+
- **Cross-Reference**: Look for patterns across the repository
132+
- **Historical Context**: Consider if duplication is new or existing
133+
134+
## Issue Template
135+
136+
For each distinct duplication pattern found, create a separate issue using this structure:
137+
138+
````markdown
139+
# 🔍 Duplicate Code Detected: [Pattern Name]
140+
141+
*Analysis of commit ${{ github.event.head_commit.id }}*
142+
143+
**Assignee**: @copilot
144+
145+
## Summary
146+
147+
[Brief overview of this specific duplication pattern]
148+
149+
## Duplication Details
150+
151+
### Pattern: [Description]
152+
- **Severity**: High/Medium/Low
153+
- **Occurrences**: [Number of instances]
154+
- **Locations**:
155+
- `path/to/file1.ext` (lines X-Y)
156+
- `path/to/file2.ext` (lines A-B)
157+
- **Code Sample**:
158+
````[language]
159+
[Example of duplicated code]
160+
````
161+
162+
## Impact Analysis
163+
164+
- **Maintainability**: [How this affects code maintenance]
165+
- **Bug Risk**: [Potential for inconsistent fixes]
166+
- **Code Bloat**: [Impact on codebase size]
167+
168+
## Refactoring Recommendations
169+
170+
1. **[Recommendation 1]**
171+
- Extract common functionality to: `suggested/path/utility.ext`
172+
- Estimated effort: [hours/complexity]
173+
- Benefits: [specific improvements]
174+
175+
2. **[Recommendation 2]**
176+
[... additional recommendations ...]
177+
178+
## Implementation Checklist
179+
180+
- [ ] Review duplication findings
181+
- [ ] Prioritize refactoring tasks
182+
- [ ] Create refactoring plan
183+
- [ ] Implement changes
184+
- [ ] Update tests
185+
- [ ] Verify no functionality broken
186+
187+
## Analysis Metadata
188+
189+
- **Analyzed Files**: [count]
190+
- **Detection Method**: Semantic code analysis
191+
- **Commit**: ${{ github.event.head_commit.id }}
192+
- **Analysis Date**: [timestamp]
193+
````
194+
195+
## Operational Guidelines
196+
197+
### Security
198+
- Never execute untrusted code or commands
199+
- Only use read-only analysis tools
200+
- Do not modify files during analysis
201+
202+
### Efficiency
203+
- Focus on recently changed files first
204+
- Use semantic analysis for meaningful duplication, not superficial matches
205+
- Stay within timeout limits (balance thoroughness with execution time)
206+
207+
### Accuracy
208+
- Verify findings before reporting
209+
- Distinguish between acceptable patterns and true duplication
210+
- Consider language-specific idioms and best practices
211+
- Provide specific, actionable recommendations
212+
213+
### Issue Creation
214+
- Create **one issue per distinct duplication pattern** - do NOT bundle multiple patterns in a single issue
215+
- Limit to the top 3 most significant patterns if more are found
216+
- Only create issues if significant duplication is found
217+
- Include sufficient detail for coding agents to understand and act on findings
218+
- Provide concrete examples with file paths and line numbers
219+
- Suggest practical refactoring approaches
220+
- Assign issue to @copilot for automated remediation
221+
- Use descriptive titles that clearly identify the specific pattern (e.g., "Duplicate Code: Error Handling Pattern in Parser Module")
222+
223+
**Objective**: Improve code quality by identifying and reporting meaningful code duplication that impacts maintainability. Focus on actionable findings that enable automated or manual refactoring.

0 commit comments

Comments
 (0)