Skip to content

Commit 7a42a7e

Browse files
jorge-martinez-giljorge-martinez-gil
committed
New release
1 parent c85fdcd commit 7a42a7e

65 files changed

Lines changed: 3457 additions & 73 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/ISSUE_TEMPLATE/bug_report.yml‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
name: Bug report
2+
description: Report a problem with FeatFuse
3+
labels: [bug]
4+
body:
5+
- type: textarea
6+
id: what
7+
attributes: { label: What happened?, description: Describe the bug and what you expected. }
8+
validations: { required: true }
9+
- type: textarea
10+
id: repro
11+
attributes: { label: Reproduction, description: Command/config and steps. Paste the manifest.json if available. }
12+
- type: input
13+
id: version
14+
attributes: { label: FeatFuse version, placeholder: "0.2.0" }
15+
- type: textarea
16+
id: env
17+
attributes: { label: Environment, description: OS, Python, torch/transformers versions. }

‎.github/ISSUE_TEMPLATE/config.yml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
blank_issues_enabled: true
2+
contact_links:
3+
- name: Paper (arXiv:2408.08903)
4+
url: https://arxiv.org/abs/2408.08903
5+
about: Read the paper this benchmark accompanies.

‎.github/ISSUE_TEMPLATE/feature_contribution.yml‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
name: New feature / fusion / model / dataset
2+
description: Propose a new engineered feature, fusion strategy, encoder, or dataset
3+
labels: [enhancement, plugin]
4+
body:
5+
- type: dropdown
6+
id: kind
7+
attributes:
8+
label: Plugin kind
9+
options: [engineered feature, fusion strategy, code model/encoder, dataset, task/metric]
10+
validations: { required: true }
11+
- type: textarea
12+
id: motivation
13+
attributes: { label: Motivation, description: What scientific question does this enable? Any reference? }
14+
validations: { required: true }
15+
- type: textarea
16+
id: design
17+
attributes: { label: Proposed design, description: Sketch the class/registry name and inputs/outputs. }

‎.github/PULL_REQUEST_TEMPLATE.md‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
## Summary
2+
3+
<!-- What does this PR add/change? Link any related issue. -->
4+
5+
## Type
6+
- [ ] New engineered feature
7+
- [ ] New fusion strategy
8+
- [ ] New model / encoder
9+
- [ ] New dataset / task
10+
- [ ] Bug fix / docs / infra
11+
12+
## Checklist
13+
- [ ] Registered the plugin via the appropriate `@REGISTRY.register(...)` decorator
14+
- [ ] Added/updated tests (`pytest -q` passes locally)
15+
- [ ] Updated relevant docs (`docs/` and/or README)
16+
- [ ] No fabricated results; any reported numbers are reproducible from a config
17+
- [ ] `featfuse run -c configs/smoke.yaml` still succeeds

‎.github/workflows/ci.yml‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
name: CI
2+
3+
on:
4+
push:
5+
branches: [main, master]
6+
pull_request:
7+
workflow_dispatch:
8+
9+
jobs:
10+
test:
11+
runs-on: ubuntu-latest
12+
strategy:
13+
fail-fast: false
14+
matrix:
15+
python-version: ["3.9", "3.10", "3.11", "3.12"]
16+
steps:
17+
- uses: actions/checkout@v4
18+
- uses: actions/setup-python@v5
19+
with:
20+
python-version: ${{ matrix.python-version }}
21+
cache: pip
22+
- name: Install (core + dev)
23+
run: |
24+
python -m pip install --upgrade pip
25+
pip install -e ".[dev]"
26+
- name: Run test suite
27+
run: pytest -q
28+
- name: Smoke benchmark (CPU, no network)
29+
run: |
30+
featfuse info
31+
featfuse run -c configs/smoke.yaml

‎.github/workflows/reproduce.yml‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# Manually triggerable reproduction of the engineered-feature baseline on IR-Plag.
2+
# Produces the run directory (tables + figures) as a downloadable artifact.
3+
name: Reproduce baseline
4+
5+
on:
6+
workflow_dispatch:
7+
8+
jobs:
9+
reproduce:
10+
runs-on: ubuntu-latest
11+
steps:
12+
- uses: actions/checkout@v4
13+
- uses: actions/setup-python@v5
14+
with: { python-version: "3.11", cache: pip }
15+
- run: pip install -e ".[dev]"
16+
- run: featfuse run -c configs/classical_features_irplag.yaml --ablate
17+
- uses: actions/upload-artifact@v4
18+
with:
19+
name: irplag-baseline-run
20+
path: runs/classical_features_irplag/

‎.gitignore‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
__pycache__/
2+
*.py[cod]
3+
*.egg-info/
4+
build/
5+
dist/
6+
.pytest_cache/
7+
.coverage
8+
htmlcov/
9+
runs/
10+
!runs/.gitkeep
11+
.ipynb_checkpoints/
12+
.DS_Store
13+
*.class

‎CHANGELOG.md‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
# Changelog
2+
3+
All notable changes to this project are documented here. The format is based on
4+
[Keep a Changelog](https://keepachangelog.com/) and the project aims to follow
5+
[Semantic Versioning](https://semver.org/).
6+
7+
## [0.2.0] — Research infrastructure release
8+
9+
Turns the single-paper script into a reusable, extensible benchmark platform.
10+
11+
### Added
12+
- **Installable package** `featfuse` with a `featfuse` command-line interface
13+
(`run`, `ablate`, `importance`, `list`, `info`) and a plugin **registry** for
14+
features, fusion strategies, models and datasets.
15+
- **11 engineered features** across lexical, structural and execution families
16+
(generalising the paper's single execution-similarity feature). Features are
17+
pairwise, deterministic and reproducible without a JVM.
18+
- **5 neural fusion strategies**: `concat` (the paper's design), `gated`,
19+
`attention`, `residual`, `film`. The classification head now accepts an
20+
arbitrary feature vector + a swappable fusion module.
21+
- **Encoders**: HuggingFace wrapper (GraphCodeBERT/CodeBERT/UniXcoder/…) plus a
22+
dependency-free `ToyEncoder` so the pipeline runs without GPUs or downloads.
23+
- **Full metric suite**: accuracy, precision, recall, F1, MCC, balanced accuracy,
24+
ROC-AUC, PR-AUC, Brier score and Expected Calibration Error.
25+
- **Statistics**: bootstrap confidence intervals, McNemar's test and paired
26+
bootstrap difference tests.
27+
- **Automatic ablation studies** (leave-one-out + single-feature) and
28+
**feature importance** (permutation + native).
29+
- **Auto-generated outputs**: Markdown + LaTeX (booktabs) tables, publication-quality
30+
figures (calibration, metric comparison, feature correlation) and a `REPORT.md`.
31+
- **Reproducibility**: deterministic seeding and a `manifest.json` per run recording
32+
seed, config hash, git commit, platform and library versions.
33+
- **Tooling**: tests (pytest), GitHub Actions CI (Py 3.9–3.12), Dockerfile, Makefile,
34+
issue/PR templates, `CITATION.cff`/`CITATION.bib`, and how-to documentation.
35+
36+
### Fixed
37+
- The runner no longer ignores the published fixed splits and re-splits randomly;
38+
splits are loaded deterministically and *enriched* with the precomputed execution
39+
feature that the split files omitted.
40+
- Removed the hard-coded Windows dataset path; data location is configuration-driven.
41+
42+
### Changed
43+
- The original training script and notebook moved to `legacy/` for provenance.

‎CITATION.bib‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
@article{martinezgil2024graphcodebert,
2+
title = {Improving Source Code Similarity Detection Through GraphCodeBERT and Integration of Additional Features},
3+
author = {Martinez-Gil, Jorge},
4+
journal = {arXiv preprint arXiv:2408.08903},
5+
year = {2024},
6+
url = {https://arxiv.org/abs/2408.08903}
7+
}
8+
9+
@software{featfuse2025,
10+
title = {FeatFuse: An Open Benchmark for Feature Integration in Code Language Models},
11+
author = {Martinez-Gil, Jorge and contributors},
12+
year = {2025},
13+
version = {0.2.0},
14+
url = {https://github.com/jorge-martinez-gil/graphcodebert-feature-integration},
15+
note = {Software accompanying arXiv:2408.08903}
16+
}

‎CITATION.cff‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
cff-version: 1.2.0
2+
message: "If you use FeatFuse or this benchmark in your research, please cite the paper below."
3+
title: "FeatFuse: An Open Benchmark for Feature Integration in Code Language Models"
4+
abstract: >-
5+
Research infrastructure for studying how engineered features (lexical, structural,
6+
execution-based) complement pretrained code language models such as GraphCodeBERT,
7+
CodeBERT and UniXcoder, with reproducible benchmarks, ablation studies, statistical
8+
significance testing and automatic report generation.
9+
type: software
10+
authors:
11+
- family-names: Martinez-Gil
12+
given-names: Jorge
13+
repository-code: "https://github.com/jorge-martinez-gil/graphcodebert-feature-integration"
14+
license: MIT
15+
version: "0.2.0"
16+
keywords:
17+
- GraphCodeBERT
18+
- feature integration
19+
- feature fusion
20+
- code clone detection
21+
- code representation learning
22+
- code language models
23+
- benchmark
24+
preferred-citation:
25+
type: article
26+
title: "Improving Source Code Similarity Detection Through GraphCodeBERT and Integration of Additional Features"
27+
authors:
28+
- family-names: Martinez-Gil
29+
given-names: Jorge
30+
year: 2024
31+
journal: "arXiv preprint"
32+
url: "https://arxiv.org/abs/2408.08903"
33+
identifiers:
34+
- type: other
35+
value: "arXiv:2408.08903"
36+
description: "arXiv identifier"

0 commit comments

Comments
 (0)