Skip to content

Commit 6cc6671

Browse files
committed
[refactor] rename ACE-Bench to FeatureBench
1 parent 8c485fe commit 6cc6671

111 files changed

Lines changed: 420 additions & 708 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.gitignore

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -7,11 +7,11 @@ cases_filter/*
77
*outputs/
88
*log/
99
*logs/
10-
acebench/resources/repos
11-
acebench/resources/constants/*
12-
!acebench/resources/constants/python_example.py
13-
!acebench/resources/constants/full_images.txt
14-
!acebench/resources/constants/lite_images.txt
10+
featurebench/resources/repos
11+
featurebench/resources/constants/*
12+
!featurebench/resources/constants/python_example.py
13+
!featurebench/resources/constants/full_images.txt
14+
!featurebench/resources/constants/lite_images.txt
1515
config.toml
1616
tmp/
1717
runs/

README.md

Lines changed: 22 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -6,32 +6,33 @@
66
<a href=""><img src="https://img.shields.io/badge/arXiv-25xx.xxxxx-b31b1b.svg" alt="arXiv"></a>
77
<a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-green.svg" alt="License"></a>
88
<a href="https://hub.docker.com/u/libercoders"><img src="https://img.shields.io/badge/DockerHub-Images-blue.svg" alt="DockerHub"></a>
9-
<a href="https://huggingface.co/datasets/LiberCoders/ACE-Bench"><img src="https://img.shields.io/badge/HuggingFace-datasets-yellow.svg" alt="HuggingFace"></a>
10-
<a href="https://LiberCoders.github.io/ACE-Bench/"><img src="https://img.shields.io/badge/Leaderboard-view-purple.svg" alt="Leaderboard"></a>
9+
<a href="https://huggingface.co/datasets/LiberCoders/FeatureBench"><img src="https://img.shields.io/badge/HuggingFace-datasets-yellow.svg" alt="HuggingFace"></a>
10+
<a href="https://LiberCoders.github.io/FeatureBench/"><img src="https://img.shields.io/badge/Leaderboard-view-purple.svg" alt="Leaderboard"></a>
1111
</p>
1212

1313
---
1414

15-
ACE-Bench is a test-driven data generation and evaluation pipeline for feature-level coding benchmarks.
15+
FeatureBench is a test-driven data generation and evaluation pipeline for feature-level coding benchmarks.
1616
It provides a unified CLI to run inference, evaluation, and dataset generation.
1717

1818
## 📰 News
1919

20-
🎁 **2026.02.06**: We now support one-click inference for mainstream agent frameworks, including **OpenHands, Claude Code, Codex, Gemini CLI, and mini-swe-agent**. All supported agent frameworks can be found [here](acebench/infer/agents/). We have also open-sourced the ACE-Bench **data pipeline**.
20+
🎁 **2026.02.06**: We now support one-click inference for mainstream agent frameworks, including **OpenHands, Claude Code, Codex, Gemini CLI, and mini-swe-agent**. All supported agent frameworks can be found [here](featurebench/infer/agents/). We have also open-sourced the FeatureBench **data pipeline**.
2121

2222
## 🚀 Quickstart
2323

2424
**Prerequisites:**
25-
- `uv` for Python environment management
26-
- `docker` for reproducible builds and evaluation
25+
- [uv](https://docs.astral.sh/uv/getting-started/installation/) for Python environment management
26+
- [docker](https://docs.docker.com/engine/install/) for reproducible builds and evaluation
2727

2828
```bash
2929
# pypi
30-
pip install ace-bench
30+
pip install featurebench
31+
# or uv add featurebench
3132

3233
# local
33-
git clone https://github.com/LiberCoders/ACE-Bench.git
34-
cd ACE-Bench
34+
git clone https://github.com/LiberCoders/FeatureBench.git
35+
cd FeatureBench
3536
uv sync
3637
```
3738

@@ -43,17 +44,17 @@ See [docs/config.md](docs/config.md) for a comprehensive reference (harness, inf
4344

4445
**Optional: pre-pull images to reduce network variance:**
4546
```bash
46-
ace pull --mode lite # lite split image list (13 images)
47-
ace pull --mode full # full split image list (24 images)
48-
ace pull --mode /path/to/images.txt # one image name per line
47+
fb pull --mode lite # lite split image list (13 images)
48+
fb pull --mode full # full split image list (24 images)
49+
fb pull --mode /path/to/images.txt # one image name per line
4950

50-
# full list: acebench/resources/constants/full_images.txt
51-
# lite list: acebench/resources/constants/lite_images.txt
51+
# full list: featurebench/resources/constants/full_images.txt
52+
# lite list: featurebench/resources/constants/lite_images.txt
5253
```
5354

5455
**Run inference:**
5556
```bash
56-
ace infer \
57+
fb infer \
5758
--config-path config.toml \
5859
--agent mini_swe_agent \
5960
--model openai/qwen3-coder-480b-a35b-instruct \
@@ -62,21 +63,21 @@ ace infer \
6263

6364
**Run evaluation:**
6465
```bash
65-
ace eval \
66+
fb eval \
6667
-p runs/<timestamp>/output.jsonl \
6768
--split lite
6869
```
6970

7071
## 🧭 CLI Overview
7172

72-
`ace` provides three core commands:
73-
- `ace infer` runs `acebench.infer.run_infer` (docs: [docs/infer_cli_arg.md](docs/infer_cli_arg.md))
74-
- `ace eval` runs `acebench.harness.run_evaluation` (docs: [docs/harness_cli_arg.md](docs/harness_cli_arg.md))
75-
- `ace data` runs `acebench.pipeline` (docs: [docs/pipeline.md](docs/pipeline.md))
73+
`fb` provides three core commands:
74+
- `fb infer` runs `featurebench.infer.run_infer` (docs: [docs/infer_cli_arg.md](docs/infer_cli_arg.md))
75+
- `fb eval` runs `featurebench.harness.run_evaluation` (docs: [docs/harness_cli_arg.md](docs/harness_cli_arg.md))
76+
- `fb data` runs `featurebench.pipeline` (docs: [docs/pipeline.md](docs/pipeline.md))
7677

7778
## ✍️ Citation
7879

79-
If you found ACE-Bench useful, please cite us as:
80+
If you found FeatureBench useful, please cite us as:
8081

8182
```bibtex
8283
xxx

README_bk.md

Lines changed: 0 additions & 287 deletions
This file was deleted.

0 commit comments

Comments
 (0)