Skip to content

Commit 67061ae

Browse files
authored
Update CWE Bench (#61)
* add new cwe bench * fix project_info * update readme * add CWE/CVE table in README
1 parent cfe522b commit 67061ae

4 files changed

Lines changed: 1696 additions & 1418 deletions

File tree

README.md

Lines changed: 17 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@
1010
⚠️ Code and data for the [ICLR 2025 Paper](https://arxiv.org/pdf/2405.17238) can be found in the v1 branch, license and citation below.
1111

1212
## 📰 News
13+
* **[Aug. 30, 2025]**: Updated CWE-Bench-Java with 93 new CVEs and 38 CWEs.
1314
* **[Jul. 10, 2025]**: IRIS v2 released, added support for 7 new CWEs.
1415

1516
## 👋 Overview
@@ -21,10 +22,25 @@ At a high level, IRIS takes a project and a CWE (vulnerability class, such as pa
2122

2223
### CWE-Bench-Java
2324
This repository also contains the dataset CWE-Bench-Java, presented in the paper [LLM-Assisted Static Analysis for Detecting Security Vulnerabilities](https://arxiv.org/abs/2405.17238).
24-
At a high level, this dataset contains 120 CVEs spanning 4 CWEs, namely path-traversal, OS-command injection, cross-site scripting, and code-injection. Each CVE includes the buggy and fixed source code of the project, along with the information of the fixed files and functions. We provide the seed information in this repository, and we provide scripts for fetching, patching, and building the repositories. The dataset collection process is illustrated in the figure below:
25+
At a high level, this dataset contains 213 CVEs spanning 49 CWEs. Some examples include path-traversal, OS-command injection, cross-site scripting, and code-injection. Each CVE includes the buggy and fixed source code of the project, along with the information of the fixed files and functions. We provide the seed information in this repository, and we provide scripts for fetching, patching, and building the repositories. The dataset collection process is illustrated in the figure below:
2526

2627
![cwe-bench graphic](docs/assets/dataset-collection.png)
2728

29+
The table below summarizes the number of CVEs in our dataset grouped by CWE category, with smaller categories (fewer than 5 CVEs) grouped together for compactness.
30+
31+
| CWE-ID | CVE Count |
32+
|--------|-----------|
33+
| CWE-22 | 60 |
34+
| CWE-79 | 38 |
35+
| CWE-94 | 23 |
36+
| CWE-78 | 13 |
37+
| CWE-502 | 7 |
38+
| CWE-611 | 6 |
39+
| CWE-200 | 5 |
40+
| CWE-287 | 5 |
41+
| CWE-400 | 5 |
42+
| Other CWEs (36 total) | 51 |
43+
2844
## 🚀 Set Up
2945
### Using Docker (Recommended)
3046
```bash

0 commit comments

Comments
 (0)