Skip to content

Commit dc63fea

Browse files
committed
[Bugfix] Add pred and choices parsing to fix the issue of score=0 for mmstar datasets
1 parent 3ce3c94 commit dc63fea

2 files changed

Lines changed: 1 addition & 11 deletions

File tree

.gitignore

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -6,8 +6,3 @@ test_reports
66
.venv
77
.eggs
88
ais_bench_benchmark.egg-info
9-
10-
ais_bench/datasets/**/*/*
11-
!ais_bench/datasets/**/*/*.py
12-
outputs
13-
tmp

ais_bench/benchmark/datasets/mmstar.py

Lines changed: 1 addition & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -20,15 +20,10 @@
2020
logger = AISLogger()
2121

2222

23-
def extract_options_from_question(question_text):
23+
def extract_options_from_question(question_text:str):
2424
options = {}
25-
26-
# 查找"Options:"部分
2725
if "Options:" in question_text:
28-
# 获取Options部分
2926
options_part = question_text.split("Options:")[1].strip()
30-
31-
# 使用正则表达式匹配所有A-Z的选项
3227
pattern = r"([A-Z]):\s*([^,]+(?:,\s*[^,]+)*?)(?=(?:,\s*[A-Z]:|$))"
3328

3429
matches = re.findall(pattern, options_part)

0 commit comments

Comments
 (0)