-
Notifications
You must be signed in to change notification settings - Fork 450
Pull requests: huggingface/lighteval
Author
Label
Projects
Milestones
Reviews
Assignee
Sort
Pull requests list
fix(vllm): Enhance VLLMModel context size handling for batch inputs
#1205
opened Apr 6, 2026 by
paulovsantanas
Loading…
examples: add RAIL Score responsible AI custom task template
#1203
opened Apr 2, 2026 by
SumitVermakgp
Loading…
Add --load-tasks-multilingual and fix --custom-tasks for inspect backend
#1199
opened Mar 25, 2026 by
dzautner
Loading…
4 tasks done
[Bugfix] Check all responses when n>1 instead of only the first one
#1197
opened Mar 23, 2026 by
eldarkurtic
Contributor
Loading…
[Litellm Enhancement] Enable extra sampling args for litellm backend
#1195
opened Mar 20, 2026 by
eldarkurtic
Contributor
Loading…
[Bugfix] presence_penalty is silently dropped from sampling args in litellm backend
#1193
opened Mar 18, 2026 by
eldarkurtic
Contributor
Loading…
[Bugfix] litellm backend should iterate over docs in a split not entire dataset
#1192
opened Mar 18, 2026 by
eldarkurtic
Contributor
Loading…
Remove deprecated prompt_token_ids wrapping in vLLM backend
#1191
opened Mar 18, 2026 by
sihyeonn
Loading…
Fix litellm connection pool limiting concurrent_requests
#1190
opened Mar 18, 2026 by
sihyeonn
Loading…
feat(utils): show count of evaluated samples in Markdown summary table
#1188
opened Mar 13, 2026 by
anzzyspeaksgit
Loading…
Fix typos in math_comparison.py and sample_comparison.py
#1186
opened Mar 12, 2026 by
joshuaswanson
Loading…
squad_v2: include
unanswerable questions in evaluation
#1185
opened Mar 9, 2026 by
Matteovanypersele
Loading…
Redact model config credentials in saved and returned results
#1181
opened Mar 8, 2026 by
yangbaechu
Loading…
fix(normalizations): guard against index out of range in LogProbToken…
#1180
opened Mar 6, 2026 by
inakiLakunza
Loading…
[LiteLLM] Add cross-provider reasoning_effort support (first step) + token budget fixes
#1178
opened Mar 5, 2026 by
dyurchenko98
Loading…
Fix LiteLLM split iteration in greedy_until to avoid duplicate API requests
#1177
opened Mar 5, 2026 by
dyurchenko98
Loading…
Fix corpus reference orientation for chrF/chrF++/TER metrics
#1176
opened Mar 5, 2026 by
dyurchenko98
Loading…
FIX : handle empty choices in Doc.get_golds() to prevent IndexError
#1174
opened Feb 23, 2026 by
nandeanie
Loading…
Fix: pass through custom_tasks and enable multilingual in eval command
#1172
opened Feb 19, 2026 by
dzautner
Loading…
2 tasks done
Previous Next
ProTip!
Exclude everything labeled
bug with -label:bug.