Skip to content

Pull requests: EleutherAI/lm-evaluation-harness

New pull request New
334 Open 1,900 Closed
334 Open 1,900 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Assigned to nobody Loading
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

fix(afrixnli): use Jinja braces in prompt_1 doc_to_text so premise and hypothesis are substituted
#3932 opened Jul 13, 2026 by AmosBunde Loading…
2
Add LooGLE long-context generation tasks
#3929 opened Jul 12, 2026 by zbl1998-sdjn Loading…
4 tasks done
4
Greek basic language modeling benchmark suite
#3928 opened Jul 12, 2026 by alexliap Loading…
1
Add L-Eval closed-ended long-context tasks
#3927 opened Jul 12, 2026 by zbl1998-sdjn Loading…
4 tasks done
4
docs: add AI agent guidance
#3925 opened Jul 11, 2026 by pollychen-lab Loading…
2
1
fix(jsonschema_bench): add per-sample validation timeout to prevent eval hangs
#3923 opened Jul 10, 2026 by hancheolcho Loading…
1
style: fix pre-commit lint failures in anthropic_llms.py and global_piqa template (#3900)
#3922 opened Jul 9, 2026 by Anai-Guo Contributor Loading…
fix(longbench): skip blank leading line in code_sim_score extraction
#3921 opened Jul 8, 2026 by cameronshinn Loading…
2 tasks done
3
Add FLORES+ multilingual translation tasks and common configuration
#3920 opened Jul 8, 2026 by djstrong Contributor Loading…
Add SycoBench-600 baseline task
#3919 opened Jul 8, 2026 by debu-sinha Loading…
1
Add MedFailBench medical AI safety task
#3918 opened Jul 7, 2026 by goktugozkanmd Loading…
1
feat: add blimp_it italian grammar evaluation tasks
#3917 opened Jul 7, 2026 by cristianochesi Loading…
1
fix(huggingface): detect max_length from nested text_config (Gemma3 multimodal)
#3916 opened Jul 6, 2026 by OrionArchitekton Loading…
1
1
fix(infinitebench): convert feature spec to datasets.Features and correct split names
#3914 opened Jul 6, 2026 by KrishVenky Loading…
1
feat(metrics): add ECE and RMS calibration error metrics
#3913 opened Jul 5, 2026 by shadowmodder Loading…
1
1
add missing docstrings to print_writeout and get_sample_size in evaluator_utils.py
#3910 opened Jul 4, 2026 by RavSinghChandan Loading…
1
add missing docstrings to is_torch_available, is_transformers_available, hash_string, join_iters and group
#3909 opened Jul 4, 2026 by RavSinghChandan Loading…
1
Filter think cases
#3908 opened Jul 4, 2026 by gadavidd Loading…
1
Add README.md for medmcqa and medqa tasks
#3907 opened Jul 3, 2026 by PrakyathMC Loading…
1
Add Helium Market Resolution and Model Worldview tasks
#3906 opened Jul 3, 2026 by connerlambden Loading…
1 of 2 tasks
2
feat: add Turkish Clinical Source Support evaluation task (clinical_safety group)
#3903 opened Jul 2, 2026 by goktugozkanmd Loading…
1
2
Kangaroo math
#3902 opened Jul 1, 2026 by hugson-d Loading…
1
fix(filters): POSFilter.apply returns a list, not a generator
#3896 opened Jun 30, 2026 by WatchTree-19 Loading…
Add BIG-Bench Extra Hard (BBEH) task
#3894 opened Jun 29, 2026 by cgloriacc Loading…
2 tasks done
1
ci: pin all GitHub Actions to full commit SHAs to prevent supply-chain attacks
#3893 opened Jun 28, 2026 by XananasX7 Loading…
1
ProTip! no:milestone will show everything without a milestone.