Skip to content

Pull requests: NVIDIA-NeMo/Evaluator

New pull request New
15 Open 902 Closed
15 Open 902 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Assigned to nobody Loading
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

Add completions_logprob endpoint and extend BYOB scorers/datasets for Sovereign benchmarks community-request documentation Improvements or additions to documentation nemo-evaluator tests waiting-on-maintainers Waiting on maintainers to respond
#953 opened Apr 30, 2026 by kanishks-23 Loading…
1
fix(launcher): give auto-export more CPU time nemo-evaluator-launcher tests
#951 opened Apr 29, 2026 by piojanu Contributor Loading…
1
feat: add shared metric contract for scorer functions tests
#950 opened Apr 29, 2026 by SandyChapman • Draft
8
feat(scoring): add scorer_as_metric to close the two-way bridge tests
#946 opened Apr 28, 2026 by wprazuch Contributor • Draft
2
feat(tasks): Add RASB 26H1 evaluator integration CI community-request documentation Improvements or additions to documentation nemo-evaluator nemo-evaluator-launcher tests waiting-on-customer Waiting on the original author to respond
#943 opened Apr 27, 2026 by pbelcak Loading…
2
chore(beep boop 🤖): Bump uv.lock (main) (2026-04-27) nemo-evaluator nemo-evaluator-launcher
#941 opened Apr 27, 2026 by svcnvidia-nemo-ci Contributor Loading…
1 task
feat(scoring): expose item/sample/jinja_context on ScorerInput tests
#937 opened Apr 24, 2026 by wprazuch Contributor • Draft
2
feat(byob): let @benchmark accept TemplateMetric classes + instances tests
#932 opened Apr 23, 2026 by wprazuch Contributor • Draft
2
feat(scoring): add ergonomics helpers for TemplateMetric authoring tests
#931 opened Apr 23, 2026 by wprazuch Contributor • Draft
2
feat(scoring): add metric abstractions for NEL/NMP interop tests
#930 opened Apr 23, 2026 by wprazuch Contributor • Draft
2
feat(scoring): add NEL-NMP integration contracts (Approach 3) tests
#924 opened Apr 22, 2026 by wprazuch Contributor • Draft
2
Wprazuch/sdk reshape tests
#923 opened Apr 22, 2026 by wprazuch Contributor • Draft
2
fix(slurm): request zero GPUs when export falls back to GPU partition nemo-evaluator-launcher
#915 opened Apr 17, 2026 by AdamRajfer Contributor Loading…
feat: add generic heartbeat for periodic script execution during evaluation documentation Improvements or additions to documentation nemo-evaluator-launcher tests
#876 opened Mar 20, 2026 by AdamRajfer Contributor Loading…
4 tasks done
@AdamRajfer
8
feat: add local no-docker mode for nemo-evaluator-launcher community-request documentation Improvements or additions to documentation nemo-evaluator-launcher tests waiting-on-customer Waiting on the original author to respond
#786 opened Mar 1, 2026 by mishana Loading…
@prokotg
26
ProTip! Type g i on any issue or pull request to go back to the issue listing page.