fix lm-eval version conflict in puzzletron requirements

[j-rausch]

Summary

• lm-eval 0.4.10 has API changes that break lm_eval_hf.py, which monkey-patches HFLM.create_from_arg_obj
• Pin examples/puzzletron/requirements.txt from lm-eval==0.4.10 to lm-eval==0.4.8 to match examples/llm_eval/requirements.txt
• Add version compatibility warning to lm_eval_hf.py for early detection if versions drift again

Summary by CodeRabbit

• Chores
  • Pinned lm-eval to version 0.4.8 for the Puzzletron example to ensure consistent behavior across environments.
  • Added a startup compatibility check in the LM Eval example that emits a warning if an unexpected lm-eval version is detected, helping surface mismatches early.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 437d8445-c26a-4f4b-a5d4-8def499999fa

📥 Commits

Reviewing files that changed from the base of the PR and between 6bbb9d6 and 746f669.

📒 Files selected for processing (1)

• examples/llm_eval/lm_eval_hf.py

🚧 Files skipped from review as they are similar to previous changes (1)

• examples/llm_eval/lm_eval_hf.py

---

📝 Walkthrough

Walkthrough

The PR downgrades the lm-eval pin in a puzzletron requirements file and adds a top-level import plus a startup warning in examples/llm_eval/lm_eval_hf.py if lm_eval.__version__ does not start with "0.4.8".

Changes

Cohort / File(s)	Summary
Dependency version update examples/puzzletron/requirements.txt	Pinned lm-eval dependency changed from 0.4.10 to 0.4.8.
Runtime version check examples/llm_eval/lm_eval_hf.py	Added import lm_eval and a startup warnings.warn(...) if lm_eval.__version__ does not start with "0.4.8"; no other execution logic changed.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

🚥 Pre-merge checks | ✅ 4

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'fix lm-eval version conflict in puzzletron requirements' accurately and specifically describes the main change: resolving a version incompatibility by pinning lm-eval in the puzzletron requirements file.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Security Anti-Patterns	✅ Passed	The PR contains no instances of the 6 critical security anti-patterns defined in SECURITY.md. All trust_remote_code usages properly default to False and are user-configurable.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch jrausch/fix-lm-eval-version

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[kevalmorabia97]

/ok to test 6bbb9d6

[github-actions]

PR Preview Action v1.8.1
🚀 View preview at https://NVIDIA.github.io/Model-Optimizer/pr-preview/pr-1257/
Built to branch gh-pages at 2026-04-14 15:18 UTC. Preview will be ready when the GitHub Pages deployment is complete.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 76.45%. Comparing base (3f41819) to head (746f669).

Additional details and impacted files

@@                 Coverage Diff                 @@
##           feature/puzzletron    #1257   +/-   ##
===================================================
  Coverage               76.44%   76.45%           
===================================================
  Files                     454      454           
  Lines                   47913    47913           
===================================================
+ Hits                    36629    36632    +3     
+ Misses                  11284    11281    -3     

Flag	Coverage Δ
examples	41.95% <ø> (+<0.01%)	⬆️
unit	51.84% <ø> (-0.02%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.