Skip to content

Commit 6ac2e55

Browse files
jingxiaozhengcopybara-github
authored andcommitted
Internal code change.
PiperOrigin-RevId: 921091672
1 parent 21aa46d commit 6ac2e55

7 files changed

Lines changed: 887 additions & 1 deletion

README.md

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@
1919
- [⚡ Running Evaluations](#-running-evaluations)
2020
- [LiteRT LM Runners](#🤖-litert-lm-runners)
2121
- [Direct Native Library Runners](#🚀-direct-native-library-runners-huggingface-etc)
22+
- [Lighteval Framework](#🪶-lighteval-framework)
2223
- [🛠️ Custom Task CUJ](#️-custom-task-cuj)
2324
- [1. Prepare the Dataset](#1-prepare-the-dataset)
2425
- [2. Task Definition](#2-task-definition)
@@ -111,6 +112,9 @@ uv pip install "ai-edge-eval[hf]"
111112
# Install HuggingFace multimodal runner support (includes TorchVision)
112113
uv pip install "ai-edge-eval[hf-multimodal]"
113114

115+
# Install the Lighteval evaluation framework (alternative to the default lm-eval)
116+
uv pip install "ai-edge-eval[lighteval]"
117+
114118
# Install everything for local evaluation
115119
uv pip install "ai-edge-eval[all]"
116120
```
@@ -123,6 +127,9 @@ pip install "ai-edge-eval[hf]"
123127
# Install HuggingFace multimodal runner support (includes TorchVision)
124128
pip install "ai-edge-eval[hf-multimodal]"
125129

130+
# Install the Lighteval evaluation framework (alternative to the default lm-eval)
131+
pip install "ai-edge-eval[lighteval]"
132+
126133
# Install everything for local evaluation
127134
pip install "ai-edge-eval[all]"
128135
```
@@ -216,6 +223,51 @@ ai-edge-eval \
216223
> [!IMPORTANT]
217224
> For HuggingFace runners, `huggingface/repo` refers to the HuggingFace model ID, such as `Qwen/Qwen2.5-7B-Instruct` or `google/gemma-3-270m`.
218225
226+
### 🪶 Lighteval Framework
227+
228+
`--framework lighteval` is an alternative evaluation framework alongside the default `lm-eval`. Install via `ai-edge-eval[lighteval]` (see [Optional Dependency Groups](#-optional-dependency-groups)).
229+
230+
Supports two runner paths:
231+
232+
- `--runner litert-lm` — scores via the LiteRT-LM server's `/v1/chat/score` endpoint (auto-launched).
233+
- `--runner accelerate` — scores natively via lighteval's HuggingFace/Accelerate backend.
234+
235+
Supported tasks (from `model_eval/config/tasks.yaml`): `mmlu`, `arc:easy`, `arc:challenge`, `winogrande`, `ifeval`, `bigbench_hard`.
236+
237+
#### LiteRT-LM Runner
238+
239+
```bash
240+
ai-edge-eval \
241+
--runner litert-lm \
242+
--model-path litert-community/SmolLM2-360M-Instruct/SmolLM2_360M_instruct.litertlm \
243+
--device cpu \
244+
--tasks arc:easy \
245+
--framework lighteval \
246+
--batch-size 1 \
247+
--limit 20 \
248+
--output-dir your_result_directory
249+
```
250+
251+
#### Accelerate Runner (HuggingFace native)
252+
253+
```bash
254+
ai-edge-eval \
255+
--runner accelerate \
256+
--model-path HuggingFaceTB/SmolLM2-360M-Instruct \
257+
--device cpu \
258+
--tasks arc:easy \
259+
--framework lighteval \
260+
--batch-size 1 \
261+
--limit 20 \
262+
--output-dir your_result_directory
263+
```
264+
265+
> [!IMPORTANT]
266+
> Always pin `--batch-size 1` when using `--framework lighteval`. Without it, lighteval auto-picks the largest batch that fits, which can OOM on the LiteRT-LM runner and produces padding-dependent results across runs. We recommend using a small `--limit` (e.g., `--limit 1-5`) for generation and sampling tasks (`ifeval`, `bigbench_hard`) to perform quick smoke checks. This allows you to verify task configuration and gauge the overall evaluation size (including token generation volume) before launching comprehensive runs.
267+
268+
> [!NOTE]
269+
> Some known cross-path differences when running the same model under both lighteval runners: (1) the accelerate path injects the tokenizer's default system message; the litert-lm path doesn't — prompts differ on tasks without an explicit system message. (2) The accelerate path is currently non-deterministic across process invocations (upstream lighteval issue); the litert-lm path is byte-deterministic.
270+
219271
---
220272

221273
## 🛠️ Custom Task CUJ
Lines changed: 122 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,122 @@
1+
# Copyright 2026 The ODML Authors.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
"""CLI subprocess smoke test for the Lighteval framework integration.
16+
17+
Verifies that the `ai-edge-eval list-tasks --framework lighteval` entry point
18+
correctly loads and reports tasks from the internal allowlist, catching
19+
import-order regressions and CLI registration bugs in fresh interpreter
20+
instances.
21+
"""
22+
23+
import os
24+
import shutil
25+
import subprocess
26+
27+
from absl.testing import absltest
28+
29+
try:
30+
from google3.third_party.bazel_rules.rules_python.python.runfiles import runfiles # pylint: disable=g-import-not-at-top
31+
except ImportError:
32+
runfiles = None
33+
34+
35+
def _resolve_cli_bin() -> str:
36+
"""Find the ai-edge-eval entry point via Blaze runfiles or pip install."""
37+
if runfiles:
38+
r = runfiles.Create()
39+
if r:
40+
found = r.Rlocation("google3/third_party/py/ai_edge_eval/cli")
41+
if found and os.path.exists(found):
42+
return found
43+
44+
found = shutil.which("ai-edge-eval")
45+
if found and os.path.exists(found):
46+
return found
47+
48+
raise absltest.SkipTest("ai-edge-eval CLI binary not found via Bazel or PATH")
49+
50+
51+
class CliLightevalSmokeTest(absltest.TestCase):
52+
53+
def setUp(self):
54+
super().setUp()
55+
self.bin = _resolve_cli_bin()
56+
# Don't inherit a noisy parent env that could mask import errors.
57+
self.env = {**os.environ, "TRANSFORMERS_VERBOSITY": "error"}
58+
59+
def _run(self, *args: str) -> subprocess.CompletedProcess[str]:
60+
"""Executes the `ai-edge-eval` CLI binary with arguments in an isolated subprocess.
61+
62+
Captures stdout and stderr, enforces text-mode decoding, explicitly disables
63+
strict returncode checking (`check=False`), and applies a 60-second timeout.
64+
65+
Args:
66+
*args: Command-line arguments and flags to forward to the executable.
67+
68+
Returns:
69+
A `subprocess.CompletedProcess[str]` instance representing the finished
70+
invocation.
71+
"""
72+
return subprocess.run(
73+
[self.bin, *args],
74+
capture_output=True,
75+
text=True,
76+
env=self.env,
77+
timeout=60,
78+
check=False,
79+
)
80+
81+
def test_list_tasks_lighteval(self):
82+
"""list-tasks --framework lighteval returns the yaml allowlist and exits 0."""
83+
p = self._run("list-tasks", "--framework", "lighteval")
84+
self.assertEqual(p.returncode, 0, msg=p.stderr)
85+
out = p.stdout
86+
# Known tasks from model_eval/config/tasks.yaml (lighteval section).
87+
for task in ("arc:easy", "arc:challenge", "mmlu", "winogrande", "ifeval"):
88+
self.assertIn(
89+
task, out, msg=f"task {task!r} missing from list-tasks output"
90+
)
91+
92+
def test_list_runners_lighteval(self):
93+
"""list-runners --framework lighteval includes both runner paths and exits 0."""
94+
p = self._run("list-runners", "--framework", "lighteval")
95+
self.assertEqual(p.returncode, 0, msg=p.stderr)
96+
out = p.stdout
97+
for runner in ("litert-lm", "accelerate"):
98+
self.assertIn(
99+
runner, out, msg=f"runner {runner!r} missing from list-runners output"
100+
)
101+
102+
def test_list_args_lighteval(self):
103+
"""list-args --framework lighteval surfaces lighteval PipelineParameters fields."""
104+
p = self._run("list-args", "--framework", "lighteval")
105+
self.assertEqual(p.returncode, 0, msg=p.stderr)
106+
out = p.stdout
107+
# 'max_samples' is the field the adapter maps --limit onto; if this is
108+
# missing, the introspection path is broken.
109+
self.assertIn("max_samples", out)
110+
111+
def test_unknown_framework_is_handled(self):
112+
"""An unknown --framework value exits non-zero with a clear error.
113+
114+
This guards against silent fall-through that could mask a real
115+
misregistration of the lighteval framework type.
116+
"""
117+
p = self._run("list-tasks", "--framework", "definitely-not-a-framework")
118+
self.assertNotEqual(p.returncode, 0)
119+
120+
121+
if __name__ == "__main__":
122+
absltest.main()

0 commit comments

Comments
 (0)