Skip to content

Commit 25891cb

Browse files
Update README.md
Signed-off-by: chichun-charlie-liu <57839396+chichun-charlie-liu@users.noreply.github.com>
1 parent 957173c commit 25891cb

1 file changed

Lines changed: 4 additions & 1 deletion

File tree

examples/GPTQ/README.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -72,7 +72,7 @@ This end-to-end example utilizes the common set of interfaces provided by `fms_m
7272
7373
```bash
7474
lm_eval --model hf \
75-
--model_args pretrained="Meta-Llama-3-8B-GPTQ,dtype=float16,autogptq=True,enforce_eager=True" \
75+
--model_args pretrained="Meta-Llama-3-8B-GPTQ,dtype=float16,gptqmodel=True=True,enforce_eager=True" \
7676
--tasks lambada_openai \
7777
--num_fewshot 5 \
7878
--device cuda:0 \
@@ -82,18 +82,21 @@ This end-to-end example utilizes the common set of interfaces provided by `fms_m
8282
## Example Test Results
8383
8484
- Unquantized Model
85+
-
8586
|Model | Tasks |Version|Filter|n-shot| Metric | |Value | |Stderr|
8687
|------------|--------------|------:|------|-----:|----------|---|-----:|---|-----:|
8788
| LLAMA3-8B |lambada_openai| 1|none | 5|acc |↑ |0.7103|± |0.0063|
8889
| | | |none | 5|perplexity|↓ |3.7915|± |0.0727|
8990
9091
- Quantized model with the settings showed above (`desc_act` default to False.)
92+
-
9193
|Model | Tasks |Version|Filter|n-shot| Metric | |Value | |Stderr|
9294
|------------|--------------|------:|------|-----:|----------|---|------:|---|-----:|
9395
| LLAMA3-8B |lambada_openai| 1|none | 5|acc |↑ |0.6365 |± |0.0067|
9496
| | | |none | 5|perplexity|↓ |5.9307 |± |0.1830|
9597
9698
- Quantized model with `desc_act` set to `True` (could improve the model quality, but at the cost of inference speed.)
99+
-
97100
|Model | Tasks |Version|Filter|n-shot| Metric | |Value | |Stderr|
98101
|------------|--------------|------:|------|-----:|----------|---|------:|---|-----:|
99102
| LLAMA3-8B |lambada_openai| 1|none | 5|acc |↑ |0.6193 |± |0.0068|

0 commit comments

Comments
 (0)