forked from r00ckky/Module2
-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathoutputs.txt
More file actions
63 lines (52 loc) · 4.6 KB
/
Copy pathoutputs.txt
File metadata and controls
63 lines (52 loc) · 4.6 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
Gemma 4: 2 Billlion Parameters (Int-8 Quantization) (2 Months old)
======================================================================
PROMPT DESIGN EVALUATION RESULTS
======================================================================
+------------------+-----------+------------+------------------+---------------+-----------------+
| Approach | Correct | Accuracy | Avg Prompt Tok | Avg Gen Tok | Avg Total Tok |
+==================+===========+============+==================+===============+=================+
| Zero-Shot | 48/50 | 96.0% | 82.1 | 81.1 | 163.2 |
+------------------+-----------+------------+------------------+---------------+-----------------+
| Few-Shot | 47/50 | 94.0% | 340.1 | 32.7 | 372.8 |
+------------------+-----------+------------+------------------+---------------+-----------------+
| Chain-of-Thought | 50/50 | 100.0% | 89.1 | 500.7 | 589.8 |
+------------------+-----------+------------+------------------+---------------+-----------------+
Llama3: Latest 8 Billlion Parameters (Int-4 Quantization) (2 Years old)
======================================================================
PROMPT DESIGN EVALUATION RESULTS
======================================================================
+------------------+-----------+------------+------------------+---------------+-----------------+
| Approach | Correct | Accuracy | Avg Prompt Tok | Avg Gen Tok | Avg Total Tok |
+==================+===========+============+==================+===============+=================+
| Zero-Shot | 46/50 | 92.0% | 82.9 | 2 | 84.9 |
+------------------+-----------+------------+------------------+---------------+-----------------+
| Few-Shot | 48/50 | 96.0% | 358.9 | 2 | 360.9 |
+------------------+-----------+------------+------------------+---------------+-----------------+
| Chain-of-Thought | 46/50 | 92.0% | 86.9 | 129.7 | 216.6 |
+------------------+-----------+------------+------------------+---------------+-----------------+
Mistral: Latest 7.25 Billion Parameters (Int-4 Quantization) (11 Months old)
======================================================================
PROMPT DESIGN EVALUATION RESULTS
======================================================================
+------------------+-----------+------------+------------------+---------------+-----------------+
| Approach | Correct | Accuracy | Avg Prompt Tok | Avg Gen Tok | Avg Total Tok |
+==================+===========+============+==================+===============+=================+
| Zero-Shot | 45/50 | 90.0% | 90.7 | 2 | 92.7 |
+------------------+-----------+------------+------------------+---------------+-----------------+
| Few-Shot | 48/50 | 96.0% | 448.7 | 2.6 | 451.3 |
+------------------+-----------+------------+------------------+---------------+-----------------+
| Chain-of-Thought | 49/50 | 98.0% | 92.7 | 110.1 | 202.8 |
+------------------+-----------+------------+------------------+---------------+-----------------+
Gemma4: Latest 8 Billion Parameters (Int-4 Quantization) (2 Months old)
======================================================================
PROMPT DESIGN EVALUATION RESULTS
======================================================================
+------------------+-----------+------------+------------------+---------------+-----------------+
| Approach | Correct | Accuracy | Avg Prompt Tok | Avg Gen Tok | Avg Total Tok |
+==================+===========+============+==================+===============+=================+
| Zero-Shot | 48/50 | 96.0% | 82.1 | 2 | 84.1 |
+------------------+-----------+------------+------------------+---------------+-----------------+
| Few-Shot | 49/50 | 98.0% | 340.1 | 2 | 342.1 |
+------------------+-----------+------------+------------------+---------------+-----------------+
| Chain-of-Thought | 49/50 | 98.0% | 89.1 | 437.1 | 526.2 |
+------------------+-----------+------------+------------------+---------------+-----------------+