Skip to content

CI: Update baseline.json (from PR #102) #103

Open
MingxuZh wants to merge 3 commits intomainfrom
ci/update-baseline-pr-102
Open

CI: Update baseline.json (from PR #102) #103
MingxuZh wants to merge 3 commits intomainfrom
ci/update-baseline-pr-102

Conversation

@MingxuZh
Copy link
Copy Markdown
Collaborator

@MingxuZh MingxuZh commented Feb 2, 2026

Benchmark Comparison

Ratio = log / baseline (lower is better)

LOWER (log < baseline)

num_tokens - num_experts - topk - hidden_size - shard_intermediate_size log baseline ratio
1-8-2-4096-3584 0.291 0.291278 -0.06%
1-8-2-4096-7168 0.485 0.485212 -0.02%
1024-64-8-3584-640 2.742 2.75374 -0.41%
1024-8-2-4096-7168 3.038 3.04832 -0.35%
128-8-2-4096-3584 1.106 1.11192 -0.51%
2048-64-8-3584-1280 5.899 5.90057 -0.03%
4096-64-8-3584-640 8.042 8.0614 -0.24%
8192-64-8-3584-640 14.772 14.7794 -0.05%

HIGHER (log > baseline)

num_tokens - num_experts - topk - hidden_size - shard_intermediate_size log baseline ratio
1-64-8-3584-1280 0.356 0.356382 +0.01%
1-64-8-3584-640 0.237 0.236964 +0.13%
1024-64-8-3584-1280 3.966 3.673 +7.98%
1024-8-2-4096-3584 1.887 1.869 +0.96%
128-64-8-3584-1280 2.380 2.37856 +0.05%
128-64-8-3584-640 1.301 1.30005 +0.06%
128-8-2-4096-7168 2.314 2.09 +10.70%
2048-64-8-3584-640 4.641 4.573 +1.49%
2048-8-2-4096-3584 3.229 3.199 +0.93%
2048-8-2-4096-7168 6.269 5.973 +4.95%
4096-64-8-3584-1280 10.481 10.016 +4.64%
4096-8-2-4096-3584 5.893 5.576 +5.69%
4096-8-2-4096-7168 10.859 10.284 +5.59%
512-64-8-3584-1280 2.807 2.79651 +0.39%
512-64-8-3584-640 1.792 1.79057 +0.09%
512-8-2-4096-3584 1.639 1.224 +33.92%
512-8-2-4096-7168 2.378 2.196 +8.28%
8192-64-8-3584-1280 18.834 18.6474 +1.00%
8192-8-2-4096-3584 10.428 10.4043 +0.23%
8192-8-2-4096-7168 20.574 20.211 +1.80%

EQUAL

None

@MingxuZh
Copy link
Copy Markdown
Collaborator Author

MingxuZh commented Feb 2, 2026

Benchmark Comparison

Ratio = log / baseline (lower is better)

LOWER (log < baseline)

num_tokens - num_experts - topk - hidden_size - shard_intermediate_size log baseline ratio
1-8-2-4096-3584 0.291 0.291278 -0.03%
1024-64-8-3584-640 2.749 2.75374 -0.17%
1024-8-2-4096-7168 3.042 3.04832 -0.20%
128-64-8-3584-640 1.299 1.30005 -0.07%
128-8-2-4096-3584 1.111 1.11192 -0.12%
4096-64-8-3584-640 8.046 8.0614 -0.19%
512-64-8-3584-640 1.789 1.79057 -0.07%
8192-64-8-3584-640 14.777 14.7794 -0.01%

HIGHER (log > baseline)

num_tokens - num_experts - topk - hidden_size - shard_intermediate_size log baseline ratio
1-64-8-3584-1280 0.356 0.356382 +0.02%
1-64-8-3584-640 0.237 0.236964 +0.22%
1-8-2-4096-7168 0.485 0.485212 +0.03%
1024-64-8-3584-1280 3.970 3.673 +8.07%
1024-8-2-4096-3584 1.887 1.869 +0.97%
128-64-8-3584-1280 2.389 2.37856 +0.44%
128-8-2-4096-7168 2.288 2.09 +9.45%
2048-64-8-3584-1280 5.901 5.90057 +0.01%
2048-64-8-3584-640 4.646 4.573 +1.59%
2048-8-2-4096-3584 3.233 3.199 +1.06%
2048-8-2-4096-7168 6.340 5.973 +6.14%
4096-64-8-3584-1280 10.479 10.016 +4.62%
4096-8-2-4096-3584 6.022 5.576 +8.00%
4096-8-2-4096-7168 10.839 10.284 +5.40%
512-64-8-3584-1280 2.806 2.79651 +0.32%
512-8-2-4096-3584 1.640 1.224 +33.95%
512-8-2-4096-7168 2.377 2.196 +8.26%
8192-64-8-3584-1280 18.805 18.6474 +0.85%
8192-8-2-4096-3584 10.423 10.4043 +0.18%
8192-8-2-4096-7168 20.630 20.211 +2.07%

EQUAL

None

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant