Skip to content

CI: Update baseline.json (from PR #104) #105

Open
MingxuZh wants to merge 1 commit into
mainfrom
ci/update-baseline-pr-104
Open

CI: Update baseline.json (from PR #104) #105
MingxuZh wants to merge 1 commit into
mainfrom
ci/update-baseline-pr-104

Conversation

@MingxuZh
Copy link
Copy Markdown
Collaborator

@MingxuZh MingxuZh commented Feb 2, 2026

Benchmark Comparison

Ratio = log / baseline (lower is better)

LOWER (log < baseline)

num_tokens - num_experts - topk - hidden_size - shard_intermediate_size log baseline ratio
1-64-8-3584-640 0.237 0.236964 -0.04%
1-8-2-4096-3584 0.291 0.291278 -0.03%
1024-64-8-3584-640 2.750 2.75374 -0.14%
1024-8-2-4096-7168 3.046 3.04832 -0.08%
128-64-8-3584-640 1.299 1.30005 -0.05%
128-8-2-4096-3584 1.107 1.11192 -0.41%
512-64-8-3584-640 1.790 1.79057 -0.05%
8192-64-8-3584-640 14.777 14.7794 -0.02%

HIGHER (log > baseline)

num_tokens - num_experts - topk - hidden_size - shard_intermediate_size log baseline ratio
1-64-8-3584-1280 0.357 0.356382 +0.17%
1024-64-8-3584-1280 3.972 3.673 +8.13%
1024-8-2-4096-3584 1.888 1.869 +1.01%
128-64-8-3584-1280 2.387 2.37856 +0.34%
128-8-2-4096-7168 2.301 2.09 +10.08%
2048-64-8-3584-1280 5.905 5.90057 +0.07%
2048-64-8-3584-640 4.642 4.573 +1.50%
2048-8-2-4096-3584 3.238 3.199 +1.21%
2048-8-2-4096-7168 6.326 5.973 +5.92%
4096-64-8-3584-1280 10.472 10.016 +4.55%
4096-64-8-3584-640 8.106 8.0614 +0.55%
4096-8-2-4096-3584 5.870 5.576 +5.27%
4096-8-2-4096-7168 10.859 10.284 +5.59%
512-64-8-3584-1280 2.805 2.79651 +0.31%
512-8-2-4096-3584 1.643 1.224 +34.26%
512-8-2-4096-7168 2.378 2.196 +8.30%
8192-64-8-3584-1280 18.675 18.6474 +0.15%
8192-8-2-4096-3584 10.459 10.4043 +0.52%
8192-8-2-4096-7168 20.620 20.211 +2.02%

EQUAL

num_tokens - num_experts - topk - hidden_size - shard_intermediate_size log baseline ratio
1-8-2-4096-7168 0.485 0.485212 +0.00%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant