Skip to content

Sycltla BF16 GEMM#294

Draft
xinyu-intel wants to merge 1 commit into
vllm-project:mainfrom
xinyu-intel:dev/gemm-sycltla
Draft

Sycltla BF16 GEMM#294
xinyu-intel wants to merge 1 commit into
vllm-project:mainfrom
xinyu-intel:dev/gemm-sycltla

Conversation

@xinyu-intel
Copy link
Copy Markdown
Collaborator

Essential Elements of an Effective PR Description Checklist

  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS ABOVE HAVE BEEN CONSIDERED.

Purpose

measured on B60

M oneDNN us oneDNN TFLOPS TLA us TLA TFLOPS TLA tile Speedup
1 81.7 0.377 75.3 0.433 8x256 1.08x
2 79.1 0.743 75.5 0.888 8x256 1.05x
4 92.8 1.672 76.3 1.762 8x256 1.22x
8 83.2 2.918 76.5 3.502 8x256 1.09x
16 88.0 6.260 77.5 6.945 16x256 1.14x
32 91.4 12.648 77.9 13.811 32x256 1.17x
64 90.4 23.864 80.7 26.569 64x256 1.12x
128 89.7 47.696 85.4 50.147 128x256 1.05x
256 128.0 68.233 118.4 72.252 256x256 1.08x
384 177.6 72.471 177.0 72.770 384x128 1.00x
512 236.7 73.384 234.6 73.338 128x128 1.01x
640 285.3 75.223 263.2 81.720 128x128 1.08x
768 342.0 76.050 295.0 87.200 256x128 1.16x
896 343.7 87.460 362.4 82.887 128x128 0.95x
1024 395.7 86.732 410.5 83.668 256x128 0.96x
4096 1456.2 94.490 1500.9 91.505 256x256 0.97x

Test Plan

Test Result

(Optional) Documentation Update

BEFORE SUBMITTING, PLEASE READ https://docs.vllm.ai/en/latest/contributing (anything written below this line will be removed by GitHub Actions)

Signed-off-by: Xinyu Chen <xinyu1.chen@intel.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant