Commit 90ec8ad
authored
[opt](function) speed up md5 with AVX2 batch path (#63484)
Root cause: md5/md5sum evaluated every row through Md5Digest and
OpenSSL, which leaves the vectorized string function path dominated by
per-row scalar digest setup and hex materialization.
Fix: add an AVX2 multi-buffer MD5 helper with scalar fallback, expose a
batch hex API, and route single-argument md5/md5sum over
ColumnString/ColumnVarbinary through the batch path while keeping
multi-argument md5sum and sm3 on the existing digest implementation.
test with sql:
```sql
SET parallel_pipeline_task_num=1;
SET enable_query_cache=false;
SELECT SUM(ASCII(SUBSTRING(MD5(CAST(number AS STRING)), 1, 1)))
FROM numbers("number" = "50000000");
```
result:
| version | times | avg | median |
|---|---:|---:|---:|
| upstream/master baseline | 8.59, 10.21, 9.52, 9.93, 8.85s | 9.42s |
9.52s |
| after AVX2 batch | 2.83, 2.84, 2.82, 2.79, 2.82s | 2.82s | 2.82s |1 parent d7d516f commit 90ec8ad
5 files changed
Lines changed: 677 additions & 7 deletions
File tree
- be
- src
- exprs/function
- util
- test
- exprs/function
- util
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
16 | 16 | | |
17 | 17 | | |
18 | 18 | | |
| 19 | + | |
19 | 20 | | |
| 21 | + | |
| 22 | + | |
20 | 23 | | |
21 | 24 | | |
22 | 25 | | |
| |||
98 | 101 | | |
99 | 102 | | |
100 | 103 | | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
101 | 112 | | |
102 | 113 | | |
103 | 114 | | |
| |||
114 | 125 | | |
115 | 126 | | |
116 | 127 | | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
117 | 164 | | |
118 | 165 | | |
119 | 166 | | |
| |||
0 commit comments