You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/en/cuda-memory-cheatsheet.md
+25-12Lines changed: 25 additions & 12 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,15 +4,19 @@ title: CUDA Memory Cheat Sheet
4
4
5
5
# CUDA Memory Cheat Sheet
6
6
7
-
A compact memory reference for reading SGEMM kernels faster
7
+
This quick sheet is now part of the [Resources Hub](/en/resources/). Use it when you need to re-orient yourself before reopening kernel code, profiler output, or WMMA constraints.
8
8
9
+
## When this page is most useful
9
10
11
+
- Right before reading [Memory Flow](/en/architecture/memory-flow) or [Tiled Kernel](/en/kernel-tiled) again.
12
+
- When a benchmark changed and you need a fast checklist before blaming occupancy or Tensor Cores.
13
+
- When you want to explain memory behavior in an interview without reopening the full CUDA manuals.
10
14
11
15
## Coalescing quick rules
12
16
13
17
- Consecutive threads in a warp should touch consecutive addresses whenever possible.
14
18
- Accessing `B[k * N + col]` with large `N` can become stride-heavy for neighboring threads.
15
-
- Tiling is not just reuse; it is also a way to reshape access into coalesced loads.
19
+
- Tiling is not only about reuse; it also reshapes access into more coalesced loads.
16
20
17
21
```mermaid
18
22
flowchart LR
@@ -21,25 +25,34 @@ flowchart LR
21
25
C --> D[Global memory writeback]
22
26
```
23
27
28
+
## Shared-memory watchpoints
24
29
30
+
| Question | Why it matters |
31
+
|---|---|
32
+
| Are threads writing a tile layout that later reads back contiguously? | Shared memory only helps when it fixes a global-memory access problem instead of creating a local one. |
33
+
| Did padding or index remapping remove bank conflicts on the hot path? | Bank conflicts can erase the benefit of otherwise-good tiling choices. |
34
+
| Did the extra shared-memory footprint change occupancy enough to matter? | Some tiling wins disappear if the launch geometry becomes too constrained. |
25
35
26
36
## Tensor Core memory notes
27
37
28
38
| Topic | What to remember |
29
-
|-------|------------------|
30
-
| Alignment constraints | WMMA path expects dimensions aligned to 16 for efficient fragment handling |
31
-
| Data conversion | End-to-end timing includes conversion and wrapper logic |
32
-
| Safe behavior | Non-friendly shapes should fall back to FP32 path |
33
-
| Reporting | Distinguish end-to-end and compute-only results |
34
-
35
-
39
+
|---|---|
40
+
| Alignment constraints | WMMA paths expect dimensions aligned to fragment-friendly sizes, typically 16, for efficient fragment handling. |
41
+
| Data conversion | End-to-end timing includes conversion and wrapper logic, not just the fused matrix multiply. |
42
+
| Safe behavior | Non-friendly shapes should fall back to the FP32 path instead of forcing a misleading WMMA result. |
43
+
| Reporting | Distinguish end-to-end numbers from compute-only numbers before comparing implementations. |
36
44
37
45
## Fast checklist when reading a kernel
38
46
39
-
1. Can I explain the globalmemory access order for one warp?
40
-
2.Is sharedmemory layout conflict-aware?
47
+
1. Can I explain the global-memory access order for one warp?
48
+
2.Does shared-memory layout reduce conflicts rather than merely move data around?
41
49
3. Are register accumulators bounded and intentional?
42
50
4. Is Tensor Core fallback behavior explicit?
43
51
5. Do benchmark labels match what the kernel path really measures?
<p>Use the support and validation surfaces together to understand what to run locally, what CI already proves, and how results should be interpreted.</p>
170
170
<div class="route-links">
171
-
<a href="/en/getting-started">Getting Started</a>
172
-
<a href="/en/validation/">Validation overview</a>
171
+
<a href="./getting-started">Getting Started</a>
172
+
<a href="./validation/">Validation overview</a>
173
173
</div>
174
174
</div>
175
175
</div>
176
176
177
177
## Resource hub entry points
178
178
179
179
<divclass="knowledge-grid">
180
-
<aclass="knowledge-card"href="/en/architecture/">
180
+
<aclass="knowledge-card"href="./architecture/">
181
181
<h3>Architecture</h3>
182
182
<p>The structural map of the kernel ladder, interface boundaries, and the decisions that hold the implementation together.</p>
183
183
</a>
184
-
<aclass="knowledge-card"href="/en/methodology/">
184
+
<aclass="knowledge-card"href="./methodology/">
185
185
<h3>Methodology</h3>
186
186
<p>The canonical optimization workflow: staged learning, benchmark discipline, and diagnosis logic tied back to the architecture.</p>
187
187
</a>
188
-
<aclass="knowledge-card"href="/en/references">
188
+
<aclass="knowledge-card"href="./resources/">
189
189
<h3>Resources</h3>
190
-
<p>The source trail behind the project: official docs, papers, and mature repositories mapped to concrete decisions.</p>
190
+
<p>A curated learning hub that explains which docs, papers, tools, and follow-up routes matter after each part of the whitepaper.</p>
191
191
</a>
192
-
<aclass="knowledge-card"href="/en/validation/">
192
+
<aclass="knowledge-card"href="./validation/">
193
193
<h3>Validation</h3>
194
194
<p>The evidence surface for correctness budgets, benchmark scope, fallback behavior, and what the published numbers actually mean.</p>
0 commit comments