Skip to content

Commit 546961b

Browse files
committed
Update lecture21 content and regenerate PDFs for week 7
- Modified the mask rate description in lecture21.md to clarify the selection process. - Regenerated lecture21.pdf and lecture23.pdf to reflect the latest changes.
1 parent 6c4d881 commit 546961b

4 files changed

Lines changed: 28 additions & 28 deletions

File tree

slides/week7/lecture21.html

Lines changed: 27 additions & 27 deletions
Large diffs are not rendered by default.

slides/week7/lecture21.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -254,7 +254,7 @@ Recall from Lecture 18: BERT trains by masking 15% of tokens and predicting the
254254

255255
| | BERT (Lecture 18) | Discrete diffusion |
256256
|---|---|---|
257-
| Mask rate | Fixed at 15% | Varies from 0% to 100% over a schedule |
257+
| Mask rate | Select 15%, then 80/10/10 | Varies from 0% to 100% over a schedule |
258258
| Prediction | One-shot: predict all masks at once | Iterative: unmask a few tokens at a time |
259259
| Training | Single forward pass per example | Sample random mask rate $t$, predict masked tokens |
260260
| Generation | Not designed for generation | Built for generation: start at 100% masked, iteratively unmask |

slides/week7/lecture21.pdf

11 Bytes
Binary file not shown.

slides/week7/lecture23.pdf

-3 Bytes
Binary file not shown.

0 commit comments

Comments
 (0)