Skip to content

Commit 5d81eaa

Browse files
jstacclaude
andauthored
Polish Bayesian lecture and rename prob_dist title (#760)
- Add an overview flowchart (prior belief + data -> updated belief), drawn inline with matplotlib in a hide-input cell - Tighten prose around the prior and normalizing constant; define c(y) explicitly and present the trapezoidal rule as a numbered list - Fix a missing period and trailing whitespace - Rename "Distributions and Probabilities" to "Common Distributions" Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 937e37a commit 5d81eaa

2 files changed

Lines changed: 56 additions & 21 deletions

File tree

lectures/bayes_intro.md

Lines changed: 54 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -15,14 +15,49 @@ kernelspec:
1515

1616
## Overview
1717

18-
In this lecture we study one of the most important ideas in statistics: how to update our beliefs about an unknown quantity as new data arrives.
18+
In this lecture we study one of the most important ideas in statistics: how to
19+
update our beliefs about an unknown quantity as new data arrives.
1920

20-
The technique we will use is called **Bayesian updating**.
21+
The technique we will use is called **Bayesian updating**, named after [Thomas
22+
Bayes](https://en.wikipedia.org/wiki/Thomas_Bayes).
2123

2224
We start with a belief about some unknown number.
2325

2426
As we observe data, we revise that belief in a way that is mathematically precise.
2527

28+
The figure below illustrates the idea: we combine a **prior belief** with
29+
observed **data** to arrive at an **updated belief**.
30+
31+
```{code-cell} ipython3
32+
:tags: [hide-input]
33+
34+
import matplotlib.pyplot as plt
35+
from matplotlib.patches import Ellipse, FancyArrowPatch
36+
37+
def add_node(ax, xy, text, color):
38+
ax.add_patch(Ellipse(xy, width=0.34, height=0.20,
39+
facecolor=color, edgecolor='black', lw=1.0, alpha=0.35))
40+
ax.annotate(text, xy, ha='center', va='center', fontsize=12)
41+
42+
def add_arrow(ax, start, end):
43+
ax.add_patch(FancyArrowPatch(start, end, arrowstyle='-|>',
44+
mutation_scale=18, lw=1.2, color='black'))
45+
46+
fig, ax = plt.subplots(figsize=(7, 3.5))
47+
48+
add_node(ax, (0.22, 0.75), "prior belief", 'C0')
49+
add_node(ax, (0.22, 0.25), "data", 'C1')
50+
add_node(ax, (0.78, 0.50), "updated belief", 'C2')
51+
52+
add_arrow(ax, (0.39, 0.70), (0.61, 0.54))
53+
add_arrow(ax, (0.39, 0.30), (0.61, 0.46))
54+
55+
ax.set_xlim(0, 1)
56+
ax.set_ylim(0, 1)
57+
ax.axis('off')
58+
plt.show()
59+
```
60+
2661
We will develop these ideas through an example drawn from development
2762
finance: estimating the default rate on a new type of loan.
2863

@@ -279,21 +314,29 @@ def pi(θ):
279314
return beta.pdf(θ, a_0, b_0)
280315
```
281316
282-
This prior puts most of its weight on default rates below 0.5, with a peak around 0.2, reflecting cautious optimism together with genuine uncertainty.
317+
This prior was shown above.
318+
319+
It puts most of its weight on default rates below 0.5, with a peak around 0.2, reflecting cautious optimism (most borrowers don't default) together with significant uncertainty.
283320
284321
285322
### Normalizing constant
286323
287-
Next we need to compute the integral in the denominator of {eq}`eq:bayes_density`.
324+
Next we need to compute the constant in the denominator of {eq}`eq:bayes_density`,
325+
which is the integrated likelihood times the prior:
326+
327+
$$
328+
c(y) := \int_0^1 p(y \mid t)\, \pi(t)\, dt .
329+
$$
288330
289-
One general approach is to compute it numerically, using a technique
290-
such as the [trapezoidal rule](https://en.wikipedia.org/wiki/Trapezoidal_rule).
291331
292-
We fix a grid of points across $[0, 1]$ and represent each density by its values at those grid points.
332+
One general approach is to compute $c(y)$ numerically, using a technique
333+
such as the [trapezoidal rule](https://en.wikipedia.org/wiki/Trapezoidal_rule).
293334
294-
Every integral then becomes a sum that `numpy` can evaluate for us.
335+
The idea of the trapezoidal rule is to
295336
296-
The idea of the trapezoidal rule is to join neighboring grid points by straight lines and sum the areas of the resulting trapezoids.
337+
1. fix a grid of points across $[0, 1]$,
338+
2. join neighboring grid points by straight lines, and
339+
3. sum the areas of the resulting trapezoids.
297340
298341
The figure below illustrates this for the integrand $p(y \mid \theta)\, \pi(\theta)$ with $y = 1$, using a coarse grid so the trapezoids are visible.
299342
@@ -311,7 +354,7 @@ ax.plot(fine, integrand, lw=2, label=r"$p(y \mid \theta)\,\pi(\theta)$")
311354
ax.fill_between(coarse, heights, alpha=0.3,
312355
label="trapezoidal approximation")
313356
ax.plot(coarse, heights, 'o-', color='C1', lw=1, ms=4)
314-
for x, h in zip(coarse, heights): # draw the trapezoid edges
357+
for x, h in zip(coarse, heights):
315358
ax.plot([x, x], [0, h], color='C1', lw=0.8, alpha=0.6)
316359
ax.set_xlabel(r"$\theta$")
317360
ax.set_ylabel("integrand")
@@ -321,15 +364,7 @@ plt.show()
321364
322365
The finer the grid, the closer the shaded region gets to the true area under the curve.
323366
324-
Let's build the update in two steps.
325-
326-
First, recall that the denominator in {eq}`eq:bayes_density` is the integrated likelihood times the prior:
327-
328-
$$
329-
\int_0^1 p(y \mid t)\, \pi(t)\, dt .
330-
$$
331-
332-
The function below computes this constant on the grid, approximating the integral with `np.trapezoid`.
367+
The function below uses this method to compute an approximation of $c(y)$.
333368
334369
```{code-cell} ipython3
335370
def normalizing_constant(y):

lectures/prob_dist.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,9 +11,9 @@ kernelspec:
1111
name: python3
1212
---
1313

14-
# Distributions and Probabilities
14+
# Common Distributions
1515

16-
```{index} single: Distributions and Probabilities
16+
```{index} single: Common Distributions
1717
```
1818

1919
## Outline

0 commit comments

Comments
 (0)