Commit 1fe9e2c
authored
spec: add calibration-curve specification (#2337)
## New Specification: `calibration-curve`
Related to #2331
---
### specification.md
# calibration-curve: Calibration Curve
## Description
A calibration curve (reliability diagram) visualizes how well the
predicted probabilities of a binary classifier match actual outcomes. By
plotting the fraction of positives against mean predicted probability in
binned intervals, it reveals whether a model is well-calibrated,
overconfident, or underconfident. A perfectly calibrated model follows
the diagonal line where predicted probability equals observed frequency.
## Applications
- Evaluating probability predictions in credit scoring models where
accurate risk estimates determine loan pricing
- Assessing diagnostic confidence in medical screening systems where
probability calibration affects treatment decisions
- Comparing calibration across multiple classifiers to select models
that produce reliable probability estimates
- Validating risk assessment models in insurance and fraud detection
where miscalibrated probabilities lead to financial losses
## Data
- `y_true` (binary array) - Ground truth binary labels (0 or 1)
- `y_prob` (numeric array) - Predicted probabilities from classifier (0
to 1)
- Size: 500-10000 samples recommended for reliable binning
- Example: Binary classification predictions from sklearn classifier
with `predict_proba()` output
## Notes
- Include a diagonal reference line representing perfect calibration
(predicted probability = observed frequency)
- Use 10 bins by default, with bin edges at equal probability intervals
(0.0-0.1, 0.1-0.2, etc.)
- Display Brier score or Expected Calibration Error (ECE) as a summary
metric
- Optional: include histogram of predicted probabilities as a secondary
subplot to show prediction distribution
- For multiple models comparison, use distinct colors with clear legend
---
**Next:** Add `approved` label to the issue to merge this PR.
---
:robot: *[spec-create
workflow](https://github.com/MarkusNeusinger/pyplots/actions/runs/20528130950)*
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>1 parent 2dc4a06 commit 1fe9e2c
4 files changed
Lines changed: 59 additions & 0 deletions
File tree
- plots/calibration-curve
- implementations
- metadata
Whitespace-only changes.
Whitespace-only changes.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
0 commit comments