Skip to content

More snapshots in tests #347

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jgabry merged 1 commit into from
Apr 7, 2026
Merged

More snapshots in tests #347

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 31 additions & 3 deletions tests/testthat/_snaps/compare.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,24 @@
# loo_compare returns expected results (2 models)

WAoAAAACAAQFAAACAwAAAAMOAAAAEAAAAAAAAAAAwBA6U1+cRe4AAAAAAAAAAD+2ake0LxMB
Code
print(comp1)
Output
elpd_diff se_diff
model1 0.0 0.0
model2 0.0 0.0

---

Code
print(comp2)
Output
elpd_diff se_diff
model1 0.0 0.0
model2 -4.1 0.1

---

WAoAAAACAAQFAgACAwAAAAMOAAAAEAAAAAAAAAAAwBA6U1+cRe4AAAAAAAAAAD+2ake0LxMB
wFTh8N3JQljAVeWWE8MGuUARCD2zEXBfQBEalRIN2T9ACijAYdW5U0AmZ5XrANCKP/H9Zexy
814/8ZtgnG1nx0Bk4fDdyUJYQGXllhPDBrlAIQg9sxFwX0AhGpUSDdk/AAAEAgAAAAEABAAJ
AAAAA2RpbQAAAA0AAAACAAAAAgAAAAgAAAQCAAAAAQAEAAkAAAAIZGltbmFtZXMAAAATAAAA
Expand All @@ -12,8 +30,18 @@

# loo_compare returns expected result (3 models)

WAoAAAACAAQFAAACAwAAAAMOAAAAGAAAAAAAAAAAwBA6U1+cRe7AMA3KkbYEGAAAAAAAAAAA
P7ZqR7QvEwE/y6/t4TTtXsBU4fDdyUJYwFXllhPDBrnAWOVjgjbDYkARCD2zEXBfQBEalRIN
Code
print(comp1)
Output
elpd_diff se_diff
model1 0.0 0.0
model2 -4.1 0.1
model3 -16.1 0.2

---

WAoAAAACAAQFAgACAwAAAAMOAAAAGAAAAAAAAAAAwBA6U1+cRe7AMA3KkbYEGAAAAAAAAAAA
P7ZqR7QvEwE/y6/t4TTtYMBU4fDdyUJYwFXllhPDBrnAWOVjgjbDYkARCD2zEXBfQBEalRIN
2T9AEPIF3GigE0AKKMBh1blTQCZnlesA0IpAQcjYUhrdCj/x/WXscvNeP/GbYJxtZ8c/8YDQ
kmfJX0Bk4fDdyUJYQGXllhPDBrlAaOVjgjbDYkAhCD2zEXBfQCEalRIN2T9AIPIF3GigEwAA
BAIAAAABAAQACQAAAANkaW0AAAANAAAAAgAAAAMAAAAIAAAEAgAAAAEABAAJAAAACGRpbW5h
Expand Down
7 changes: 7 additions & 0 deletions tests/testthat/_snaps/kfold_helpers.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# print_dims.kfold works

Code
print_dims(xx)
Output
Based on 17-fold cross-validation.

137 changes: 137 additions & 0 deletions tests/testthat/_snaps/loo_subsampling_cases.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
# Test the vignette

Code
print(looss_1)
Output
Computed from 4000 by 100 subsampled log-likelihood
values from 3020 total observations.
Estimate SE subsampling SE
elpd_loo -1968.5 15.6 0.3
p_loo 3.1 0.1 0.4
looic 3936.9 31.2 0.6
------
MCSE of elpd_loo is 0.0.
MCSE and ESS estimates assume independent draws (r_eff=1).
All Pareto k estimates are good (k < 0.7).
See help('pareto-k-diagnostic') for details.

---

Code
print(looss_1b)
Output
Computed from 4000 by 200 subsampled log-likelihood
values from 3020 total observations.
Estimate SE subsampling SE
elpd_loo -1968.3 15.6 0.2
p_loo 3.2 0.1 0.4
looic 3936.7 31.2 0.5
------
MCSE of elpd_loo is 0.0.
MCSE and ESS estimates assume independent draws (r_eff=1).
All Pareto k estimates are good (k < 0.7).
See help('pareto-k-diagnostic') for details.

---

Code
print(aploo_1)
Output
Computed from 2000 by 3020 log-likelihood matrix.
Estimate SE
elpd_loo -1968.4 15.6
p_loo 3.2 0.2
looic 3936.8 31.2
------
Posterior approximation correction used.
MCSE of elpd_loo is 0.0.
MCSE and ESS estimates assume independent draws (r_eff=1).
All Pareto k estimates are good (k < 0.7).
See help('pareto-k-diagnostic') for details.

---

Code
print(looapss_1)
Output
Computed from 2000 by 100 subsampled log-likelihood
values from 3020 total observations.
Estimate SE subsampling SE
elpd_loo -1968.2 15.6 0.4
p_loo 2.9 0.1 0.5
looic 3936.4 31.1 0.8
------
Posterior approximation correction used.
MCSE of elpd_loo is 0.0.
MCSE and ESS estimates assume independent draws (r_eff=1).
All Pareto k estimates are good (k < 0.7).
See help('pareto-k-diagnostic') for details.

---

Code
print(looss_2)
Output
Computed from 4000 by 100 subsampled log-likelihood
values from 3020 total observations.
Estimate SE subsampling SE
elpd_loo -1952.0 16.2 0.2
p_loo 2.6 0.1 0.3
looic 3903.9 32.4 0.4
------
MCSE of elpd_loo is 0.0.
MCSE and ESS estimates assume independent draws (r_eff=1).
All Pareto k estimates are good (k < 0.7).
See help('pareto-k-diagnostic') for details.

---

Code
print(comp)
Output
elpd_diff se_diff subsampling_se_diff
model2 0.0 0.0 0.0
model1 16.5 22.5 0.4

---

Code
print(comp)
Output
elpd_diff se_diff subsampling_se_diff
model2 0.0 0.0 0.0
model1 16.1 4.4 0.1

---

Code
print(comp2)
Output
elpd_diff se_diff subsampling_se_diff
model2 0.0 0.0 0.0
model1 16.3 4.4 0.1

---

Code
print(comp3)
Output
elpd_diff se_diff subsampling_se_diff
model2 0.0 0.0 0.0
model1 16.5 4.4 0.3

40 changes: 38 additions & 2 deletions tests/testthat/_snaps/model_weighting.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,44 @@
# loo_model_weights (stacking and pseudo-BMA) gives expected result

WAoAAAACAAQFAAACAwAAAAAOAAAAAz/KEXngFjO6P+l7oXTIUDU+YzIi3AAAAA==
WAoAAAACAAQFAgACAwAAAAAOAAAAAz/KEXnf1DM9P+l7oXTYyi4+YzIpAwAAAA==

---

WAoAAAACAAQFAAACAwAAAAAOAAAAAz+xA6UGtqDFP+3eFS5zKzY/J2MLYAsc4w==
Code
print(w1)
Output
Method: stacking
------
weight
model1 0.204
model2 0.796
model3 0.000

---

WAoAAAACAAQFAgACAwAAAAAOAAAAAz+xA6UGtqDDP+3eFS5zKzY/J2MLYAsc4A==

---

Code
print(w2)
Output
Method: pseudo-BMA+ with Bayesian bootstrap
------
weight
model1 0.066
model2 0.933
model3 0.000

---

Code
print(w3)
Output
Method: pseudo-BMA
------
weight
model1 0.000
model2 1.000
model3 0.000

64 changes: 64 additions & 0 deletions tests/testthat/_snaps/print_plot.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,67 @@
# print.waic output is ok

Code
print(waic1)
Output
Computed from 1000 by 32 log-likelihood matrix.
Estimate SE
elpd_waic -83.5 4.3
p_waic 3.3 1.1
waic 167.1 8.5
3 (9.4%) p_waic estimates greater than 0.4. We recommend trying loo instead.

# print.psis_loo and print.psis output ok

Code
print(psis1)
Output
Computed from 1000 by 32 log-weights matrix.
MCSE and ESS estimates assume independent draws (r_eff=1).
All Pareto k estimates are good (k < 0.67).
See help('pareto-k-diagnostic') for details.

---

Code
print(loo1)
Output
Computed from 1000 by 32 log-likelihood matrix.
Estimate SE
elpd_loo -83.6 4.3
p_loo 3.3 1.2
looic 167.2 8.6
------
MCSE of elpd_loo is 0.1.
MCSE and ESS estimates assume independent draws (r_eff=1).
All Pareto k estimates are good (k < 0.67).
See help('pareto-k-diagnostic') for details.

---

Code
print(loo1_r_eff)
Output
Computed from 1000 by 32 log-likelihood matrix.
Estimate SE
elpd_loo -83.6 4.3
p_loo 3.3 1.2
looic 167.2 8.6
------
MCSE of elpd_loo is 0.1.
MCSE and ESS estimates assume MCMC draws (r_eff in [0.6, 1.0]).
All Pareto k estimates are good (k < 0.67).
See help('pareto-k-diagnostic') for details.

# mcse_loo extractor gives correct value

WAoAAAACAAQFAAACAwAAAAAOAAAAAT+2J8YDcP5s
Expand Down
28 changes: 28 additions & 0 deletions tests/testthat/_snaps/tisis.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,31 @@
All Pareto k estimates are good (k < 0.67).
See help('pareto-k-diagnostic') for details.

# tis_loo and sis_loo are returned

Code
print(loo_tis)
Output

Computed from 1000 by 32 log-likelihood matrix using tis_loo .

Estimate SE
elpd_loo -83.6 4.3
p_loo 3.3 1.2
looic 167.2 8.6
------

---

Code
print(loo_sis)
Output

Computed from 1000 by 32 log-likelihood matrix using sis_loo .

Estimate SE
elpd_loo -83.6 4.3
p_loo 3.3 1.2
looic 167.2 8.6
------

4 changes: 3 additions & 1 deletion tests/testthat/test_compare.R
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,13 +89,14 @@ test_that("loo_compare returns expected results (2 models)", {
expect_s3_class(comp1, "compare.loo")
expect_equal(colnames(comp1), comp_colnames)
expect_equal(rownames(comp1), c("model1", "model2"))
expect_output(print(comp1), "elpd_diff")
expect_snapshot(print(comp1))
expect_equal(comp1[1:2, 1], c(0, 0), ignore_attr = TRUE)
expect_equal(comp1[1:2, 2], c(0, 0), ignore_attr = TRUE)

comp2 <- loo_compare(w1, w2)
expect_s3_class(comp2, "compare.loo")
expect_equal(colnames(comp2), comp_colnames)
expect_snapshot(print(comp2))

expect_snapshot_value(comp2, style = "serialize")

Expand All @@ -113,6 +114,7 @@ test_that("loo_compare returns expected result (3 models)", {
expect_equal(comp1[1, 1], 0)
expect_s3_class(comp1, "compare.loo")
expect_s3_class(comp1, "matrix")
expect_snapshot(print(comp1))

expect_snapshot_value(comp1, style = "serialize")

Expand Down
2 changes: 1 addition & 1 deletion tests/testthat/test_kfold_helpers.R
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,7 @@ test_that("kfold helpers throw correct errors", {

test_that("print_dims.kfold works", {
xx <- structure(list(), K = 17, class = c("kfold", "loo"))
expect_output(print_dims(xx), "Based on 17-fold cross-validation")
expect_snapshot(print_dims(xx))

attr(xx, "K") <- NULL
expect_silent(print_dims(xx))
Expand Down
Loading
Loading