Skip to content

Add documentation on partial specification of a multivariate variable (#2239)#1434

Merged
sunxd3 merged 12 commits into
mainfrom
docs/predict-fix-whole-variable-note
Jul 14, 2026
Merged

Add documentation on partial specification of a multivariate variable (#2239)#1434
sunxd3 merged 12 commits into
mainfrom
docs/predict-fix-whole-variable-note

Conversation

@yebai

@yebai yebai commented Jul 8, 2026

Copy link
Copy Markdown
Member

Fixes the documentation and the behaviour side of TuringLang/Turing.jl#2239.

A variable drawn from a multivariate distribution in a single tilde-statement (e.g. x ~ MvNormal(...), filldist) is a single random variable, not a collection of i.i.d. components. Supplying only part of such a variable — via predict with a chain from a differently sized model, or fix/condition on a subset of indices — should produce an informative error.


Update: (sunxd3) scoped down to documentation only, per the discussion below — the tilde-site guard, version bump, and error tests are reverted; the docstring notes on whole-variable semantics for predict and fix remain. The behaviour-side check for predict will follow in a separate PR via InitFromParams; condition/fix enforcement (if worth doing) will get its own design issue.

A variable drawn from a multivariate distribution in a single tilde-statement
(e.g. `x ~ MvNormal(...)` / `filldist`) is a single random variable, not i.i.d.
components. `predict` therefore cannot fix a subset of its components while
resampling the rest, and `fix` cannot fix them independently. Add warning
admonitions documenting this, referencing TuringLang/Turing.jl#2239.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@codecov

codecov Bot commented Jul 8, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.64%. Comparing base (d7e84ce) to head (762856b).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1434   +/-   ##
=======================================
  Coverage   81.64%   81.64%           
=======================================
  Files          50       50           
  Lines        3579     3579           
=======================================
  Hits         2922     2922           
  Misses        657      657           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

DynamicPPL.jl documentation for PR #1434 is available at:
https://TuringLang.github.io/DynamicPPL.jl/previews/PR1434/

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Benchmarks @ 762856b

Performance Ratio: gradient time divided by log-density time.

For very small models these ratios are noisy across runs and machines; raw primal and gradient timings are more reliable. The benchmarks are aimed at DynamicPPL developers and mainly catch obvious allocation or type-stability regressions. See benchmark notes for details.

===================================================================================================
                                               eval                       gradient                 
                                            ----------  -------------------------------------------
Model                        dim    linked      primal     FwdDiff    RvsDiff    Mooncake    Enzyme
---------------------------------------------------------------------------------------------------
Simple assume observe*         1     false     3.29 ns       13.82    1494.27       47.79     18.92
Simple assume observe*         1      true     3.28 ns       14.29    1660.61       47.84     19.15
Smorgasbord                  201     false     4.46 μs       90.10     135.59        7.26      7.53
Smorgasbord                  201      true      5.8 μs      102.54     139.44        6.35      5.52
Loop univariate 1k          1000     false     12.6 μs     1492.19     331.92        9.75      8.08
Loop univariate 1k          1000      true     15.4 μs     1231.08     272.85        8.15      6.61
Multivariate 1k             1000     false     19.5 μs      457.40      64.79        8.34      1.92
Multivariate 1k             1000      true     18.6 μs      427.90      69.82        8.89      1.96
Loop univariate 10k        10000     false    122.0 μs    19269.43     374.00       10.17      7.99
Loop univariate 10k        10000      true    150.0 μs    11745.88     307.95        8.35      6.49
Multivariate 10k           10000     false    173.0 μs     7339.66      77.99        9.59      1.76
Multivariate 10k           10000      true    175.0 μs     7112.84      76.36        9.43      1.74
Dynamic                       15     false     1.12 μs         err      42.41       10.93      9.83
Dynamic                       10      true     1.51 μs        2.10      57.97       11.90     19.43
Submodel*                      1     false     3.29 ns       14.50    1763.90       48.78     17.93
Submodel*                      1      true     3.28 ns       14.45    1876.04       48.31     19.08
LDA                           12      true     15.6 μs        0.59       2.38       35.36       err
===================================================================================================
Main @ d7e84ce
===================================================================================================
                                               eval                       gradient                 
                                            ----------  -------------------------------------------
Model                        dim    linked      primal     FwdDiff    RvsDiff    Mooncake    Enzyme
---------------------------------------------------------------------------------------------------
Simple assume observe*         1     false     3.83 ns       13.31    1849.44       86.03     27.95
Simple assume observe*         1      true     3.85 ns       29.84    2219.30       86.80     27.10
Smorgasbord                  201     false     12.0 μs       38.48      62.87        6.54      4.93
Smorgasbord                  201      true     14.7 μs       38.95      75.21        5.69      3.64
Loop univariate 1k          1000     false     43.3 μs      406.67     126.10        4.30      3.39
Loop univariate 1k          1000      true     43.2 μs      614.60     125.41        4.33      3.52
Multivariate 1k             1000     false     40.9 μs      252.08      37.85        4.43      1.63
Multivariate 1k             1000      true     38.9 μs      226.37      35.69        5.83      1.61
Loop univariate 10k        10000     false    196.0 μs    13173.42     284.19        7.34      6.54
Loop univariate 10k        10000      true    205.0 μs    14867.79     294.94        6.89      6.08
Multivariate 10k           10000     false    240.0 μs     7213.95      78.23        9.98      1.87
Multivariate 10k           10000      true    239.0 μs     7175.53      77.28        9.16      1.87
Dynamic                       15     false     2.35 μs         err      38.00       13.82      9.34
Dynamic                       10      true     3.17 μs        2.01      43.90       11.18     20.98
Submodel*                      1     false     3.84 ns       29.90    2599.44       91.65     26.96
Submodel*                      1      true     3.86 ns       28.99    2605.61       92.87     19.16
LDA                           12      true     30.0 μs        0.61       1.94       24.24       err
===================================================================================================
Environment
Julia Version 1.11.9
Commit 53a02c0720c (2026-02-06 00:27 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 4 × AMD EPYC 9V74 80-Core Processor
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, znver4)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

yebai and others added 5 commits July 8, 2026 12:58
…s specified

Supplying a subset of a variable that the model samples as a single multivariate
draw (e.g. `x[1:10]` when the model draws `x ~ MvNormal(zeros(20), ...)`) has never
worked: `predict` silently resampled the whole variable from the prior, `fix`
silently collapsed it to the supplied length, and `condition` threw an opaque
`DimensionMismatch`. This is therefore not a behaviour change — it only replaces
those silent or confusing failures with a single, informative error.

Add `_check_supplied_shape(dist, supplied, vn)`, dispatched on the supplied
representation (a materialised value for `condition`/`fix`, or the parameter
`VarNamedTuple` for `predict`/`InitFromParams`), which throws one clear error
referencing TuringLang/Turing.jl#2239. Only multivariate distributions are checked;
per-index (`x[i] ~ ...`) declarations and correctly-sized whole-variable values are
unaffected.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Cover the #2239 misuse cases: predicting with a chain from a differently-sized
model, and `fix`/`condition` of a single index of a multivariate variable, all
now raise the informative error. A whole-variable `fix` of the correct size is
kept as a positive control.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
No behaviour change. Shorten the `_check_supplied_shape` docstring and the
`init` comment, and reduce the predict misuse test to the smallest model that
still triggers the error (just the multivariate variable, no extra latent or
observation).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Document the new informative error for partial specification of a multivariate
variable via condition/fix/predict (Turing#2239). Non-breaking: the case never
worked, so this only replaces a silent or opaque failure with a clear message.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The docstring notes previously described the old silent behaviour (predict
resampling the whole variable from the prior); update them to state that
supplying only part of a multivariate variable now raises an error.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@yebai yebai changed the title docs: document whole-variable semantics of predict and fix Error informatively on partial specification of a multivariate variable (#2239) Jul 8, 2026
yebai and others added 3 commits July 8, 2026 13:00
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Keep the error self-contained and actionable (the loop workaround); the #2239
reference stays in the docstrings and HISTORY. Tests now match on a stable
phrase from the message rather than the issue number.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Address review feedback: raise `ArgumentError` (consistent with `check_tilde_rhs`
/`check_dot_tilde_rhs`) instead of a generic `error`, and match it by type in the
tests rather than by a fragile message substring. Add a regression test that
per-index (`.~`) variables still grow correctly under `predict` without being
falsely flagged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@yebai

yebai commented Jul 8, 2026

Copy link
Copy Markdown
Member Author

@sunxd3, can you help review this?

@sunxd3

sunxd3 commented Jul 8, 2026

Copy link
Copy Markdown
Member

Yes!

@sunxd3

sunxd3 commented Jul 13, 2026

Copy link
Copy Markdown
Member

I'm not sure the macro-level guard is the right place for this check — it both misses cases and catches one it shouldn't.

It only runs when hasconditioned_nested/hasfixed_nested finds a value, but partial specification usually makes the lookup fail (the PartialArray mask isn't fully set), so the variable is treated as an assumption and the guard never fires. Matrix-variate and submodel-prefixed variables slip through similarly.

On the other side, a conditioned value is an observation, and the observe pipeline accepts batches via Distributions.loglikelihood — the exact-size check turns those into errors:

@model mvc(n) = m ~ MvNormal(zeros(n), I)

# errors on this branch:
fix(mvc(3), Dict(@varname(m[1]) => 1.0))()
# silently ignored, m sampled from the prior:
fix(mvc(3), Dict(@varname(m[1]) => 1.0, @varname(m[3]) => 3.0))()

# two draws, summed by loglikelihood — works on main, errors on this branch:
loglikelihood(condition(mvc(3), (; m = [1.0 2.0; 3.0 4.0; 5.0 6.0])), (;))

Getting condition/fix right would need the tilde pipeline to tell "absent" apart from "partially present", and to treat observe-side shapes differently from assume-side — that's on the hot path. Honestly not sure it's worth fixing at all; if it is, definitely its own PR.

So maybe we scope this down: keep the PR documentation-only (note the whole-variable semantics in the fix/predict docstrings — condition's docstring already has this), or perhaps also fix predict from the init interface — a check in InitFromParams (src/contexts/init.jl) can be made complete there: raw values must be exactly one draw of size(dist) for any ArrayLikeVariate, and error when params cover only part of a variable, submodel paths included. I can take over and implement this.

(prepared with CC + Fable)

@yebai

yebai commented Jul 13, 2026

Copy link
Copy Markdown
Member Author

Thanks, @sunxd3. It would be great to find a robust, principled solution that catches unintended uses of fix/condition/predict, etc.

One approach that often works well is to brainstorm with coding agents: give them a clear design goal or constraint, ask them to explore feasible solutions, and then evaluate the trade-offs of their proposals. In a sense, the process resembles the Metropolis–Hastings accept/reject step in MCMC, with humans deciding which proposed solutions to keep.

I'm happy to merge the docstring fixes first, as you proposed, and then have you take on the remaining changes in a separate PR.

sunxd3 added 3 commits July 14, 2026 10:44
Revert the tilde-site guard, the 0.42.2 bump, and the error tests, per
the PR discussion: the guard misses most partial cases (noncontiguous
indices, matrix-variate, submodel prefixes) and rejects valid batch
observations. Keep the docstring notes on whole-variable semantics for
predict and fix. The behaviour-side check moves to a follow-up PR via
InitFromParams.
predict: partial chain values mean the whole variable is silently
resampled and the predictions look plausible while ignoring the chain.
fix: a partial fix may silently collapse the variable or be ignored.
condition: partial conditioning may abort with an unrelated
DimensionMismatch or be silently ignored.
@sunxd3 sunxd3 changed the title Error informatively on partial specification of a multivariate variable (#2239) Add documentation on partial specification of a multivariate variable (#2239) Jul 14, 2026
@sunxd3 sunxd3 merged commit 5989b33 into main Jul 14, 2026
24 checks passed
@sunxd3 sunxd3 deleted the docs/predict-fix-whole-variable-note branch July 14, 2026 13:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants