Skip to content

Error informatively on partial specification of a multivariate variable (#2239)#1434

Open
yebai wants to merge 9 commits into
mainfrom
docs/predict-fix-whole-variable-note
Open

Error informatively on partial specification of a multivariate variable (#2239)#1434
yebai wants to merge 9 commits into
mainfrom
docs/predict-fix-whole-variable-note

Conversation

@yebai

@yebai yebai commented Jul 8, 2026

Copy link
Copy Markdown
Member

Fixes the documentation and the behaviour side of TuringLang/Turing.jl#2239.

A variable drawn from a multivariate distribution in a single tilde-statement (e.g. x ~ MvNormal(...), filldist) is a single random variable, not a collection of i.i.d. components. Supplying only part of such a variable — via predict with a chain from a differently sized model, or fix/condition on a subset of indices — should produce an informative error.

A variable drawn from a multivariate distribution in a single tilde-statement
(e.g. `x ~ MvNormal(...)` / `filldist`) is a single random variable, not i.i.d.
components. `predict` therefore cannot fix a subset of its components while
resampling the rest, and `fix` cannot fix them independently. Add warning
admonitions documenting this, referencing TuringLang/Turing.jl#2239.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@codecov

codecov Bot commented Jul 8, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 88.23529% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.69%. Comparing base (d7e84ce) to head (4a9e25a).

Files with missing lines Patch % Lines
src/compiler.jl 87.50% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1434      +/-   ##
==========================================
+ Coverage   81.64%   81.69%   +0.05%     
==========================================
  Files          50       50              
  Lines        3579     3595      +16     
==========================================
+ Hits         2922     2937      +15     
- Misses        657      658       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

DynamicPPL.jl documentation for PR #1434 is available at:
https://TuringLang.github.io/DynamicPPL.jl/previews/PR1434/

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Benchmarks @ 4a9e25a

Performance Ratio: gradient time divided by log-density time.

For very small models these ratios are noisy across runs and machines; raw primal and gradient timings are more reliable. The benchmarks are aimed at DynamicPPL developers and mainly catch obvious allocation or type-stability regressions. See benchmark notes for details.

===================================================================================================
                                               eval                       gradient                 
                                            ----------  -------------------------------------------
Model                        dim    linked      primal     FwdDiff    RvsDiff    Mooncake    Enzyme
---------------------------------------------------------------------------------------------------
Simple assume observe*         1     false     4.64 ns       12.46    1508.83       34.94     12.37
Simple assume observe*         1      true     4.63 ns       12.67    1643.37       35.49     12.19
Smorgasbord                  201     false     5.98 μs       71.55     133.43        6.69      9.54
Smorgasbord                  201      true     7.61 μs       72.96     140.56        6.19      6.88
Loop univariate 1k          1000     false     17.7 μs      976.71     307.77        8.17      6.36
Loop univariate 1k          1000      true     19.1 μs     1404.67     290.90        7.73      5.91
Multivariate 1k             1000     false     23.7 μs      332.48      72.76        8.93      2.85
Multivariate 1k             1000      true     26.6 μs      271.35      59.31        8.50      2.95
Loop univariate 10k        10000     false    172.0 μs    11786.99     335.40        8.47      6.44
Loop univariate 10k        10000      true    187.0 μs    11638.18     310.34        7.92      6.10
Multivariate 10k           10000     false    193.0 μs     5998.09      91.09       11.30      2.34
Multivariate 10k           10000      true    193.0 μs     5750.22      91.26       11.33      2.29
Dynamic                       15     false     1.34 μs         err      43.27       14.79     12.06
Dynamic                       10      true     1.86 μs        1.92      58.34       18.63     19.02
Submodel*                      1     false     4.64 ns       12.52    1615.19       35.00     12.40
Submodel*                      1      true     4.64 ns       12.66    1741.68       35.60     12.25
LDA                           12      true     22.2 μs        0.59       2.01       34.22       err
===================================================================================================
Main @ d7e84ce
===================================================================================================
                                               eval                       gradient                 
                                            ----------  -------------------------------------------
Model                        dim    linked      primal     FwdDiff    RvsDiff    Mooncake    Enzyme
---------------------------------------------------------------------------------------------------
Simple assume observe*         1     false     4.63 ns       12.53    1571.72       35.17     12.66
Simple assume observe*         1      true     4.63 ns       12.52    1693.90       35.33     12.75
Smorgasbord                  201     false      6.0 μs       70.66     134.23        6.85      9.67
Smorgasbord                  201      true     7.58 μs       75.08     149.15        6.18      6.99
Loop univariate 1k          1000     false     17.6 μs      894.24     304.32        8.22      6.77
Loop univariate 1k          1000      true     19.0 μs     1411.55     286.23        7.47      6.26
Multivariate 1k             1000     false     23.3 μs      344.15      61.65        9.46      3.05
Multivariate 1k             1000      true     28.9 μs      267.88      57.80        7.78      3.12
Loop univariate 10k        10000     false    171.0 μs    11366.52     334.19        8.51      6.85
Loop univariate 10k        10000      true    186.0 μs    11998.43     311.86        7.70      6.31
Multivariate 10k           10000     false    195.0 μs     5817.81      89.54       11.47      2.28
Multivariate 10k           10000      true    195.0 μs     5104.85      89.44       11.44      2.30
Dynamic                       15     false     1.42 μs         err      42.94       15.67     11.03
Dynamic                       10      true     1.96 μs        1.90      60.16       17.66     20.18
Submodel*                      1     false     4.64 ns       12.50    1705.78       36.24     12.52
Submodel*                      1      true     4.64 ns       10.18    1857.75       33.37     12.62
LDA                           12      true     23.5 μs        0.57       1.99       32.29       err
===================================================================================================
Environment
Julia Version 1.11.9
Commit 53a02c0720c (2026-02-06 00:27 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 4 × AMD EPYC 7763 64-Core Processor
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

yebai and others added 5 commits July 8, 2026 12:58
…s specified

Supplying a subset of a variable that the model samples as a single multivariate
draw (e.g. `x[1:10]` when the model draws `x ~ MvNormal(zeros(20), ...)`) has never
worked: `predict` silently resampled the whole variable from the prior, `fix`
silently collapsed it to the supplied length, and `condition` threw an opaque
`DimensionMismatch`. This is therefore not a behaviour change — it only replaces
those silent or confusing failures with a single, informative error.

Add `_check_supplied_shape(dist, supplied, vn)`, dispatched on the supplied
representation (a materialised value for `condition`/`fix`, or the parameter
`VarNamedTuple` for `predict`/`InitFromParams`), which throws one clear error
referencing TuringLang/Turing.jl#2239. Only multivariate distributions are checked;
per-index (`x[i] ~ ...`) declarations and correctly-sized whole-variable values are
unaffected.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Cover the #2239 misuse cases: predicting with a chain from a differently-sized
model, and `fix`/`condition` of a single index of a multivariate variable, all
now raise the informative error. A whole-variable `fix` of the correct size is
kept as a positive control.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
No behaviour change. Shorten the `_check_supplied_shape` docstring and the
`init` comment, and reduce the predict misuse test to the smallest model that
still triggers the error (just the multivariate variable, no extra latent or
observation).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Document the new informative error for partial specification of a multivariate
variable via condition/fix/predict (Turing#2239). Non-breaking: the case never
worked, so this only replaces a silent or opaque failure with a clear message.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The docstring notes previously described the old silent behaviour (predict
resampling the whole variable from the prior); update them to state that
supplying only part of a multivariate variable now raises an error.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@yebai yebai changed the title docs: document whole-variable semantics of predict and fix Error informatively on partial specification of a multivariate variable (#2239) Jul 8, 2026
yebai and others added 3 commits July 8, 2026 13:00
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Keep the error self-contained and actionable (the loop workaround); the #2239
reference stays in the docstrings and HISTORY. Tests now match on a stable
phrase from the message rather than the issue number.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Address review feedback: raise `ArgumentError` (consistent with `check_tilde_rhs`
/`check_dot_tilde_rhs`) instead of a generic `error`, and match it by type in the
tests rather than by a fragile message substring. Add a regression test that
per-index (`.~`) variables still grow correctly under `predict` without being
falsely flagged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@yebai

yebai commented Jul 8, 2026

Copy link
Copy Markdown
Member Author

@sunxd3, can you help review this?

@sunxd3

sunxd3 commented Jul 8, 2026

Copy link
Copy Markdown
Member

Yes!

@sunxd3

sunxd3 commented Jul 13, 2026

Copy link
Copy Markdown
Member

I'm not sure the macro-level guard is the right place for this check — it both misses cases and catches one it shouldn't.

It only runs when hasconditioned_nested/hasfixed_nested finds a value, but partial specification usually makes the lookup fail (the PartialArray mask isn't fully set), so the variable is treated as an assumption and the guard never fires. Matrix-variate and submodel-prefixed variables slip through similarly.

On the other side, a conditioned value is an observation, and the observe pipeline accepts batches via Distributions.loglikelihood — the exact-size check turns those into errors:

@model mvc(n) = m ~ MvNormal(zeros(n), I)

# errors on this branch:
fix(mvc(3), Dict(@varname(m[1]) => 1.0))()
# silently ignored, m sampled from the prior:
fix(mvc(3), Dict(@varname(m[1]) => 1.0, @varname(m[3]) => 3.0))()

# two draws, summed by loglikelihood — works on main, errors on this branch:
loglikelihood(condition(mvc(3), (; m = [1.0 2.0; 3.0 4.0; 5.0 6.0])), (;))

Getting condition/fix right would need the tilde pipeline to tell "absent" apart from "partially present", and to treat observe-side shapes differently from assume-side — that's on the hot path. Honestly not sure it's worth fixing at all; if it is, definitely its own PR.

So maybe we scope this down: keep the PR documentation-only (note the whole-variable semantics in the fix/predict docstrings — condition's docstring already has this), or perhaps also fix predict from the init interface — a check in InitFromParams (src/contexts/init.jl) can be made complete there: raw values must be exactly one draw of size(dist) for any ArrayLikeVariate, and error when params cover only part of a variable, submodel paths included. I can take over and implement this.

(prepared with CC + Fable)

@yebai

yebai commented Jul 13, 2026

Copy link
Copy Markdown
Member Author

Thanks, @sunxd3. It would be great to find a robust, principled solution that catches unintended uses of fix/condition/predict, etc.

One approach that often works well is to brainstorm with coding agents: give them a clear design goal or constraint, ask them to explore feasible solutions, and then evaluate the trade-offs of their proposals. In a sense, the process resembles the Metropolis–Hastings accept/reject step in MCMC, with humans deciding which proposed solutions to keep.

I'm happy to merge the docstring fixes first, as you proposed, and then have you take on the remaining changes in a separate PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants