We should have a tutorial which goes through the following aspects of a model:
- How to benchmark and profile a model.
- How to choose the right AD backend.
- How to write models in a performant manner.
This is what I'm imagining (please do make suggestions!)
1. Benchmarking and profiling
Should include:
- Usage of TuringBenchmarking.jl.
- Usage of a profiler, e.g. PProf.jl, both in the evaluation of the model itself and in gradient computations.
2. Choosing AD backend
This will be informed by the steps in the previous section.
Also, maybe have some description (or at least point to another resource which outlines) the pros and cons of the different AD backends.
3. Writing performant models
This comes down to a few tricks:
- Keep LHS of
~ "simple", i.e. x ~ filldist(Normal(), n) instead of x[i] ~ Normal() inside a for-loop.
- Use typing information, e.g.
::Type{TV}=Vector{Float64} in the model definition, and so on.
- Checking of type stability, e.g. using
DynamicPPL.DebugUtils.model_typed (or maybe using Cthulu.jl?)
We should have a tutorial which goes through the following aspects of a model:
This is what I'm imagining (please do make suggestions!)
1. Benchmarking and profiling
Should include:
2. Choosing AD backend
This will be informed by the steps in the previous section.
Also, maybe have some description (or at least point to another resource which outlines) the pros and cons of the different AD backends.
3. Writing performant models
This comes down to a few tricks:
~"simple", i.e.x ~ filldist(Normal(), n)instead ofx[i] ~ Normal()inside a for-loop.::Type{TV}=Vector{Float64}in the model definition, and so on.DynamicPPL.DebugUtils.model_typed(or maybe using Cthulu.jl?)